YJIT Benchmarks

Details for Benchmarks at 2023-11-15 06:08:15 GMT

YJIT metrics from the yjit-bench suite using Ruby 536649f819.

Overall YJIT is 63.6% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 68.3% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 No JIT YJIT activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 CRuby 3.3.0dev YJIT 3.3.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 No JIT YJIT binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 CRuby 3.3.0dev YJIT 3.3.0dev binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.3.0dev YJIT 3.3.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 81.3 1.91% 40.6 4.11% 2.00x 4.54% 99.96%
chunky-png 1051.6 0.29% 621.1 0.54% 1.69x 0.61% 100.00%
erubi-rails 2619.2 0.71% 1380.4 1.31% 1.90x 1.49% 99.93%
hexapdf 3484.2 0.60% 2045.4 2.83% 1.70x 2.90% 94.24%
liquid-c 81.5 0.69% 57.9 0.70% 1.41x 0.99% 99.74%
liquid-compile 81.5 4.08% 58.4 4.72% 1.39x 6.24% 99.95%
liquid-render 203.5 0.22% 85.2 0.49% 2.39x 0.54% 99.86%
lobsters 1274.7 2.01% 845.7 3.93% 1.51x 4.41% 99.53%
mail 174.5 0.23% 121.7 0.40% 1.43x 0.47% 99.90%
psych-load 2818.3 0.06% 1821.8 0.10% 1.55x 0.11% 100.00%
railsbench 2773.1 0.51% 1647.5 1.03% 1.68x 1.15% 99.55%
ruby-lsp 80.0 2.46% 51.4 5.34% 1.56x 5.88% 92.82%
sequel 91.7 1.10% 68.5 1.67% 1.34x 2.00% 99.92%
binarytrees 492.0 0.09% 237.4 0.16% 2.07x 0.18% 100.00%
erubi 307.5 0.13% 252.8 0.14% 1.22x 0.20% 99.98%
etanni 433.1 2.07% 376.7 0.08% 1.15x 2.07% 99.98%
fannkuchredux 2213.0 0.11% 779.9 0.14% 2.84x 0.18% 90.92%
fluentd 2493.7 0.78% 2291.1 0.62% 1.09x 1.00% 99.99%
graphql 4051.9 0.02% 3388.8 0.04% 1.20x 0.05% 99.75%
graphql-native 565.2 0.08% 470.0 0.10% 1.20x 0.13% 99.97%
lee 1375.8 1.03% 981.2 1.42% 1.40x 1.76% 99.98%
nbody 127.5 0.08% 69.2 0.03% 1.84x 0.09% 99.95%
optcarrot 7496.8 0.57% 2043.4 0.65% 3.67x 0.86% 99.50%
rack 136.3 0.89% 107.3 1.16% 1.27x 1.46% 99.72%
ruby-json 3962.6 0.03% 3316.8 0.02% 1.19x 0.04% 99.99%
rubykon 13746.4 1.14% 6669.2 0.50% 2.06x 1.25% 99.95%
tinygql 889.5 0.16% 458.8 0.09% 1.94x 0.18% 99.99%
30k_ifelse 2392.1 0.06% 357.2 0.08% 6.70x 0.10% 99.99%
30k_methods 6626.4 0.02% 831.3 0.03% 7.97x 0.03% 99.99%
cfunc_itself 115.8 0.16% 26.5 0.04% 4.37x 0.17% 99.30%
fib 283.2 0.05% 47.7 0.44% 5.93x 0.44% 100.00%
getivar 137.6 0.04% 21.1 0.56% 6.53x 0.56% 97.87%
keyword_args 314.0 0.05% 40.3 0.15% 7.79x 0.15% 99.55%
respond_to 293.2 0.20% 15.0 0.15% 19.54x 0.25% 99.71%
setivar 89.1 0.17% 12.0 0.07% 7.43x 0.19% 98.73%
setivar_object 122.2 15.76% 51.1 30.48% 2.39x 34.31% 94.84%
setivar_young 121.4 14.71% 50.3 27.22% 2.41x 30.94% 94.92%
str_concat 85.6 0.76% 41.6 1.39% 2.06x 1.58% 99.98%
throw 31.6 0.27% 23.9 0.36% 1.32x 0.45% 99.99%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.3.0dev (2023-11-15T05:58:51Z :detached: 536649f819) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.0dev mem (MiB) YJIT 3.3.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 48 54 1 1 10.8%
chunky-png 57 72 1 1 27.2%
erubi-rails 92 101 2 2 9.2%
hexapdf 128 293 2 2 128.4%
liquid-c 36 39 1 1 8.6%
liquid-compile 36 38 1 1 6.8%
liquid-render 34 38 1 1 11.4%
lobsters 275 324 7 7 17.8%
mail 52 55 1 1 6.4%
psych-load 35 37 1 1 6.6%
railsbench 90 104 3 3 15.6%
ruby-lsp 94 129 5 5 36.6%
sequel 37 40 1 1 9.3%
binarytrees 25 24 1 1 -6.4%
erubi 29 31 1 1 7.1%
etanni 23 24 1 1 1.8%
fannkuchredux 20 20 1 1 1.3%
fluentd 606 657 1 1 8.4%
graphql 38 42 1 1 12.3%
graphql-native 35 38 1 1 6.9%
lee 34 36 1 1 6.1%
nbody 20 20 1 1 1.1%
optcarrot 59 65 1 1 11.4%
rack 30 32 1 1 8.1%
ruby-json 21 21 1 1 1.0%
rubykon 48 49 1 1 3.0%
tinygql 28 30 1 1 7.1%
30k_ifelse 60 91 5 5 49.9%
30k_methods 49 58 2 2 19.4%
cfunc_itself 20 20 1 1 1.0%
fib 20 20 1 1 1.0%
getivar 20 20 1 1 1.0%
keyword_args 20 20 1 1 1.0%
respond_to 20 20 1 1 1.0%
setivar 20 20 1 1 1.0%
setivar_object 20 20 1 1 1.0%
setivar_young 20 20 1 1 1.0%
str_concat 46 47 1 1 1.0%
throw 20 20 1 1 1.0%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

Benchmark YJIT Stats

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.