YJIT Benchmarks

Details for Benchmarks at 2023-02-12 06:08:06 GMT

YJIT metrics from the yjit-bench suite using Ruby 6517d1a188.

Overall YJIT is 41.4% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 48.9% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 No JIT YJIT activerecord hexapdf liquid-c liquid-render mail psych-load railsbench ruby-lsp sequel
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 CRuby 3.3.0dev YJIT 3.3.0dev activerecord hexapdf liquid-c liquid-render mail psych-load railsbench ruby-lsp sequel geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 No JIT YJIT binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.3.0dev YJIT 3.3.0dev binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
0.0 2.0 4.0 6.0 8.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object str_concat
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 CRuby 3.3.0dev YJIT 3.3.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object str_concat geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 183.3 0.80% 108.1 0.51% 1.70x 0.95% 89.49%
hexapdf 3170.8 0.64% 2039.3 2.05% 1.55x 2.14% 88.97%
liquid-c 80.3 1.30% 59.6 2.05% 1.35x 2.43% 93.08%
liquid-render 198.9 1.59% 112.3 2.65% 1.77x 3.09% 87.61%
mail 167.4 0.11% 136.8 0.27% 1.22x 0.29% 99.06%
psych-load 2532.7 0.15% 1806.4 0.15% 1.40x 0.21% 99.99%
railsbench 2767.6 0.69% 1859.3 1.10% 1.49x 1.30% 93.19%
ruby-lsp 82.7 9.31% 65.8 18.98% 1.26x 21.14% 69.46%
sequel 163.9 1.42% 146.1 1.79% 1.12x 2.29% 82.36%
binarytrees 460.2 0.03% 230.0 0.07% 2.00x 0.07% 100.00%
chunky_png 965.4 0.04% 630.6 0.04% 1.53x 0.06% 100.00%
erubi 302.3 0.71% 255.4 0.92% 1.18x 1.16% 100.00%
erubi_rails 25.9 2.41% 16.3 3.33% 1.59x 4.11% 89.49%
etanni 407.9 0.50% 405.2 0.47% 1.01x 0.69% 7.03%
fannkuchredux 2175.2 0.15% 896.5 0.08% 2.43x 0.17% 90.20%
lee 1250.2 0.11% 897.4 0.11% 1.39x 0.16% 99.97%
nbody 131.4 0.69% 71.9 0.07% 1.83x 0.69% 100.00%
optcarrot 8971.6 1.89% 8402.7 0.21% 1.07x 1.90% 18.28%
rubykon 12712.2 0.36% 6606.8 0.22% 1.92x 0.42% 99.77%
30k_ifelse 2346.8 0.01% 422.9 0.07% 5.55x 0.07% 99.99%
30k_methods 6410.1 0.06% 955.4 0.04% 6.71x 0.07% 100.00%
cfunc_itself 101.8 0.15% 42.5 0.11% 2.39x 0.19% 100.00%
fib 240.9 0.03% 62.2 0.06% 3.88x 0.07% 100.00%
getivar 115.6 0.13% 45.8 0.20% 2.52x 0.24% 95.53%
keyword_args 292.0 0.11% 52.7 0.12% 5.54x 0.16% 100.00%
respond_to 258.5 0.58% 29.2 0.48% 8.85x 0.75% 100.00%
setivar 68.4 0.36% 14.9 0.09% 4.59x 0.37% 98.43%
setivar_object 110.5 1.20% 47.5 1.16% 2.33x 1.67% 95.18%
str_concat 80.4 0.65% 43.2 1.30% 1.86x 1.46% 99.62%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.3.0dev (2023-02-11T00:57:56Z :detached: 6517d1a188) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.0dev mem (MiB) YJIT 3.3.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 53 60 1 1 12.4%
hexapdf 235 377 1 1 60.3%
liquid-c 40 44 1 1 10.2%
liquid-render 36 41 1 1 12.3%
mail 48 52 1 1 8.0%
psych-load 36 38 1 1 5.9%
railsbench 101 117 3 3 15.1%
ruby-lsp 92 129 4 4 40.4%
sequel 42 47 1 1 13.0%
binarytrees 29 30 1 1 1.3%
chunky_png 50 47 1 1 -6.2%
erubi 116 124 1 1 7.0%
erubi_rails 100 107 2 2 7.5%
etanni 101 156 1 1 54.6%
fannkuchredux 23 24 1 1 3.0%
lee 34 37 1 1 9.7%
nbody 23 24 1 1 2.6%
optcarrot 58 61 1 1 6.2%
rubykon 59 60 1 1 1.5%
30k_ifelse 65 107 5 5 63.9%
30k_methods 55 69 2 2 25.3%
cfunc_itself 23 24 1 1 2.2%
fib 23 24 1 1 2.2%
getivar 23 24 1 1 2.2%
keyword_args 23 24 1 1 2.2%
respond_to 23 24 1 1 1.8%
setivar 23 24 1 1 1.7%
setivar_object 23 24 1 1 2.2%
str_concat 97 104 1 1 7.6%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

Benchmark YJIT Stats

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.