YJIT Benchmarks

Details for Benchmarks at 2022-12-12 20:09:59 GMT

YJIT metrics from the yjit-bench suite using Ruby 3de7ff8eb9.

Overall YJIT is 41.4% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 37.0% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 No JIT YJIT activerecord hexapdf liquid-render mail psych-load railsbench ruby-lsp
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.2.0dev YJIT 3.2.0dev activerecord hexapdf liquid-render mail psych-load railsbench ruby-lsp geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 No JIT YJIT binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 CRuby 3.2.0dev YJIT 3.2.0dev binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
0.0 2.0 4.0 6.0 8.0 10.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object str_concat
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 CRuby 3.2.0dev YJIT 3.2.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object str_concat geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 162.3 0.29% 101.5 0.99% 1.60x 1.03% 90.65%
hexapdf 3217.9 0.65% 2152.8 1.91% 1.49x 2.02% 93.18%
liquid-render 198.1 0.54% 113.4 1.56% 1.75x 1.65% 86.97%
mail 172.6 0.08% 142.3 0.09% 1.21x 0.12% 97.21%
psych-load 2566.4 0.06% 1841.8 0.05% 1.39x 0.08% 99.99%
railsbench 2772.3 0.57% 2024.0 1.01% 1.37x 1.16% 90.69%
ruby-lsp 81.9 8.76% 70.0 18.34% 1.17x 20.33% 65.60%
binarytrees 470.3 0.05% 224.9 0.09% 2.09x 0.10% 100.00%
chunky_png 942.0 0.03% 613.2 0.06% 1.54x 0.07% 100.00%
erubi 360.9 4.73% 286.3 0.86% 1.26x 4.81% 100.00%
erubi_rails 26.0 2.71% 17.5 3.88% 1.49x 4.74% 87.69%
etanni 469.6 0.65% 473.2 0.56% 0.99x 0.85% 7.03%
fannkuchredux 2128.1 0.13% 1025.7 0.10% 2.07x 0.17% 100.00%
lee 1229.5 1.38% 895.9 1.84% 1.37x 2.30% 99.97%
nbody 127.2 0.11% 71.1 0.04% 1.79x 0.12% 100.00%
optcarrot 6260.6 0.43% 2599.2 0.42% 2.41x 0.60% 96.81%
rubykon 12700.2 0.59% 6681.0 0.34% 1.90x 0.68% 99.69%
30k_ifelse 2419.2 0.02% 399.8 0.13% 6.05x 0.13% 100.00%
30k_methods 6371.9 0.02% 966.2 0.03% 6.59x 0.03% 100.00%
cfunc_itself 97.0 0.05% 41.4 0.33% 2.34x 0.33% 100.00%
fib 247.0 0.06% 62.8 0.06% 3.93x 0.08% 100.00%
getivar 107.2 0.40% 45.8 0.32% 2.34x 0.51% 100.00%
keyword_args 284.6 0.07% 52.0 0.31% 5.47x 0.32% 100.00%
respond_to 287.0 0.55% 28.5 0.57% 10.05x 0.79% 100.00%
setivar 61.2 0.24% 14.7 0.09% 4.16x 0.26% 100.00%
setivar_object 106.6 12.94% 52.9 21.14% 2.01x 24.78% 100.00%
str_concat 79.9 0.90% 40.6 1.62% 1.97x 1.85% 99.81%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.2.0dev (2022-12-13T09:07:41Z master 3de7ff8eb9) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.2.0dev mem (MiB) YJIT 3.2.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 63 81 3 3 28.6%
hexapdf 241 353 2 2 46.2%
liquid-render 29 35 1 1 18.2%
mail 48 57 2 2 17.9%
psych-load 38 42 1 1 12.9%
railsbench 102 133 6 6 30.9%
ruby-lsp 93 137 6 6 46.1%
binarytrees 29 31 1 1 4.6%
chunky_png 43 50 1 1 16.8%
erubi 63 84 1 1 33.7%
erubi_rails 101 124 5 5 22.8%
etanni 110 113 1 1 2.2%
fannkuchredux 23 25 1 1 5.6%
lee 34 43 1 1 24.1%
nbody 24 25 1 1 5.3%
optcarrot 61 68 1 1 11.0%
rubykon 54 58 1 1 5.7%
30k_ifelse 64 108 6 6 67.9%
30k_methods 56 71 2 2 27.2%
cfunc_itself 23 25 1 1 6.4%
fib 24 25 1 1 5.4%
getivar 24 25 1 1 5.9%
keyword_args 23 25 1 1 6.1%
respond_to 23 25 1 1 6.0%
setivar 23 25 1 1 5.4%
setivar_object 23 25 1 1 7.4%
str_concat 102 111 1 1 8.7%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

bench No JIT warmups No JIT iters YJIT warmups YJIT iters
activerecord 5 200 5 200
hexapdf 5 15 5 15
liquid-render 5 154 5 154
mail 5 140 5 140
psych-load 5 15 5 15
railsbench 5 15 5 15
ruby-lsp 5 245 5 245
binarytrees 5 87 5 87
chunky_png 5 32 5 32
erubi 5 68 5 68
erubi_rails 5 1128 5 1128
etanni 5 42 5 42
fannkuchredux 5 21 5 21
lee 5 22 5 22
nbody 5 278 5 278
optcarrot 5 15 5 15
rubykon 5 15 5 15
30k_ifelse 5 50 5 50
30k_methods 5 20 5 20
cfunc_itself 5 481 5 481
fib 5 324 5 324
getivar 5 438 5 438
keyword_args 5 383 5 383
respond_to 5 673 5 673
setivar 5 1359 5 1359
setivar_object 5 436 5 436
str_concat 5 500 5 500

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps
activerecord (click) 2276381 2276190 25 374 5 1% 0 0 0
hexapdf (click) 1722389 1721328 819 10428 217 2% 0 0 0
liquid-render (click) 561612 560005 156 1852 32 1% 0 0 0
mail (click) 1247880 1246826 373 4683 176 3% 0 0 0
psych-load (click) 779396 778996 70 651 19 2% 0 0 0
railsbench (click) 5524354 5522600 1779 14282 482 3% 0 0 0
ruby-lsp (click) 5400782 5399176 6428 41787 1537 3% 13554 0 0
binarytrees (click) 96267 94377 16 140 3 2% 0 0 0
chunky_png (click) 797000 796142 89 1185 31 2% 0 0 0
erubi (click) 739411 738614 13 149 4 2% 0 0 0
erubi_rails (click) 4665034 4663163 329 2824 111 3% 0 0 0
etanni (click) 144321 142697 13 108 3 2% 0 0 0
fannkuchredux (click) 108481 107857 14 273 3 1% 0 0 0
lee (click) 794312 794203 79 813 24 2% 0 0 0
nbody (click) 109148 108249 15 245 5 2% 0 0 0
optcarrot (click) 528108 526866 203 4439 92 2% 0 0 0
rubykon (click) 235243 233446 146 1631 30 1% 0 0 0
30k_ifelse (click) 5245212 5243303 9268 57200 3 0% 0 0 0
30k_methods (click) 1993486 1992185 5787 19427 3 0% 0 0 0
cfunc_itself (click) 90316 90207 13 91 3 3% 0 0 0
fib (click) 92325 91583 13 112 3 2% 0 0 0
getivar (click) 94099 92845 13 139 3 2% 0 0 0
keyword_args (click) 94538 93277 14 125 3 2% 0 0 0
respond_to (click) 94263 93008 13 139 4 2% 0 0 0
setivar (click) 93138 92045 13 119 3 2% 0 0 0
setivar_object (click) 93480 92213 13 119 3 2% 0 0 0
str_concat (click) 86795 85302 15 138 5 3% 0 0 0

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.