YJIT Benchmarks

Details for Benchmarks at 2022-10-30 04:26:24 GMT

YJIT metrics from the yjit-bench suite using Ruby 91c28ab2ee.

Overall YJIT is 34.6% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 37.1% faster than CRuby!

Performance on Headline Benchmarks

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 No JIT YJIT activerecord hexapdf liquid-render mail psych-load railsbench ruby-lsp
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 CRuby 3.2.0dev YJIT 3.2.0dev activerecord hexapdf liquid-render mail psych-load railsbench ruby-lsp geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

0.0 0.5 1.0 1.5 2.0 No JIT YJIT binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.2.0dev YJIT 3.2.0dev binarytrees chunky_png erubi erubi_rails etanni fannkuchredux lee nbody optcarrot rubykon geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

0.0 2.0 4.0 6.0 8.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar str_concat
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 CRuby 3.2.0dev YJIT 3.2.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar str_concat geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 154.1 0.58% 97.7 0.74% 1.58x 0.94% 90.08%
hexapdf 3247.0 0.92% 2342.8 1.95% 1.39x 2.15% 86.95%
liquid-render 192.3 0.97% 112.3 1.86% 1.71x 2.10% 86.95%
mail 176.1 0.15% 156.5 0.12% 1.13x 0.19% 98.53%
psych-load 2484.5 0.04% 1914.7 0.02% 1.30x 0.05% 99.99%
railsbench 2771.0 0.88% 2021.2 1.40% 1.37x 1.65% 88.59%
ruby-lsp 85.0 12.19% 79.6 15.89% 1.07x 20.03% 63.47%
binarytrees 464.8 1.82% 230.2 3.71% 2.02x 4.13% 100.00%
chunky_png 960.6 0.04% 624.3 0.05% 1.54x 0.07% 99.99%
erubi 347.2 0.63% 296.4 0.82% 1.17x 1.04% 100.00%
erubi_rails 26.6 2.71% 18.6 4.07% 1.43x 4.89% 86.79%
etanni 480.2 0.41% 479.0 0.49% 1.00x 0.64% 7.03%
fannkuchredux 5251.3 0.04% 2285.5 0.23% 2.30x 0.23% 100.00%
lee 1321.1 0.14% 971.2 1.69% 1.36x 1.70% 99.97%
nbody 132.0 0.11% 81.3 0.07% 1.62x 0.13% 100.00%
optcarrot 6439.5 0.45% 3035.5 0.39% 2.12x 0.59% 96.70%
rubykon 12623.9 0.46% 6798.1 0.43% 1.86x 0.63% 99.89%
30k_ifelse 2271.9 0.02% 395.4 0.10% 5.75x 0.10% 100.00%
30k_methods 6246.1 0.02% 951.2 0.05% 6.57x 0.05% 100.00%
cfunc_itself 108.0 0.29% 40.9 0.13% 2.64x 0.31% 100.00%
fib 258.1 0.03% 62.0 0.08% 4.16x 0.09% 100.00%
getivar 111.1 0.22% 46.9 0.05% 2.37x 0.23% 100.00%
keyword_args 292.1 0.07% 52.0 0.11% 5.61x 0.13% 100.00%
respond_to 266.3 0.47% 28.5 0.43% 9.34x 0.63% 100.00%
setivar 82.9 0.50% 48.3 0.36% 1.72x 0.62% 100.00%
str_concat 83.9 1.73% 46.1 1.52% 1.82x 2.30% 99.83%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.2.0dev (2022-10-29T19:47:16Z master 91c28ab2ee) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

bench CRuby 3.2.0dev mem (MiB) YJIT 3.2.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 65 88 3 3 35.9%
hexapdf 286 412 2 2 43.8%
liquid-render 31 38 1 1 22.5%
mail 49 61 2 2 23.4%
psych-load 38 45 1 1 16.5%
railsbench 103 157 6 6 52.2%
ruby-lsp 90 152 6 6 67.7%
binarytrees 29 31 1 1 7.1%
chunky_png 44 53 1 1 19.8%
erubi 93 91 1 1 -2.7%
erubi_rails 101 144 5 5 42.7%
etanni 98 102 1 1 4.0%
fannkuchredux 25 27 1 1 7.8%
lee 33 45 1 1 37.6%
nbody 25 27 1 1 10.0%
optcarrot 63 70 1 1 11.2%
rubykon 57 61 1 1 6.8%
30k_ifelse 67 128 6 6 89.8%
30k_methods 58 79 2 2 37.1%
cfunc_itself 25 27 1 1 7.7%
fib 25 27 1 1 8.1%
getivar 25 27 1 1 8.9%
keyword_args 25 26 1 1 6.7%
respond_to 25 27 1 1 7.0%
setivar 25 27 1 1 7.5%
str_concat 106 108 1 1 1.4%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

bench No JIT warmups No JIT iters YJIT warmups YJIT iters
activerecord 5 203 5 203
hexapdf 5 15 5 15
liquid-render 5 172 5 172
mail 5 120 5 120
psych-load 5 15 5 15
railsbench 5 15 5 15
ruby-lsp 5 246 5 246
binarytrees 5 87 5 87
chunky_png 5 32 5 32
erubi 5 65 5 65
erubi_rails 5 1082 5 1082
etanni 5 41 5 41
fannkuchredux 5 15 5 15
lee 5 20 5 20
nbody 5 245 5 245
optcarrot 5 15 5 15
rubykon 5 15 5 15
30k_ifelse 5 50 5 50
30k_methods 5 20 5 20
cfunc_itself 5 494 5 494
fib 5 327 5 327
getivar 5 427 5 427
keyword_args 5 386 5 386
respond_to 5 691 5 691
setivar 5 413 5 413
str_concat 5 459 5 459

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps
activerecord (click) 2316158 2313318 23 366 6 1% 0 0 0
hexapdf (click) 1773141 1771533 817 10054 189 1% 0 0 0
liquid-render (click) 590838 590296 154 1768 34 1% 0 0 0
mail (click) 1338131 1336865 371 5083 176 3% 0 0 0
psych-load (click) 806853 805537 68 573 19 3% 0 0 0
railsbench (click) 5536565 5535702 1777 13703 483 3% 0 0 0
ruby-lsp (click) 5603443 5602057 6423 41569 1521 3% 12554 0 0
binarytrees (click) 128452 126491 14 79 4 5% 0 0 0
chunky_png (click) 822108 820931 87 1061 31 2% 0 0 0
erubi (click) 749969 748549 11 87 4 4% 0 0 0
erubi_rails (click) 4635714 4634797 327 2619 111 4% 0 0 0
etanni (click) 129355 126501 11 47 4 8% 0 0 0
fannkuchredux (click) 140491 139998 11 209 4 1% 0 0 0
lee (click) 833467 831447 77 758 25 3% 0 0 0
nbody (click) 136187 134584 13 188 6 3% 0 0 0
optcarrot (click) 562979 561263 201 4014 93 2% 0 0 0
rubykon (click) 279445 278987 144 1592 32 2% 0 0 0
30k_ifelse (click) 5420144 5419577 9266 57136 4 0% 0 0 0
30k_methods (click) 2088352 2087875 5785 19365 4 0% 0 0 0
cfunc_itself (click) 126387 125081 11 62 4 6% 0 0 0
fib (click) 117838 116641 11 51 4 7% 0 0 0
getivar (click) 127011 125607 11 78 4 5% 0 0 0
keyword_args (click) 127178 125485 12 64 4 6% 0 0 0
respond_to (click) 126914 125448 11 78 5 6% 0 0 0
setivar (click) 120073 117975 11 51 4 7% 0 0 0
str_concat (click) 119517 118078 13 77 6 7% 0 0 0

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.