YJIT Benchmarks

2021-11-30 19:10:50

As of 2021-11-30 19:10:50:

Overall YJIT is 26.1% faster than interpreted CRuby, or 21.3% faster than MJIT (3.0)!
On Railsbench specifically, YJIT is 19.8% faster than CRuby, 16.6% faster than MJIT (3.0)!

The basic "faster" measurement is the geomean of all "headlining" benchmarks on this page.

Headlining Benchmarks

These are "headlining" because the "overall" speedup above is based on these benchmarks specifically.

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 No JIT MJIT3.0 MJIT3.1 YJIT activerecord hexapdf jekyll liquid-render mail psych-load railsbench Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

MicroBenchmarks

0.0 2.0 4.0 6.0 8.0 No JIT MJIT3.0 MJIT3.1 YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

Other Benchmarks

0.0 0.5 1.0 1.5 2.0 No JIT MJIT3.0 MJIT3.1 YJIT binarytrees erubi erubi_rails fannkuchredux lee nbody optcarrot rubykon Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

The y axis is CRuby's pure-interpreter time for the benchmark divided by the named interpreter's time (YJIT, MJIT, TruffleRuby). Taller bars mean better performance. Each benchmark is scaled to the CRuby's interpreter's mean result, so CRuby's results are always exactly 1.0.

bench No JIT (ms) No JIT RSD MJIT3.0 (ms) MJIT3.0 RSD MJIT3.1 (ms) MJIT3.1 RSD YJIT (ms) YJIT RSD MJIT3.0 spd MJIT3.0 spd RSD MJIT3.1 spd MJIT3.1 spd RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 146.4 0.89% 164.2 1.36% 176.4 4.50% 108.6 1.50% 0.89x 1.62% 0.83x 4.59% 1.35x 1.74% 81.84%
hexapdf 2815.5 1.11% 2563.7 1.77% 2194.4 2.10% 1.10x 2.09% 1.28x 2.37% 74.81%
jekyll 8512.7 1.81% 8210.5 4.04% 8420.9 3.73% 7443.1 2.56% 1.04x 4.43% 1.01x 4.15% 1.14x 3.14% 82.13%
liquid-render 174.4 2.03% 159.1 2.31% 158.0 3.86% 118.3 2.46% 1.10x 3.07% 1.10x 4.36% 1.47x 3.19% 84.87%
mail 155.9 0.37% 148.5 0.80% 161.0 2.08% 142.2 0.29% 1.05x 0.88% 0.97x 2.11% 1.10x 0.47% 98.33%
psych-load 2156.6 1.56% 1964.6 0.68% 1854.4 0.36% 1626.7 0.24% 1.10x 1.70% 1.16x 1.60% 1.33x 1.58% 82.03%
railsbench 3037.9 1.14% 2956.9 0.93% 3264.9 1.15% 2535.4 1.35% 1.03x 1.47% 0.93x 1.62% 1.20x 1.77% 80.25%
binarytrees 371.4 0.11% 234.3 0.13% 236.5 0.07% 281.2 2.87% 1.59x 0.17% 1.57x 0.13% 1.32x 2.87% 84.27%
erubi 430.9 1.70% 426.9 1.15% 395.9 3.82% 431.8 1.23% 1.01x 2.05% 1.09x 4.18% 1.00x 2.10% 5.62%
erubi_rails 42.2 13.67% 49.0 4.99% 42.3 3.31% 33.6 3.69% 0.86x 14.55% 1.00x 14.06% 1.26x 14.16% 82.86%
fannkuchredux 5517.6 0.18% 4071.9 0.07% 5393.3 0.13% 5538.1 0.43% 1.36x 0.19% 1.02x 0.22% 1.00x 0.47% 0.02%
lee 1020.0 0.95% 952.8 0.88% 834.7 1.17% 745.8 1.46% 1.07x 1.29% 1.22x 1.51% 1.37x 1.74% 99.93%
nbody 102.4 0.07% 60.6 0.61% 58.3 0.06% 73.2 0.04% 1.69x 0.61% 1.76x 0.09% 1.40x 0.08% 100.00%
optcarrot 5272.5 0.45% 2206.7 0.71% 2156.8 0.83% 3063.4 0.60% 2.39x 0.84% 2.44x 0.94% 1.72x 0.75% 96.13%
rubykon 10322.6 0.44% 6472.8 0.95% 6805.3 0.54% 5746.7 0.37% 1.59x 1.04% 1.52x 0.69% 1.80x 0.57% 99.90%
30k_ifelse 2461.9 3.47% 2201.2 2.23% 3870.7 5.97% 319.8 0.08% 1.12x 4.13% 0.64x 6.91% 7.70x 3.47% 100.00%
30k_methods 6324.7 1.18% 6018.7 0.91% 6730.5 0.98% 741.4 0.05% 1.05x 1.49% 0.94x 1.53% 8.53x 1.18% 100.00%
cfunc_itself 89.3 0.22% 60.1 0.10% 64.2 0.13% 39.0 0.27% 1.49x 0.25% 1.39x 0.26% 2.29x 0.35% 100.00%
fib 215.3 0.04% 59.9 0.04% 62.5 0.09% 52.1 0.05% 3.59x 0.06% 3.45x 0.10% 4.13x 0.07% 100.00%
getivar 96.5 0.06% 90.3 0.17% 96.5 0.05% 35.5 0.03% 1.07x 0.18% 1.00x 0.08% 2.72x 0.06% 98.94%
keyword_args 258.2 0.06% 187.4 0.06% 190.4 0.05% 47.2 0.11% 1.38x 0.08% 1.36x 0.08% 5.47x 0.13% 100.00%
respond_to 247.4 0.40% 189.7 0.28% 201.1 0.41% 157.1 0.15% 1.30x 0.49% 1.23x 0.57% 1.57x 0.43% 100.00%
setivar 64.9 0.13% 68.1 0.09% 65.4 0.18% 39.2 0.32% 0.95x 0.16% 0.99x 0.23% 1.66x 0.35% 98.42%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means it runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.1.0dev (2021-11-30T11:54:05Z master 7fd88da935) +YJIT [x86_64-linux]
Tested Ruby version for Ruby 3.1 MJIT: ruby 3.1.0dev (2021-11-30T11:54:05Z master 7fd88da935) +JIT [x86_64-linux]
Tested Ruby version for Ruby 3.0 MJIT: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) +JIT [x86_64-linux]
(We got much better MJIT results with released than prerelease, so we used those.)

Benchmark Memory Usage Details

bench No JIT mem (MiB) MJIT3.0 mem (MiB) MJIT3.1 mem (MiB) YJIT mem (MiB)
activerecord 61 60 69 322
hexapdf 295 349 684
jekyll 835 1451 2037 1673
liquid-render 30 29 34 289
mail 48 48 52 306
psych-load 37 35 44 295
railsbench 104 102 139 376
binarytrees 28 28 28 284
erubi 71 74 107 337
erubi_rails 108 118 124 377
fannkuchredux 23 21 23 279
lee 36 33 35 293
nbody 23 22 23 279
optcarrot 76 79 85 335
rubykon 66 81 72 324
30k_ifelse 64 63 185 351
30k_methods 55 55 179 322
cfunc_itself 23 22 23 279
fib 23 21 23 279
getivar 23 22 23 279
keyword_args 23 22 23 279
respond_to 22 22 23 279
setivar 23 22 23 279

Memory is shown in mebibytes (1024 * 1024 bytes.)

By default, YJIT allocates an additional 256MiB for generated code. The additional size allocated can be tuned with a command-line parameter, but allocating too little memory will result in YJIT crashing.

Number of Iterations and Warmups Tested

bench No JIT warmups No JIT iters MJIT3.0 warmups MJIT3.0 iters MJIT3.1 warmups MJIT3.1 iters YJIT warmups YJIT iters
activerecord 5 186 20 186 75 186 20 186
hexapdf 5 15 20 15 20 15
jekyll 5 15 20 15 36 15 20 15
liquid-render 5 172 20 172 75 172 20 172
mail 5 140 20 140 75 140 20 140
psych-load 5 15 20 15 75 15 20 15
railsbench 5 15 20 15 75 15 20 15
binarytrees 5 88 20 88 75 88 20 88
erubi 5 51 20 51 75 51 20 51
erubi_rails 5 558 20 558 75 558 20 558
fannkuchredux 5 15 20 15 54 15 20 15
lee 5 26 20 26 75 26 20 26
nbody 5 342 20 342 75 342 20 342
optcarrot 5 15 20 15 75 15 20 15
rubykon 5 15 20 15 42 15 20 15
30k_ifelse 5 62 20 62 74 62 20 62
30k_methods 5 26 20 26 44 26 20 26
cfunc_itself 5 492 20 492 75 492 20 492
fib 5 383 20 383 75 383 20 383
getivar 5 563 20 563 75 563 20 563
keyword_args 5 429 20 429 75 429 20 429
respond_to 5 126 20 126 75 126 20 126
setivar 5 511 20 511 75 511 20 511

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the column headings for descriptions of each statistic.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps
activerecord (click) 972010 730551 185 1194 0 0% 0 0 0
hexapdf (click) 1076074 857247 595 7993 616 7% 0 0 321
jekyll (click) 1884906 1497990 336 3925 179 4% 0 0 0
liquid-render (click) 472119 366383 143 1459 96 6% 0 0 1
mail (click) 848810 625981 325 5206 124 2% 0 0 39
psych-load (click) 291361 226160 57 423 1 0% 0 0 0
railsbench (click) 2577058 1922778 1293 9740 263 2% 0 0 26
binarytrees (click) 112293 85408 9 48 0 0% 0 0 0
erubi (click) 269537 209076 8 35 0 0% 0 0 0
erubi_rails (click) 2437090 1742413 260 1995 17 0% 0 0 3
fannkuchredux (click) 128869 97767 6 202 0 0% 0 0 0
lee (click) 330218 256770 44 562 67 11% 0 0 12
nbody (click) 121701 92379 8 148 0 0% 0 0 0
optcarrot (click) 451109 392134 196 3566 20 0% 0 0 0
rubykon (click) 254181 204277 138 1526 1 0% 0 0 0
30k_ifelse (click) 5502053 4306081 9261 57794 0 0% 0 0 0
30k_methods (click) 2116581 1614194 5780 19351 0 0% 0 0 0
cfunc_itself (click) 111461 84386 6 39 0 0% 0 0 0
fib (click) 109349 83645 6 28 0 0% 0 0 0
getivar (click) 111653 86505 6 55 0 0% 0 0 0
keyword_args (click) 111717 84678 7 41 0 0% 0 0 0
respond_to (click) 113317 85819 6 54 0 0% 0 0 0
setivar (click) 111013 84680 6 28 0 0% 0 0 0

YJIT stats correspond to the YJIT stats exit report.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.