YJIT Benchmarks

2022-05-09 07:13:14

As of 2022-05-09 07:13:14:

Overall YJIT is 33.3% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 31.8% faster than CRuby!

The basic "faster" measurement is the geomean of all "headlining" benchmarks on this page.

Headlining Benchmarks

These are "headlining" because the "overall" speedup above is based on these benchmarks specifically.

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 No JIT MJIT3.1 YJIT activerecord hexapdf liquid-render mail psych-load railsbench Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

MicroBenchmarks

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 No JIT MJIT3.1 YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar str_concat Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

Other Benchmarks

0.0 0.5 1.0 1.5 2.0 No JIT MJIT3.1 YJIT binarytrees chunky_png discourse erubi erubi_rails fannkuchredux lee nbody optcarrot rubykon Speed of each Ruby implementation (iterations/second) relative to the CRuby interpreter. Higher is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

The y axis is CRuby's pure-interpreter time for the benchmark divided by the named interpreter's time (YJIT, MJIT, TruffleRuby). Taller bars mean better performance. Each benchmark is scaled to the CRuby's interpreter's mean result, so CRuby's results are always exactly 1.0.

bench No JIT (ms) No JIT RSD MJIT3.1 (ms) MJIT3.1 RSD YJIT (ms) YJIT RSD MJIT3.1 spd MJIT3.1 spd RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 151.0 0.35% 175.2 3.62% 105.2 0.61% 0.86x 3.63% 1.44x 0.70% 85.01%
hexapdf 3217.1 0.99% 2365.0 1.32% 1.36x 1.65% 82.49%
liquid-render 192.8 1.08% 182.1 1.47% 127.7 1.88% 1.06x 1.82% 1.51x 2.17% 85.70%
mail 177.1 0.07% 174.9 1.18% 160.8 0.42% 1.01x 1.18% 1.10x 0.42% 99.41%
psych-load 2525.2 0.02% 2260.4 0.04% 1930.4 0.04% 1.12x 0.05% 1.31x 0.04% 88.45%
railsbench 2799.9 1.04% 3122.6 0.93% 2125.1 1.59% 0.90x 1.39% 1.32x 1.90% 86.09%
binarytrees 468.4 1.40% 322.8 0.34% 359.0 1.44% 1.45x 1.44% 1.30x 2.01% 84.27%
chunky_png 1016.1 0.30% 796.2 0.45% 663.2 0.13% 1.28x 0.54% 1.53x 0.33% 99.95%
discourse 544.0 7.18% 590.1 10.89% 495.9 9.17% 0.92x 13.04% 1.10x 11.64% 73.83%
erubi 460.4 0.63% 427.5 0.65% 418.8 0.87% 1.08x 0.90% 1.10x 1.07% 100.00%
erubi_rails 27.8 2.46% 29.4 4.11% 19.8 3.56% 0.95x 4.79% 1.40x 4.33% 86.29%
fannkuchredux 7174.0 0.04% 4066.5 0.07% 7006.7 1.04% 1.76x 0.08% 1.02x 1.04% 0.01%
lee 1220.9 0.36% 1005.5 0.65% 899.5 0.33% 1.21x 0.74% 1.36x 0.48% 99.97%
nbody 123.6 0.05% 72.8 0.03% 91.4 0.42% 1.70x 0.06% 1.35x 0.42% 100.00%
optcarrot 6232.8 0.43% 2620.7 0.59% 3504.3 0.59% 2.38x 0.73% 1.78x 0.73% 96.13%
rubykon 13468.1 0.65% 10518.7 1.60% 7855.2 0.58% 1.28x 1.73% 1.71x 0.87% 99.52%
30k_ifelse 2296.2 0.01% 3467.0 3.76% 368.5 0.18% 0.66x 3.76% 6.23x 0.18% 100.00%
30k_methods 6305.2 0.01% 10083.9 15.79% 880.7 0.06% 0.63x 15.79% 7.16x 0.06% 100.00%
cfunc_itself 99.7 0.20% 69.3 0.37% 44.5 0.94% 1.44x 0.42% 2.24x 0.97% 100.00%
fib 240.0 0.03% 86.1 0.03% 62.6 0.64% 2.79x 0.04% 3.84x 0.64% 100.00%
getivar 116.4 0.07% 26.2 70.99% 42.6 0.71% 4.45x 70.99% 2.73x 0.71% 99.21%
keyword_args 282.1 0.08% 218.5 0.17% 53.0 0.67% 1.29x 0.19% 5.33x 0.68% 100.00%
respond_to 266.0 0.17% 225.3 0.99% 182.1 0.39% 1.18x 1.00% 1.46x 0.43% 100.00%
setivar 78.2 0.31% 10.8 102.09% 53.9 0.63% 7.23x 102.09% 1.45x 0.70% 99.55%
str_concat 132.2 0.67% 101.9 1.35% 108.8 1.45% 1.30x 1.51% 1.22x 1.60% 99.68%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means it runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.2.0dev (2022-05-09T05:19:01Z master cea34bd808) +YJIT [x86_64-linux]
Tested Ruby version for Ruby 3.1 MJIT: ruby 3.2.0dev (2022-05-09T05:19:01Z master cea34bd808) +MJIT [x86_64-linux]
(We got much better MJIT results with released than prerelease, so we used those.)

Benchmark Memory Usage Details

bench No JIT mem (MiB) MJIT3.1 mem (MiB) YJIT mem (MiB)
activerecord 64 72 330
hexapdf 286 613
liquid-render 31 36 293
mail 49 52 311
psych-load 35 45 296
railsbench 103 134 382
binarytrees 28 31 288
chunky_png 42 49 303
discourse 388 395 720
erubi 82 121 373
erubi_rails 103 107 376
fannkuchredux 24 27 282
lee 35 39 295
nbody 24 26 282
optcarrot 58 66 320
rubykon 48 80 308
30k_ifelse 66 167 375
30k_methods 55 182 327
cfunc_itself 24 26 282
fib 24 26 282
getivar 24 26 282
keyword_args 24 27 282
respond_to 24 27 282
setivar 24 26 282
str_concat 90 92 354

Memory is shown in mebibytes (1024 * 1024 bytes.)

By default, YJIT allocates an additional 256MiB for generated code. The additional size allocated can be tuned with a command-line parameter, but allocating too little memory will result in YJIT no longer JITting new functions, so performance reverts toward interpreter speeds.

Number of Iterations and Warmups Tested

bench No JIT warmups No JIT iters MJIT3.1 warmups MJIT3.1 iters YJIT warmups YJIT iters
activerecord 5 191 75 191 20 191
hexapdf 5 15 20 15
liquid-render 5 159 75 159 20 159
mail 5 124 75 124 20 124
psych-load 5 15 75 15 20 15
railsbench 5 15 75 15 20 15
binarytrees 5 63 75 63 20 63
chunky_png 5 30 75 30 20 30
discourse 5 38 75 38 20 38
erubi 5 47 75 47 20 47
erubi_rails 5 1007 75 1007 20 1007
fannkuchredux 5 15 73 15 20 15
lee 5 21 75 21 20 21
nbody 5 262 75 262 20 262
optcarrot 5 15 75 15 20 15
rubykon 5 15 27 15 20 15
30k_ifelse 5 56 75 56 20 56
30k_methods 5 22 29 22 20 22
cfunc_itself 5 446 75 446 20 446
fib 5 319 75 319 20 319
getivar 5 764 75 764 20 764
keyword_args 5 378 75 378 20 378
respond_to 5 106 75 106 20 106
setivar 5 1836 75 1836 20 1836
str_concat 5 189 75 189 20 189

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the column headings for descriptions of each statistic.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps
activerecord (click) 1014107 753082 131 1248 0 0% 0 0 0
hexapdf (click) 1115227 874544 683 7699 9 0% 0 0 0
liquid-render (click) 509339 390169 150 1593 2 0% 0 0 0
mail (click) 879515 637349 374 5218 15 0% 0 0 0
psych-load (click) 339995 255478 68 457 1 0% 0 0 0
railsbench (click) 2617883 1911495 1435 9786 16 0% 0 0 0
binarytrees (click) 172251 129555 13 58 0 0% 0 0 0
chunky_png (click) 379163 283292 88 989 0 0% 0 0 0
discourse (click) 5851419 4337179 3162 24927 81 0% 2 0 0
erubi (click) 322843 242807 10 79 0 0% 0 0 0
erubi_rails (click) 1930907 1389651 300 1971 4 0% 0 0 0
fannkuchredux (click) 185627 139692 10 188 0 0% 0 0 0
lee (click) 377947 283480 58 618 0 0% 0 0 0
nbody (click) 182747 137081 12 158 0 0% 0 0 0
optcarrot (click) 513179 441956 207 3571 20 0% 0 0 0
rubykon (click) 316443 249786 144 1518 1 0% 0 0 0
30k_ifelse (click) 5561563 4349830 9265 57804 0 0% 0 0 0
30k_methods (click) 2183771 1660888 5784 19405 0 0% 0 0 0
cfunc_itself (click) 171035 128135 10 49 0 0% 0 0 0
fib (click) 169499 127743 10 38 0 0% 0 0 0
getivar (click) 171291 130254 10 65 0 0% 0 0 0
keyword_args (click) 171867 128776 11 51 0 0% 0 0 0
respond_to (click) 172827 129373 10 64 0 0% 0 0 0
setivar (click) 170651 128347 10 38 0 0% 0 0 0
str_concat (click) 171227 129098 12 62 0 0% 0 0 0

YJIT stats correspond to the YJIT stats exit report.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.