YJIT Benchmarks

Details for Benchmarks at 2023-06-29 06:08:13 GMT

YJIT metrics from the yjit-bench suite using Ruby 26b69fd407.

Overall YJIT is 49.4% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 53.7% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 No JIT YJIT activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render mail psych-load railsbench ruby-lsp sequel
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 CRuby 3.3.0dev YJIT 3.3.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render mail psych-load railsbench ruby-lsp sequel geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 2.5 No JIT YJIT binarytrees erubi etanni fannkuchredux lee nbody optcarrot ruby-json rubykon
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
0.0 2.0 4.0 6.0 8.0 10.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 CRuby 3.3.0dev YJIT 3.3.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 80.6 1.83% 42.5 3.94% 1.90x 4.35% 94.75%
chunky-png 1009.0 0.20% 649.1 0.41% 1.55x 0.45% 100.00%
erubi-rails 26.4 1.75% 15.6 4.12% 1.69x 4.48% 91.60%
hexapdf 3264.6 0.62% 2099.8 2.25% 1.55x 2.34% 90.54%
liquid-c 80.1 0.59% 58.1 0.80% 1.38x 0.99% 93.41%
liquid-compile 74.9 2.51% 57.5 1.46% 1.30x 2.91% 93.96%
liquid-render 197.2 0.21% 103.9 0.45% 1.90x 0.50% 87.95%
mail 173.4 1.23% 134.9 0.43% 1.29x 1.30% 99.03%
psych-load 2596.3 0.07% 1855.3 0.08% 1.40x 0.11% 99.99%
railsbench 2774.1 0.65% 1805.3 1.16% 1.54x 1.33% 93.75%
ruby-lsp 7.9 7.35% 6.0 11.32% 1.31x 13.50% 89.78%
sequel 88.8 0.82% 68.9 1.20% 1.29x 1.46% 95.28%
binarytrees 477.5 0.05% 227.7 0.09% 2.10x 0.11% 100.00%
erubi 305.4 0.05% 253.8 0.06% 1.20x 0.08% 100.00%
etanni 408.4 0.03% 410.2 0.04% 1.00x 0.05% 7.03%
fannkuchredux 2138.2 0.09% 768.1 0.05% 2.78x 0.10% 90.92%
lee 1320.8 0.30% 939.5 0.36% 1.41x 0.47% 99.97%
nbody 133.5 0.06% 69.8 0.07% 1.91x 0.09% 100.00%
optcarrot 6353.0 0.65% 2289.7 0.59% 2.77x 0.88% 96.78%
ruby-json 3832.7 0.04% 3321.6 0.05% 1.15x 0.06% 99.82%
rubykon 12586.2 0.42% 6831.2 0.43% 1.84x 0.60% 99.78%
30k_ifelse 2387.4 0.04% 359.9 0.07% 6.63x 0.08% 99.99%
30k_methods 6627.0 0.06% 857.1 0.05% 7.73x 0.08% 100.00%
cfunc_itself 106.8 0.30% 39.0 0.39% 2.74x 0.49% 100.00%
fib 234.4 0.04% 44.6 0.06% 5.26x 0.07% 100.00%
getivar 112.9 0.24% 21.1 0.27% 5.34x 0.37% 97.88%
keyword_args 279.3 0.11% 48.0 0.14% 5.82x 0.18% 100.00%
respond_to 273.1 0.28% 25.7 0.67% 10.61x 0.73% 100.00%
setivar 69.1 0.05% 12.0 0.11% 5.76x 0.12% 98.74%
setivar_object 101.2 0.09% 42.2 0.05% 2.40x 0.10% 95.69%
setivar_young 101.3 0.30% 42.2 1.42% 2.40x 1.45% 95.69%
str_concat 83.1 0.20% 43.6 0.44% 1.90x 0.48% 99.97%
throw 29.7 0.21% 24.3 0.34% 1.22x 0.40% 64.81%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.3.0dev (2023-06-29T01:23:37Z :detached: 26b69fd407) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.0dev mem (MiB) YJIT 3.3.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 58 63 1 1 8.8%
chunky-png 60 68 1 1 11.9%
erubi-rails 92 99 2 2 8.0%
hexapdf 243 311 2 2 28.2%
liquid-c 43 46 1 1 8.5%
liquid-compile 37 39 1 1 7.5%
liquid-render 39 43 1 1 11.2%
mail 49 51 1 1 5.5%
psych-load 37 39 1 1 5.2%
railsbench 111 125 3 2 12.6%
ruby-lsp 46 51 1 1 12.2%
sequel 40 44 1 1 8.6%
binarytrees 30 30 1 1 1.3%
erubi 32 34 1 1 5.9%
etanni 27 28 1 1 1.9%
fannkuchredux 24 24 1 1 2.2%
lee 33 37 1 1 13.0%
nbody 24 24 1 1 1.9%
optcarrot 62 68 1 1 9.1%
ruby-json 25 25 1 1 1.6%
rubykon 252 255 1 1 1.4%
30k_ifelse 68 104 6 5 53.3%
30k_methods 59 70 2 2 19.5%
cfunc_itself 24 24 1 1 1.4%
fib 24 24 1 1 1.4%
getivar 24 24 1 1 1.4%
keyword_args 24 24 1 1 1.4%
respond_to 24 24 1 1 1.5%
setivar 24 24 1 1 1.4%
setivar_object 24 24 1 1 1.4%
setivar_young 24 24 1 1 1.4%
str_concat 50 81 1 1 62.0%
throw 24 24 1 1 1.4%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

Benchmark YJIT Stats

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.