YJIT Benchmarks

Details for Benchmarks at 2023-11-30 06:08:15 GMT

YJIT metrics from the yjit-bench suite using Ruby f2ad246071.

Overall YJIT is 64.1% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 66.9% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 No JIT YJIT activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 CRuby 3.3.0dev YJIT 3.3.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 No JIT YJIT binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 CRuby 3.3.0dev YJIT 3.3.0dev binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 No JIT YJIT 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.3.0dev YJIT 3.3.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 82.5 1.92% 41.4 3.86% 1.99x 4.31% 99.96%
chunky-png 1066.3 0.30% 618.1 0.50% 1.73x 0.58% 100.00%
erubi-rails 2583.4 0.66% 1366.6 1.14% 1.89x 1.32% 99.93%
hexapdf 3465.4 0.81% 2047.0 1.39% 1.69x 1.61% 93.86%
liquid-c 81.5 0.84% 57.6 1.20% 1.41x 1.47% 99.74%
liquid-compile 85.3 0.91% 57.1 4.92% 1.49x 5.00% 99.95%
liquid-render 198.8 0.31% 82.6 0.74% 2.41x 0.80% 99.86%
lobsters 1265.8 1.75% 841.9 3.17% 1.50x 3.62% 99.53%
mail 171.5 0.25% 120.0 0.40% 1.43x 0.47% 99.90%
psych-load 2726.7 0.06% 1808.5 0.09% 1.51x 0.11% 100.00%
railsbench 2755.0 0.50% 1650.9 0.96% 1.67x 1.08% 99.55%
ruby-lsp 78.7 1.94% 50.7 5.88% 1.55x 6.19% 92.82%
sequel 91.1 1.13% 68.2 1.71% 1.34x 2.05% 99.88%
binarytrees 491.8 1.07% 232.7 2.22% 2.11x 2.46% 100.00%
erubi 306.6 0.13% 254.2 0.14% 1.21x 0.19% 99.98%
etanni 432.7 0.08% 376.9 0.08% 1.15x 0.11% 99.98%
fannkuchredux 2194.5 0.12% 778.5 0.21% 2.82x 0.24% 90.92%
fluentd 2508.6 0.93% 2296.7 1.01% 1.09x 1.37% 99.99%
graphql 4165.1 0.12% 3530.8 0.03% 1.18x 0.12% 99.75%
graphql-native 543.5 0.08% 477.7 0.08% 1.14x 0.12% 99.97%
lee 1378.0 0.99% 980.3 1.42% 1.41x 1.73% 99.98%
nbody 128.1 0.04% 67.2 0.04% 1.91x 0.06% 99.95%
optcarrot 7194.1 0.57% 1989.4 0.70% 3.62x 0.90% 99.50%
rack 135.0 0.92% 108.8 1.16% 1.24x 1.48% 99.71%
ruby-json 3969.9 0.02% 3430.6 0.06% 1.16x 0.07% 99.99%
rubykon 13472.8 0.69% 6696.5 1.49% 2.01x 1.65% 99.95%
tinygql 882.1 0.26% 478.1 0.11% 1.85x 0.29% 99.99%
30k_ifelse 2396.8 0.06% 357.7 0.06% 6.70x 0.08% 99.99%
30k_methods 6588.2 0.02% 831.3 0.03% 7.93x 0.03% 99.99%
cfunc_itself 108.4 0.06% 26.5 0.05% 4.09x 0.08% 99.30%
fib 265.1 0.16% 47.7 0.46% 5.55x 0.48% 100.00%
getivar 127.7 0.14% 21.2 1.31% 6.03x 1.32% 97.87%
keyword_args 300.9 0.06% 40.5 0.33% 7.43x 0.33% 99.55%
respond_to 291.3 0.27% 16.8 0.43% 17.39x 0.51% 99.67%
setivar 78.6 0.40% 12.0 0.05% 6.55x 0.40% 98.74%
setivar_object 106.7 0.40% 42.0 0.02% 2.54x 0.40% 94.66%
setivar_young 107.3 0.62% 42.2 1.59% 2.54x 1.70% 94.82%
str_concat 86.6 0.71% 44.2 1.67% 1.96x 1.82% 99.97%
throw 30.5 0.24% 23.8 0.32% 1.28x 0.40% 99.99%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.3.0dev (2023-11-30T04:38:07Z :detached: f2ad246071) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.0dev mem (MiB) YJIT 3.3.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 49 54 1 1 10.5%
chunky-png 66 65 1 1 -1.5%
erubi-rails 91 100 2 2 9.4%
hexapdf 156 287 2 2 83.1%
liquid-c 35 39 1 1 10.8%
liquid-compile 33 38 1 1 14.9%
liquid-render 34 38 1 1 12.0%
lobsters 274 324 8 6 18.0%
mail 50 55 1 1 11.1%
psych-load 33 35 1 1 6.3%
railsbench 90 104 3 3 15.3%
ruby-lsp 94 127 6 5 35.1%
sequel 36 40 1 1 9.8%
binarytrees 26 26 1 1 -2.2%
erubi 29 31 1 1 6.5%
etanni 23 23 1 1 1.6%
fannkuchredux 20 20 1 1 1.4%
fluentd 607 657 1 1 8.4%
graphql 39 43 1 1 8.5%
graphql-native 35 39 1 1 10.5%
lee 34 36 1 1 6.6%
nbody 20 20 1 1 1.1%
optcarrot 61 67 1 1 9.9%
rack 29 31 1 1 6.4%
ruby-json 21 21 1 1 1.4%
rubykon 47 49 1 1 4.7%
tinygql 28 30 1 1 7.2%
30k_ifelse 61 91 5 5 49.3%
30k_methods 49 58 2 2 19.6%
cfunc_itself 20 20 1 1 1.0%
fib 20 20 1 1 1.0%
getivar 20 20 1 1 0.8%
keyword_args 20 20 1 1 1.0%
respond_to 20 20 1 1 1.0%
setivar 20 20 1 1 1.0%
setivar_object 20 20 1 1 1.0%
setivar_young 20 20 1 1 1.2%
str_concat 46 59 1 1 28.5%
throw 20 20 1 1 1.0%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

Benchmark YJIT Stats

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.