YJIT Benchmarks

Details for Benchmarks at 2023-10-18 19:08:15 GMT

YJIT metrics from the yjit-bench suite using Ruby 4b909bdbae.

Overall YJIT is 62.0% faster than interpreted CRuby!
On Railsbench specifically, YJIT is 67.5% faster than CRuby!

Performance on Headline Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 No JIT YJIT activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 CRuby 3.3.0dev YJIT 3.3.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench ruby-lsp sequel geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 2.5 3.0 No JIT YJIT binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 CRuby 3.3.0dev YJIT 3.3.0dev binarytrees erubi etanni fannkuchredux fluentd graphql graphql-native lee nbody optcarrot rack ruby-json rubykon tinygql geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRuby 3.3.0dev YJIT 3.3.0dev 30k_ifelse 30k_methods cfunc_itself fib getivar keyword_args respond_to setivar setivar_object setivar_young str_concat throw geomean*
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform
bench No JIT (ms) No JIT RSD YJIT (ms) YJIT RSD YJIT spd YJIT spd RSD % in YJIT
activerecord 82.0 1.94% 40.7 4.07% 2.01x 4.51% 99.96%
chunky-png 1054.5 0.29% 613.6 0.54% 1.72x 0.61% 100.00%
erubi-rails 26.0 10.10% 14.0 20.82% 1.86x 23.15% 99.94%
hexapdf 3418.7 0.83% 2069.8 1.80% 1.65x 1.98% 94.60%
liquid-c 80.5 0.63% 56.4 0.83% 1.43x 1.04% 99.74%
liquid-compile 83.8 0.90% 59.2 1.71% 1.42x 1.93% 99.95%
liquid-render 197.1 0.23% 82.0 0.55% 2.40x 0.59% 99.86%
lobsters 1266.5 2.13% 852.0 3.99% 1.49x 4.53% 99.71%
mail 171.5 0.25% 125.9 0.35% 1.36x 0.43% 99.90%
psych-load 2683.1 0.28% 1829.4 0.14% 1.47x 0.31% 100.00%
railsbench 2760.4 0.57% 1647.7 1.03% 1.68x 1.17% 99.55%
ruby-lsp 79.4 3.29% 51.2 7.39% 1.55x 8.09% 92.83%
sequel 88.5 1.26% 66.8 1.68% 1.33x 2.10% 99.91%
binarytrees 492.8 1.09% 235.3 2.26% 2.09x 2.51% 100.00%
erubi 309.3 0.11% 257.1 0.14% 1.20x 0.18% 99.98%
etanni 431.4 0.08% 380.0 0.08% 1.14x 0.12% 99.98%
fannkuchredux 2232.7 0.08% 784.1 0.11% 2.85x 0.13% 90.92%
fluentd 2467.7 0.64% 2244.8 0.66% 1.10x 0.92% 99.99%
graphql 3953.9 0.04% 3390.0 0.03% 1.17x 0.05% 99.75%
graphql-native 531.8 0.11% 475.7 0.08% 1.12x 0.13% 99.97%
lee 1341.0 1.07% 935.5 1.53% 1.43x 1.87% 99.98%
nbody 122.7 0.09% 70.3 0.03% 1.75x 0.09% 99.95%
optcarrot 6680.2 0.59% 2042.0 0.67% 3.27x 0.89% 99.50%
rack 132.3 0.89% 106.6 1.12% 1.24x 1.44% 99.71%
ruby-json 3851.2 0.07% 3297.0 0.07% 1.17x 0.10% 99.99%
rubykon 13350.1 0.60% 6607.2 1.05% 2.02x 1.20% 99.95%
tinygql 849.4 0.09% 448.4 0.08% 1.89x 0.12% 99.99%
30k_ifelse 2377.2 0.06% 356.5 0.05% 6.67x 0.08% 99.99%
30k_methods 6610.2 0.02% 832.8 0.02% 7.94x 0.02% 99.99%
cfunc_itself 111.9 0.05% 26.5 0.04% 4.22x 0.06% 99.30%
fib 270.6 0.17% 47.7 0.44% 5.67x 0.47% 100.00%
getivar 126.2 0.19% 21.4 1.76% 5.88x 1.77% 97.82%
keyword_args 307.5 0.31% 40.3 0.13% 7.63x 0.33% 99.55%
respond_to 289.2 0.58% 15.2 0.05% 19.05x 0.58% 99.70%
setivar 84.3 0.32% 12.0 0.06% 7.03x 0.32% 98.73%
setivar_object 114.1 0.50% 42.1 0.02% 2.71x 0.50% 95.70%
setivar_young 114.1 0.55% 43.1 1.39% 2.65x 1.49% 95.61%
str_concat 86.9 0.72% 43.4 1.59% 2.00x 1.75% 99.97%
throw 31.4 0.28% 24.2 0.37% 1.30x 0.46% 99.99%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.3.0dev (2023-10-18T16:51:25Z :detached: 4b909bdbae) +YJIT [x86_64-linux]

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.0dev mem (MiB) YJIT 3.3.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 48 54 1 1 11.0%
chunky-png 59 62 1 1 4.2%
erubi-rails 88 97 2 2 10.1%
hexapdf 154 278 2 2 80.2%
liquid-c 36 40 1 1 11.6%
liquid-compile 33 36 1 1 9.2%
liquid-render 34 38 1 1 12.9%
lobsters 276 332 8 7 20.5%
mail 51 55 1 1 8.6%
psych-load 35 37 1 1 6.7%
railsbench 90 104 3 3 16.1%
ruby-lsp 94 131 5 5 38.8%
sequel 37 40 1 1 8.4%
binarytrees 26 26 1 1 1.1%
erubi 29 31 1 1 7.0%
etanni 23 23 1 1 2.2%
fannkuchredux 20 20 1 1 1.8%
fluentd 606 657 1 1 8.4%
graphql 39 43 1 1 9.7%
graphql-native 36 39 1 1 7.4%
lee 34 36 1 1 7.2%
nbody 20 20 1 1 1.4%
optcarrot 60 67 1 1 11.2%
rack 29 31 1 1 7.9%
ruby-json 21 21 1 1 1.2%
rubykon 47 48 1 1 2.4%
tinygql 27 30 1 1 7.8%
30k_ifelse 63 92 5 5 46.0%
30k_methods 51 59 2 2 15.3%
cfunc_itself 20 20 1 1 1.3%
fib 20 20 1 1 1.3%
getivar 20 20 1 1 1.3%
keyword_args 20 20 1 1 1.2%
respond_to 20 20 1 1 1.3%
setivar 20 20 1 1 1.3%
setivar_object 20 20 1 1 1.3%
setivar_young 20 20 1 1 1.3%
str_concat 46 61 1 1 32.9%
throw 20 20 1 1 1.3%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

Benchmark YJIT Stats

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.