YJIT Benchmarks

Details for Benchmarks at 2024-10-28 03:46:51 GMT

YJIT metrics from the yjit-bench suite using Ruby 70d20ce473.

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 91.3% faster than CRuby 3.4.0dev
  • 5.2% faster than YJIT 3.3.4
On railsbench it is
  • 105.0% faster than CRuby 3.4.0dev
  • 4.2% faster than YJIT 3.3.4
x86_64 runtime: 3 hours, 34 minutes
aarch64 runtime: 3 hours, 32 minutes

Performance on Headline Benchmarks

Select Platform
0.0 0.5 1.0 1.5 2.0 2.5 CRuby 3.3.4 CRuby 3.4.0dev YJIT 3.3.4 YJIT 3.4.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench rubocop ruby-lsp sequel geomean* 0.95 2.27 2.37 1.00 1.81 1.96 0.98 2.23 2.30 1.00 1.83 1.97 1.01 1.51 1.64 0.97 1.42 1.45 0.99 2.63 2.78 0.98 1.66 1.67 0.96 1.53 1.52 0.98 1.47 1.66 0.97 1.92 2.00 0.99 1.95 2.07 1.00 1.76 1.94 1.00 1.54 1.51 0.99 1.79 1.89
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 CRuby 3.3.4 CRuby 3.4.0dev YJIT 3.3.4 YJIT 3.4.0dev activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench rubocop ruby-lsp sequel geomean* 0.98 1.16 1.05 1.27 1.65 1.40 1.00 1.07 1.08 0.95 1.95 1.83 0.80 1.14 1.01 0.83 1.10 0.90 0.81 1.15 0.98 0.94 1.16 1.12 0.92 1.09 1.02 0.76 1.05 0.83 0.95 1.17 1.09 0.89 1.35 1.33 0.89 1.33 1.00 0.83 1.13 0.96 0.91 1.23 1.09
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Select Platform
0.0 1.0 2.0 3.0 4.0 5.0 CRuby 3.3.4 CRuby 3.4.0dev YJIT 3.3.4 YJIT 3.4.0dev binarytrees blurhash erubi etanni fannkuchredux fluentd graphql graphql-native lee matmul nbody nqueens optcarrot protoboeuf protoboeuf-encode rack ruby-json rubykon sudoku tinygql geomean* 1.00 2.22 2.24 1.02 2.05 2.25 1.16 1.15 0.69 1.07 1.07 1.16 1.04 4.33 4.43 1.00 1.10 1.14 1.00 1.19 1.17 1.31 1.11 1.42 0.97 1.40 1.42 1.01 2.46 2.65 1.05 2.55 3.00 1.02 1.00 5.05 0.97 3.84 4.01 0.99 3.74 4.09 1.08 2.05 4.59 1.09 1.72 1.89 0.97 1.12 1.10 1.01 2.13 2.33 1.01 4.06 3.98 1.00 1.78 1.85 1.04 1.87 2.17
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Select Platform
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 CRuby 3.3.4 CRuby 3.4.0dev YJIT 3.3.4 YJIT 3.4.0dev binarytrees blurhash erubi etanni fannkuchredux fluentd graphql graphql-native lee matmul nbody nqueens optcarrot protoboeuf protoboeuf-encode rack ruby-json rubykon sudoku tinygql geomean* 0.65 1.04 0.68 0.67 1.03 0.71 1.26 1.07 0.80 0.84 1.02 0.86 0.65 1.02 0.69 1.04 1.03 1.11 0.83 1.01 0.87 0.95 1.05 0.98 0.72 1.07 0.81 0.84 1.01 0.87 0.65 1.02 0.69 0.66 1.01 0.72 0.77 1.11 0.84 0.75 1.13 0.78 1.13 1.07 1.80 0.72 1.05 0.81 0.67 1.02 0.70 0.70 1.06 0.81 0.66 1.02 0.70 0.76 1.05 0.84 0.78 1.04 0.83
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Select Platform
Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Select Platform
Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Select Platform

Benchmark Memory Usage Details

Select Platform
bench CRuby 3.3.4 mem (MiB) CRuby 3.4.0dev mem (MiB) YJIT 3.3.4 mem (MiB) YJIT 3.4.0dev mem (MiB) Inline Code Outlined Code YJIT Mem overhead
activerecord 63 62 74 66 2 2 6.7%
chunky-png 40 51 67 57 1 1 10.4%
erubi-rails 105 106 112 114 2 2 7.8%
hexapdf 144 138 282 264 2 2 91.9%
liquid-c 34 27 39 34 1 1 25.0%
liquid-compile 32 26 35 29 1 1 8.7%
liquid-render 34 28 39 33 1 1 20.3%
lobsters 280 262 324 314 8 7 19.8%
mail 52 48 57 53 1 1 9.9%
psych-load 32 24 34 27 1 1 9.9%
railsbench 105 100 123 115 4 3 15.4%
rubocop 91 81 123 121 6 5 49.3%
ruby-lsp 69 61 92 69 1 1 12.4%
sequel 36 30 40 34 1 1 15.6%
binarytrees 25 16 27 17 1 1 3.7%
blurhash 21 14 22 15 1 1 6.1%
erubi 28 36 30 22 1 1 -36.8%
etanni 23 19 24 20 1 1 2.4%
fannkuchredux 20 13 20 14 1 1 5.3%
fluentd 192 200 199 214 1 1 6.8%
graphql 39 32 40 34 1 1 5.4%
graphql-native 39 37 41 38 1 1 3.3%
lee 34 24 36 27 1 1 12.3%
matmul 25 21 26 22 1 1 2.8%
nbody 20 13 20 14 1 1 6.5%
nqueens 20 13 21 15 1 1 10.2%
optcarrot 61 47 68 51 1 1 7.9%
protoboeuf 36 27 40 28 1 1 4.3%
protoboeuf-encode 64 73 68 116 1 1 58.7%
rack 31 22 32 25 1 1 11.8%
ruby-json 21 13 21 14 1 1 4.6%
rubykon 24 17 26 20 1 1 15.4%
sudoku 20 13 21 14 1 1 5.4%
tinygql 27 20 28 23 1 1 10.6%
30k_ifelse 60 110 91 121 7 6 9.9%
30k_methods 49 53 59 58 3 2 9.9%
cfunc_itself 20 13 20 13 1 1 4.6%
fib 20 13 20 13 1 1 4.5%
getivar 20 13 20 13 1 1 4.5%
keyword_args 20 13 20 13 1 1 4.5%
object-new 20 13 20 13 1 1 4.4%
respond_to 20 13 20 13 1 1 4.9%
ruby-xor 21 14 21 15 1 1 4.3%
setivar 20 13 20 13 1 1 4.8%
setivar_object 20 13 20 13 1 1 4.5%
setivar_young 20 13 20 13 1 1 4.5%
str_concat 46 39 59 67 1 1 69.0%
throw 20 13 20 13 1 1 4.5%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 5 83 5 79 50 149 50 158
chunky-png 5 31 5 31 50 15 50 20
erubi-rails 5 16 5 15 50 10 50 10
hexapdf 5 10 5 10 50 10 50 10
liquid-c 5 411 5 417 50 577 50 630
liquid-compile 5 415 5 404 50 543 50 556
liquid-render 5 168 5 166 50 403 50 429
lobsters 5 20 5 20 50 10 50 10
mail 5 193 5 186 50 250 50 250
psych-load 5 10 5 10 50 10 50 10
railsbench 5 10 5 10 50 10 50 10
rubocop 5 156 5 154 50 252 50 267
ruby-lsp 5 156 5 156 50 233 50 260
sequel 5 417 5 418 50 599 50 586
binarytrees 5 84 5 83 50 146 50 148
blurhash 5 82 5 84 50 127 50 144
erubi 5 122 5 142 50 95 50 38
etanni 5 90 5 97 50 52 50 60
fannkuchredux 5 52 5 54 50 101 50 108
fluentd 5 61 5 61 50 23 50 23
graphql 5 85 5 86 50 57 50 56
graphql-native 5 39 5 53 50 10 50 13
lee 5 23 5 22 50 10 50 10
matmul 5 51 5 52 50 88 50 99
nbody 5 248 5 261 50 595 50 706
nqueens 5 119 5 121 50 74 50 562
optcarrot 5 10 5 10 50 10 50 10
protoboeuf 5 174 5 173 50 616 50 678
protoboeuf-encode 5 179 5 194 50 328 50 793
rack 5 578 5 628 50 952 50 1048
ruby-json 5 85 5 82 50 51 50 49
rubykon 5 21 5 21 50 10 50 10
sudoku 5 48 5 49 50 164 50 159
tinygql 5 37 5 37 50 24 50 26
30k_ifelse 5 28 5 28 50 215 50 194
30k_methods 5 39 5 38 50 482 50 478
cfunc_itself 5 344 5 344 50 1046 50 1136
fib 5 126 5 127 50 900 50 962
getivar 5 206 5 216 50 2267 50 2620
keyword_args 5 108 5 103 50 959 50 1231
object-new 5 321 5 272 50 405 50 309
respond_to 5 137 5 139 50 3257 50 4037
ruby-xor 5 243 5 238 50 803 50 1376
setivar 5 360 5 355 50 3126 50 5718
setivar_object 5 372 5 347 50 766 50 786
setivar_young 5 371 5 344 50 771 50 845
str_concat 5 399 5 410 50 990 50 1166
throw 5 1272 5 1161 50 1730 50 1415

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1476252 1255190 213 2296 0 0% 0 0 65.312872
chunky-png (click) 287469 271331 86 1133 1 0% 0 0 33.028688
erubi-rails (click) 1927788 1603557 249 2857 8 0% 0 0 77.592591
hexapdf (click) 1523703 1341262 597 13852 40 0% 0 0 392.445906
liquid-c (click) 502787 424008 122 1906 5 0% 0 0 52.960079
liquid-compile (click) 444479 383723 150 2197 2 0% 0 0 67.315317
liquid-render (click) 639366 564060 139 2520 12 0% 0 0 69.573731
lobsters (click) 8316777 7106940 3213 54377 181 0% 0 0 1817.33481
mail (click) 742113 676858 346 5481 16 0% 0 0 157.551345
psych-load (click) 257862 223538 64 656 3 0% 0 0 20.4425
railsbench (click) 3324648 2904262 1635 16746 41 0% 0 0 480.678056
rubocop (click) 5835433 4945522 2891 52116 110 0% 4 0 1521.116685
ruby-lsp (click) 850986 743515 308 4990 26 0% 0 0 139.649044
sequel (click) 467047 441048 15 117 0 0% 0 0 4.112995
binarytrees (click) 9627 8546 8 75 0 0% 0 0 3.032935
blurhash (click) 57136 49713 29 501 0 0% 0 0 16.212166
erubi (click) 251445 235512 9 108 0 0% 0 0 3.683105
etanni (click) 27632 25855 9 81 0 0% 0 0 2.737985
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.7351
fluentd (click) 350595 308577 10 109 0 0% 0 0 3.775166
graphql (click) 427384 382156 69 695 0 0% 0 0 20.331162
graphql-native (click) 393483 336182 40 270 0 0% 0 0 8.695288
lee (click) 310446 256143 48 803 0 0% 0 0 24.540972
matmul (click) 11838 4563 10 119 0 0% 0 0 3.95469
nbody (click) 14884 19325 8 196 0 0% 0 0 5.109024
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.200985
optcarrot (click) 327944 312320 190 4549 34 0% 0 0 105.392892
protoboeuf (click) 163392 155036 14 1610 0 0% 0 0 45.945917
protoboeuf-encode (click) 234584 230620 16 1249 0 0% 0 0 31.980853
rack (click) 234007 200730 36 417 0 0% 0 0 10.919417
ruby-json (click) 25474 21941 10 187 0 0% 0 0 5.359361
rubykon (click) 141232 121417 138 1575 3 0% 0 0 45.220837
sudoku (click) 48090 43120 9 536 0 0% 0 0 15.979532
tinygql (click) 275776 247699 61 787 5 0% 0 0 22.440623
30k_ifelse (click) 6315665 5791705 9261 56473 0 0% 0 0 1581.539081
30k_methods (click) 2235566 1593704 5780 19359 0 0% 0 0 455.524249
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 1.93987
fib (click) 4041 4120 5 37 0 0% 0 0 1.398074
getivar (click) 5037 6014 6 64 0 0% 0 0 1.92075
keyword_args (click) 7637 6022 7 60 0 0% 0 0 2.010085
object-new (click) 3469 3394 5 35 0 0% 0 0 1.35652
respond_to (click) 9322 2039 6 95 0 0% 0 0 2.689422
ruby-xor (click) 6437 6880 6 72 0 0% 0 0 2.198733
setivar (click) 4188 4690 6 44 0 0% 0 0 1.544752
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.697487
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.835915
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.719376
throw (click) 7384 5592 8 59 0 0% 0 0 2.150995

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.