YJIT Benchmarks

Details for Benchmarks at 2024-11-17 03:48:04 UTC

YJIT metrics from the yjit-bench suite using Ruby 6588cc4e29

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 89.9% faster than CRuby 3.4.0dev
  • 5.2% faster than YJIT 3.3.4
On railsbench it is
  • 92.9% faster than CRuby 3.4.0dev
  • 2.2% faster than YJIT 3.3.4

Performance on Headline Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Benchmark Memory Usage Details

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 5 82 5 83 50 156 50 163
chunky-png 5 31 5 31 50 15 50 35
erubi-rails 5 16 5 15 50 10 50 10
hexapdf 5 10 5 10 50 10 50 10
liquid-c 5 417 5 424 50 572 50 625
liquid-compile 5 412 5 411 50 542 50 556
liquid-render 5 171 5 166 50 404 50 422
lobsters 5 20 5 19 50 10 50 10
mail 5 194 5 189 50 238 50 252
psych-load 5 10 5 10 50 10 50 10
railsbench 5 10 5 10 50 10 50 10
rubocop 5 156 5 155 50 250 50 263
ruby-lsp 5 154 5 159 50 247 50 251
sequel 5 420 5 418 50 581 50 588
binarytrees 5 84 5 83 50 146 50 153
blurhash 5 82 5 82 50 127 50 144
erubi 5 121 5 149 50 89 50 145
etanni 5 90 5 99 50 52 50 70
fannkuchredux 5 52 5 55 50 100 50 109
fluentd 5 61 5 61 50 22 50 29
graphql 5 85 5 84 50 57 50 56
graphql-native 5 40 5 54 50 10 50 14
lee 5 22 5 22 50 10 50 10
matmul 5 51 5 48 50 88 50 99
nbody 5 247 5 248 50 596 50 716
nqueens 5 119 5 119 50 74 50 557
optcarrot 5 10 5 10 50 10 50 10
protoboeuf 5 170 5 175 50 616 50 727
protoboeuf-encode 5 179 5 199 50 326 50 809
rack 5 586 5 606 50 943 50 1052
ruby-json 5 84 5 84 50 50 50 50
rubykon 5 21 5 21 50 10 50 10
sudoku 5 48 5 48 50 163 50 162
tinygql 5 36 5 37 50 23 50 27
30k_ifelse 5 28 5 28 50 215 50 193
30k_methods 5 38 5 38 50 482 50 478
cfunc_itself 5 340 5 339 50 1046 50 1136
fib 5 127 5 126 50 900 50 963
getivar 5 209 5 220 50 2267 50 2630
keyword_args 5 105 5 104 50 959 50 1231
object-new 5 322 5 319 50 407 50 386
respond_to 5 137 5 137 50 3257 50 4029
ruby-xor 5 241 5 248 50 803 50 1380
setivar 5 360 5 358 50 3125 50 5709
setivar_object 5 374 5 341 50 772 50 844
setivar_young 5 373 5 343 50 772 50 847
str_concat 5 397 5 382 50 988 50 1203
throw 5 1265 5 1323 50 1734 50 1734

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1476982 1285422 213 2351 0 0% 0 0 67.783323
chunky-png (click) 338881 284519 87 1222 1 0% 0 0 35.823134
erubi-rails (click) 1978627 1657030 263 3032 19 0% 0 0 82.366088
hexapdf (click) 1518460 1242809 598 13939 40 0% 0 0 393.984863
liquid-c (click) 523551 459498 119 1852 5 0% 0 0 52.480535
liquid-compile (click) 448573 405430 150 2215 2 0% 0 0 68.361048
liquid-render (click) 602046 517725 136 2474 12 0% 0 0 69.963625
lobsters (click) 8416267 7096778 3214 54837 188 0% 0 0 1842.322897
mail (click) 785693 686323 346 5590 16 0% 0 0 161.793074
psych-load (click) 284100 265806 64 656 3 0% 0 0 20.474873
railsbench (click) 3357038 2855743 1646 16986 52 0% 0 0 486.865144
rubocop (click) 5872646 5080995 2891 52426 110 0% 4 0 1618.966977
ruby-lsp (click) 877621 800815 309 5004 26 0% 0 0 141.270471
sequel (click) 498016 444504 15 121 0 0% 0 0 4.793373
binarytrees (click) 9721 8668 8 76 0 0% 0 0 3.111711
blurhash (click) 57136 49713 29 501 0 0% 0 0 16.181843
erubi (click) 227331 193362 8 113 0 0% 0 0 3.773081
etanni (click) 32243 30528 9 94 0 0% 0 0 3.217498
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.652648
fluentd (click) 359844 332631 10 109 0 0% 0 0 3.813418
graphql (click) 442174 390359 69 695 0 0% 0 0 20.361746
graphql-native (click) 408525 344672 40 270 0 0% 0 0 8.632744
lee (click) 302892 260297 48 841 0 0% 0 0 25.833942
matmul (click) 11838 4563 10 119 0 0% 0 0 3.963696
nbody (click) 14884 19325 8 196 0 0% 0 0 5.110258
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.248191
optcarrot (click) 327944 312320 190 4549 34 0% 0 0 106.646695
protoboeuf (click) 163392 155036 14 1610 0 0% 0 0 45.728363
protoboeuf-encode (click) 234584 230620 16 1249 0 0% 0 0 32.111823
rack (click) 255535 236305 36 417 0 0% 0 0 11.019203
ruby-json (click) 27119 23942 10 187 0 0% 0 0 5.367423
rubykon (click) 142198 122470 139 1584 3 0% 0 0 45.661637
sudoku (click) 48052 35463 9 538 0 0% 0 0 16.015679
tinygql (click) 290867 248708 61 787 5 0% 0 0 22.373787
30k_ifelse (click) 6315665 5791705 9261 56473 0 0% 0 0 1558.035872
30k_methods (click) 2235566 1593704 5780 19359 0 0% 0 0 451.652615
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 1.973522
fib (click) 4041 4120 5 37 0 0% 0 0 1.435912
getivar (click) 5037 6014 6 64 0 0% 0 0 1.902745
keyword_args (click) 7637 6022 7 60 0 0% 0 0 2.045843
object-new (click) 3469 3394 5 35 0 0% 0 0 1.353761
respond_to (click) 9322 2039 6 95 0 0% 0 0 2.752866
ruby-xor (click) 6297 6740 6 72 0 0% 0 0 2.156137
setivar (click) 4188 4690 6 44 0 0% 0 0 1.594454
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.661033
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.898378
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.73707
throw (click) 7384 5592 8 59 0 0% 0 0 2.115018

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.