YJIT Benchmarks

Details for Benchmarks at 2024-11-05 03:44:33 UTC

YJIT metrics from the yjit-bench suite using Ruby ee4599dbe7

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 86.7% faster than CRuby 3.4.0dev
  • 3.5% faster than YJIT 3.3.4
On railsbench it is
  • 95.0% faster than CRuby 3.4.0dev
  • 5.2% faster than YJIT 3.3.4

Performance on Headline Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Benchmark Memory Usage Details

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 5 83 5 83 50 149 50 162
chunky-png 5 31 5 32 50 15 50 19
erubi-rails 5 15 5 15 50 10 50 10
hexapdf 5 10 5 10 50 10 50 10
liquid-c 5 409 5 419 50 568 50 626
liquid-compile 5 411 5 409 50 539 50 555
liquid-render 5 167 5 164 50 401 50 418
lobsters 5 20 5 20 50 10 50 10
mail 5 194 5 187 50 250 50 235
psych-load 5 10 5 10 50 10 50 10
railsbench 5 10 5 10 50 10 50 10
rubocop 5 157 5 155 50 251 50 264
ruby-lsp 5 154 5 151 50 246 50 234
sequel 5 429 5 412 50 597 50 555
binarytrees 5 84 5 84 50 145 50 150
blurhash 5 82 5 83 50 128 50 145
erubi 5 120 5 144 50 29 50 125
etanni 5 90 5 98 50 52 50 60
fannkuchredux 5 52 5 55 50 100 50 107
fluentd 5 61 5 65 50 22 50 31
graphql 5 85 5 87 50 57 50 60
graphql-native 5 49 5 52 50 10 50 14
lee 5 23 5 22 50 10 50 10
matmul 5 51 5 52 50 88 50 99
nbody 5 248 5 249 50 595 50 706
nqueens 5 119 5 120 50 74 50 565
optcarrot 5 10 5 10 50 10 50 10
protoboeuf 5 174 5 176 50 617 50 737
protoboeuf-encode 5 179 5 203 50 325 50 785
rack 5 579 5 623 50 960 50 1050
ruby-json 5 84 5 82 50 50 50 48
rubykon 5 21 5 21 50 10 50 10
sudoku 5 48 5 49 50 163 50 161
tinygql 5 37 5 36 50 24 50 26
30k_ifelse 5 28 5 28 50 214 50 194
30k_methods 5 38 5 38 50 481 50 479
cfunc_itself 5 346 5 341 50 1046 50 1136
fib 5 120 5 127 50 898 50 959
getivar 5 206 5 214 50 2270 50 2616
keyword_args 5 108 5 107 50 959 50 1231
object-new 5 320 5 274 50 405 50 310
respond_to 5 137 5 139 50 3258 50 4041
ruby-xor 5 242 5 248 50 801 50 1381
setivar 5 361 5 357 50 3126 50 5714
setivar_object 5 372 5 341 50 761 50 847
setivar_young 5 371 5 344 50 771 50 849
str_concat 5 399 5 399 50 986 50 1165
throw 5 1269 5 1333 50 1741 50 1719

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1453506 1249587 213 2296 0 0% 0 0 66.066175
chunky-png (click) 301148 290572 86 1133 1 0% 0 0 33.535701
erubi-rails (click) 1912303 1615801 249 2872 8 0% 0 0 78.64402
hexapdf (click) 1493823 1246915 597 13852 40 0% 0 0 393.942959
liquid-c (click) 497235 420125 119 1852 5 0% 0 0 51.793375
liquid-compile (click) 421933 349538 150 2210 2 0% 0 0 69.587054
liquid-render (click) 604091 497441 136 2466 12 0% 0 0 69.204906
lobsters (click) 8340887 7073944 3215 54523 182 0% 0 0 1882.966736
mail (click) 775121 721195 346 5481 16 0% 0 0 159.980627
psych-load (click) 253170 212200 64 656 3 0% 0 0 20.641531
railsbench (click) 3285293 2853549 1632 16675 41 0% 0 0 479.590787
rubocop (click) 5810350 4916802 2891 52142 110 0% 4 0 1520.229686
ruby-lsp (click) 851516 755176 308 4986 26 0% 0 0 139.320725
sequel (click) 464917 423476 15 121 0 0% 0 0 4.355837
binarytrees (click) 9627 8546 8 75 0 0% 0 0 3.052497
blurhash (click) 57136 49713 29 501 0 0% 0 0 16.326223
erubi (click) 226114 186338 8 103 0 0% 0 0 3.456127
etanni (click) 24125 22834 10 89 0 0% 0 0 3.056736
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.718858
fluentd (click) 374966 327393 10 109 0 0% 0 0 3.948605
graphql (click) 403358 354726 69 695 0 0% 0 0 20.577694
graphql-native (click) 393713 328233 40 270 0 0% 0 0 8.818802
lee (click) 280850 244277 48 803 0 0% 0 0 24.769131
matmul (click) 11838 4563 10 119 0 0% 0 0 3.954222
nbody (click) 14884 19325 8 196 0 0% 0 0 5.098473
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.258137
optcarrot (click) 327944 312320 190 4549 34 0% 0 0 106.550438
protoboeuf (click) 163392 155036 14 1610 0 0% 0 0 46.141286
protoboeuf-encode (click) 234584 230620 16 1249 0 0% 0 0 32.097387
rack (click) 244827 226432 36 420 0 0% 0 0 11.283324
ruby-json (click) 25799 22401 10 188 0 0% 0 0 5.464071
rubykon (click) 140297 120385 138 1566 3 0% 0 0 44.709086
sudoku (click) 48090 43120 9 536 0 0% 0 0 16.178797
tinygql (click) 275570 243169 61 787 5 0% 0 0 22.345705
30k_ifelse (click) 6315665 5791705 9261 56473 0 0% 0 0 1583.032882
30k_methods (click) 2235566 1593704 5780 19359 0 0% 0 0 457.194691
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 2.027782
fib (click) 4041 4120 5 37 0 0% 0 0 1.457514
getivar (click) 5037 6014 6 64 0 0% 0 0 1.892577
keyword_args (click) 7637 6022 7 60 0 0% 0 0 2.086005
object-new (click) 3469 3394 5 35 0 0% 0 0 1.261351
respond_to (click) 9322 2039 6 95 0 0% 0 0 2.75852
ruby-xor (click) 6437 6880 6 72 0 0% 0 0 2.239141
setivar (click) 4188 4690 6 44 0 0% 0 0 1.55527
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.686751
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.894387
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.795193
throw (click) 7384 5592 8 59 0 0% 0 0 2.150502

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.