YJIT Benchmarks

Details for Benchmarks at 2024-10-15 14:29:01 UTC

YJIT metrics from the yjit-bench suite using Ruby 3da3cabf98

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 89.3% faster than CRuby 3.4.0dev
  • 4.7% faster than YJIT 3.3.4
On railsbench it is
  • 95.3% faster than CRuby 3.4.0dev
  • 3.5% faster than YJIT 3.3.4

Performance on Headline Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Benchmark Memory Usage Details

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 10 77 10 78 10 197 10 201
chunky-png 10 26 10 26 10 55 10 62
erubi-rails 10 11 10 10 10 35 10 36
hexapdf 10 10 10 10 10 10 10 11
liquid-c 10 403 10 414 10 629 10 662
liquid-compile 10 404 10 419 10 626 10 637
liquid-render 10 167 10 160 10 444 10 466
lobsters 10 15 10 15 10 29 10 29
mail 10 188 10 187 10 291 10 293
psych-load 10 10 10 10 10 10 10 12
railsbench 10 10 10 10 10 10 10 10
rubocop 10 151 10 150 10 289 10 308
ruby-lsp 10 152 10 153 10 289 10 300
sequel 10 422 10 422 10 632 10 618
binarytrees 10 79 10 78 10 187 10 191
blurhash 10 79 10 78 10 167 10 186
erubi 10 89 10 134 10 135 10 161
etanni 10 85 10 96 10 92 10 104
fannkuchredux 10 48 10 51 10 148 10 159
fluentd 10 57 10 57 10 64 10 77
graphql 10 82 10 82 10 100 10 100
graphql-native 10 35 10 47 10 40 10 55
lee 10 18 10 17 10 29 10 29
matmul 10 49 10 50 10 129 10 139
nbody 10 265 10 249 10 669 10 776
nqueens 10 112 10 116 10 111 10 620
optcarrot 10 10 10 10 10 11 10 12
protoboeuf 10 168 10 174 10 671 10 780
protoboeuf-encode 10 167 10 192 10 352 10 832
rack 10 561 10 621 10 975 10 1113
ruby-json 10 80 10 84 10 92 10 96
rubykon 10 16 10 16 10 44 10 49
sudoku 10 43 10 43 10 210 10 218
tinygql 10 31 10 31 10 65 10 66
30k_ifelse 10 23 10 23 10 254 10 233
30k_methods 10 33 10 33 10 523 10 512
cfunc_itself 10 331 10 334 10 1086 10 1175
fib 10 123 10 120 10 939 10 1002
getivar 10 213 10 208 10 2334 10 2656
keyword_args 10 98 10 98 10 999 10 1270
object-new 10 317 10 267 10 430 10 349
respond_to 10 131 10 132 10 3293 10 4074
ruby-xor 10 235 10 239 10 843 10 1414
setivar 10 354 10 355 10 3168 10 5754
setivar_object 10 363 10 335 10 829 10 935
setivar_young 10 363 10 336 10 830 10 996
str_concat 10 389 10 397 10 1026 10 1235
throw 10 1267 10 1098 10 1676 10 1408

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1451016 1234100 214 2301 0 0% 0 0 66.097749
chunky-png (click) 325389 285466 82 1095 1 0% 0 0 32.338216
erubi-rails (click) 1910660 1629010 245 2795 8 0% 0 0 74.878424
hexapdf (click) 1393504 1219774 507 12565 40 0% 0 0 356.650331
liquid-c (click) 487454 428202 119 1859 5 0% 0 0 52.194777
liquid-compile (click) 446682 369295 150 2211 2 0% 0 0 68.286351
liquid-render (click) 605599 559842 136 2473 12 0% 0 0 68.552641
lobsters (click) 8027447 6752542 3088 51683 182 0% 0 0 1733.509563
mail (click) 779348 716454 346 5481 16 0% 0 0 159.902963
psych-load (click) 251801 226142 60 615 3 0% 0 0 18.924526
railsbench (click) 3300615 2875395 1631 16669 41 0% 0 0 473.080872
rubocop (click) 5813590 5000649 2892 52168 110 0% 4 0 1527.443136
ruby-lsp (click) 870488 793894 308 4986 26 0% 0 0 138.93697
sequel (click) 464651 396298 15 116 0 0% 0 0 4.036783
binarytrees (click) 6981 6331 5 51 0 0% 0 0 2.179679
blurhash (click) 38438 31808 20 350 0 0% 0 0 11.070501
erubi (click) 237533 217482 9 108 0 0% 0 0 3.619648
etanni (click) 28241 26371 10 86 0 0% 0 0 2.963308
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.801094
fluentd (click) 349134 304917 11 114 0 0% 0 0 4.057277
graphql (click) 427970 366251 70 700 0 0% 0 0 20.680502
graphql-native (click) 365603 309615 41 275 0 0% 0 0 9.097122
lee (click) 285235 263042 33 567 0 0% 0 0 17.334167
matmul (click) 7323 8338 6 76 0 0% 0 0 2.581798
nbody (click) 15428 19810 9 201 0 0% 0 0 5.3056
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.262147
optcarrot (click) 324183 308580 186 4508 34 0% 0 0 102.866799
protoboeuf (click) 163936 155521 15 1615 0 0% 0 0 46.414647
protoboeuf-encode (click) 235128 231105 17 1254 0 0% 0 0 32.240178
rack (click) 239615 204345 36 417 0 0% 0 0 11.215994
ruby-json (click) 27649 24400 11 194 0 0% 0 0 5.641105
rubykon (click) 137360 133862 132 1535 3 0% 0 0 43.748243
sudoku (click) 44908 40454 6 507 0 0% 0 0 15.062315
tinygql (click) 269037 236645 57 743 5 0% 0 0 21.187449
30k_ifelse (click) 6316209 5792190 9262 56478 0 0% 0 0 1572.229116
30k_methods (click) 2236110 1594189 5781 19364 0 0% 0 0 454.227531
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 1.973008
fib (click) 4041 4120 5 37 0 0% 0 0 1.442072
getivar (click) 5037 6014 6 64 0 0% 0 0 1.943326
keyword_args (click) 8181 6507 8 65 0 0% 0 0 2.337276
object-new (click) 3469 3394 5 35 0 0% 0 0 1.338527
respond_to (click) 9866 2524 7 100 0 0% 0 0 2.941609
ruby-xor (click) 6981 7365 7 77 0 0% 0 0 2.409186
setivar (click) 4188 4690 6 44 0 0% 0 0 1.600074
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.71664
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.90551
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.776194
throw (click) 7384 5592 8 59 0 0% 0 0 2.177156

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.