YJIT Benchmarks

Details for Benchmarks at 2024-11-09 03:43:32 UTC

YJIT metrics from the yjit-bench suite using Ruby c8c94bfb1e

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 86.1% faster than CRuby 3.4.0dev
  • 3.7% faster than YJIT 3.3.4
On railsbench it is
  • 99.1% faster than CRuby 3.4.0dev
  • 6.0% faster than YJIT 3.3.4

Performance on Headline Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Benchmark Memory Usage Details

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 5 82 5 83 50 152 50 160
chunky-png 5 31 5 32 50 15 50 19
erubi-rails 5 15 5 15 50 10 50 10
hexapdf 5 10 5 10 50 10 50 10
liquid-c 5 410 5 414 50 593 50 623
liquid-compile 5 411 5 420 50 545 50 583
liquid-render 5 168 5 169 50 404 50 418
lobsters 5 20 5 20 50 10 50 10
mail 5 187 5 188 50 238 50 240
psych-load 5 10 5 10 50 10 50 10
railsbench 5 10 5 10 50 10 50 10
rubocop 5 156 5 155 50 250 50 265
ruby-lsp 5 156 5 156 50 241 50 235
sequel 5 417 5 409 50 600 50 562
binarytrees 5 83 5 83 50 146 50 150
blurhash 5 82 5 83 50 127 50 145
erubi 5 114 5 141 50 86 50 120
etanni 5 90 5 97 50 52 50 60
fannkuchredux 5 52 5 55 50 100 50 105
fluentd 5 61 5 65 50 21 50 31
graphql 5 85 5 85 50 56 50 58
graphql-native 5 49 5 49 50 10 50 14
lee 5 23 5 23 50 10 50 10
matmul 5 51 5 52 50 87 50 98
nbody 5 247 5 260 50 596 50 704
nqueens 5 119 5 119 50 74 50 563
optcarrot 5 10 5 10 50 10 50 10
protoboeuf 5 173 5 171 50 614 50 667
protoboeuf-encode 5 179 5 200 50 328 50 769
rack 5 585 5 630 50 957 50 1071
ruby-json 5 85 5 81 50 50 50 47
rubykon 5 21 5 21 50 10 50 10
sudoku 5 48 5 49 50 163 50 160
tinygql 5 37 5 37 50 23 50 24
30k_ifelse 5 28 5 28 50 214 50 192
30k_methods 5 39 5 38 50 483 50 468
cfunc_itself 5 339 5 343 50 1046 50 1136
fib 5 127 5 113 50 897 50 961
getivar 5 208 5 216 50 2266 50 2614
keyword_args 5 105 5 104 50 959 50 1231
object-new 5 321 5 319 50 407 50 396
respond_to 5 137 5 139 50 3257 50 4035
ruby-xor 5 241 5 245 50 803 50 1377
setivar 5 360 5 356 50 3124 50 5700
setivar_object 5 374 5 340 50 772 50 844
setivar_young 5 374 5 354 50 773 50 847
str_concat 5 398 5 404 50 989 50 1174
throw 5 1264 5 1330 50 1725 50 1685

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1453406 1252406 213 2296 0 0% 0 0 65.792162
chunky-png (click) 293593 273404 86 1133 1 0% 0 0 33.874082
erubi-rails (click) 1911375 1654839 249 2824 8 0% 0 0 75.043634
hexapdf (click) 1487580 1236629 598 13856 38 0% 0 0 394.504221
liquid-c (click) 491526 439183 119 1852 5 0% 0 0 51.356498
liquid-compile (click) 413756 340984 150 2213 2 0% 0 0 68.68958
liquid-render (click) 632861 566624 136 2466 12 0% 0 0 70.211126
lobsters (click) 8311046 7112162 3216 54294 183 0% 0 0 1830.22618
mail (click) 742665 644592 346 5481 16 0% 0 0 157.459776
psych-load (click) 286499 243035 64 656 3 0% 0 0 20.442591
railsbench (click) 3291719 2784452 1633 16697 41 0% 0 0 478.741323
rubocop (click) 5836250 4905624 2891 52169 110 0% 4 0 1517.623933
ruby-lsp (click) 844962 718965 308 4991 26 0% 0 0 140.77409
sequel (click) 493371 439246 15 114 0 0% 0 0 4.176977
binarytrees (click) 9627 8546 8 75 0 0% 0 0 3.005227
blurhash (click) 57136 49713 29 501 0 0% 0 0 16.056223
erubi (click) 216695 172759 9 108 0 0% 0 0 3.633387
etanni (click) 24125 22834 10 89 0 0% 0 0 3.050823
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.585065
fluentd (click) 352116 299393 10 109 0 0% 0 0 3.876906
graphql (click) 398117 353733 69 695 0 0% 0 0 20.40513
graphql-native (click) 391638 334897 40 270 0 0% 0 0 8.719429
lee (click) 307759 278713 48 803 0 0% 0 0 24.683477
matmul (click) 11838 4563 10 119 0 0% 0 0 3.976666
nbody (click) 14884 19325 8 196 0 0% 0 0 5.177239
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.183567
optcarrot (click) 327944 312320 190 4549 34 0% 0 0 104.111646
protoboeuf (click) 163392 155036 14 1610 0 0% 0 0 45.790131
protoboeuf-encode (click) 234584 230620 16 1249 0 0% 0 0 31.857522
rack (click) 268446 244998 36 417 0 0% 0 0 10.915719
ruby-json (click) 21296 18539 10 187 0 0% 0 0 5.389656
rubykon (click) 140297 120385 138 1566 3 0% 0 0 44.446747
sudoku (click) 48090 43120 9 536 0 0% 0 0 15.883464
tinygql (click) 298904 270405 61 787 5 0% 0 0 22.611601
30k_ifelse (click) 6315665 5791705 9261 56473 0 0% 0 0 1573.101659
30k_methods (click) 2235566 1593704 5780 19359 0 0% 0 0 452.504145
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 1.98899
fib (click) 4041 4120 5 37 0 0% 0 0 1.427431
getivar (click) 5037 6014 6 64 0 0% 0 0 1.89313
keyword_args (click) 7637 6022 7 60 0 0% 0 0 2.046148
object-new (click) 3469 3394 5 35 0 0% 0 0 1.33187
respond_to (click) 9322 2039 6 95 0 0% 0 0 2.782446
ruby-xor (click) 6437 6880 6 72 0 0% 0 0 2.246805
setivar (click) 4188 4690 6 44 0 0% 0 0 1.582463
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.66476
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.896079
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.673667
throw (click) 7384 5592 8 59 0 0% 0 0 2.109922

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.