YJIT Benchmarks

Details for Benchmarks at 2024-11-06 03:44:09 UTC

YJIT metrics from the yjit-bench suite using Ruby c690ca03f3

Using the geomean of the headline benchmarks for x86 YJIT 3.4.0dev is
  • 87.7% faster than CRuby 3.4.0dev
  • 3.9% faster than YJIT 3.3.4
On railsbench it is
  • 98.6% faster than CRuby 3.4.0dev
  • 4.1% faster than YJIT 3.3.4

Performance on Headline Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Headline Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on Other Benchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on Other Benchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Performance on MicroBenchmarks

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

Memory Usage on MicroBenchmarks

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

Want Raw Graphs and CSV?

Benchmarks Speed Details

Benchmark Memory Usage Details

Number of Iterations and Warmups Tested

bench CRuby 3.3.4 warmups CRuby 3.3.4 iters CRuby 3.4.0dev warmups CRuby 3.4.0dev iters YJIT 3.3.4 warmups YJIT 3.3.4 iters YJIT 3.4.0dev warmups YJIT 3.4.0dev iters
activerecord 5 83 5 83 50 158 50 163
chunky-png 5 31 5 32 50 15 50 20
erubi-rails 5 15 5 15 50 10 50 10
hexapdf 5 10 5 10 50 10 50 10
liquid-c 5 410 5 425 50 568 50 626
liquid-compile 5 411 5 409 50 540 50 557
liquid-render 5 169 5 166 50 403 50 436
lobsters 5 20 5 20 50 10 50 10
mail 5 188 5 186 50 251 50 235
psych-load 5 10 5 10 50 10 50 10
railsbench 5 10 5 10 50 10 50 10
rubocop 5 156 5 155 50 252 50 267
ruby-lsp 5 156 5 151 50 244 50 232
sequel 5 429 5 421 50 597 50 575
binarytrees 5 84 5 84 50 145 50 149
blurhash 5 82 5 83 50 128 50 145
erubi 5 114 5 148 50 95 50 129
etanni 5 90 5 98 50 52 50 62
fannkuchredux 5 52 5 55 50 101 50 107
fluentd 5 60 5 64 50 22 50 24
graphql 5 85 5 86 50 57 50 58
graphql-native 5 39 5 52 50 10 50 14
lee 5 23 5 22 50 10 50 10
matmul 5 51 5 53 50 88 50 99
nbody 5 248 5 261 50 595 50 706
nqueens 5 119 5 119 50 74 50 561
optcarrot 5 10 5 10 50 10 50 10
protoboeuf 5 172 5 175 50 616 50 743
protoboeuf-encode 5 178 5 201 50 325 50 779
rack 5 589 5 622 50 962 50 1053
ruby-json 5 84 5 82 50 50 50 48
rubykon 5 20 5 21 50 10 50 10
sudoku 5 48 5 49 50 163 50 160
tinygql 5 37 5 37 50 24 50 25
30k_ifelse 5 28 5 28 50 214 50 191
30k_methods 5 38 5 38 50 481 50 476
cfunc_itself 5 346 5 341 50 1046 50 1136
fib 5 125 5 127 50 901 50 962
getivar 5 206 5 217 50 2269 50 2620
keyword_args 5 108 5 108 50 959 50 1231
object-new 5 321 5 276 50 406 50 400
respond_to 5 137 5 140 50 3258 50 4042
ruby-xor 5 243 5 245 50 803 50 1381
setivar 5 360 5 356 50 3124 50 5714
setivar_object 5 372 5 348 50 764 50 847
setivar_young 5 371 5 344 50 771 50 847
str_concat 5 398 5 409 50 991 50 1185
throw 5 1266 5 1318 50 1728 50 1730

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

bench Exit Report Inline Outlined Comp iSeqs Comp Blocks Inval Inval Ratio Bind Alloc Bind Set Const Bumps Compile Time MS
activerecord (click) 1444056 1228571 213 2296 0 0% 0 0 66.737169
chunky-png (click) 294035 226382 86 1133 1 0% 0 0 33.842624
erubi-rails (click) 1916347 1634353 249 2829 8 0% 0 0 77.090819
hexapdf (click) 1489273 1270543 598 13853 38 0% 0 0 395.060212
liquid-c (click) 495867 440081 119 1852 5 0% 0 0 50.916611
liquid-compile (click) 413523 347015 150 2194 2 0% 0 0 67.197153
liquid-render (click) 605343 539477 136 2466 12 0% 0 0 69.120921
lobsters (click) 8309490 6914663 3215 54160 181 0% 0 0 1822.929518
mail (click) 778995 691684 346 5481 16 0% 0 0 157.19789
psych-load (click) 260256 214440 64 656 3 0% 0 0 20.929901
railsbench (click) 3308351 2833175 1632 16671 41 0% 0 0 480.312798
rubocop (click) 5812998 4973558 2891 52169 110 0% 4 0 1511.342966
ruby-lsp (click) 847188 713293 308 4986 26 0% 0 0 139.324289
sequel (click) 463523 405387 15 125 0 0% 0 0 4.546731
binarytrees (click) 9627 8546 8 75 0 0% 0 0 3.055305
blurhash (click) 57136 49713 29 501 0 0% 0 0 16.131103
erubi (click) 232289 186621 9 108 0 0% 0 0 3.573937
etanni (click) 27632 25826 9 85 0 0% 0 0 2.903099
fannkuchredux (click) 24038 30826 5 257 0 0% 0 0 7.704305
fluentd (click) 349145 314898 10 109 0 0% 0 0 3.841552
graphql (click) 396747 337848 69 695 0 0% 0 0 20.385357
graphql-native (click) 393713 328201 40 270 0 0% 0 0 8.5585
lee (click) 283918 255476 48 803 0 0% 0 0 24.807405
matmul (click) 11838 4563 10 119 0 0% 0 0 3.949574
nbody (click) 14884 19325 8 196 0 0% 0 0 5.170839
nqueens (click) 21155 28164 7 247 0 0% 0 0 7.254959
optcarrot (click) 327944 312320 190 4549 34 0% 0 0 105.809771
protoboeuf (click) 163392 155036 14 1610 0 0% 0 0 46.209064
protoboeuf-encode (click) 234584 230620 16 1249 0 0% 0 0 32.220719
rack (click) 234288 199178 36 417 0 0% 0 0 10.650015
ruby-json (click) 27119 23942 10 187 0 0% 0 0 5.373696
rubykon (click) 140693 120664 138 1568 3 0% 0 0 44.815577
sudoku (click) 48090 43120 9 536 0 0% 0 0 16.024495
tinygql (click) 275972 244826 61 787 5 0% 0 0 22.087101
30k_ifelse (click) 6315665 5791705 9261 56473 0 0% 0 0 1576.846403
30k_methods (click) 2235566 1593704 5780 19359 0 0% 0 0 455.97172
cfunc_itself (click) 6912 5554 7 62 0 0% 0 0 1.980486
fib (click) 4041 4120 5 37 0 0% 0 0 1.426293
getivar (click) 5037 6014 6 64 0 0% 0 0 1.905162
keyword_args (click) 7637 6022 7 60 0 0% 0 0 2.050864
object-new (click) 3469 3394 5 35 0 0% 0 0 1.396164
respond_to (click) 9322 2039 6 95 0 0% 0 0 2.730172
ruby-xor (click) 6437 6880 6 72 0 0% 0 0 2.237045
setivar (click) 4188 4690 6 44 0 0% 0 0 1.527523
setivar_object (click) 4717 4838 6 46 0 0% 0 0 1.705198
setivar_young (click) 5427 5560 7 54 0 0% 0 0 1.871938
str_concat (click) 7646 7893 9 81 0 0% 0 0 2.730703
throw (click) 7384 5592 8 59 0 0% 0 0 2.153702

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.