YJIT Benchmarks

Details for Benchmarks at 2022-06-09 07:15:15 GMT

YJIT metrics from the yjit-bench suite

¶Performance on Headline Benchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on Headline Benchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Performance on Other Benchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on Other Benchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Performance on MicroBenchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on MicroBenchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Want Raw Graphs and CSV?

• Headlining benchmarks (x86_64)
• All benchmarks (x86_64)
• CSV speed results (x86_64)

¶Benchmarks Speed Details

X

bench	No JIT (ms)	No JIT RSD	MJIT (ms)	MJIT RSD	YJIT (ms)	YJIT RSD	MJIT spd	MJIT spd RSD	YJIT spd	YJIT spd RSD	% in YJIT
activerecord	158.0	0.36%	176.6	3.80%	105.8	0.51%	0.89x	3.81%	1.49x	0.62%	85.54%
hexapdf	3358.8	0.98%			2371.3	1.39%			1.42x	1.70%	83.55%
liquid-render	207.5	1.40%	191.7	1.46%	136.2	2.05%	1.08x	2.02%	1.52x	2.49%	85.70%
mail	188.3	1.29%	182.9	1.92%	165.2	0.37%	1.03x	2.31%	1.14x	1.34%	99.35%
psych-load	2575.4	0.02%	2400.1	0.05%	1920.4	0.04%	1.07x	0.06%	1.34x	0.04%	88.45%
railsbench	2844.8	0.91%	3096.2	1.40%	2105.7	1.46%	0.92x	1.67%	1.35x	1.73%	86.12%
binarytrees	501.2	0.09%	330.3	0.38%	399.9	0.19%	1.52x	0.39%	1.25x	0.21%	84.27%
chunky_png	1094.1	0.03%	847.5	0.54%	712.5	0.18%	1.29x	0.54%	1.54x	0.19%	99.95%
discourse	596.6	6.20%	585.1	2.01%	562.8	3.38%	1.02x	6.52%	1.06x	7.07%	73.60%
erubi	473.9	0.62%	446.6	0.78%	404.3	0.70%	1.06x	1.00%	1.17x	0.93%	100.00%
erubi_rails	28.6	2.17%	29.4	3.90%	20.0	3.32%	0.97x	4.47%	1.43x	3.97%	86.30%
fannkuchredux	8585.6	0.06%	5841.3	0.18%	8772.5	0.04%	1.47x	0.19%	0.98x	0.07%	0.02%
lee	1413.6	0.98%	1132.1	1.36%	1004.4	1.40%	1.25x	1.67%	1.41x	1.71%	99.97%
nbody	107.4	0.11%	62.9	0.03%	88.3	0.46%	1.71x	0.12%	1.22x	0.47%	100.00%
optcarrot	6455.4	0.44%	2604.2	0.59%	3688.5	0.59%	2.48x	0.73%	1.75x	0.73%	96.13%
rubykon	13970.6	2.81%	9446.9	4.72%	7980.0	5.44%	1.48x	5.49%	1.75x	6.12%	99.88%
30k_ifelse	2333.9	0.03%	3126.2	2.82%	360.5	0.19%	0.75x	2.82%	6.47x	0.19%	100.00%
30k_methods	6585.8	0.00%	9730.0	1.32%	969.5	0.04%	0.68x	1.32%	6.79x	0.04%	100.00%
cfunc_itself	127.0	0.25%	77.0	0.97%	49.9	0.94%	1.65x	1.00%	2.54x	0.97%	100.00%
fib	273.6	0.05%	79.0	0.16%	62.5	0.56%	3.46x	0.17%	4.38x	0.57%	100.00%
getivar	102.9	0.04%	25.8	61.18%	42.6	0.68%	3.99x	61.18%	2.42x	0.68%	99.22%
keyword_args	313.0	0.08%	226.8	0.49%	57.8	0.62%	1.38x	0.50%	5.41x	0.63%	100.00%
respond_to	321.3	0.42%	263.4	0.46%	204.8	0.47%	1.22x	0.62%	1.57x	0.63%	100.00%
setivar	73.9	0.25%	14.4	75.01%	73.2	0.46%	5.14x	75.01%	1.01x	0.52%	99.40%
str_concat	136.7	1.31%	102.0	1.46%	112.0	1.38%	1.34x	1.97%	1.22x	1.91%	99.97%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.2.0dev (2022-06-09T05:39:18Z master 90b240d127) +YJIT [x86_64-linux]
Tested Ruby version for Ruby latest MJIT: ruby 3.2.0dev (2022-06-09T05:39:18Z master 90b240d127) +MJIT [x86_64-linux]

¶Benchmark Memory Usage Details

X

bench	CRuby 3.2.0dev mem (MiB)	MJIT mem (MiB)	YJIT 3.2.0dev mem (MiB)	Inline Code	Outlined Code	YJIT Mem overhead
activerecord	55	63	321	1	1	474.9%
hexapdf	294		617	2	1	110.0%
liquid-render	22	28	284	1	1	1157.2%
mail	41	45	303	1	1	631.6%
psych-load	30	39	290	1	1	861.4%
railsbench	93	122	374	3	2	300.9%
binarytrees	20	27	278	1	1	1266.8%
chunky_png	32	40	292	1	1	788.6%
discourse	386	411	708	6	5	83.4%
erubi	49	92	329	1	1	559.6%
erubi_rails	92	96	365	2	2	295.8%
fannkuchredux	16	21	274	1	1	1536.5%
lee	26	36	289	1	1	977.5%
nbody	16	19	274	1	1	1553.6%
optcarrot	54	63	317	1	1	487.1%
rubykon	46	54	305	1	1	559.9%
30k_ifelse	58	112	367	6	5	525.3%
30k_methods	49	138	324	3	2	554.6%
cfunc_itself	16	20	274	1	1	1555.0%
fib	16	19	274	1	1	1549.2%
getivar	16	19	274	1	1	1529.8%
keyword_args	16	21	274	1	1	1550.4%
respond_to	16	20	274	1	1	1569.2%
setivar	16	19	274	1	1	1542.5%
str_concat	76	85	353	1	1	364.1%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

¶Number of Iterations and Warmups Tested

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

¶Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

¶Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.

• prod_ruby_no_jit
• prod_ruby_with_mjit
• prod_ruby_with_yjit
• yjit_stats