YJIT Benchmarks

Details for Benchmarks at 2022-05-16 07:13:56 GMT

YJIT metrics from the yjit-bench suite

¶Performance on Headline Benchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on Headline Benchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Performance on Other Benchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on Other Benchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Performance on MicroBenchmarks

X

Speed of each Ruby implementation relative to the baseline CRuby measurement. Higher is better.

¶Memory Usage on MicroBenchmarks

X

Memory usage of each Ruby implementation relative to the baseline CRuby measurement. Lower is better.

¶Want Raw Graphs and CSV?

• Headlining benchmarks (x86_64)
• All benchmarks (x86_64)
• CSV speed results (x86_64)

¶Benchmarks Speed Details

X

bench	No JIT (ms)	No JIT RSD	MJIT (ms)	MJIT RSD	YJIT (ms)	YJIT RSD	MJIT spd	MJIT spd RSD	YJIT spd	YJIT spd RSD	% in YJIT
activerecord	151.9	0.35%	176.8	4.27%	103.3	0.44%	0.86x	4.29%	1.47x	0.57%	85.54%
hexapdf	3223.9	1.37%			2359.2	1.81%			1.37x	2.27%	82.62%
liquid-render	211.8	1.08%	203.5	1.37%	145.3	1.61%	1.04x	1.75%	1.46x	1.94%	85.70%
mail	200.4	2.08%	203.3	3.26%	185.8	2.57%	0.99x	3.87%	1.08x	3.31%	99.34%
psych-load	2835.4	0.06%	2330.1	0.04%	1982.0	0.28%	1.22x	0.07%	1.43x	0.28%	88.45%
railsbench	3109.3	0.54%	3506.5	0.64%	2391.7	1.13%	0.89x	0.84%	1.30x	1.26%	86.19%
binarytrees	478.0	0.04%	313.9	0.31%	367.2	0.15%	1.52x	0.31%	1.30x	0.16%	84.27%
chunky_png	980.4	0.27%	795.4	0.30%	675.1	0.52%	1.23x	0.40%	1.45x	0.58%	99.95%
discourse	580.5	13.15%	620.5	14.79%	529.8	18.42%	0.94x	19.79%	1.10x	22.63%	73.52%
erubi	485.5	0.68%	460.6	1.11%	416.0	0.81%	1.05x	1.31%	1.17x	1.06%	100.00%
erubi_rails	32.3	39.94%	35.0	43.42%	25.3	50.64%	0.92x	59.00%	1.28x	64.50%	86.30%
fannkuchredux	8636.8	0.15%	4158.4	0.08%	8881.6	0.09%	2.08x	0.17%	0.97x	0.18%	0.02%
lee	1237.0	0.18%	969.1	0.27%	929.3	0.27%	1.28x	0.33%	1.33x	0.32%	99.97%
nbody	122.5	0.04%	72.9	0.18%	89.0	0.49%	1.68x	0.19%	1.38x	0.49%	100.00%
optcarrot	6220.1	0.43%	2602.9	0.58%	3447.4	0.60%	2.39x	0.73%	1.80x	0.74%	96.13%
rubykon	13681.4	1.88%	11752.1	1.57%	7970.5	2.12%	1.16x	2.45%	1.72x	2.84%	99.73%
30k_ifelse	2280.4	0.03%	3605.5	3.89%	358.6	0.19%	0.63x	3.90%	6.36x	0.19%	100.00%
30k_methods	6364.4	0.02%	10149.0	16.92%	876.4	0.06%	0.63x	16.92%	7.26x	0.06%	100.00%
cfunc_itself	100.1	0.17%	68.4	0.30%	44.6	0.93%	1.46x	0.34%	2.25x	0.95%	100.00%
fib	239.9	0.08%	86.1	0.03%	62.5	0.65%	2.79x	0.08%	3.84x	0.65%	100.00%
getivar	116.3	0.06%	26.1	71.47%	42.6	0.72%	4.45x	71.47%	2.73x	0.72%	99.21%
keyword_args	286.8	0.09%	224.5	0.37%	53.1	0.67%	1.28x	0.38%	5.40x	0.68%	100.00%
respond_to	264.3	0.14%	232.2	0.45%	188.8	0.43%	1.14x	0.47%	1.40x	0.45%	100.00%
setivar	78.3	0.34%	10.8	101.32%	54.3	0.84%	7.26x	101.32%	1.44x	0.90%	99.55%
str_concat	132.7	0.86%	102.3	1.35%	108.5	1.45%	1.30x	1.60%	1.22x	1.68%	99.68%

RSD is relative standard deviation - the standard deviation divided by the mean, expressed as a percentage.
% in YJIT is the percentage of instructions that complete in YJIT rather than exiting to the non-JITted interpreter. YJIT performs better when this is higher.
Speedup is relative to interpreted CRuby. So an "MJIT speedup" of 1.21x means MJIT runs at 1.21 times the iters/second of CRuby with JIT disabled.

You can find our benchmark code in the yjit-bench Github repo and the yjit-extra-benchmarks Github repo.
Our benchmark-runner and reporting code is in the yjit-metrics Github repo.

Tested Ruby version for YJIT and No-JIT: ruby 3.2.0dev (2022-05-16T03:24:26Z master 36efb2a146) +YJIT [x86_64-linux]
Tested Ruby version for Ruby latest MJIT: ruby 3.2.0dev (2022-05-16T03:24:26Z master 36efb2a146) +MJIT [x86_64-linux]

¶Benchmark Memory Usage Details

X

bench	CRuby 3.2.0dev mem (MiB)	MJIT mem (MiB)	YJIT 3.2.0dev mem (MiB)	Inline Code	Outlined Code	YJIT Mem overhead
activerecord	64	72	330	1	1	409.3%
hexapdf	234		581	2	1	148.2%
liquid-render	30	35	292	1	1	848.0%
mail	49	52	311	1	1	528.0%
psych-load	37	50	300	1	1	709.1%
railsbench	102	137	384	3	2	273.4%
binarytrees	31	33	288	1	1	830.9%
chunky_png	41	49	301	1	1	624.0%
discourse	387	403	706	6	5	82.5%
erubi	39	43	298	1	1	651.9%
erubi_rails	102	106	375	2	2	265.7%
fannkuchredux	25	27	283	1	1	1022.9%
lee	32	40	293	1	1	810.7%
nbody	24	27	282	1	1	1034.5%
optcarrot	58	67	320	1	1	443.6%
rubykon	50	80	310	1	1	509.9%
30k_ifelse	66	172	375	6	5	461.7%
30k_methods	57	184	331	3	2	474.4%
cfunc_itself	24	27	282	1	1	1031.5%
fib	25	27	282	1	1	1029.3%
getivar	25	27	282	1	1	1028.7%
keyword_args	24	27	282	1	1	1034.5%
respond_to	24	27	283	1	1	1035.5%
setivar	25	27	282	1	1	1028.0%
str_concat	112	93	354	1	1	215.9%

Memory is shown in mebibytes (1024 * 1024 bytes.)

Older YJIT allocated an additional 256MiB for generated code. Current YJIT allocates executable memory on demand, so this overhead should no longer be present.

¶Number of Iterations and Warmups Tested

Different Ruby configurations want different amounts of warmup. With no JIT, CRuby needs hardly any. YJIT and MJIT 3.0 both warm up quite quickly, while MJIT in 3.1 often slows down for a time as it compiles, after an unpredictable delay.

¶Benchmark YJIT Stats

Hover your cursor over the benchmark names for descriptions of each benchmark.

YJIT stats correspond to the YJIT stats exit report.

Note: currently, all stats are collected on x86_64, not ARM.

¶Raw JSON data files

All graphs and table data in this page comes from processing these data files, which come from benchmark runs.

• prod_ruby_no_jit
• prod_ruby_with_mjit
• prod_ruby_with_yjit
• yjit_stats