2023-06-27 Triage Log

Relatively quiet week outside of a large win on one incremental benchmark in a regression test (i.e., not real world code). Bimodality in a number of benchmarks continues to be an issue.

Triage done by @simulacrum. Revision range: b9d608c979ad3c0700f9f0237a8c12feb0ba44fb..b5e51db16dfbf5685e32dfe2d9a835a5c695afe4

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.0%	[0.4%, 3.7%]	186
Regressions ❌ (secondary)	1.1%	[0.2%, 3.0%]	95
Improvements ✅ (primary)	-1.9%	[-3.7%, -0.9%]	3
Improvements ✅ (secondary)	-16.1%	[-78.8%, -0.3%]	5
All ❌✅ (primary)	0.9%	[-3.7%, 3.7%]	189

5 Regressions, 6 Improvements, 3 Mixed; 5 of them in rollups 44 artifact comparisons made in total

Regressions

proof tree nits #112835 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.3%, 0.9%]	7
Regressions ❌ (secondary)	1.2%	[0.5%, 1.7%]	10
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.7%	[0.3%, 0.9%]	7

The regression in diesel seems to have persisted in future commits, but PR authors note that the code changed in this PR is not expected to affect benchmarks, so it is likely to be optimizer noise of some kind.

Disable two mir opts that are known to be unsound #112834 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.3%, 0.9%]	48
Regressions ❌ (secondary)	0.5%	[0.2%, 1.1%]	30
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.2%	[-0.3%, -0.1%]	2
All ❌✅ (primary)	0.4%	[0.3%, 0.9%]	48

Regression on multiple benchmarks, seems to be real, likely to be expected with disabling of optimization passes.

Inline before merging cgus #112695 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 3.6%]	98
Regressions ❌ (secondary)	0.5%	[0.3%, 2.7%]	37
Improvements ✅ (primary)	-1.0%	[-1.2%, -0.9%]	2
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	2
All ❌✅ (primary)	0.5%	[-1.2%, 3.6%]	100

Instruction counts (shown in this table) are not the best metric for this change. See comment on the PR with more details for why this change makes sense.

Rollup of 9 pull requests #112957 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.3%]	8
Regressions ❌ (secondary)	0.4%	[0.4%, 0.4%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.3%]	8

Regressions are limited to subset of incremental benchmarks, multiple of which are benchmarks prone to bimodality, with relatively low significance factors. No further investigation warranted.

Rollup of 2 pull requests #113057 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.4%, 0.8%]	5
Regressions ❌ (secondary)	0.8%	[0.3%, 1.1%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.4%, 0.8%]	5

See comment for details.

Improvements

Avoid Lrc<Box<dyn CodegenBackend>>. #112913 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.2%]	10
All ❌✅ (primary)	-	-	0

rustc_session: default to -Z plt=yes on non-x86_64 #109982 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.2%]	5
Improvements ✅ (secondary)	-0.5%	[-0.6%, -0.3%]	3
All ❌✅ (primary)	-0.3%	[-0.3%, -0.2%]	5

Rollup of 5 pull requests #113014 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-1.2%, -0.4%]	39
Improvements ✅ (secondary)	-0.6%	[-0.9%, -0.3%]	20
All ❌✅ (primary)	-0.7%	[-1.2%, -0.4%]	39

Do not emit coerce_suggestions for expr from destructuring assignment desugaring #112476 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.6%	[-0.8%, -0.5%]	6
Improvements ✅ (secondary)	-1.0%	[-1.1%, -1.0%]	4
All ❌✅ (primary)	-0.6%	[-0.8%, -0.5%]	6

Move windows-sys arm32 shim to c.rs #113001 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-1.2%	[-1.3%, -1.1%]	6
All ❌✅ (primary)	-0.7%	[-0.7%, -0.7%]	1

Rollup of 6 pull requests #113049 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-1.2%, -0.5%]	17
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.8%	[-1.2%, -0.5%]	17

Mixed

Rollup of 6 pull requests #112877 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	4
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	2
Improvements ✅ (primary)	-2.9%	[-2.9%, -2.9%]	1
Improvements ✅ (secondary)	-78.9%	[-78.9%, -78.9%]	1
All ❌✅ (primary)	-0.3%	[-2.9%, 0.4%]	5

Various impl trait in assoc tys cleanups #112891 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.3%]	1
Regressions ❌ (secondary)	0.7%	[0.6%, 0.7%]	2
Improvements ✅ (primary)	-0.5%	[-0.7%, -0.4%]	6
Improvements ✅ (secondary)	-1.0%	[-1.3%, -0.6%]	8
All ❌✅ (primary)	-0.4%	[-0.7%, 0.3%]	7

Performance is mixed, but the cleanups are useful in their own right. Marking as triaged.

Specialize StepBy<Range<{integer}>> #111850 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.5%	[0.5%, 0.7%]	3
Improvements ✅ (primary)	-1.7%	[-2.1%, -1.4%]	2
Improvements ✅ (secondary)	-0.5%	[-0.6%, -0.4%]	6
All ❌✅ (primary)	-1.7%	[-2.1%, -1.4%]	2