2024-01-23 Triage Log

This week saw a bunch of regressions caused by correctness fixes and in general doing more work in the compiler. These were offset by many improvements (especially around hashing in the compiler) that improved performance by ~2% across a large number of benchmarks. Don't get too excited about the large 45+% wins though, these were only for tiny benchmarks like helloworld. They were caused by a change in Cargo which introduces stripping of debug symbols from Rust release binaries by default, and in turn also improves compilation time for small crates.

Triage done by @kobzol. Revision range: f9c2421a..d6b151fc

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.2%, 1.5%]	11
Regressions ❌ (secondary)	2.2%	[0.2%, 9.9%]	26
Improvements ✅ (primary)	-3.2%	[-47.5%, -0.2%]	191
Improvements ✅ (secondary)	-7.9%	[-46.5%, -0.1%]	123
All ❌✅ (primary)	-3.0%	[-47.5%, 1.5%]	202

4 Regressions, 4 Improvements, 9 Mixed; 4 of them in rollups 48 artifact comparisons made in total

Regressions

fix fn/const items implied bounds and wf check (rebase) #120019 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.5%]	26
Regressions ❌ (secondary)	1.3%	[0.3%, 1.6%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.5%]	26

This was a correctness fix that had an expected performance hit.

Rollup of 8 pull requests #120187 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.3%, 1.0%]	8
Regressions ❌ (secondary)	0.9%	[0.3%, 1.8%]	16
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.3%, 1.0%]	8

This was a mixture of several PRs that had a perf. effect.
#119461 and #118811 slightly regressed the deep-vector benchmark, which was however quite noisy recently, so it was probably just a blip.
#116090 caused a real regression on doc benchmarks, which was caused simply by adding a bunch of new methods to the standard library.

Rollup of 8 pull requests #120196 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.3%, 1.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.3%, 1.3%]	1

Caused by #120145, which was a correctness fix, so marked as triaged.

Rollup of 10 pull requests #120242 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.1%	[3.1%, 3.1%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Caused by #120137, which added more validation to the compiler, and the perf. hit was deemed acceptable, so marked as triaged.

Improvements

Cache local DefId-keyed queries without hashing #119977 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.7%, 0.7%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.2%	[-8.2%, -0.2%]	161
Improvements ✅ (secondary)	-1.5%	[-3.7%, -0.3%]	64
All ❌✅ (primary)	-1.2%	[-8.2%, 0.7%]	162

Incredible wins across many benchmarks, caused by improving hashing in the compiler.
The wall-time results of this PR were even more impressive, resulting in a ~4% mean improvement across more than a hundred benchmark configurations!

Get rid of the hir_owner query. #120006 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.3%	[0.2%, 0.4%]	3
Improvements ✅ (primary)	-1.8%	[-5.5%, -0.3%]	129
Improvements ✅ (secondary)	-2.1%	[-4.4%, -0.4%]	46
All ❌✅ (primary)	-1.8%	[-5.5%, -0.3%]	129

Again, great wins across a lot of benchmarks.

Use UnhashMap for a few more maps #120076 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.2%]	13
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.2%]	4
All ❌✅ (primary)	-0.3%	[-0.3%, -0.2%]	13

Continuation of hashing improvements.

Always use RevealAll for const eval queries #119821 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.3%]	3
Improvements ✅ (secondary)	-1.4%	[-1.8%, -1.2%]	7
All ❌✅ (primary)	-0.3%	[-0.3%, -0.3%]	3

This PR improved mainly the performance of const evaluation in the compiler.

Mixed

Sandwich MIR optimizations between DSE. #119672 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.2%, 1.4%]	14
Regressions ❌ (secondary)	0.5%	[0.2%, 2.7%]	14
Improvements ✅ (primary)	-1.0%	[-2.2%, -0.2%]	31
Improvements ✅ (secondary)	-0.9%	[-2.2%, -0.2%]	10
All ❌✅ (primary)	-0.4%	[-2.2%, 1.4%]	45

Several nice wins on larger benchmarks, overall the wins outweighed the losses.

large_assignments: Lint on specific large args passed to functions #116520 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 0.8%]	45
Regressions ❌ (secondary)	0.6%	[0.2%, 2.1%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-6.0%	[-6.0%, -6.0%]	1
All ❌✅ (primary)	0.4%	[0.2%, 0.8%]	45

This new lint introduced more tracking of spans in the compiler, which expectedly slightly regressed compile times. It is expected that the new spans will be used by future lints/errors to provide better diagnostics.

Update cargo #120036 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.5%]	2
Regressions ❌ (secondary)	1.4%	[1.4%, 1.4%]	1
Improvements ✅ (primary)	-16.8%	[-47.6%, -0.4%]	13
Improvements ✅ (secondary)	-20.5%	[-46.9%, -2.2%]	37
All ❌✅ (primary)	-14.5%	[-47.6%, 0.5%]	15

A few tiny regressions, but there's not much to be done about these since this was a Cargo update.
The large instruction count improvements came from https://github.com/rust-lang/cargo/pull/13257, which introduced stripping of --release binaries by default. This coincidentally also reduces linking time on Linux quite a lot for tiny programs, which caused these improvements on helloworld-like crates.

error on incorrect implied bounds in wfcheck except for Bevy dependents #118553 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.2%, 1.6%]	30
Regressions ❌ (secondary)	0.4%	[0.2%, 0.6%]	13
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.0%	[-1.2%, -0.8%]	6
All ❌✅ (primary)	0.7%	[0.2%, 1.6%]	30

Regressions have outweighted the improvements here, however, this was a correctness fix, so we will have to take the hit. A part of the regressions reverted wins from #120123.

Rollup of 9 pull requests #120112 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.2%	[0.3%, 4.1%]	7
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.7%	[-0.7%, -0.7%]	1

This was caused by #120037, which seems to have just undone a previous perf. win on the same benchmarks caused by code being reshuffled around.

use implied bounds compat mode in MIR borrowck #120123 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.9%	[0.9%, 1.0%]	4
Improvements ✅ (primary)	-0.7%	[-0.9%, -0.6%]	6
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.7%	[-0.9%, -0.6%]	6

This PR had more wins than losses, however its effects were later mostly reverted in #118553.

LLVM 18 x86 data layout update #116672 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.7%, 0.7%]	1
Regressions ❌ (secondary)	1.0%	[0.3%, 3.3%]	6
Improvements ✅ (primary)	-0.4%	[-0.6%, -0.3%]	8
Improvements ✅ (secondary)	-0.6%	[-1.1%, -0.4%]	11
All ❌✅ (primary)	-0.3%	[-0.6%, 0.7%]	9

The wins have outweighted the losses.
This change is required to update to LLVM 18.
It introduced a few Max-RSS regressions, which were later fixed in #120080.

Rollup of 6 pull requests #120170 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.6%	[0.6%, 0.6%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.9%	[-1.4%, -0.5%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.4%	[-1.4%, 0.6%]	3

Tiny regression on a single benchmark, marked as triaged.

Pack u128 in the compiler to mitigate new alignment #120080 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.5%]	12
Regressions ❌ (secondary)	1.2%	[0.4%, 2.1%]	2
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-0.6%	[-0.7%, -0.3%]	5
All ❌✅ (primary)	0.3%	[-0.7%, 0.5%]	13

This PR mitigates the Max-RSS hits caused by #116672, at the cost of a few small instruction count regressions.