2022-11-02 Triage Log

Noise continues to make triaging a bit tedious. We've become good at identifying noise, but we may need to invest in trying to reduce it or automate some of the triaging needed to identify it. In terms of performance, this week ending up being positive albeit with improvements only outweighing regressions by a little. Some of the largest improvements were in reverts of previous regressions as well.

Triage done by @rylev. Revision range: 629a414d..822f8

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.7%	[0.2%, 7.9%]	28
Regressions ❌ (secondary)	1.7%	[0.2%, 7.0%]	97
Improvements ✅ (primary)	-1.2%	[-4.6%, -0.2%]	73
Improvements ✅ (secondary)	-1.3%	[-2.6%, -0.3%]	61
All ❌✅ (primary)	-0.4%	[-4.6%, 7.9%]	101

13 Regressions, 9 Improvements, 5 Mixed; 9 of them in rollups 41 artifact comparisons made in total

Regressions

Rollup of 6 pull requests #103572 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.3%]	3
Regressions ❌ (secondary)	0.3%	[0.3%, 0.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.3%]	3

Small number and severity of regressions means that we don't need to follow up on this.

Rollup of 5 pull requests #103671 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 1.5%]	14
Regressions ❌ (secondary)	2.0%	[0.5%, 4.1%]	18
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[0.2%, 1.5%]	14

Most of the secondary regressions seem to be noise, but the primary regressions are unlikely to be.
A perf run was done for #103641, but nothing showed up.
A new attempt for #103550 was kicked off.

Fix line numbers for MIR inlined code #103071 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.0%	[0.6%, 2.2%]	8
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[0.6%, 2.2%]	8

Perf regression seems real and @wesleywiser has committed to investigate.

poll_fn and Unpin: fix pinning #102737 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	2.0%	[0.4%, 4.1%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

Introduce UnordMap, UnordSet, and UnordBag (MCP 533) #102698 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	1.8%	[0.4%, 4.1%]	14
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

Rollup of 8 pull requests #103727 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	1.9%	[0.4%, 4.1%]	13
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

Bump to 1.67.0 #103731 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	2.1%	[0.5%, 4.1%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

update Miri #103721 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.1%	[0.4%, 4.1%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

ci: Bring back ninja for dist builders #103295 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.5%	[0.2%, 7.2%]	22
Regressions ❌ (secondary)	2.0%	[0.2%, 6.5%]	68
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.0%	[-1.0%, -1.0%]	1
All ❌✅ (primary)	1.5%	[0.2%, 7.2%]	22

Currently unclear why this would cause regressions, but it seems real.
A PR reverting this one was filed and improvements in performance were clear.
A call to revert has been made.

Don't use usub.with.overflow intrinsic #103299 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.6%	[3.2%, 4.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

Include both benchmarks and tests in the numbers given to TeFiltered{,Out} #103795 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.6%	[3.2%, 4.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

This is very likely noise as described here.

(almost) Always use ObligationCtxt when dealing with canonical queries #103590 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.2%	[1.6%, 2.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

This could be noise, but it might not be - either way it's too small of a regression to worry about as discussed here.

Track where diagnostics were created. #103217 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.2%	[1.6%, 2.8%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

The likelihood that this is noise is pretty high as you can see from the end of this graph.

Improvements

Revert “Unify tcx.constness and param env constness checks” #103284 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-1.0%, -0.2%]	43
Improvements ✅ (secondary)	-1.0%	[-2.7%, -0.1%]	36
All ❌✅ (primary)	-0.4%	[-1.0%, -0.2%]	43

privacy: Rename “accessibility levels” to “effective visibilities” #102233 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-2.0%	[-4.0%, -0.4%]	12
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

Do not consider repeated lifetime params for elision. #103450 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.9%	[-1.4%, -0.2%]	3
Improvements ✅ (secondary)	-1.9%	[-4.0%, -0.4%]	13
All ❌✅ (primary)	-0.9%	[-1.4%, -0.2%]	3

Rollup of 8 pull requests #103745 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-2.0%	[-4.0%, -0.5%]	12
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

Rollup of 5 pull requests #103755 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-3.5%	[-4.0%, -3.1%]	6
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

rustdoc: Simplify modifications of effective visibility table #103010 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.1%	[1.1%, 1.1%]	1
Improvements ✅ (primary)	-0.6%	[-1.3%, -0.3%]	21
Improvements ✅ (secondary)	-1.1%	[-1.5%, -0.3%]	21
All ❌✅ (primary)	-0.6%	[-1.3%, -0.3%]	21

Update LLVM submodule #103479 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-3.4%	[-3.9%, -3.1%]	6
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

Use br instead of switch in more cases. #103331 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.0%	[-4.6%, -0.4%]	34
Improvements ✅ (secondary)	-2.2%	[-3.3%, -1.0%]	12
All ❌✅ (primary)	-2.0%	[-4.6%, -0.4%]	34

Enable x.py check for miri #102950 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-3.5%	[-4.0%, -3.1%]	6
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

Mixed

Remove allow(rustc::potential_query_instability) in rustc_const_eval #102674 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	0.6%	[0.3%, 0.8%]	18
Improvements ✅ (primary)	-0.6%	[-1.4%, -0.2%]	5
Improvements ✅ (secondary)	-2.7%	[-3.9%, -0.3%]	8
All ❌✅ (primary)	-0.5%	[-1.4%, 0.4%]	6

A mix of noise and some accetpable regressions as described here.

Rollup of 7 pull requests #103714 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.5%	[0.4%, 0.7%]	5
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-1.9%	[-4.0%, -0.4%]	12
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

A mix of noise and some accetpable regressions as described here.

Rollup of 10 pull requests #103829 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.6%	[3.2%, 4.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.0%	[-2.0%, -0.2%]	13
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

A mix of noise and some accetpable regressions as described here.

Rollup of 6 pull requests #103832 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[0.4%, 2.2%]	2
Regressions ❌ (secondary)	0.6%	[0.3%, 0.9%]	2
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-1.9%	[-3.9%, -0.3%]	14
All ❌✅ (primary)	0.0%	[-1.4%, 2.2%]	4

A mix of noise and some accetpable regressions as described here.

Rollup of 5 pull requests #103841 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.4%, 0.6%]	8
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-2.8%, -1.7%]	7
All ❌✅ (primary)	0.5%	[0.4%, 0.6%]	8

Seems like #103760 is the culprit.