2023-02-07 Triage Log

Much noise in benchmarks this week, which makes it hard to tell what the real improvements were and what they were due to. A query cache change (PR #107667) is part of the story. In addition, much improvement was reaped from the change to not deaggregate MIR (PR #107267). Finally, microoptimizing fold_ty (PR #107627) yielded a small improvement to a broad set of benchmarks.

Triage done by @pnkfelix. Revision range: a64ef7d0..e4dd9edb

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	2.4%	[0.3%, 18.8%]	18
Regressions ❌ (secondary)	1.8%	[0.2%, 4.1%]	21
Improvements ✅ (primary)	-1.0%	[-3.2%, -0.3%]	88
Improvements ✅ (secondary)	-4.0%	[-13.1%, -0.1%]	47
All ❌✅ (primary)	-0.4%	[-3.2%, 18.8%]	106

3 Regressions, 3 Improvements, 8 Mixed; 3 of them in rollups 41 artifact comparisons made in total

Regressions

Fix handling of items inside a doc(hidden) block #107000 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	4.1%	[0.2%, 18.9%]	9
Regressions ❌ (secondary)	1.1%	[0.2%, 1.7%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	4.1%	[0.2%, 18.9%]	9

• a number of doc benchmarks regressed, but only doc benchmarks.
• the big hit was a 18.9% regression to doc on hyper.
• rustdoc developer says the issue cannot be resolved without some amount of regression, so marked as triaged.

don't point at nonexisting code beyond EOF when warning about delims #107663 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	1
Regressions ❌ (secondary)	0.9%	[0.2%, 1.3%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.2%	[0.2%, 0.2%]	1

• already marked as triaged

Run expand-yaml-anchors in x test tidy #107704 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.0%	[0.4%, 1.5%]	3
Regressions ❌ (secondary)	3.7%	[3.4%, 4.3%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[0.4%, 1.5%]	3

• PR author says PR touched only CI code, and therefore this must be noise
• primary regressions are on cranelift variations; secondary are on keccak
• graphs for cranelift and keccak both have up-and-down swings that both begun with #107627; will check with nnethercote about that
• marked as triaged.

Improvements

emit ConstEquate in TypeRelating<D> #107434 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.3%]	2
Improvements ✅ (secondary)	-0.8%	[-1.3%, -0.2%]	10
All ❌✅ (primary)	-0.4%	[-0.4%, -0.3%]	2

Recover form missing expression in for loop #107526 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.3%]	2
Improvements ✅ (secondary)	-1.0%	[-1.3%, -0.6%]	7
All ❌✅ (primary)	-0.3%	[-0.3%, -0.3%]	2

Remove OnHit callback from query caches. #107667 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.6%	[-1.7%, -0.2%]	49
Improvements ✅ (secondary)	-2.0%	[-5.9%, -0.3%]	36
All ❌✅ (primary)	-0.6%	[-1.7%, -0.2%]	49

• as already noted by nnethercote, these results appear better than reality
• much of the delta is noise (namely inverse of #107627 (see below)).

Mixed

Don't generate unecessary &&self.field in deriving Debug #107599 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.3%, 0.4%]	2
Regressions ❌ (secondary)	1.0%	[0.2%, 4.9%]	7
Improvements ✅ (primary)	-1.0%	[-2.9%, -0.4%]	9
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.8%	[-2.9%, 0.4%]	11

• already triaged as noise

Rollup of 6 pull requests #107642 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.4%	[0.1%, 0.8%]	9
Improvements ✅ (primary)	-0.3%	[-0.5%, -0.2%]	8
Improvements ✅ (secondary)	-0.8%	[-1.1%, -0.5%]	7
All ❌✅ (primary)	-0.3%	[-0.5%, -0.2%]	8

• sole regressions are to (secondary) deep-vector and match-stress.
• perf graph for deep-vector hints that this might not be noise, but it is also minor.
• marking as triaged.

Rollup of 8 pull requests #107650 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.4%	[0.3%, 6.2%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-7.4%	[-7.4%, -7.4%]	1
All ❌✅ (primary)	-	-	0

• the vast bulk regressions are on six variations of secondary benchmarks match-stress
• and all of those appear to to have all been resolved by follow-on PR #107667 (see above)
• marking as triaged.

Do not deaggregate MIR #107267 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.1%]	51
Regressions ❌ (secondary)	0.8%	[0.1%, 2.0%]	30
Improvements ✅ (primary)	-1.4%	[-2.6%, -0.3%]	25
Improvements ✅ (secondary)	-6.0%	[-12.9%, -0.6%]	27
All ❌✅ (primary)	-0.1%	[-2.6%, 1.1%]	76

• already marked as triaged (wins far outweigh losses)

Rollup of 3 pull requests #107672 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.8%	[0.8%, 0.8%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.6%	[-1.1%, -0.4%]	7
All ❌✅ (primary)	-	-	0

• regressions are to secondary benchmark externs.
• whatever this regression was, it has since been more than recovered by #107667

Less import overhead for errors #107679 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.3%, 0.5%]	2
Regressions ❌ (secondary)	0.9%	[0.3%, 1.3%]	11
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.7%	[-0.7%, -0.7%]	2
All ❌✅ (primary)	0.4%	[0.3%, 0.5%]	2

• already marked as triaged (its obvious noise)

rustdoc: change trait bound formatting #102842 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.7%	[0.7%, 0.7%]	2
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.2%]	2
Improvements ✅ (secondary)	-0.9%	[-1.3%, -0.3%]	11
All ❌✅ (primary)	-0.3%	[-0.4%, -0.2%]	2

• already marked as triaged (and its noise)

Optimize fold_ty #107627 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.1%	[1.0%, 1.1%]	2
Regressions ❌ (secondary)	2.3%	[0.6%, 4.2%]	12
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.3%]	18
Improvements ✅ (secondary)	-0.7%	[-1.6%, -0.2%]	30
All ❌✅ (primary)	-0.3%	[-0.5%, 1.1%]	20

• already marked as triaged

Untriaged Pull Requests

• #107704 Run expand-yaml-anchors in x test tidy
• #107672 Rollup of 3 pull requests
• #107650 Rollup of 8 pull requests
• #107642 Rollup of 6 pull requests
• #107408 Rollup of 9 pull requests
• #107143 Rollup of 9 pull requests
• #107103 Use new solver in evaluate_obligation query (when new solver is enabled)
• #107101 Filter param-env predicates for errors before calling to_opt_poly_trait_pred
• #107000 Fix handling of items inside a doc(hidden) block
• #106757 Rollup of 10 pull requests
• #105657 Guard ProjectionTy creation against passing the wrong number of substs
• #105472 Make encode_info_for_trait_item use queries instead of accessing the HIR
• #105426 Catch panics/unwinding in destruction of TLS values
• #105378 Rollup of 9 pull requests
• #105323 Perform SimplifyLocals before ConstProp.
• #105147 Allow unsafe through inline const
• #104566 couple of clippy::perf fixes
• #104533 Clean up and harden various methods around trait substs
• #104017 Rollup of 5 pull requests
• #103998 Rollup of 6 pull requests
• #103975 Some tracing and comment cleanups
• #103934 std: sync “Dependencies of the backtrace crate” with backtrace
• #103880 Use non-ascribed type as field's type in mir
• #103841 Rollup of 5 pull requests
• #103650 rustdoc: change .src-line-numbers > span to .src-line-numbers > a
• #103562 Rollup of 10 pull requests
• #103439 Show note where the macro failed to match
• #103295 ci: Bring back ninja for dist builders
• #103071 Fix line numbers for MIR inlined code
• #102975 Rollup of 7 pull requests

Nags requiring follow up

TODO: Nags