2022-09-14 Triage Log

From the viewpoint of metrics gathering, this was an absolutely terrible week, because the vast majority of this week's report is dominated by noise. Several benchmarks (html5ever, cranelift-codegen, and keccak) have all been exhibiting bimodal behavior where their compile-times would regress and improve randomly from run to run. Looking past that, we had one small win from adding an inline directive.

Triage done by @pnkfelix. Revision range: e7cdd4c0..17cbdfd0

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.1%	[0.2%, 6.2%]	26
Regressions ❌ (secondary)	1.9%	[0.1%, 5.6%]	34
Improvements ✅ (primary)	-1.8%	[-29.4%, -0.2%]	42
Improvements ✅ (secondary)	-1.3%	[-5.3%, -0.2%]	50
All ❌✅ (primary)	-0.7%	[-29.4%, 6.2%]	68

11 Regressions, 11 Improvements, 13 Mixed; 11 of them in rollups 71 artifact comparisons made in total

Regressions

Simplify hir::PathSegment #101228 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.3%, 1.3%]	16
Regressions ❌ (secondary)	1.3%	[0.9%, 1.8%]	14
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.7%	[0.3%, 1.3%]	16

• already triaged as “these regressions aren't real; they are an artifact of some current bimodality”

Rollup of 7 pull requests #101485 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.9%	[0.3%, 1.6%]	15
Regressions ❌ (secondary)	1.2%	[0.8%, 1.6%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.9%	[0.3%, 1.6%]	15

• already triaged as “these regressions aren't real; they are an artifact of some current bimodality”

Track PGO profiles in depinfo #100801 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.0%	[1.8%, 2.4%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

• already triaged as “these regressions aren't real; they are an artifact of some current bimodality”

Make HandleCycleError an enum instead of a macro-generated closure #101303 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.2%	[1.1%, 1.3%]	2
Regressions ❌ (secondary)	3.4%	[3.1%, 3.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.2%	[1.1%, 1.3%]	2

• already triaged as “these regressions aren't real; they are an artifact of some current bimodality”

Shrink hir::Ty and hir::Pat #101467 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.8%, 1.0%]	6
Regressions ❌ (secondary)	7.5%	[7.5%, 7.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.8%	[0.8%, 1.0%]	6

• already triaged as noise (all regressions are noise in html5ever).

Normalize before erasing late-bound regions in equal_up_to_regions #101437 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.2%	[1.1%, 1.3%]	2
Regressions ❌ (secondary)	3.4%	[3.0%, 3.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.2%	[1.1%, 1.3%]	2

• already triaged as noise; cranelift-codegen and keccak have been noisy lately.

Rollup of 6 pull requests #101639 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.1%	[0.8%, 4.0%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

• already triaged as noise from bimodality.

Only compute captures once when building MIR. #100968 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.2%	[1.1%, 1.3%]	2
Regressions ❌ (secondary)	3.4%	[3.1%, 3.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.2%	[1.1%, 1.3%]	2

• already triaged as noise.

Fix LLVM IR type mismatch reported in #99551 #101647 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.3%, 1.3%]	1
Regressions ❌ (secondary)	3.4%	[3.1%, 3.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.3%, 1.3%]	1

• already triaged as noise from cranelift-codegen and keccak.

Simplify codeblock and their associated tooltip #101613 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.3%	[1.3%, 1.3%]	1
Regressions ❌ (secondary)	3.4%	[3.1%, 3.9%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.3%, 1.3%]	1

• already triaged as noise.

Compute information about function parameters on THIR #101086 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 0.5%]	10
Regressions ❌ (secondary)	0.6%	[0.3%, 1.6%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[0.2%, 0.5%]	10

• between 0.38% and 0.47% regression for stm32f4 in various incremental scenarios. Approximately 0.25% on serde,clap,image for incremental.
• it seems plausible that this is just overhead that we have to pay, in terms of adding more state to the THIR, in exchange for simpler MIR construction code.
• (its also possible that this is just noise, though the listed benchmarks are not the same as the other ones that have been exhibiting bimodality this last week.)
• marking as triaged.

Improvements

Inline <T as From<T>>::from #100733 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-2.2%, -0.2%]	25
Improvements ✅ (secondary)	-1.4%	[-2.2%, -0.5%]	22
All ❌✅ (primary)	-0.8%	[-2.2%, -0.2%]	25

Rollup of 8 pull requests #101508 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.2%	[-1.3%, -1.1%]	2
Improvements ✅ (secondary)	-3.3%	[-3.8%, -2.9%]	6
All ❌✅ (primary)	-1.2%	[-1.3%, -1.1%]	2

• (bimodality in other direction, cranelift-codegen and keccak)

Lower the assume intrinsic to a MIR statement #98332 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-0.9%, -0.7%]	6
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.8%	[-0.9%, -0.7%]	6

• presume bimodality in other direction for html5ever

Update miri submodule #101522 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.8%	[-0.9%, -0.7%]	6
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.8%	[-0.9%, -0.7%]	6

• presume bimodality in other direction for html5ever

Rollup of 14 pull requests #101544 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.2%	[-1.3%, -1.1%]	2
Improvements ✅ (secondary)	-3.3%	[-3.8%, -3.0%]	6
All ❌✅ (primary)	-1.2%	[-1.3%, -1.1%]	2

• (bimodality in other direction, cranelift-codegen and keccak)

const_generics: correctly deal with bound variables #98900 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.9%	[-1.3%, -0.8%]	8
Improvements ✅ (secondary)	-3.3%	[-3.8%, -2.9%]	6
All ❌✅ (primary)	-0.9%	[-1.3%, -0.8%]	8

• (bimodality in other direction, html5ever, cranelift-codegen, and keccak)

Rollup of 7 pull requests #101577 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.1%]	1
Improvements ✅ (secondary)	-7.5%	[-7.5%, -7.5%]	1
All ❌✅ (primary)	-1.1%	[-1.1%, -1.1%]	1

Rollup of 6 pull requests #101603 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.4%, -1.2%]	2
Improvements ✅ (secondary)	-3.3%	[-3.8%, -2.8%]	7
All ❌✅ (primary)	-1.3%	[-1.4%, -1.2%]	2

• (bimodality in other direction, cranelift-codegen, keccak, and coercions)

Rollup of 6 pull requests #101652 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-1.3%, -0.2%]	7
Improvements ✅ (secondary)	-2.6%	[-3.8%, -0.4%]	8
All ❌✅ (primary)	-0.4%	[-1.3%, -0.2%]	7

• largely cranelift-codegen, keccak

Remove ReEmpty #98559 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-1.3%, -0.3%]	5
Improvements ✅ (secondary)	-3.3%	[-3.8%, -3.0%]	6
All ❌✅ (primary)	-0.7%	[-1.3%, -0.3%]	5

• largely cranelift-codegen, keccak

Make compare_predicate_entailment no longer a query #101615 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.4%]	6
Improvements ✅ (secondary)	-0.4%	[-0.5%, -0.3%]	5
All ❌✅ (primary)	-0.4%	[-0.5%, -0.4%]	6

Mixed

Make const_eval_select a real intrinsic #100759 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.3%, 1.4%]	12
Regressions ❌ (secondary)	2.4%	[0.2%, 4.1%]	9
Improvements ✅ (primary)	-1.2%	[-3.0%, -0.6%]	7
Improvements ✅ (secondary)	-1.4%	[-1.6%, -1.0%]	10
All ❌✅ (primary)	0.1%	[-3.0%, 1.4%]	19

• If you take html5ever, cranelift-codegen, keccak out of the mix here, ...
• ... you are left with the following four primary regressions: ripgrep by 1.37%, clap by 1.34%, cargo by 0.80%, serde_derive by 0.33%.
• The regression to ripgrep does look like it might be legitimate, at least from skimming its graph and zooming in.

proc_macro/bridge: use the cross-thread executor for nested proc-macros #101414 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.9%	[0.4%, 1.4%]	10
Regressions ❌ (secondary)	1.1%	[0.6%, 1.7%]	9
Improvements ✅ (primary)	-1.2%	[-1.3%, -1.2%]	2
Improvements ✅ (secondary)	-3.4%	[-3.9%, -3.0%]	6
All ❌✅ (primary)	0.5%	[-1.3%, 1.4%]	12

• As noted elsewhere https://github.com/rust-lang/rust/pull/101461#issuecomment-1238192638 ...
• ... the regressions flagged here are almost certainly noise from recently arising bimodality in certain benchmarks.
• marked as triaged

Rollup of 6 pull requests #101439 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.9%	[0.9%, 1.0%]	2
Regressions ❌ (secondary)	2.1%	[1.8%, 2.4%]	6
Improvements ✅ (primary)	-2.9%	[-29.7%, -0.3%]	18
Improvements ✅ (secondary)	-1.2%	[-1.8%, -0.7%]	9
All ❌✅ (primary)	-2.5%	[-29.7%, 1.0%]	20

• the regressions flagged here are almost certainly noise from recently arising bimodality in certain benchmarks.
• already marked as triaged

Rollup of 7 pull requests #101464 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.6%	[1.6%, 1.6%]	1
Improvements ✅ (primary)	-0.8%	[-1.3%, -0.3%]	13
Improvements ✅ (secondary)	-1.3%	[-1.7%, -0.8%]	8
All ❌✅ (primary)	-0.8%	[-1.3%, -0.3%]	13

• already marked as triaged due to ongoing bimodality noise issue.

Rollup of 5 pull requests #101479 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.9%	[0.4%, 1.3%]	10
Regressions ❌ (secondary)	1.3%	[0.8%, 1.8%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.4%	[-1.6%, -1.2%]	6
All ❌✅ (primary)	0.9%	[0.4%, 1.3%]	10

• already marked as triaged due to ongoing bimodality noise issue.

change stdlib circular dependencies handling #100404 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.2%	[1.1%, 1.3%]	2
Regressions ❌ (secondary)	3.4%	[3.0%, 3.9%]	6
Improvements ✅ (primary)	-0.8%	[-0.9%, -0.7%]	6
Improvements ✅ (secondary)	-1.0%	[-1.0%, -1.0%]	1
All ❌✅ (primary)	-0.3%	[-0.9%, 1.3%]	8

• marked as triaged due to ongoing bimodality noise issue.

Add test for #98294 #101232 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.7%, 0.9%]	6
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.1%	[-1.1%, -1.1%]	1
All ❌✅ (primary)	0.8%	[0.7%, 0.9%]	6

• already marked as triaged due to ongoing bimodality noise issue.

Shrink PredicateS #101432 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.7%, 0.9%]	6
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.4%]	5
Improvements ✅ (secondary)	-1.4%	[-1.4%, -1.4%]	1
All ❌✅ (primary)	0.2%	[-0.5%, 0.9%]	11

• already marked as triaged due to ongoing bimodality noise issue.

Use niche-filling optimization even when multiple variants have data. #94075 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	3.2%	[0.8%, 9.4%]	5
Regressions ❌ (secondary)	1.8%	[0.2%, 5.2%]	27
Improvements ✅ (primary)	-0.6%	[-1.1%, -0.2%]	23
Improvements ✅ (secondary)	-1.5%	[-2.9%, -0.2%]	16
All ❌✅ (primary)	0.1%	[-1.1%, 9.4%]	28

The syn regression sounds bad. Left a comment.

Rollup of 7 pull requests #101592 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.6%	[4.6%, 4.6%]	1
Improvements ✅ (primary)	-1.2%	[-1.3%, -1.1%]	2
Improvements ✅ (secondary)	-3.3%	[-3.8%, -2.9%]	6
All ❌✅ (primary)	-1.2%	[-1.3%, -1.1%]	2

• Already triaged saying “The coercions regression is only for a single profile/scenario combination, so I don‘t think it’s meaningful.”

Initial implementation of return-position impl Trait in traits #101224 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.4%]	19
Regressions ❌ (secondary)	2.0%	[0.2%, 4.1%]	13
Improvements ✅ (primary)	-0.9%	[-1.1%, -0.8%]	6
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	1
All ❌✅ (primary)	0.2%	[-1.1%, 1.4%]	25

• Many of regressions fall in the bimodal noise category.
• The serde regression did not, but it is addressed by the follow-up PR #101615
• Marking as triaged.

Update LLVM used in x86 CI dist builds to 15.0.0 #101527 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.3%	[1.3%, 1.3%]	1
Improvements ✅ (primary)	-0.6%	[-0.8%, -0.5%]	12
Improvements ✅ (secondary)	-0.5%	[-0.6%, -0.3%]	4
All ❌✅ (primary)	-0.6%	[-0.8%, -0.5%]	12

• Already triaged

The <*const T>::guaranteed_* methods now return an option for the unknown case #101483 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 0.6%]	5
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.1%	[-1.4%, -0.9%]	5
Improvements ✅ (secondary)	-2.9%	[-3.8%, -0.7%]	8
All ❌✅ (primary)	-0.4%	[-1.4%, 0.6%]	10

• The big improvements are just noise from some bimodal behavior we've been observing this week.
• Its possible that there was some regression to cargo here, but I don't think the data shows it to be large enough to be worth investigating. (I admit, my attitude there might be colored by my frustration with the huge noise impact of the aforementioned bimodality on all of the perf results from the last week.)
• marked as triaged

Untriaged Pull Requests

• #101483 The <*const T>::guaranteed_* methods now return an option for the unknown case
• #101414 proc_macro/bridge: use the cross-thread executor for nested proc-macros
• #101224 Initial implementation of return-position impl Trait in traits
• #101086 Compute information about function parameters on THIR
• #100963 Rollup of 15 pull requests
• #100869 Replace rustc_data_structures::thin_vec::ThinVec with thin_vec::ThinVec
• #100841 update Miri
• #100759 Make const_eval_select a real intrinsic
• #100748 add depth_limit in QueryVTable to avoid entering a new tcx in layout_of
• #100676 implied bounds: explicitly state which types are assumed to be wf
• #100652 Remove deferred sized checks (make them eager)
• #100497 Avoid cloning a collection only to iterate over it
• #100429 rustdoc: Merge source code pages HTML elements together
• #100404 change stdlib circular dependencies handling
• #100293 Add inline-llvm option for disabling/enabling LLVM inlining
• #99792 Rollup of 8 pull requests
• #99520 Rollup of 9 pull requests
• #99251 Upgrade indexmap and thorin-dwp to use hashbrown 0.12
• #99231 Rollup of 5 pull requests
• #99210 Rollup of 5 pull requests
• #99126 remove allow(rustc::potential_query_instability) in rustc_span
• #99123 proc_macro: use crossbeam channels for the proc_macro cross-thread bridge
• #99102 Rework definition of MIR phases to more closely reflect semantic concerns
• #99047 Rollup of 6 pull requests
• #99014 Rollup of 8 pull requests
• #98987 Rollup of 7 pull requests
• #98957 don't allow ZST in ScalarInt
• #98904 Rollup of 8 pull requests
• #98874 Rollup of 5 pull requests
• #98612 Rollup of 9 pull requests