2023-12-19 Triage Log

A lot of noise in the results this week; there was an lull in the noise recently, so our auto-inferred noise threshold went down, and thus five PR's were artificially flagged this week (and three supposed improvements were just reverting to the mean). Beyond that, we had three nice improvements: the first to debug builds in #117962 (by ceasing emission of expensive+unused .debug_pubnames and .debug_pubtypes), a second to diesel and serde in #119048 (by avoiding some unnecessary work), and a third to several benchmarks in #117749 (by adding some caching of an internal compiler structure).

Triage done by @pnkfelix. Revision range: 57010939..bf9229a2

Summary:

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.3%[0.2%, 0.4%]9
Regressions ❌
(secondary)		0.4%[0.2%, 0.9%]21
Improvements ✅
(primary)		-1.8%[-10.8%, -0.1%]103
Improvements ✅
(secondary)		-1.2%[-2.6%, -0.2%]93
All ❌✅ (primary)-1.6%[-10.8%, 0.4%]112

6 Regressions, 9 Improvements, 3 Mixed; 5 of them in rollups 67 artifact comparisons made in total

Regressions

rustc_passes: Enforce rustc::potential_query_instability lint #118870 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		13.5%[0.5%, 44.5%]8
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)13.5%[0.5%, 44.5%]8
  • already marked as triaged by lqd; this PR did not regress things, the reported change is due to a periodic “slow s3 uploads on the collector”, i.e. instrument noise/bias.

fix --dry-run when the change-id warning is printed #118789 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.9%[0.8%, 1.0%]4
Regressions ❌
(secondary)		0.5%[0.3%, 0.8%]18
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.9%[0.8%, 1.0%]4
  • already marked as triaged by lqd; this PR did not regress things. lqd hypothesized that the inferred noise threshold was lowered (due to a period of low noisiness?), and thus normal amounts of noise are tripping the threshold.

Rollup of 3 pull requests #118982 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.8%[0.8%, 0.9%]4
Regressions ❌
(secondary)		0.4%[0.0%, 0.8%]32
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.8%[0.8%, 0.9%]4
  • already marked as triaged by lqd. We think this is another case where the inferred noise threshold was low and we are seeing regression reports as the noise ramps up again.

Rollup of 5 pull requests #119002 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.2%[0.1%, 0.5%]40
Regressions ❌
(secondary)		0.4%[0.1%, 0.7%]36
Improvements ✅
(primary)		-0.3%[-0.3%, -0.3%]1
Improvements ✅
(secondary)		-0.1%[-0.1%, -0.1%]2
All ❌✅ (primary)0.2%[-0.3%, 0.5%]41
  • lqd identified PR #118396, “Collect lang items from AST, get rid of GenericBound::LangItemTrait” as the root cause for the regressions here.
  • not marking as triaged

More unescaping cleanups #118897 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.6%[0.1%, 0.8%]5
Regressions ❌
(secondary)		0.4%[0.2%, 0.7%]26
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.6%[0.1%, 0.8%]5
  • lqd theorized this is another instance of noise threshold wriggling
  • pnkfelix agrees and marked as triaged.

coverage: Regression test for assert!(!false) #119011 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.7%[0.7%, 0.8%]4
Regressions ❌
(secondary)		0.5%[0.3%, 0.6%]2
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.7%[0.7%, 0.8%]4
  • yet again, helloworld check is being flagged as regressing (in four scenarios), probably due to wriggling noise threshold.
  • pnkfelix marking as triaged

Improvements

Rollup of 8 pull requests #118838 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-0.9%[-1.0%, -0.9%]6
All ❌✅ (primary)--0

fix: stop emitting .debug_pubnames and .debug_pubtypes #117962 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-2.1%[-5.2%, -0.3%]24
Improvements ✅
(secondary)		-1.6%[-3.1%, -0.2%]61
All ❌✅ (primary)-2.1%[-5.2%, -0.3%]24
  • nice improvement to debug builds, especially for ripgrep and serde_derive

Unbreak non-unix non-windows bootstrap #118894 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-10.4%[-30.8%, -0.6%]8
Improvements ✅
(secondary)		--0
All ❌✅ (primary)-10.4%[-30.8%, -0.6%]8
  • (don't get excited; this is just a return-to-normal after the artificial regression flagged with PR #118870 above)

cache param env canonicalization #117749 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-2.0%[-4.8%, -0.2%]32
Improvements ✅
(secondary)		-1.3%[-1.4%, -1.3%]2
All ❌✅ (primary)-2.0%[-4.8%, -0.2%]32
  • pretty solid improvement here to serde, diesel, bitmaps, hyper, image.

-Ztrait-solver=next to -Znext-solver #118937 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.9%[-1.0%, -0.9%]4
Improvements ✅
(secondary)		-0.5%[-0.8%, -0.3%]21
All ❌✅ (primary)-0.9%[-1.0%, -0.9%]4
  • (just noise)

Use the LLVM option NoTrapAfterNoreturn #110494 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.9%[-0.9%, -0.8%]4
Improvements ✅
(secondary)		-0.5%[-0.8%, -0.3%]25
All ❌✅ (primary)-0.9%[-0.9%, -0.8%]4
  • (just noise)

Add support for --env on tracked_env::var #118830 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.7%[-0.8%, -0.7%]3
Improvements ✅
(secondary)		-0.5%[-0.7%, -0.3%]7
All ❌✅ (primary)-0.7%[-0.8%, -0.7%]3
  • (just noise)

don't fold ParamEnv in register_predicate_obligation #119048 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-2.1%[-7.3%, -0.2%]23
Improvements ✅
(secondary)		-0.3%[-0.3%, -0.3%]1
All ❌✅ (primary)-2.1%[-7.3%, -0.2%]23
  • nice improvement to diesel and serde.

resolve: Replace visibility table in resolver outputs with query feeding #118657 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.3%[-0.5%, -0.1%]70
Improvements ✅
(secondary)		-0.6%[-1.2%, -0.1%]42
All ❌✅ (primary)-0.3%[-0.5%, -0.1%]70

Mixed

codegen_llvm: set DW_AT_accessibility #115165 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.2%[0.2%, 0.3%]4
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.7%[-0.7%, -0.6%]4
Improvements ✅
(secondary)		-0.4%[-0.7%, -0.2%]22
All ❌✅ (primary)-0.2%[-0.7%, 0.3%]8
  • the improvements are helloworld noise; the regressions are minor.

Rollup of 6 pull requests #119009 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.2%[0.2%, 0.2%]1
Regressions ❌
(secondary)		0.4%[0.2%, 1.2%]8
Improvements ✅
(primary)		-1.3%[-1.9%, -0.9%]8
Improvements ✅
(secondary)		--0
All ❌✅ (primary)-1.1%[-1.9%, 0.2%]9
  • The single tiny regression to exa doc full does not warrant spending time trying to tease apart this rollup, IMO.
  • marked as triaged

Rollup of 5 pull requests #119069 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		0.3%[0.3%, 0.3%]2
Improvements ✅
(primary)		-0.2%[-0.3%, -0.1%]12
Improvements ✅
(secondary)		-0.3%[-0.5%, -0.2%]13
All ❌✅ (primary)-0.2%[-0.3%, -0.1%]12
  • The two minor regressions to secondary benchmarks do not warrant spending time trying to tease apart this rollup, IMO.
  • marked as triaged