2023-05-03 Triage Log

This week the good outweighed the bad. In particular, we had three different PRs that made improvements to a wide range of benchmarks. Special call out to PR #111026, which yielded 3% to 8% improvement for incremental compile times on a large set of benchmarks, by avoiding unnecessary caching in the type checker.

Triage done by @pnkfelix. Revision range: fdeef3ed..a368898d

Summary:

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.5%[0.2%, 0.9%]30
Regressions ❌
(secondary)		0.7%[0.4%, 1.1%]12
Improvements ✅
(primary)		-1.6%[-8.8%, -0.3%]115
Improvements ✅
(secondary)		-1.5%[-10.3%, -0.2%]92
All ❌✅ (primary)-1.1%[-8.8%, 0.9%]145

4 Regressions, 6 Improvements, 7 Mixed; 6 of them in rollups 56 artifact comparisons made in total

Regressions

Add loongarch64 asm! support #101069 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		2.2%[1.1%, 2.9%]3
Regressions ❌
(secondary)		4.9%[0.3%, 8.2%]8
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)2.2%[1.1%, 2.9%]3
  • primary regressions are all in cranelift-codegen; secondary are in keccak and externs.
  • already triaged.

Do not bother optimizing impossible functions. #110728 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.4%[0.4%, 0.5%]3
Regressions ❌
(secondary)		0.3%[0.2%, 0.4%]8
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.4%[0.4%, 0.5%]3
  • Seems like it didn't have the expected positive impact
  • primary regressions are to libc; most of secondary are to variants many-assoc-items (plus some check incr-unchanged on three other secondaries).
  • in any case, the measured impact looks negligible to me.
  • marked as triaged

Rollup of 7 pull requests #110896 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.4%[0.4%, 0.4%]2
Regressions ❌
(secondary)		1.1%[0.2%, 2.4%]7
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-0.2%[-0.2%, -0.2%]1
All ❌✅ (primary)0.4%[0.4%, 0.4%]2
  • primary regressions are to doc for syn and webrender. secondary that regressed by 2.4% was incr-unchanged for wg-grammar benchmarks.
  • marking as triaged.

Rollup of 6 pull requests #110924 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.6%[0.4%, 0.7%]7
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)0.6%[0.4%, 0.7%]7
  • primary regressions are to debug profiles for serde_derive, hyper (plus noise from clap and cargo).
  • we've recovered the loss on serde_derive since this commit landed, but not on hyper.
  • from reviewing the PRs in the rollup, it does not seem like there's any real culprit here.
  • marking as triaged.

Improvements

Allow older LLVM versions to have missing components #110232 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-1.7%[-2.8%, -0.7%]4
Improvements ✅
(secondary)		-6.0%[-7.5%, -4.9%]6
All ❌✅ (primary)-1.7%[-2.8%, -0.7%]4
  • I don't understand why this had any impact at all, albeit positive. It looks like just some environment variable settings in the Dockerfiles.

Rewrite MemDecoder around pointers not a slice #110634 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.4%[0.4%, 0.4%]1
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.5%[-0.7%, -0.3%]52
Improvements ✅
(secondary)		-0.5%[-0.8%, -0.3%]16
All ❌✅ (primary)-0.4%[-0.7%, 0.4%]53
  • a broad collection of small improvements, nice!

Rollup of 10 pull requests #110852 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.4%[-0.5%, -0.4%]4
Improvements ✅
(secondary)		-0.4%[-0.4%, -0.4%]1
All ❌✅ (primary)-0.4%[-0.5%, -0.4%]4

Remove QueryEngine trait #109611 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		0.3%[0.3%, 0.3%]3
Improvements ✅
(primary)		-0.6%[-0.8%, -0.2%]84
Improvements ✅
(secondary)		-0.8%[-1.5%, -0.4%]32
All ❌✅ (primary)-0.6%[-0.8%, -0.2%]84
  • another broad collection of small improvements, good to see!

Only cache typeck results if it's the typeck root #111026 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-1.8%[-7.9%, -0.4%]62
Improvements ✅
(secondary)		-4.6%[-11.5%, -1.0%]12
All ❌✅ (primary)-1.8%[-7.9%, -0.4%]62
  • nice work! (sizeable, 3% to 8% improvement on a number of benchmarks for incr runs)

Make some simple queries no longer cache on disk #111028 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.3%[-0.4%, -0.3%]17
Improvements ✅
(secondary)		-0.5%[-0.6%, -0.4%]9
All ❌✅ (primary)-0.3%[-0.4%, -0.3%]17

Mixed

Use MIR's Offset for pointer add too #110837 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		1.2%[0.9%, 1.4%]2
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.8%[-1.4%, -0.5%]14
Improvements ✅
(secondary)		-1.3%[-2.0%, -0.5%]11
All ❌✅ (primary)-0.6%[-1.4%, 1.4%]16
  • cute idea for a PR (“reduce your useless MIR with this small trick!”)
  • primary regressions are to serde_derive and webrender (opt-full for both).
  • already triaged; improvements clearly outweigh the regressions here, at least based on the counts of number of impacted benchmarks.

Rollup of 8 pull requests #110967 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		1.1%[0.4%, 2.0%]17
Regressions ❌
(secondary)		1.1%[1.1%, 1.3%]6
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-0.3%[-0.3%, -0.3%]2
All ❌✅ (primary)1.1%[0.4%, 2.0%]17
  • primary regression are to diesel, clap, and cranelift-codegen.
  • lqd and compiler-errors are discussing on the PR.
  • (pnkfelix is not convinced its a real regression that we can address, based on their conversation.)

Rollup of 6 pull requests #110978 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		0.3%[0.3%, 0.3%]2
Improvements ✅
(primary)		-0.3%[-0.3%, -0.3%]2
Improvements ✅
(secondary)		-0.4%[-0.4%, -0.4%]1
All ❌✅ (primary)-0.3%[-0.3%, -0.3%]2
  • regressions are solely to secondary externs benchmark.
  • its so minor. marking as triaged.

Improve niche placement by trying two strategies and picking the better result #108106 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.4%[0.2%, 0.6%]7
Regressions ❌
(secondary)		0.7%[0.4%, 1.0%]12
Improvements ✅
(primary)		-2.6%[-8.5%, -0.2%]4
Improvements ✅
(secondary)		-1.2%[-2.8%, -0.3%]31
All ❌✅ (primary)-0.7%[-8.5%, 0.6%]11
  • addresses a real performance regression (#105371) to how we laid out niches.
  • primary big benefit to syn. rest of primary changes may be noise; the secondary improvements are nice to have.
  • marking as triaged.

Rollup of 6 pull requests #110994 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.5%[0.4%, 0.7%]3
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-0.3%[-0.3%, -0.3%]2
All ❌✅ (primary)0.5%[0.4%, 0.7%]3
  • primary regressions to doc for clap, stm32f4, cranelift-codegen
  • already marked as triaged.

Make mem::replace simpler in codegen #111010 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.4%[0.2%, 0.6%]3
Regressions ❌
(secondary)		0.3%[0.2%, 0.5%]2
Improvements ✅
(primary)		-1.3%[-1.8%, -1.0%]3
Improvements ✅
(secondary)		-0.5%[-0.7%, -0.4%]3
All ❌✅ (primary)-0.5%[-1.8%, 0.6%]6
  • primary regressions to ripgrep opt-full, plus doc for hyper and html5ever.
  • already marked as triaged.

Remove type ascription from parser and diagnostics #109128 (Comparison Link)

(instructions:u)meanrangecount
Regressions ❌
(primary)		0.3%[0.2%, 0.4%]9
Regressions ❌
(secondary)		0.3%[0.2%, 0.5%]9
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-1.4%[-1.4%, -1.4%]1
All ❌✅ (primary)0.3%[0.2%, 0.4%]9
  • primary regressions are to serde, hyper, diesel; mostly to incr-unchanged scenarios.
  • surprising that this had this impact.
  • i'm guessing this is noise (this change should be solely simplifying the compiler, AFAICT).
  • marking as triaged.

Untriaged Pull Requests