2024-05-21 Triage Log

Fairly quiet week with the exception of a very large improvement coming from the switch to rust-lld on nightly Linux. This can have very large impacts on benchmarks where linking dominates the build time (e.g., ripgrep, exa, small binaries like hello-world). Aside from that change, there were a few small regressions that were either deemed worth it or are still being investigated.

Triage done by @rylev. Revision range: 9105c57b..1d0e4afd

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.7%	[0.1%, 2.5%]	30
Regressions ❌ (secondary)	0.5%	[0.2%, 0.8%]	5
Improvements ✅ (primary)	-30.4%	[-71.7%, -0.4%]	35
Improvements ✅ (secondary)	-25.6%	[-70.9%, -0.5%]	75
All ❌✅ (primary)	-16.1%	[-71.7%, 2.5%]	65

4 Regressions, 1 Improvement, 4 Mixed; 2 of them in rollups 66 artifact comparisons made in total

Regressions

Add x86_64-unknown-linux-none target #125023 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.6%	[0.2%, 1.0%]	3
Regressions ❌ (secondary)	0.4%	[0.4%, 0.4%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.6%	[0.2%, 1.0%]	3

• Noise

Rollup of 6 pull requests #125144 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.2%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.2%, 1.2%]	3

Update expr matcher for Edition 2024 and add expr_2021 nonterminal #123865 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.3%]	5
Regressions ❌ (secondary)	0.5%	[0.2%, 1.7%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.2%	[0.2%, 0.3%]	5

• It seems the primary benchmark affected by this html5ever seems to spend lots of time in expand_crate which could be impacted by this change.
• Asked authors and reviewer whether this is worth an investigation or not.

Update unexpected_cfgs lint for Cargo new check-cfg config #125219 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.2%, 2.5%]	21
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.8%	[0.2%, 2.5%]	21

• Somewhat further fall out from https://github.com/rust-lang/rust/pull/124684. This adds additional output which will slow down benchmarks which emit this warning particularly often.
• We're likely ok with this given this is the “unhappy path”.

Improvements

Enable rust-lld on nightly x86_64-unknown-linux-gnu #124129 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-30.8%	[-71.8%, -0.5%]	31
Improvements ✅ (secondary)	-25.6%	[-70.9%, -0.5%]	75
All ❌✅ (primary)	-30.8%	[-71.8%, -0.5%]	31

Mixed

Rollup of 7 pull requests #125120 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	2
Regressions ❌ (secondary)	0.5%	[0.3%, 1.0%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-0.5%, -0.4%]	2
All ❌✅ (primary)	0.2%	[0.2%, 0.2%]	2

• Caused by https://github.com/rust-lang/rust/pull/125088. Asked the author/reviewers whether this is worth following up on.

Remove bound checks from BorrowedBuf and BorrowedCursor methods #123786 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.8%, 0.8%]	1
Regressions ❌ (secondary)	1.3%	[1.3%, 1.3%]	1
Improvements ✅ (primary)	-1.7%	[-1.7%, -1.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.5%	[-1.7%, 0.8%]	2

• Noise

optimize inplace collection of Vec #123878 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.4%	[0.3%, 2.9%]	3
Regressions ❌ (secondary)	0.3%	[0.2%, 0.3%]	2
Improvements ✅ (primary)	-22.7%	[-37.4%, -0.4%]	5
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-13.7%	[-37.4%, 2.9%]	8

• Improvements vastly outweigh the regressions which seemed to have returned to baseline after this PR.

Follow-up fixes to report_return_mismatched_types #123812 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.1%	[0.6%, 1.8%]	3
Regressions ❌ (secondary)	0.2%	[0.1%, 0.2%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.1%	[-1.1%, -1.1%]	2
All ❌✅ (primary)	1.1%	[0.6%, 1.8%]	3

• Single benchmark with a large regression. Asked author to investigate.