2020-07-21 Triage Log

See the README for details on how this log was gathered.

This week was a disaster, perf-wise. 28 revisions checked. 7 regressions, several of them ranging from large to huge, many in rollups. Some additional regressions may have occurred in rollups that were masked by other regressions/improvements. 3 improvements, one of which was a reversion of a regression. Thanks for Mark-Simulacrum and eddyb for follow-up measurements and adding new tooling to help investigate regressions in rollups. A follow-up thread on Zulip is here.

In better news, rustdoc performance is now being benchmarked, thanks to the efforts of Joshua Nelson.

Triage done by njn. Revision range: 9d09331e00b02f81c714b0c41ce3a38380dd36a2..71384101ea3b030b80f7def80a37f67e148518b0.

Regressions

• Rollup of 18 pull requests #74461 (instructions): Up to 73.0% losses on many benchmarks. #74416 is at fault; it was reverted in #74478, see below.
• std: Switch from libbacktrace to gimli #73441 (instructions, max-rss): Up to 44.8% instruction increases on many benchmarks, and 5-10% max-rss increases on many benchmarks. Being reverted in #74613.
• Support const args in type dependent paths (Take 2) #74113 (instructions): Up to 1.5% losses on numerous benchmarks, which was expected.
• Reduce the amount of interning and layout_of calls in const eval. #74202 (instructions): Up to 2% losses on one benchmark.
• Rollup of 9 pull requests #74543 (instructions): Up to 12% losses on rustdoc for many benchmarks.
• Rollup of 4 pull requests #74518 (instructions): Up to 6.4% losses on rustdoc for two benchmarks.

Improvements

• Rollup of 7 pull requests #74493 (instructions): Up to 33.5% improvements, at least partly due to the backout of #74416 in #74478, but it's also possible that another PR in that rollup caused a regression that was masked by the improvement from the backout.
• Change SymbolName::name to a &str. #74214 (instructions): Up to 2.5% wins on numerous benchmarks.
• Don't run everybody_loops for rustdoc; instead ignore resolution errors #73566 (instructions): Wins of up to 62.6% and losses of up to 8.5%, all on rustdoc builds. Overall, the improvements greatly outweigh the losses. (Landed in rollup #74408.)

Notes

Rollup of 11 pull requests #74468

• (instructions)

This rollup was originally judged as responsible for a 10% regression in instrutions:u. However, since then, it has been determined that the likely cause of this regression is actually perf‘s move to using x.py dist-shipped std’s rather than building one locally. Investigation into why this move caused a regression is as yet not done, but is being tracked in rustc-perf#724.

Initially #74069 and/or #72414 were suspected as the cause of the regression, but further testing showed that to not be the case.

• The original rollup, #74468, landed with a 5-10% performance regression
• In #74611, we tested as-if a rollup of #74069 and #72414 landed. (This is the diff between d3df85 and cfade73) That yielded identical results to the rollup.
• We reverted #74069 in #74611, this yielded neutral performance results.
• #74716 then tried a revert of #72414 but this also yielded neutral performance results

We have since opened a PR to re-land #74069, as well: #74802.