2022-04-12 Triage Log

A week with a large amount of changes in rollups, which makes performance triage difficult. The performance team and the infra team are working on finding ways to automate marking PRs as likely a poor choice for rolling up. Otherwise, the week overall saw a ~1% improvement in incremental check builds, with smaller improvements to incremental debug and release builds. A number of benchmarks have been updated in the last few weeks, which has meant a decrease in the automated noise assessment's algorithm performance, but that should settle out to steady state behavior on its own in the next few days.

Triage done by @simulacrum. Revision range: 949b98cab8a186b98bf87e64374b8d0848c55271..4e1927db3c399fa34dc71992bd5dbec09f945c3d

5 Regressions, 4 Improvements, 7 Mixed; 7 of them in rollups 50 artifact comparisons made in total

Regressions

Update libc to 0.2.121 #95688 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 8 | 19 | 0 | 0 | 8 | | mean | 1.2% | 0.9% | N/A | N/A | 1.2% | | max | 1.2% | 1.2% | N/A | N/A | 1.2% |

This is pretty likely to just be natural increases in compile times when key dependencies like libc increase in size.

Rollup of 6 pull requests #95840 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 10 | 21 | 0 | 3 | 10 | | mean | 0.6% | 0.9% | N/A | -0.4% | 0.6% | | max | 1.2% | 1.3% | N/A | -0.5% | 1.2% |

Primarily limited to doc builds, seems likely to be caused by #95769.

Rollup of 7 pull requests #95889 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 5 | 0 | 0 | 0 | 5 | | mean | 0.3% | N/A | N/A | N/A | 0.3% | | max | 0.5% | N/A | N/A | N/A | 0.5% |

Unclear cause, and fairly small regression. Given this is a rollup likely not worth the time to investigate -- benefit/cost tradeoff is not worth it.

Faster parsing for lower numbers for radix up to 16 (cont.) #95399 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 14 | 0 | 0 | 0 | | mean | N/A | 0.3% | N/A | N/A | N/A | | max | N/A | 0.5% | N/A | N/A | N/A |

Unclear that this will be an easy regression to investigate, changes are pretty extensive, and justification slim beyond theoretical microbenchmarks.

Rollup of 7 pull requests #95855 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 9 | 5 | 1 | 5 | | mean | N/A | 1.0% | -0.8% | -1.0% | -0.8% | | max | N/A | 1.7% | -1.1% | -1.0% | -1.1% |

Improvements

Call compute_locs once per rule #95669 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 0 | 23 | 10 | 23 | | mean | N/A | N/A | -1.4% | -1.3% | -1.4% | | max | N/A | N/A | -3.5% | -3.4% | -3.5% |

Shrink Nonterminal #95715 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 1 | 0 | 4 | 0 | | mean | N/A | 0.3% | N/A | -0.3% | N/A | | max | N/A | 0.3% | N/A | -0.5% | N/A |

This was originally categorized as a mixed change, and even though there is a regression, it seems overall to be an improvement if anything.

rustdoc: Early doc link resolution fixes and refactorings #95706 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 2 | 6 | 3 | 10 | 5 | | mean | 0.4% | 1.2% | -13.9% | -16.4% | -8.2% | | max | 0.4% | 1.3% | -24.5% | -24.3% | -24.5% |

This change has a few regressions, but those are likely to be unrelated noise. The improvements are large and significant.

CI: update rustc-perf version used in CI and also the corresponding PGO benchmarks #95724 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 13 | 102 | 101 | 102 | | mean | N/A | 1.0% | -0.7% | -1.0% | -0.7% | | max | N/A | 1.3% | -2.2% | -2.7% | -2.2% |

Overall a slight improvement, likely from better data due to updated benchmarks.

Mixed

Rollup of 5 pull requests #95667 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 17 | 39 | 14 | 0 | 31 | | mean | 6.2% | 7.9% | -0.8% | N/A | 3.1% | | max | 39.4% | 38.9% | -1.6% | N/A | 39.4% |

Investigation shows this regression to be primarily caused by new support for diagnostic translations; investigation is ongoing, but it sounds like there are ways to claw back some of the performance hit.

Rollup of 8 pull requests #95702 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 2 | 6 | 2 | 6 | 4 | | mean | 0.3% | 1.1% | -0.2% | -0.4% | 0.1% | | max | 0.4% | 1.2% | -0.2% | -0.7% | 0.4% |

Unclear cause. Regression might be noise, but it‘s hard to say -- the keccak benchmark is in line with the ~1% noise we’ve seen elsewhere this week, but the other benchmarks (particularly in the primary category) seem potentially less likely to be noise. deep-vector is also affected.

Rollup of 6 pull requests #95742 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 6 | 31 | 12 | 10 | 18 | | mean | 0.4% | 0.5% | -0.6% | -0.4% | -0.3% | | max | 0.7% | 0.8% | -2.4% | -0.6% | -2.4% |

Fairly large number of benchmarks showing small-ish changes, both positive and negative. Cause is unclear.

Avoid accessing HIR from MIR passes #95487 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 9 | 1 | 0 | 1 | | mean | N/A | 0.5% | -0.6% | N/A | -0.6% | | max | N/A | 0.9% | -0.6% | N/A | -0.6% |

Add new Deinit statement #95125 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 27 | 39 | 4 | 17 | 31 | | mean | 0.5% | 1.4% | -0.9% | -1.0% | 0.3% | | max | 1.6% | 4.1% | -1.4% | -2.2% | 1.6% |

Rollup of 7 pull requests #95944 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 6 | 5 | 9 | 5 | | mean | N/A | 0.4% | -0.2% | -0.5% | -0.2% | | max | N/A | 0.5% | -0.4% | -1.8% | -0.4% |

Respect -Z verify-llvm-ir and other flags that add extra passes when combined with -C no-prepopulate-passes in the new LLVM Pass Manager. #95893 (Comparison Link) | | Regressions 😿
(primary) | Regressions 😿
(secondary) | Improvements 🎉
(primary) | Improvements 🎉
(secondary) | All 😿 🎉
(primary) | |:---:|:---:|:---:|:---:|:---:|:---:| | count | 0 | 3 | 0 | 6 | 0 | | mean | N/A | 0.5% | N/A | -0.4% | N/A | | max | N/A | 0.6% | N/A | -0.5% | N/A |