src/tools/rustc-perf/triage/2023-03-28.md

# 2023-03-28 Triage Log

A busy week with lots of real performance gains. Most regressions seemed to be due to noise. The biggest highlight was large wins in incremental compilation leading to a lot of (albeit modest) gains of 1% performance in a majority of incremental compilation test scenarios. Other than that most performance gains were smaller and more incremental. One of the biggest performance regressions came in an update to LLVM. However, nearly just as many test cases showed improvements as regressions.

Triage done by **@rylev**.
Revision range: [ef03fda3..cbc064b3](https://perf.rust-lang.org/?start=ef03fda339923e659d3d3ca3321de887316d2807&end=cbc064b341be231403d181402a786cce7f1c73f1&absolute=false&stat=instructions%3Au)

**Summary**:

| (instructions:u)                   | mean  | range           | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 1.7%  | [0.5%, 3.5%]    | 24    |
| Regressions ❌ <br /> (secondary)  | 1.2%  | [0.2%, 2.6%]    | 18    |
| Improvements ✅ <br /> (primary)   | -1.5% | [-10.9%, -0.3%] | 168   |
| Improvements ✅ <br /> (secondary) | -4.0% | [-65.3%, -0.4%] | 119   |
| All ❌✅ (primary)                 | -1.1% | [-10.9%, 3.5%]  | 192   |


3 Regressions, 7 Improvements, 8 Mixed; 5 of them in rollups
46 artifact comparisons made in total

#### Regressions

Add RANLIB_x86_64_unknown_illumos env for dist-x86_64-illumos dockerfile [#109163](https://github.com/rust-lang/rust/pull/109163) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=6502613a8199cab4cb68b08c7ddce6f210d8d69c&end=439292bc7913399e406d9bb7e8da0f70c6317c6e&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.8% | [0.7%, 0.8%] | 4     |
| Regressions ❌ <br /> (secondary)  | 0.5% | [0.4%, 0.7%] | 9     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 0.8% | [0.7%, 0.8%] | 4     |
- Noise


Make NLL Type Relating Eager [#108861](https://github.com/rust-lang/rust/pull/108861) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=8859fde21fa4e059c71a843cc7ee995c189a97c0&end=1c771fec3329ef1d8c66697daf9517501d0c129a&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.6%  | [0.3%, 0.8%]   | 7     |
| Regressions ❌ <br /> (secondary)  | 0.5%  | [0.3%, 0.6%]   | 10    |
| Improvements ✅ <br /> (primary)   | -     | -              | 0     |
| Improvements ✅ <br /> (secondary) | -0.8% | [-0.8%, -0.8%] | 1     |
| All ❌✅ (primary)                 | 0.6%  | [0.3%, 0.8%]   | 7     |
- A good chunk of the regressions are noise, and the rest are small enough that I don't think it's worth looking too deeply into.


Refactor `try_execute_query` [#109100](https://github.com/rust-lang/rust/pull/109100) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e10eab5956f91c26dcc5ae29a19cfcd747047e4d&end=b72e896268bd741a2a2bb10028dca9e6f8d0ae63&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.7% | [0.7%, 0.8%] | 4     |
| Regressions ❌ <br /> (secondary)  | 0.5% | [0.3%, 0.7%] | 8     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 0.7% | [0.7%, 0.8%] | 4     |
- Noise


#### Improvements

Rollup of 10 pull requests [#109442](https://github.com/rust-lang/rust/pull/109442) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ef03fda339923e659d3d3ca3321de887316d2807&end=a01b4cc9f375f1b95fa8195daeea938d3d9c4c34&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.4% | [-0.5%, -0.3%] | 4     |
| Improvements ✅ <br /> (secondary) | -     | -              | 0     |
| All ❌✅ (primary)                 | -0.4% | [-0.5%, -0.3%] | 4     |


mv tests/codegen/issue-* tests/codegen/issues/ [#109172](https://github.com/rust-lang/rust/pull/109172) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1c771fec3329ef1d8c66697daf9517501d0c129a&end=cf811810fe8dc92b3ff25e7dd29fb3178c91d409&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.6% | [-0.7%, -0.6%] | 4     |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.6%, -0.4%] | 2     |
| All ❌✅ (primary)                 | -0.6% | [-0.7%, -0.6%] | 4     |


Rollup of 7 pull requests [#109517](https://github.com/rust-lang/rust/pull/109517) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9a6b0c3326f3577f330d8828d3b706c0eea9cf29&end=99c49d95cd7d704d2226859cfdc419c930cde9c6&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.4% | [-0.4%, -0.4%] | 1     |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.7%, -0.4%] | 6     |
| All ❌✅ (primary)                 | -0.4% | [-0.4%, -0.4%] | 1     |


Don't pass `TreatProjections` separately to `fast_reject` [#109202](https://github.com/rust-lang/rust/pull/109202) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1459b3128e288a85fcc4dd1fee7ada2cdcf28794&end=cf073ec2cbf0b90ded7893f429020b81dd28e6c3&stat=instructions:u)

| (instructions:u)                   | mean  | range           | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -               | 0     |
| Regressions ❌ <br /> (secondary)  | 0.4%  | [0.4%, 0.4%]    | 1     |
| Improvements ✅ <br /> (primary)   | -0.9% | [-2.3%, -0.2%]  | 17    |
| Improvements ✅ <br /> (secondary) | -2.6% | [-10.5%, -0.4%] | 24    |
| All ❌✅ (primary)                 | -0.9% | [-2.3%, -0.2%]  | 17    |


Add #[inline] to the Into for From impl [#109546](https://github.com/rust-lang/rust/pull/109546) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=d012d2f96e2959267b14812929b85e17a155f653&end=24a69af213f4bf23014dc91903e7a934c88f6b9e&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.8%  | [0.8%, 0.8%]   | 1     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.8% | [-1.5%, -0.4%] | 15    |
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.4%, -0.3%] | 19    |
| All ❌✅ (primary)                 | -0.7% | [-1.5%, 0.8%]  | 16    |


Optimize `incremental_verify_ich` [#109371](https://github.com/rust-lang/rust/pull/109371) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9fa6b3c15758e85657d5be051cfa57022a8bbe57&end=e10eab5956f91c26dcc5ae29a19cfcd747047e4d&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.9% | [-1.6%, -0.4%] | 78    |
| Improvements ✅ <br /> (secondary) | -1.1% | [-2.2%, -0.5%] | 39    |
| All ❌✅ (primary)                 | -0.9% | [-1.6%, -0.4%] | 78    |


Rollup of 7 pull requests [#109581](https://github.com/rust-lang/rust/pull/109581) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b72e896268bd741a2a2bb10028dca9e6f8d0ae63&end=96bd50dd47b645efa52d1312b9cc32e507d9cd76&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.7% | [-0.7%, -0.7%] | 2     |
| Improvements ✅ <br /> (secondary) | -0.6% | [-0.7%, -0.6%] | 5     |
| All ❌✅ (primary)                 | -0.7% | [-0.7%, -0.7%] | 2     |


#### Mixed

Only clear written-to locals in ConstProp [#109087](https://github.com/rust-lang/rust/pull/109087) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=6dc3999c2699461aa930b8c1e00f99e73dcc0174&end=5fa73a75ce94faf1d1008ec96c043134ef0df427&stat=instructions:u)

| (instructions:u)                   | mean   | range           | count |
|:----------------------------------:|:------:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -      | -               | 0     |
| Regressions ❌ <br /> (secondary)  | 0.5%   | [0.5%, 0.5%]    | 3     |
| Improvements ✅ <br /> (primary)   | -2.7%  | [-6.1%, -0.8%]  | 27    |
| Improvements ✅ <br /> (secondary) | -15.8% | [-63.2%, -0.2%] | 17    |
| All ❌✅ (primary)                 | -2.7%  | [-6.1%, -0.8%]  | 27    |
- Given the overall positive impact of this PR and the complex relationship it has with some other PRs, I think it's safe to say the perf results are fine here.


Rollup of 11 pull requests [#109496](https://github.com/rust-lang/rust/pull/109496) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=439292bc7913399e406d9bb7e8da0f70c6317c6e&end=a266f11990d9544ee408e213e1eec8cc9eb032b7&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | 0.5%  | [0.3%, 0.8%]   | 4     |
| Improvements ✅ <br /> (primary)   | -0.7% | [-0.8%, -0.7%] | 4     |
| Improvements ✅ <br /> (secondary) | -0.6% | [-0.7%, -0.4%] | 4     |
| All ❌✅ (primary)                 | -0.7% | [-0.8%, -0.7%] | 4     |
- Unfortunately most of the positives here seem to be correction from noise. The regressions are really small though so I don't think investigation is worth it.


Rollup of 9 pull requests [#109503](https://github.com/rust-lang/rust/pull/109503) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=cf811810fe8dc92b3ff25e7dd29fb3178c91d409&end=84dd6dfd9d19176cc3c94bc1448a841e44d57890&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.5%  | [0.5%, 0.5%]   | 1     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -1.5% | [-1.7%, -1.4%] | 2     |
| Improvements ✅ <br /> (secondary) | -2.4% | [-5.3%, -0.3%] | 13    |
| All ❌✅ (primary)                 | -0.9% | [-1.7%, 0.5%]  | 3     |
- The one regression is outweighed by many other improvements. Given this is a rollup which requires extra steps to investigate regressions, I think it's safe to mark this as triaged.


Add `CastKind::Transmute` to MIR [#108442](https://github.com/rust-lang/rust/pull/108442) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=df7fd9995f10627f25ccb325f693a11b3395a73c&end=e2163008763c326ec4003e07b8e6eef0c98f6204&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | 0.7%  | [0.3%, 1.5%]   | 4     |
| Improvements ✅ <br /> (primary)   | -1.6% | [-1.6%, -1.6%] | 1     |
| Improvements ✅ <br /> (secondary) | -     | -              | 0     |
| All ❌✅ (primary)                 | -1.6% | [-1.6%, -1.6%] | 1     |
- The regressions are small enough that I don't think this is worth investigating.


rustdoc: Optimize impl sorting during rendering [#109399](https://github.com/rust-lang/rust/pull/109399) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=8be3c2bda6b683f87b24714ba595e8b04faef54c&end=d012d2f96e2959267b14812929b85e17a155f653&stat=instructions:u)

| (instructions:u)                   | mean   | range            | count |
|:----------------------------------:|:------:|:----------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -      | -                | 0     |
| Regressions ❌ <br /> (secondary)  | 0.7%   | [0.7%, 0.7%]     | 1     |
| Improvements ✅ <br /> (primary)   | -10.1% | [-10.1%, -10.1%] | 1     |
| Improvements ✅ <br /> (secondary) | -      | -                | 0     |
| All ❌✅ (primary)                 | -10.1% | [-10.1%, -10.1%] | 1     |
- The one regression is noise.


Implement Default for some alloc/core iterators [#99929](https://github.com/rust-lang/rust/pull/99929) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=24a69af213f4bf23014dc91903e7a934c88f6b9e&end=9fa6b3c15758e85657d5be051cfa57022a8bbe57&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.7%  | [0.7%, 0.7%]   | 4     |
| Regressions ❌ <br /> (secondary)  | 0.5%  | [0.3%, 0.6%]   | 9     |
| Improvements ✅ <br /> (primary)   | -     | -              | 0     |
| Improvements ✅ <br /> (secondary) | -0.3% | [-0.4%, -0.3%] | 6     |
| All ❌✅ (primary)                 | 0.7%  | [0.7%, 0.7%]   | 4     |
- Given the nature of this PR (adding APIs) and the fact that the primary benchmark impacted is noisy, I think we can triage this.


Use `SmallVec` in bitsets [#109458](https://github.com/rust-lang/rust/pull/109458) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=96bd50dd47b645efa52d1312b9cc32e507d9cd76&end=2d429f3064cb67710fe64dee293329089871d92b&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | 0.8%  | [0.7%, 1.0%]   | 8     |
| Improvements ✅ <br /> (primary)   | -0.8% | [-1.4%, -0.3%] | 15    |
| Improvements ✅ <br /> (secondary) | -0.7% | [-1.4%, -0.3%] | 15    |
| All ❌✅ (primary)                 | -0.8% | [-1.4%, -0.3%] | 15    |
- Improvements outweigh the regressions and [for some of the regressions, it should be fine to ignore the result as noise](https://github.com/rust-lang/rust/pull/109458#issuecomment-1483148687). 


Upgrade to LLVM 16, again [#109474](https://github.com/rust-lang/rust/pull/109474) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2d429f3064cb67710fe64dee293329089871d92b&end=0c61c7a978fe9f7b77a1d667c77d2202dadd1c10&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 1.1%  | [0.3%, 3.6%]   | 64    |
| Regressions ❌ <br /> (secondary)  | 0.9%  | [0.2%, 2.5%]   | 23    |
| Improvements ✅ <br /> (primary)   | -1.0% | [-2.9%, -0.5%] | 49    |
| Improvements ✅ <br /> (secondary) | -1.1% | [-4.1%, -0.3%] | 75    |
| All ❌✅ (primary)                 | 0.2%  | [-2.9%, 3.6%]  | 113   |
- Given that the perf results are somewhat even (though regressions do win out), I think we sort of have to take this as is. I don't imagine we would revert an LLVM upgrade unless the perf results were really bad.