A fairly noisy week (many entries below dropped and untagged as regressions), largely driven by tt-muncher and html5ever. Our sensitivity assessment currently takes roughly a week since new noise starts to learn the noise level, so it can take some time for oscillations to stop reporting somewhat spurious results.
Otherwise, this week had a number of solid improvements, and was overall positive, with improvements across many benchmarks.
Triage done by @simulacrum. Revision range: 880646ca9c6dc21e04efe2f1940369a45b71ff2d..b3f4c3119957aa0a250cab08ab586b7a9a680ef1
Summary:
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.7% | 1.4% | 19 |
| Regressions 😿 (secondary) | 2.1% | 5.3% | 13 |
| Improvements 🎉 (primary) | -1.1% | -2.5% | 153 |
| Improvements 🎉 (secondary) | -2.1% | -9.9% | 121 |
| All 😿🎉 (primary) | -0.9% | -2.5% | 172 |
3 Regressions, 6 Improvements, 3 Mixed; 3 of them in rollups 53 artifact comparisons made in total
Make lowering a query #95573 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.5% | 1.1% | 76 |
| Regressions 😿 (secondary) | 0.8% | 2.4% | 57 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | -0.2% | -0.2% | 1 |
| All 😿🎉 (primary) | 0.5% | 1.1% | 76 |
These results may partially be noise, and are overall limited enough in impact that further investigation is not warranted at this time. Marked as triaged.
Rollup of 6 pull requests #99047 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | 1.9% | 4.0% | 13 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | N/A | N/A | 0 |
| All 😿🎉 (primary) | N/A | N/A | 0 |
Looks to be a genuine regression, far exceeding the noise bound for the deeply-nested-multi benchmark. (This is the 4% spike on that graph). Maybe caused by #98795, needs a little further investigation.
Enforce that layout size fits in isize in Layout #95295 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.5% | 1.5% | 106 |
| Regressions 😿 (secondary) | 0.6% | 1.7% | 39 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | -1.3% | -1.5% | 6 |
| All 😿🎉 (primary) | 0.5% | 1.5% | 106 |
As noted on the PR (thanks for triaging perf regressions!), this is a limited in magnitude and a justified regression for a soundness fix. See this comment for details.
continue nll transition by removing stuff #98584 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | N/A | N/A | 0 |
| Improvements 🎉 (primary) | -1.0% | -2.3% | 66 |
| Improvements 🎉 (secondary) | -1.2% | -3.6% | 63 |
| All 😿🎉 (primary) | -1.0% | -2.3% | 66 |
interpret: remove support for unsized_locals #98831 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | N/A | N/A | 0 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | -0.6% | -0.8% | 11 |
| All 😿🎉 (primary) | N/A | N/A | 0 |
Use a bitset instead of a hash map in HIR ID validator #98841 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | N/A | N/A | 0 |
| Improvements 🎉 (primary) | -0.8% | -1.7% | 123 |
| Improvements 🎉 (secondary) | -0.8% | -2.0% | 82 |
| All 😿🎉 (primary) | -0.8% | -1.7% | 123 |
don't use commit_if_ok during higher_ranked_sub #99056 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | N/A | N/A | 0 |
| Improvements 🎉 (primary) | -0.7% | -0.7% | 7 |
| Improvements 🎉 (secondary) | -0.8% | -0.9% | 6 |
| All 😿🎉 (primary) | -0.7% | -0.7% | 7 |
More derive output improvements #98758 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.6% | 0.8% | 2 |
| Regressions 😿 (secondary) | 1.6% | 2.3% | 10 |
| Improvements 🎉 (primary) | -0.4% | -0.6% | 55 |
| Improvements 🎉 (secondary) | -2.1% | -8.0% | 34 |
| All 😿🎉 (primary) | -0.4% | 0.8% | 57 |
Regressions appear primarily in tt-muncher and are spurious noise; cargo and image also show slight regressions but much smaller in comparative magnitude. Overall a small, but solid improvement.
Miscellaneous inlining improvements #99028 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | 0.7% | 1.4% | 13 |
| Improvements 🎉 (primary) | -0.4% | -0.9% | 124 |
| Improvements 🎉 (secondary) | -0.5% | -1.0% | 62 |
| All 😿🎉 (primary) | -0.4% | -0.9% | 124 |
Rollup of 7 pull requests #98987 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.6% | 0.6% | 1 |
| Regressions 😿 (secondary) | N/A | N/A | 0 |
| Improvements 🎉 (primary) | -0.2% | -0.2% | 1 |
| Improvements 🎉 (secondary) | -0.3% | -0.6% | 9 |
| All 😿🎉 (primary) | 0.2% | 0.6% | 2 |
Rollup of 8 pull requests #99014 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | 0.5% | 0.5% | 1 |
| Regressions 😿 (secondary) | 0.3% | 0.3% | 1 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | -1.3% | -1.5% | 6 |
| All 😿🎉 (primary) | 0.5% | 0.5% | 1 |
Don't allow ZST in ScalarInt #98957 (Comparison Link)
| mean | max | count | |
|---|---|---|---|
| Regressions 😿 (primary) | N/A | N/A | 0 |
| Regressions 😿 (secondary) | 4.4% | 4.4% | 2 |
| Improvements 🎉 (primary) | N/A | N/A | 0 |
| Improvements 🎉 (secondary) | -0.4% | -0.4% | 2 |
| All 😿🎉 (primary) | N/A | N/A | 0 |