A relatively noisy week (most of those have been dropped below, and comments left on GitHub), but otherwise a quiet one in terms of performance changes, with essentially no significant changes occuring.
Triage done by @simulacrum. Revision range: 822f8c22f540b12f296d844ad5bf39aaa47bfeb4..57d3c58ed6e0faf89a62411f96c000ffc9fd3937
Summary:
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | 1.2% | [0.2%, 1.8%] | 13 |
| Regressions ❌ (secondary) | 1.2% | [0.2%, 2.1%] | 47 |
| Improvements ✅ (primary) | -1.5% | [-4.0%, -0.2%] | 21 |
| Improvements ✅ (secondary) | -1.8% | [-3.9%, -0.3%] | 61 |
| All ❌✅ (primary) | -0.4% | [-4.0%, 1.8%] | 34 |
2 Regressions, 2 Improvements, 3 Mixed; 3 of them in rollups 39 artifact comparisons made in total
Rollup of 6 pull requests #103998 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | 2.5% | [0.3%, 3.9%] | 6 |
| Regressions ❌ (secondary) | 1.4% | [0.4%, 3.5%] | 43 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | -1.3% | [-1.5%, -1.0%] | 2 |
| All ❌✅ (primary) | 2.5% | [0.3%, 3.9%] | 6 |
Appears to be a genuine regression. Mostly limited to smaller benchmarks, so most likely just a slight increase in the minimum work rustc does. Suspected to be due to the changes in sysroot detection; have enqueued a build to try and find out.
std: sync “Dependencies of the backtrace crate” with backtrace #103934 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | 1.4% | [0.9%, 1.8%] | 10 |
| Regressions ❌ (secondary) | 1.5% | [0.3%, 3.3%] | 47 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | - | - | 0 |
| All ❌✅ (primary) | 1.4% | [0.9%, 1.8%] | 10 |
Some of this may be noise, but it‘s also possible that the newer dependencies are doing slightly different things causing some regressions -- it’s possible we're computing backtraces somewhere, and even modulo that, the changes to the standard library may cause different inlining decisions, etc.
Make rustdoc Item::visibility computed on-demand #103690 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | - | - | 0 |
| Improvements ✅ (primary) | -0.8% | [-1.7%, -0.4%] | 11 |
| Improvements ✅ (secondary) | -1.6% | [-2.7%, -0.6%] | 23 |
| All ❌✅ (primary) | -0.8% | [-1.7%, -0.4%] | 11 |
Rollup of 7 pull requests #103962 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | - | - | 0 |
| Improvements ✅ (primary) | -0.2% | [-0.3%, -0.2%] | 6 |
| Improvements ✅ (secondary) | -0.4% | [-0.4%, -0.3%] | 4 |
| All ❌✅ (primary) | -0.2% | [-0.3%, -0.2%] | 6 |
Revert “ci: Bring back ninja for dist builders” #103846 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | 2.5% | [2.5%, 2.5%] | 1 |
| Improvements ✅ (primary) | -1.7% | [-7.4%, -0.2%] | 19 |
| Improvements ✅ (secondary) | -2.0% | [-6.6%, -0.2%] | 71 |
| All ❌✅ (primary) | -1.7% | [-7.4%, -0.2%] | 19 |
This change was reverted precisely due to the performance regression it introduced, so this is no surprise.
Rollup of 5 pull requests #104017 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | 1.4% | [1.4%, 1.4%] | 1 |
| Regressions ❌ (secondary) | 0.5% | [0.3%, 0.8%] | 2 |
| Improvements ✅ (primary) | -1.0% | [-3.2%, -0.3%] | 13 |
| Improvements ✅ (secondary) | -2.3% | [-3.1%, -0.6%] | 19 |
| All ❌✅ (primary) | -0.9% | [-3.2%, 1.4%] | 14 |
Minor regressions in two secondary benchmarks and in libc don't seem to warrant further investigation, especially given the general improvement across the board.
Some tracing and comment cleanups #103975 (Comparison Link)
| (instructions:u) | mean | range | count |
|---|---|---|---|
| Regressions ❌ (primary) | 1.0% | [0.2%, 1.5%] | 3 |
| Regressions ❌ (secondary) | 1.9% | [0.3%, 4.2%] | 13 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | -0.6% | [-0.6%, -0.6%] | 3 |
| All ❌✅ (primary) | 1.0% | [0.2%, 1.5%] | 3 |
Most of the regressions are probably noise, so this change is likely actually an improvement or neutral.