Importing rustc-1.65.0
Bug: 250026064
Test: ./build.py --lto=thin
Change-Id: If144bbb779543363527e335b12d3ae6fbb4be8f0
diff --git a/compiler/rustc_middle/src/mir/syntax.rs b/compiler/rustc_middle/src/mir/syntax.rs
index eb90169..c7d0283 100644
--- a/compiler/rustc_middle/src/mir/syntax.rs
+++ b/compiler/rustc_middle/src/mir/syntax.rs
@@ -23,75 +23,110 @@
use rustc_span::Span;
use rustc_target::asm::InlineAsmRegOrRegClass;
-/// The various "big phases" that MIR goes through.
+/// Represents the "flavors" of MIR.
///
-/// These phases all describe dialects of MIR. Since all MIR uses the same datastructures, the
-/// dialects forbid certain variants or values in certain phases. The sections below summarize the
-/// changes, but do not document them thoroughly. The full documentation is found in the appropriate
-/// documentation for the thing the change is affecting.
+/// All flavors of MIR use the same data structure, but there are some important differences. These
+/// differences come in two forms: Dialects and phases.
///
-/// Warning: ordering of variants is significant.
+/// Dialects represent a stronger distinction than phases. This is because the transitions between
+/// dialects are semantic changes, and therefore technically *lowerings* between distinct IRs. In
+/// other words, the same [`Body`](crate::mir::Body) might be well-formed for multiple dialects, but
+/// have different semantic meaning and different behavior at runtime.
+///
+/// Each dialect additionally has a number of phases. However, phase changes never involve semantic
+/// changes. If some MIR is well-formed both before and after a phase change, it is also guaranteed
+/// that it has the same semantic meaning. In this sense, phase changes can only add additional
+/// restrictions on what MIR is well-formed.
+///
+/// When adding phases, remember to update [`MirPhase::phase_index`].
#[derive(Copy, Clone, TyEncodable, TyDecodable, Debug, PartialEq, Eq, PartialOrd, Ord)]
#[derive(HashStable)]
pub enum MirPhase {
- /// The dialect of MIR used during all phases before `DropsLowered` is the same. This is also
- /// the MIR that analysis such as borrowck uses.
+ /// The MIR that is generated by MIR building.
///
- /// One important thing to remember about the behavior of this section of MIR is that drop terminators
- /// (including drop and replace) are *conditional*. The elaborate drops pass will then replace each
- /// instance of a drop terminator with a nop, an unconditional drop, or a drop conditioned on a drop
- /// flag. Of course, this means that it is important that the drop elaboration can accurately recognize
- /// when things are initialized and when things are de-initialized. That means any code running on this
- /// version of MIR must be sure to produce output that drop elaboration can reason about. See the
- /// section on the drop terminatorss for more details.
- Built = 0,
- // FIXME(oli-obk): it's unclear whether we still need this phase (and its corresponding query).
- // We used to have this for pre-miri MIR based const eval.
- Const = 1,
- /// This phase checks the MIR for promotable elements and takes them out of the main MIR body
- /// by creating a new MIR body per promoted element. After this phase (and thus the termination
- /// of the `mir_promoted` query), these promoted elements are available in the `promoted_mir`
- /// query.
- ConstsPromoted = 2,
- /// After this projections may only contain deref projections as the first element.
- Derefered = 3,
- /// Beginning with this phase, the following variants are disallowed:
- /// * [`TerminatorKind::DropAndReplace`]
+ /// The only things that operate on this dialect are unsafeck, the various MIR lints, and const
+ /// qualifs.
+ ///
+ /// This has no distinct phases.
+ Built,
+ /// The MIR used for most analysis.
+ ///
+ /// The only semantic change between analysis and built MIR is constant promotion. In built MIR,
+ /// sequences of statements that would generally be subject to constant promotion are
+ /// semantically constants, while in analysis MIR all constants are explicit.
+ ///
+ /// The result of const promotion is available from the `mir_promoted` and `promoted_mir` queries.
+ ///
+ /// This is the version of MIR used by borrowck and friends.
+ Analysis(AnalysisPhase),
+ /// The MIR used for CTFE, optimizations, and codegen.
+ ///
+ /// The semantic changes that occur in the lowering from analysis to runtime MIR are as follows:
+ ///
+ /// - Drops: In analysis MIR, `Drop` terminators represent *conditional* drops; roughly speaking,
+ /// if dataflow analysis determines that the place being dropped is uninitialized, the drop will
+ /// not be executed. The exact semantics of this aren't written down anywhere, which means they
+ /// are essentially "what drop elaboration does." In runtime MIR, the drops are unconditional;
+ /// when a `Drop` terminator is reached, if the type has drop glue that drop glue is always
+ /// executed. This may be UB if the underlying place is not initialized.
+ /// - Packed drops: Places might in general be misaligned - in most cases this is UB, the exception
+ /// is fields of packed structs. In analysis MIR, `Drop(P)` for a `P` that might be misaligned
+ /// for this reason implicitly moves `P` to a temporary before dropping. Runtime MIR has no such
+ /// rules, and dropping a misaligned place is simply UB.
+ /// - Unwinding: in analysis MIR, unwinding from a function which may not unwind aborts. In runtime
+ /// MIR, this is UB.
+ /// - Retags: If `-Zmir-emit-retag` is enabled, analysis MIR has "implicit" retags in the same way
+ /// that Rust itself has them. Where exactly these are is generally subject to change, and so we
+ /// don't document this here. Runtime MIR has all retags explicit.
+ /// - Generator bodies: In analysis MIR, locals may actually be behind a pointer that user code has
+ /// access to. This occurs in generator bodies. Such locals do not behave like other locals,
+ /// because they eg may be aliased in surprising ways. Runtime MIR has no such special locals -
+ /// all generator bodies are lowered and so all places that look like locals really are locals.
+ /// - Const prop lints: The lint pass which reports eg `200_u8 + 200_u8` as an error is run as a
+ /// part of analysis to runtime MIR lowering. This means that transformations which may supress
+ /// such errors may not run on analysis MIR.
+ Runtime(RuntimePhase),
+}
+
+/// See [`MirPhase::Analysis`].
+#[derive(Copy, Clone, TyEncodable, TyDecodable, Debug, PartialEq, Eq, PartialOrd, Ord)]
+#[derive(HashStable)]
+pub enum AnalysisPhase {
+ Initial = 0,
+ /// Beginning in this phase, the following variants are disallowed:
/// * [`TerminatorKind::FalseUnwind`]
/// * [`TerminatorKind::FalseEdge`]
/// * [`StatementKind::FakeRead`]
/// * [`StatementKind::AscribeUserType`]
/// * [`Rvalue::Ref`] with `BorrowKind::Shallow`
///
- /// And the following variant is allowed:
- /// * [`StatementKind::Retag`]
- ///
- /// Furthermore, `Drop` now uses explicit drop flags visible in the MIR and reaching a `Drop`
- /// terminator means that the auto-generated drop glue will be invoked. Also, `Copy` operands
- /// are allowed for non-`Copy` types.
- DropsLowered = 4,
- /// Beginning with this phase, the following variant is disallowed:
- /// * [`Rvalue::Aggregate`] for any `AggregateKind` except `Array`
- ///
- /// And the following variant is allowed:
- /// * [`StatementKind::SetDiscriminant`]
- Deaggregated = 5,
- /// Before this phase, generators are in the "source code" form, featuring `yield` statements
- /// and such. With this phase change, they are transformed into a proper state machine. Running
- /// optimizations before this change can be potentially dangerous because the source code is to
- /// some extent a "lie." In particular, `yield` terminators effectively make the value of all
- /// locals visible to the caller. This means that dead store elimination before them, or code
- /// motion across them, is not correct in general. This is also exasperated by type checking
- /// having pre-computed a list of the types that it thinks are ok to be live across a yield
- /// point - this is necessary to decide eg whether autotraits are implemented. Introducing new
- /// types across a yield point will lead to ICEs becaues of this.
- ///
- /// Beginning with this phase, the following variants are disallowed:
+ /// Furthermore, `Deref` projections must be the first projection within any place (if they
+ /// appear at all)
+ PostCleanup = 1,
+}
+
+/// See [`MirPhase::Runtime`].
+#[derive(Copy, Clone, TyEncodable, TyDecodable, Debug, PartialEq, Eq, PartialOrd, Ord)]
+#[derive(HashStable)]
+pub enum RuntimePhase {
+ /// In addition to the semantic changes, beginning with this phase, the following variants are
+ /// disallowed:
+ /// * [`TerminatorKind::DropAndReplace`]
/// * [`TerminatorKind::Yield`]
/// * [`TerminatorKind::GeneratorDrop`]
+ /// * [`Rvalue::Aggregate`] for any `AggregateKind` except `Array`
+ ///
+ /// And the following variants are allowed:
+ /// * [`StatementKind::Retag`]
+ /// * [`StatementKind::SetDiscriminant`]
+ /// * [`StatementKind::Deinit`]
+ ///
+ /// Furthermore, `Copy` operands are allowed for non-`Copy` types.
+ Initial = 0,
+ /// Beginning with this phase, the following variant is disallowed:
/// * [`ProjectionElem::Deref`] of `Box`
- GeneratorsLowered = 6,
- Optimized = 7,
+ PostCleanup = 1,
+ Optimized = 2,
}
///////////////////////////////////////////////////////////////////////////
@@ -292,12 +327,40 @@
/// executed.
Coverage(Box<Coverage>),
+ /// Denotes a call to an intrinsic that does not require an unwind path and always returns.
+ /// This avoids adding a new block and a terminator for simple intrinsics.
+ Intrinsic(Box<NonDivergingIntrinsic<'tcx>>),
+
+ /// No-op. Useful for deleting instructions without affecting statement indices.
+ Nop,
+}
+
+#[derive(
+ Clone,
+ TyEncodable,
+ TyDecodable,
+ Debug,
+ PartialEq,
+ Hash,
+ HashStable,
+ TypeFoldable,
+ TypeVisitable
+)]
+pub enum NonDivergingIntrinsic<'tcx> {
+ /// Denotes a call to the intrinsic function `assume`.
+ ///
+ /// The operand must be a boolean. Optimizers may use the value of the boolean to backtrack its
+ /// computation to infer information about other variables. So if the boolean came from a
+ /// `x < y` operation, subsequent operations on `x` and `y` could elide various bound checks.
+ /// If the argument is `false`, this operation is equivalent to `TerminatorKind::Unreachable`.
+ Assume(Operand<'tcx>),
+
/// Denotes a call to the intrinsic function `copy_nonoverlapping`.
///
/// First, all three operands are evaluated. `src` and `dest` must each be a reference, pointer,
/// or `Box` pointing to the same type `T`. `count` must evaluate to a `usize`. Then, `src` and
/// `dest` are dereferenced, and `count * size_of::<T>()` bytes beginning with the first byte of
- /// the `src` place are copied to the continguous range of bytes beginning with the first byte
+ /// the `src` place are copied to the contiguous range of bytes beginning with the first byte
/// of `dest`.
///
/// **Needs clarification**: In what order are operands computed and dereferenced? It should
@@ -305,10 +368,18 @@
///
/// **Needs clarification**: Is this typed or not, ie is there a typed load and store involved?
/// I vaguely remember Ralf saying somewhere that he thought it should not be.
- CopyNonOverlapping(Box<CopyNonOverlapping<'tcx>>),
+ CopyNonOverlapping(CopyNonOverlapping<'tcx>),
+}
- /// No-op. Useful for deleting instructions without affecting statement indices.
- Nop,
+impl std::fmt::Display for NonDivergingIntrinsic<'_> {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ Self::Assume(op) => write!(f, "assume({op:?})"),
+ Self::CopyNonOverlapping(CopyNonOverlapping { src, dst, count }) => {
+ write!(f, "copy_nonoverlapping(dst = {dst:?}, src = {src:?}, count = {count:?})")
+ }
+ }
+ }
}
/// Describes what kind of retag is to be performed.
@@ -343,7 +414,7 @@
/// Some(closure_def_id).
/// Otherwise, the value of the optional LocalDefId will be None.
//
- // We can use LocaDefId here since fake read statements are removed
+ // We can use LocalDefId here since fake read statements are removed
// before codegen in the `CleanupNonCodegenStatements` pass.
ForMatchedPlace(Option<LocalDefId>),
@@ -417,7 +488,7 @@
/// must also be `cleanup`. This is a part of the type system and checked statically, so it is
/// still an error to have such an edge in the CFG even if it's known that it won't be taken at
/// runtime.
-#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq)]
+#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq, TypeFoldable, TypeVisitable)]
pub enum TerminatorKind<'tcx> {
/// Block has one successor; we continue execution there.
Goto { target: BasicBlock },
@@ -670,7 +741,7 @@
}
/// Information about an assertion failure.
-#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq, PartialOrd)]
+#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq, TypeFoldable, TypeVisitable)]
pub enum AssertKind<O> {
BoundsCheck { len: O, index: O },
Overflow(BinOp, O, O),
@@ -792,7 +863,7 @@
///
/// Rust currently requires that every place obey those two rules. This is checked by MIRI and taken
/// advantage of by codegen (via `gep inbounds`). That is possibly subject to change.
-#[derive(Copy, Clone, PartialEq, Eq, Hash, TyEncodable, HashStable)]
+#[derive(Copy, Clone, PartialEq, Eq, Hash, TyEncodable, HashStable, TypeFoldable, TypeVisitable)]
pub struct Place<'tcx> {
pub local: Local,
@@ -801,7 +872,7 @@
}
#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
-#[derive(TyEncodable, TyDecodable, HashStable)]
+#[derive(TyEncodable, TyDecodable, HashStable, TypeFoldable, TypeVisitable)]
pub enum ProjectionElem<V, T> {
Deref,
Field(Field, T),
@@ -884,7 +955,7 @@
/// **Needs clarifiation:** Is loading a place that has its variant index set well-formed? Miri
/// currently implements it, but it seems like this may be something to check against in the
/// validator.
-#[derive(Clone, PartialEq, TyEncodable, TyDecodable, Hash, HashStable)]
+#[derive(Clone, PartialEq, TyEncodable, TyDecodable, Hash, HashStable, TypeFoldable, TypeVisitable)]
pub enum Operand<'tcx> {
/// Creates a value by loading the given place.
///
@@ -915,7 +986,7 @@
/// Computing any rvalue begins by evaluating the places and operands in some order (**Needs
/// clarification**: Which order?). These are then used to produce a "value" - the same kind of
/// value that an [`Operand`] produces.
-#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq)]
+#[derive(Clone, TyEncodable, TyDecodable, Hash, HashStable, PartialEq, TypeFoldable, TypeVisitable)]
pub enum Rvalue<'tcx> {
/// Yields the operand unchanged
Use(Operand<'tcx>),
@@ -1068,11 +1139,14 @@
/// All sorts of pointer-to-pointer casts. Note that reference-to-raw-ptr casts are
/// translated into `&raw mut/const *r`, i.e., they are not actually casts.
Pointer(PointerCast),
+ /// Cast into a dyn* object.
+ DynStar,
/// Remaining unclassified casts.
Misc,
}
#[derive(Clone, Debug, PartialEq, Eq, TyEncodable, TyDecodable, Hash, HashStable)]
+#[derive(TypeFoldable, TypeVisitable)]
pub enum AggregateKind<'tcx> {
/// The type is of the element
Array(Ty<'tcx>),
@@ -1160,7 +1234,8 @@
mod size_asserts {
use super::*;
// These are in alphabetical order, which is easy to maintain.
- static_assert_size!(AggregateKind<'_>, 48);
+ #[cfg(not(bootstrap))]
+ static_assert_size!(AggregateKind<'_>, 40);
static_assert_size!(Operand<'_>, 24);
static_assert_size!(Place<'_>, 16);
static_assert_size!(PlaceElem<'_>, 24);
