| # Contribution Guide |
| |
| If you want to hack on Miri yourself, great! Here are some resources you might |
| find useful. |
| |
| ## Getting started |
| |
| Check out the issues on this GitHub repository for some ideas. In particular, |
| look for the green `E-*` labels which mark issues that should be rather |
| well-suited for onboarding. For more ideas or help with hacking on Miri, you can |
| contact us on the [Rust Zulip]. See the [Rust website](https://www.rust-lang.org/governance/teams/compiler#team-miri) |
| for a list of Miri maintainers. |
| |
| [Rust Zulip]: https://rust-lang.zulipchat.com |
| |
| ### Larger-scale contributions |
| |
| If you are thinking about making a larger-scale contribution -- in particular anything that needs |
| more than can reasonably fit in a single PR to be feature-complete -- then please talk to us before |
| writing significant amounts of code. Generally, we will ask that you follow a three-step "project" |
| process for such contributions: |
| |
| 1. Clearly define the **goal** of the project. This defines the scope of the project, i.e. which |
| part of which APIs should be supported. If this involves functions that expose a big API surface |
| with lots of flags, the project may want to support only a tiny subset of flags; that should be |
| documented. A good way to express the goal is with one or more test cases that Miri should be |
| able to successfully execute when the project is completed. It is a good idea to get feedback |
| from team members already at this stage to ensure that the project is reasonably scoped and |
| aligns with our interests. |
| 2. Make a **design** for how to realize the goal. A larger project will likely have to do global |
| changes to Miri, like adding new global state to the `Machine` type or new methods to the |
| `FileDescription` trait. Often we have to iterate on those changes, which can quite substantially |
| change how the final implementation looks like. |
| |
| The design should be reasonably concrete, i.e. for new global state or methods the corresponding |
| Rust types and method signatures should be spelled out. We realize that it can be hard to make a |
| design without doing implementation work, in particular if you are not yet familiar with the |
| codebase. Doing draft implementations in phase 2 of this process is perfectly fine, just please |
| be aware that we might request fundamental changes that can require significantly reworking what |
| you already did. If you open a PR in this stage, please clearly indicate that this project is |
| still in the design stage. |
| |
| 3. Finish the **implementation** and have it reviewed. |
| |
| This process is largely informal, and its primary goal is to more clearly communicate expectations. |
| Please get in touch with us if you have any questions! |
| |
| ## Preparing the build environment |
| |
| Miri heavily relies on internal and unstable rustc interfaces to execute MIR, |
| which means it is important that you install a version of rustc that Miri |
| actually works with. |
| |
| The `rust-version` file contains the commit hash of rustc that Miri is currently |
| tested against. Other versions will likely not work. After installing |
| [`rustup-toolchain-install-master`], you can run the following command to |
| install that exact version of rustc as a toolchain: |
| ``` |
| ./miri toolchain |
| ``` |
| This will set up a rustup toolchain called `miri` and set it as an override for |
| the current directory. |
| |
| You can also create a `.auto-everything` file (contents don't matter, can be empty), which |
| will cause any `./miri` command to automatically call `./miri toolchain`, `clippy` and `rustfmt` |
| for you. If you don't want all of these to happen, you can add individual `.auto-toolchain`, |
| `.auto-clippy` and `.auto-fmt` files respectively. |
| |
| [`rustup-toolchain-install-master`]: https://github.com/kennytm/rustup-toolchain-install-master |
| |
| ## Building and testing Miri |
| |
| Invoking Miri requires getting a bunch of flags right and setting up a custom |
| sysroot. The `miri` script takes care of that for you. With the |
| build environment prepared, compiling Miri is just one command away: |
| |
| ``` |
| ./miri build |
| ``` |
| |
| Run `./miri` without arguments to see the other commands our build tool |
| supports. |
| |
| ### Testing the Miri driver |
| |
| The Miri driver compiled from `src/bin/miri.rs` is the "heart" of Miri: it is |
| basically a version of `rustc` that, instead of compiling your code, runs it. |
| It accepts all the same flags as `rustc` (though the ones only affecting code |
| generation and linking obviously will have no effect) [and more][miri-flags]. |
| |
| [miri-flags]: README.md#miri--z-flags-and-environment-variables |
| |
| For example, you can (cross-)run the driver on a particular file by doing |
| |
| ```sh |
| ./miri run tests/pass/format.rs |
| ./miri run tests/pass/hello.rs --target i686-unknown-linux-gnu |
| ``` |
| |
| Tests in ``pass-dep`` need to be run using ``./miri run --dep <filename>``. |
| For example: |
| ```sh |
| ./miri run --dep tests/pass-dep/shims/libc-fs.rs |
| ``` |
| |
| You can (cross-)run the entire test suite using: |
| |
| ```sh |
| ./miri test |
| ./miri test --target i686-unknown-linux-gnu |
| ``` |
| |
| `./miri test FILTER` only runs those tests that contain `FILTER` in their filename (including the |
| base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters are |
| supported: `./miri test FILTER1 FILTER2` runs all tests that contain either string. |
| |
| #### Fine grained logging |
| |
| You can get a trace of which MIR statements are being executed by setting the |
| `MIRI_LOG` environment variable. For example: |
| |
| ```sh |
| MIRI_LOG=info ./miri run tests/pass/vec.rs |
| ``` |
| |
| Setting `MIRI_LOG` like this will configure logging for Miri itself as well as |
| the `rustc_middle::mir::interpret` and `rustc_mir::interpret` modules in rustc. You |
| can also do more targeted configuration, e.g. the following helps debug the |
| stacked borrows implementation: |
| |
| ```sh |
| MIRI_LOG=rustc_mir::interpret=info,miri::stacked_borrows ./miri run tests/pass/vec.rs |
| ``` |
| |
| Note that you will only get `info`, `warn` or `error` messages if you use a prebuilt compiler. |
| In order to get `debug` and `trace` level messages, you need to build miri with a locally built |
| compiler that has `debug=true` set in `config.toml`. |
| |
| #### Debugging error messages |
| |
| You can set `MIRI_BACKTRACE=1` to get a backtrace of where an |
| evaluation error was originally raised. |
| |
| |
| ### UI testing |
| |
| We use ui-testing in Miri, meaning we generate `.stderr` and `.stdout` files for the output |
| produced by Miri. You can use `./miri test --bless` to automatically (re)generate these files when |
| you add new tests or change how Miri presents certain output. |
| |
| Note that when you also use `MIRIFLAGS` to change optimizations and similar, the ui output |
| will change in unexpected ways. In order to still be able |
| to run the other checks while ignoring the ui output, use `MIRI_SKIP_UI_CHECKS=1 ./miri test`. |
| |
| For more info on how to configure ui tests see [the documentation on the ui test crate][ui_test] |
| |
| [ui_test]: https://github.com/oli-obk/ui_test/blob/main/README.md |
| |
| ### Testing `cargo miri` |
| |
| Working with the driver directly gives you full control, but you also lose all |
| the convenience provided by cargo. Once your test case depends on a crate, it |
| is probably easier to test it with the cargo wrapper. You can install your |
| development version of Miri using |
| |
| ``` |
| ./miri install |
| ``` |
| |
| and then you can use it as if it was installed by `rustup` as a component of the |
| `miri` toolchain. Note that the `miri` and `cargo-miri` executables are placed |
| in the `miri` toolchain's sysroot to prevent conflicts with other toolchains. |
| The Miri binaries in the `cargo` bin directory (usually `~/.cargo/bin`) are managed by rustup. |
| |
| There's a test for the cargo wrapper in the `test-cargo-miri` directory; run `./run-test.py` in |
| there to execute it. You can pass `--target` to execute the test for another target. |
| |
| ### Using a modified standard library |
| |
| Miri re-builds the standard library into a custom sysroot, so it is fairly easy |
| to test Miri against a modified standard library -- you do not even have to |
| build Miri yourself, the Miri shipped by `rustup` will work. All you have to do |
| is set the `MIRI_LIB_SRC` environment variable to the `library` folder of a |
| `rust-lang/rust` repository checkout. Note that changing files in that directory |
| does not automatically trigger a re-build of the standard library; you have to |
| clear the Miri build cache manually (on Linux, `rm -rf ~/.cache/miri`; |
| on Windows, `rmdir /S "%LOCALAPPDATA%\rust-lang\miri\cache"`; |
| and on macOS, `rm -rf ~/Library/Caches/org.rust-lang.miri`). |
| |
| ### Benchmarking |
| |
| Miri comes with a few benchmarks; you can run `./miri bench` to run them with the locally built |
| Miri. Note: this will run `./miri install` as a side-effect. Also requires `hyperfine` to be |
| installed (`cargo install hyperfine`). |
| |
| ## Configuring `rust-analyzer` |
| |
| To configure `rust-analyzer` and VS Code for working on Miri, save the following |
| to `.vscode/settings.json` in your local Miri clone: |
| |
| ```json |
| { |
| "rust-analyzer.rustc.source": "discover", |
| "rust-analyzer.linkedProjects": [ |
| "Cargo.toml", |
| "cargo-miri/Cargo.toml", |
| "miri-script/Cargo.toml", |
| ], |
| "rust-analyzer.check.invocationLocation": "root", |
| "rust-analyzer.check.invocationStrategy": "once", |
| "rust-analyzer.check.overrideCommand": [ |
| "env", |
| "MIRI_AUTO_OPS=no", |
| "./miri", |
| "clippy", // make this `check` when working with a locally built rustc |
| "--message-format=json", |
| ], |
| // Contrary to what the name suggests, this also affects proc macros. |
| "rust-analyzer.cargo.buildScripts.invocationLocation": "root", |
| "rust-analyzer.cargo.buildScripts.invocationStrategy": "once", |
| "rust-analyzer.cargo.buildScripts.overrideCommand": [ |
| "env", |
| "MIRI_AUTO_OPS=no", |
| "./miri", |
| "check", |
| "--message-format=json", |
| ], |
| } |
| ``` |
| |
| > #### Note |
| > |
| > If you are [building Miri with a locally built rustc][], set |
| > `rust-analyzer.rustcSource` to the relative path from your Miri clone to the |
| > root `Cargo.toml` of the locally built rustc. For example, the path might look |
| > like `../rust/Cargo.toml`. |
| |
| See the rustc-dev-guide's docs on ["Configuring `rust-analyzer` for `rustc`"][rdg-r-a] |
| for more information about configuring VS Code and `rust-analyzer`. |
| |
| [rdg-r-a]: https://rustc-dev-guide.rust-lang.org/building/suggested.html#configuring-rust-analyzer-for-rustc |
| |
| ## Advanced topic: Working on Miri in the rustc tree |
| |
| We described above the simplest way to get a working build environment for Miri, |
| which is to use the version of rustc indicated by `rustc-version`. But |
| sometimes, that is not enough. |
| |
| A big part of the Miri driver is shared with rustc, so working on Miri will |
| sometimes require also working on rustc itself. In this case, you should *not* |
| work in a clone of the Miri repository, but in a clone of the |
| [main Rust repository](https://github.com/rust-lang/rust/). There is a copy of |
| Miri located at `src/tools/miri` that you can work on directly. A maintainer |
| will eventually sync those changes back into this repository. |
| |
| When working on Miri in the rustc tree, here's how you can run tests: |
| |
| ``` |
| ./x.py test miri |
| ``` |
| |
| `--bless` will work, too. |
| |
| You can also directly run Miri on a Rust source file: |
| |
| ``` |
| ./x.py run miri --stage 1 --args src/tools/miri/tests/pass/hello.rs |
| ``` |
| |
| ## Advanced topic: Syncing with the rustc repo |
| |
| We use the [`josh` proxy](https://github.com/josh-project/josh) to transmit changes between the |
| rustc and Miri repositories. You can install it as follows: |
| |
| ```sh |
| RUSTFLAGS="--cap-lints=warn" cargo +stable install josh-proxy --git https://github.com/josh-project/josh --tag r23.12.04 |
| ``` |
| |
| Josh will automatically be started and stopped by `./miri`. |
| |
| ### Importing changes from the rustc repo |
| |
| *Note: this usually happens automatically, so these steps rarely have to be done by hand.* |
| |
| We assume we start on an up-to-date master branch in the Miri repo. |
| |
| ```sh |
| # Fetch and merge rustc side of the history. Takes ca 5 min the first time. |
| # This will also update the `rustc-version` file. |
| ./miri rustc-pull |
| # Update local toolchain and apply formatting. |
| ./miri toolchain && ./miri fmt |
| git commit -am "rustup" |
| ``` |
| |
| Now push this to a new branch in your Miri fork, and create a PR. It is worth |
| running `./miri test` locally in parallel, since the test suite in the Miri repo |
| is stricter than the one on the rustc side, so some small tweaks might be |
| needed. |
| |
| ### Exporting changes to the rustc repo |
| |
| We will use the josh proxy to push to your fork of rustc. Run the following in the Miri repo, |
| assuming we are on an up-to-date master branch: |
| |
| ```sh |
| # Push the Miri changes to your rustc fork (substitute your github handle for YOUR_NAME). |
| ./miri rustc-push YOUR_NAME miri |
| ``` |
| |
| This will create a new branch called `miri` in your fork, and the output should include a link that |
| creates a rustc PR to integrate those changes into the main repository. If that PR has conflicts, |
| you need to pull rustc changes into Miri first, and then re-do the rustc push. |
| |
| If this fails due to authentication problems, it can help to make josh push via ssh instead of |
| https. Add the following to your `.gitconfig`: |
| |
| ```toml |
| [url "[email protected]:"] |
| pushInsteadOf = https://github.com/ |
| ``` |
| |
| ## Further environment variables |
| |
| The following environment variables are relevant to `./miri`: |
| |
| * `MIRI_AUTO_OPS` indicates whether the automatic execution of rustfmt, clippy and toolchain setup |
| (as controlled by the `./auto-*` files) should be skipped. If it is set to `no`, they are skipped. |
| This is used to allow automated IDE actions to avoid the auto ops. |
| * `MIRI_LOG`, `MIRI_BACKTRACE` control logging and backtrace printing during Miri executions. |
| * `MIRI_TEST_THREADS` (recognized by `./miri test`) sets the number of threads to use for running |
| tests. By default, the number of cores is used. |
| * `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`) disables checking that the `stderr` or |
| `stdout` files match the actual output. |
| |
| Furthermore, the usual environment variables recognized by `cargo miri` also work for `./miri`, e.g. |
| `MIRI_LIB_SRC`. Note that `MIRIFLAGS` is ignored by `./miri test` as each test controls the flags it |
| is run with. |
| |
| The following environment variables are *internal* and must not be used by |
| anyone but Miri itself. They are used to communicate between different Miri |
| binaries, and as such worth documenting: |
| |
| * `CARGO_EXTRA_FLAGS` is understood by `./miri` and passed to all host cargo invocations. |
| It is reserved for CI usage; setting the wrong flags this way can easily confuse the script. |
| * `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to |
| actually not interpret the code but compile it like rustc would. With `target`, Miri sets |
| some compiler flags to prepare the code for interpretation; with `host`, this is not done. |
| This environment variable is useful to be sure that the compiled `rlib`s are compatible |
| with Miri. |
| * `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build, |
| which will re-invoke `cargo-miri` as the `rustc` to use for this build. |
| * `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is |
| running as a child process of `rustdoc`, which invokes it twice for each doc-test |
| and requires special treatment, most notably a check-only build before interpretation. |
| This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper. |
| * `MIRI_CWD` when set to any value tells the Miri driver to change to the given |
| directory after loading all the source files, but before commencing |
| interpretation. This is useful if the interpreted program wants a different |
| working directory at run-time than at build-time. |
| * `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which |
| crates should be given special treatment in diagnostics, in addition to the |
| crate currently being compiled. |
| * `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the |
| value of `RUSTDOC` from before it was overwritten. |
| * `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap |
| run the standard library tests in Miri. |
| * `MIRI_TEST_TARGET` is set by `./miri test` (and `./x.py test miri`) to tell the test harness about |
| the chosen target. |
| * `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to |
| perform verbose logging. |
| * `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host* |
| operations. |
| * `RUSTC_BLESS` is set by `./miri test` (and `./x.py test miri`) to indicate bless-mode to the test |
| harness. |