The upstream Rust project releases a new stable version of the Rust toolchain every 6 weeks. Android's Rust toolchain team must release a prebuilt for every toolchain version as new compilers often depend on features released in the previous version. This means that skipping a release would inevitably result in a failure when we attempted to update the toolchain in the future.
A high-level overview of the steps necessary to release a new Rust toolchain for Android can be found in the following section with more detailed instructions appearing in the Release Process section.
General Android development is outside the scope of this document and it is expected that readers have a working knowledge of Soong [1, 2], repo, lunch, and how to modify Android code. Information on how to debug issues that occur while conducting the release process can be found in the debugging documentation.
The Rust project uses semantic versioning for its software releases. As such, their version strings follow the <major>.<minor>.<patch> format. If a “point release” (re-compilation and distribution of the same source code built with different arguments) is performed for the Rust toolchain any new version string must adhere to the following format: <major>.<minor>.<patch>.<point> (E.g. 1.77.1.p1).
Note: Copy this Markdown source into the update‘s issue’s Description field.
The steps listed below may contain invocations of the project's custom scripts. Documentation for these tools can be found in here as well as in the tools themselves via the --help flag.
It is highly recommended to begin the update process with a freshly synchronized Android source tree.
Begin by using the fetch_source.py tool to download an archive containing the Rust toolchain source. By default it will fetch the release archive for the provided Rust version string. The --beta and --nightly flags can be used to fetch their respective source archives.
The fetch_source.py will occasionally fail to commit the new source code, with a message warning of prohibited patterns in the commit. The offending texts are keys used in CI-related configurations, not actual private keys.
These warnings can be suppressed by running the following command in the toolchainrustc directory: git config --add secrets.allowed ARTIFACTS_AWS_ACCESS_KEY_ID
Note: It is helpful to use the same Git branch name in the toolchain/android_rust project for the next two steps. This will allow you to use the -t argument to set the topic of the CLs when you upload them.
As the Rust source code changes our local patches will drift out of context or break outright. The recontextualize_patches.py tool is used to reduce the former and help fix the latter. After fetching the new source run this tool. If everything is successful you can commit the updated patch files in the toolchain/android_rust project and move to the next step.
If there was an issue with the patch update process the tool will exit with an error. You can then navigate to toolchain/rustc and finish the git am process that is in progress. You can use git or patch to re-apply the patch that failed, resolve the issues, commit the changes, and then run git am --continue.
When the git am process is finished you can run tools/recontextualize_patches.py --resume to copy the updated patches into toolchain/android_rust/patches.
It is sometimes necessary to modify the Android Rust toolchain build system to successfully build a new version of Rust. The types of changes necessary can vary from adding a new value to the config.toml.template file, to investigating the source of a new shared library dependency, to setting environment variables to enable/disable new features.
Start by using the following command to initiate a build of the new Rust source: ./tools/build.py --lto thin --llvm-linkage shared
See the debugging document for tips and suggestions on fixing build breakages. To avoid repeatedly copying and/or patching the source code during development you can use the --repatch and --no-copy-and-patch arguments.
Use the --upstream-test flag to run the upstream Rust project's host tests.
Once you have fixed any build breakages you can create a new Git commit on top of the existing update to the patches.
The vendor/ directory of the Rust release archives contain multiple versions of many crates. There are several of these crates that we must copy into the Android toolchain‘s prebuilt directory for use by Soong. Because Soong does not support having multiple versions of a project’s source we must pick which of the multiple versions in the release archive we wish to use.
During an update it is necessary to check each of the vendored crates defined in STDLIB_SOURCE_RENAMES and update the version number if necessary. If there are multiple versions of a crate available select the most recent one for initial testing. If there is a build failure when building the crate with Soong you can copy the next candidate version of the crate directly into the prebuilt's directory (e.g. prebuilts/rust/linux-x86/1.82.0/src/stdlibs/vendor/getopts) and attempt to rebuild the target.
Once you have build a Rust toolchain you can begin the process of updating the Android source code to be compatible with the new compiler. Use the test_compiler.py tool to extract the prebuilt archive and begin running the tests.
When a target breaks resolve the issue to the best of your ability. The diagnostic message from the compiler will often be enough to address the error. If you are uncertain of what changes to make to an internal Android project you can add a comment with the text of the error, upload the CL, and contact the code owners for advice on the issue.
After resolving an issue you can resume the testing process by either passing the --reuse-prebuilt argument to the test_compiler.py tool or executing export RUST_PREBUILTS_VERSION=9.99.9 before manually invoking Soong to build the test targets (e.g. m rust).
Upload any changes required by the new compiler version to Gerrit for review. Changes that are valid in both the old and new version of Rust should go in independently while any changes that require the new compiler will need to go into the topic used to update the prebuilts.
These changes need to go in before the compiler source is updated to ensure that the CI builds remain green.
Note: An example of a ticket that posted all related CLs in the Description.
If the --image flag was passed to test_compiler.py the resulting image (default target aosp_cf_x86_64_phone-trunk_staging-userdebug) can be used to boot a device. A virtual device can be created using the following command:
$ acloud create --local-image
Once it is complete, this command will print the IP address and port that can be used to connect to the virtual device. Export this is as ANDROID_SERIAL or use it as a command line argument to Android development tools.
Use the --local-instance flag to run the virtual device on the machine running the command. This is not recommended for Google developers.
The Android CI infrastructure triggers new builds of the compiler either toolchain/android_rust or toolchain/rustc are updated. This compiler is then used to invoke test_compiler.py. As such, if the Android source hasn't been updated for the new compiler the new compiler source will fail to pass presubmit checks.
Because of this dependency you will need to wait until all CLs updating the Android source code for the new compiler have been merged. Once that has occurred you can upload the CLs containing the recontextualized patches, any required build system changes, and the new Rust source.
These CLs will all need to be tested together, which requires that they are members of the same topic in Gerrit. The topics can be set manually or you can pass the -t flag to repo to use the Git branch name as the topic.
If repo complains about private keys you can pass the -o nokeycheck option to skip they key check.
Android's Rust prebuilt toolchains are compiled using PGO and BOLT. The profiles for these optimizations are generated by the build targets in the git_main-rust-profiling branch. After uploading the source for a new version of Rust you will need to wait for two builds to finish: 1) the first successful build of stage5-merge in git_main-rust-profiling and 2) the first successful build of rust-linux_pgo_bolt after the completion of the pervious stage5-merge build. This will ensure that the new prebuilts are compiled with the most recent prebuilts.
BOLT profiles are specific to a particular file name. Because Rust libraries include a checksum in their file names BOLT profiles from one version of the toolchain will fail to apply to a new version. This will result in at least one broken rust-linux_pgo_bolt built after an update, as build chaining will start an initial build of the target using the profiles from the last known good build of stage5-merge.
Once profiles have been generated and a new toolchain built the artifacts need to be fetched, unpacked, and checked in. In addition, the Rust toolchain and Soong build systems need to be updated with the new compiler version number. The update_prebuilts.py script handles each of these steps, creating commits in the build/soong, prebuilts/rust, and toolchain/android_rust projects.
For a normal release the following command can be used:
$ ./tools/update_prebuilts.py -i <issue number> <rust version>
If the automatic last known good build detection fails you can use the --bid flag to manually select a build ID to fetch artifacts from.
Upload the three commits crated by the tool and place them in a single topic (e.g. rust-update-source-1.78.0.) before proceeding with the review process.
Once the new prebuilts have been submitted send an email to the [email protected] mailing list. There is no standard template for these announcements, but including a link to the issue page or the release notes can be helpful.
Create an issue for the next Rust toolchain release and assign it to the next engineer in the release rotation.
It is a goal of the Android Rust team to only keep in-use toolchains in the prebuilts directory as any previous toolchains can be recovered using Git. After an update it can take downstream teams several days to weeks to complete their own update processes. During this time a CL can be created to delete the previous version of the toolchain but it can not be submitted until all presubmit tests pass and it has been confirmed that downstream teams have updated their build systems as necessary.
If a downstream team takes longer than a week to complete their update process it might be appropriate, after discussion with the client team, to pin their manifest to the current commit before proceeding with removing the old perbuilts.
Example commit message:
Removing Rust $OLD_RUST_VERSION prebuilts Bug: <bug ID> Test: m rust
Prebuilts exist for the following host architectures:
darwin-x86linux-x86linux-musl-x86windows-x86Once the update is complete notes should be added to the update log detailing any issues encountered or changes made.
The Rust project will publish beta releases in the weeks and days leading up to a major release. It can often be beneficial to use these beta releases for testing and to begin the update process.
The fetch_source.py tool can be used to download the beta archives using the --beta flag and then built normally with build.py. While the beta source can not be submitted, build system changes and patch updates can be prepared. If a beta compiler is successfully built and produces errors when compiling the Android codebase fixes for these errors can be submitted immediately, as long as the current compiler considers them valid.
Making changes based on a beta release too early can be detrimental to the release process, however. Patches can be reverted from the beta branch leading to large, last minute source code changes or feature removals. This could result in a need to re-recontextualize the patches or undo changes to the build system. It is therefore not recommended to test on a beta source archive more than 1 or 2 weeks in advance of a release. Lastly, it is often better to avoid beta testing in the days immediately prior to an official release and instead wait for the new version of the toolchain source to become available.