The prepare_bazel_test_env script is a proof-of-concept script to create a simulated Bazel environment within the Android source tree using targets build by the Soong build system.
The script currently support generation of a Bazel environment to run the following Soong test modules:
//platform_testing/tests/example/native:hello_world_test//platform_testing/tests/example/jarhosttest: HelloWorldHostTest//platform_testing/tests/example/instrumentation: HelloWorldTestsAdditionally, the system supports running the Tradefed console directly through the //tools/tradefederation/core:tradefed target.
There are three actions that the script can perform, which are supplied as the first argument to the script: generate, sync, and clean, discussed below. All the commands below are written with the -v flag for verbose output, however this can be safely removed if desired.
Command Line: bazel run //build/pesto/experiments/prepare_bazel_test_env -- -v generate
The generate command builds the required targets for a Bazel environment via Soong, then stages this environment and associated dependencies at out/pesto-environment/ .
The generate command performs the following actions:
data/templates directory) via Soongout/pesto-environment/prebuilts which contains symlinks to the Android Build environment provided directories (ANDROID_HOST_OUT, ANDROID_HOST_OUT_TESTCASES, ANDROID_PRODUCT_OUT, ANDROID_TARGET_OUT_TESTCASES), and will later be linked to by a .soong_prebuilts symlink that is placed adjacent to generated BUILD files.out/pesto-environment/gen using the packaged environment to determine the locations of files, which are placed in locations relative to the source tree root. For example, the out/pesto-environment/gen/tools/tradefederation/core/BUILD.bazel corresponds to a file that will eventually live at tools/tradefederation/core/BUILD.bazel..soong_prebuilts symlink that links to the aforementioned prebuilts directory.After generation, the environment create can serve as a standalone Bazel environment, or can be synced to the source tree using the sync command, discussed below.
Command Line: bazel run //build/pesto/experiments/prepare_bazel_test_env -- -v sync
The sync command scans the staging directory at out/pesto-environment/gen for all files and then creates symlinks in the source tree that point at these files, additionally each synced BUILD file is provided local access to the prebuilts directory through a .soong_prebuilts symlink.
The sync command performs the following actions:
out/pesto-environment/gen directory and create a symlink in the source tree to each file at the proper location, overwriting file in the tree if it exists. For example, the sync action would create a symlink at packages/modules/adb/BUILD.bazel that links to out/pesto-environment/gen/packages/modules/adb/BUILD.bazel..soong_prebuilts directory in every location in the source treeAfter synchronization, the Bazel environment has been merged with the source tree and can be used directly from within the source tree. Additionally, after synchronization, subsequent calls to the generate command will propogate automatically to the source tree.
Command Line: bazel run //build/pesto/experiments/prepare_bazel_test_env -- -v clean
The clean command removes all files that have been created in the tree, and also cleans up the environment directory at out/pesto-environment .
The clean command performs the following actions:
.soong_prebuilts directory for the source tree.out/pesto-environment directory.After clean, the environment should be removed from the tree. However, as some files may have been overwritten, certain repositories may need to be reset. The build/bazel/rules/BUILD.bazel file is a notable example that needs to be manually reset.
Adding support for an additional module depends on the type of module to be added. Each is discussed below. Of note, all files should be added in the templates directory and end in the .template file extension unless the file is a static, non-Bazel file.
For targets needing a new test rule, if the test is a Tradefed run test, use the existing test rules as a template, otherwise a custom Bazel rule can be added.
An example rule is provided at templates/build/bazel/rules/cc_test.bzl.template . For a new rule that leverages Tradefed, use the above rule as an example of how to package dependencies and test artifacts into the runfiles for the test. For each Tradefed rule, the required dependencies should be included as private attributes, for use by the rule implementation.
cc_test = rule(
_cc_test_impl,
attrs = {{
"_adb": attr.label(
default = Label("//packages/modules/adb"),
allow_single_file = True,
),
"_tradefed_launcher": attr.label(
default = Label("//tools/tradefederation/core:atest_tradefed"),
allow_single_file = True,
),
"_tradefed_script_help": attr.label(
default = Label("//tools/tradefederation/core:atest_script_help"),
),
"_tradefed_jars": attr.label(
default = Label("//tools/tradefederation/core:tradefed_lib"),
),
"_template": attr.label(
default = Label(
"//build/bazel/rules:tf_test_executable.sh.template",
),
allow_single_file = True,
),
"_launcher": attr.label(default = Label("//build/bazel/rules:cc_tf_test_launcher")),
"deps": attr.label_list(allow_files = True),
}},
executable = True,
test = True,
)
Additionally, the Soong produced artifacts should also be included as runfiles so they can be seen during Tradefed execution. Including a target as runfiles here, ensures that the target shows up during execution. Furthermore, it ensures that Bazel knows when to rebuild/rerun a test when artifacts change.
runfiles = ctx.runfiles(
files = ctx.files._launcher,
transitive_files = depset(
transitive = [
depset(ctx.files.deps),
depset(ctx.files._adb),
depset(ctx.files._tradefed_launcher),
depset(ctx.files._tradefed_script_help),
depset(ctx.files._tradefed_jars),
],
),
)
Finally, the test rule should use the tf_test_executable.sh file as its executable and provide the proper substitutions to this file, which can be seen in the above example rule. The tf_test_executable.sh handles setting important variables needed by Tradefed before test execution in a Bazel environment.
ctx.actions.expand_template(
template = ctx.file._template,
output = script,
substitutions = {{
"{{module_name}}": ctx.label.name,
"{{module_path}}": ctx.label.package,
"{{tradefed_launcher_module_path}}": ctx.attr._tradefed_launcher.label.package,
"{{tradefed_jars_module_path}}": ctx.attr._tradefed_jars.label.package,
"{{path_additions}}": ctx.attr._adb.label.package,
"{{launcher_path}}": "{{}}/{{}}".format(
ctx.attr._launcher.label.package,
ctx.attr._launcher.label.name,
),
}},
is_executable = True,
)
After the rule logic is added, follow the steps in the below section for how to add a test target leveraging the newly added rule.
For targets where the test rule is already provided (i.e. cc_test ), adding a new test module requires only adding a new BUILD file (with associated import logic).
All added BUILD templates should end in .template to ensure Bazel does not see these files as part of a package, and should contain the required Soong targets, defined like the following:
# SOONG_TARGET:CtsAppTestCases # SOONG_TARGET:org.apache.http.legacy
Refer to the data/templates/platform_testing/tests/example/native/BUILD.bazel.template as an example of how to import files into the Bazel environment using a genrule. The genrule logic in the example template is used to combine files from multiple locations into a single target and strip the Soong paths from the files imported to Bazel. The below genrule serves as a foundation and copies all files from srcs to the files listed in outs.
genrule(name="hello_world_test_prebuilt",
srcs=_LIB_SRCS + _TESTCASE_HOST_SRCS + _TESTCASE_DEVICE_SRCS,
outs=_LIB_OUTS + _TESTCASE_HOST_OUTS + _TESTCASE_DEVICE_OUTS,
cmd="""
src_files=($(SRCS))
out_files=($(OUTS))
for i in "$${{!src_files[@]}}"
do
src_file=$${{src_files[$$i]}}
out_file=$${{out_files[$$i]}}
mkdir -p $$(dirname $$src_file)
cp $$src_file $$out_file
done
""")
When referring to files imported from Bazel, use the {prebuilts_dir_name} substitution variable instead of referring to the .soong_prebuilts directory directly since this may change.
_LIB_SRCS = glob([
"{prebuilts_dir_name}/host/lib/**/*",
"{prebuilts_dir_name}/host/lib64/**/*"
])
Then, the newly imported module can be referenced from an existing test rule as a dependency, as is done in the example template. Ensure that the test rule is imported, such as in the example file:
load("//build/bazel/rules:cc_test.bzl", "cc_test")
.
.
.
cc_test(name="hello_world_test", deps=[":hello_world_test_prebuilt"])