doc/framework_testing.md - platform/system/chre - Git at Google

 # Testing the CHRE Framework

 [TOC]

 The CHRE framework can be tested at various levels to ensure that
 components/modules components are working correctly and API contracts are being
 met. Below are some examples of what the team currently does to verify new
 changes.

 ## Unit tests

 ### Tests run on a host machine

 Currently, unit tests exist for various core components and utilities capable
 of running on a Linux host machine. Since
 platform-specific components likely aren’t compilable/available on a host
 machine, only components that are OS independent can be tested via this path.

 In order to write new tests, add a test source file under the test directory in
 the appropriate subdirectory. e.g. `util/tests`. Then, add the file to the
 `GOOGLETEST_SRCS` variable in the appropriate .mk file for that subdir,
 `util/util.mk` for example.

 Unit tests can be built and executed using `run_tests.sh`.


 ### On-device unit tests

 #### Background

 The framework aims to provide an environment to test CHRE (and its users) code
 on-device, using [Pigweed's][PW_URL] Unit Test [Framework][PW_UT_URL]. Test
 instantiations are syntactically identical to [Googletest][GT_URL], so
 modifications to on-host unit tests to run on-device are easier.

 CHRE recommends running the same host-side gtests on-device using this
 framework, to catch subtle bugs. For example, the target CPU may raise an
 exception on unaligned access, when the same code would run without any
 problems on a local x86-based machine.

 #### Use Cases

 Example use cases of the framework include:

 * In continuous integration frameworks with device farms
 * As a superior alternative to logging and/or flag based debugging to quickly test a feature
 * As a modular set of tests to test feature or peripheral functioning (eg: a system timer implementation) during device bringup.

 ###### Note

 One key difference is to run the tests via a call to `chre::runAllTests` in
 _chre/test/common/unit_test.h_, which basically wraps the gtest `RUN_ALL_TESTS`
 macro, and implements CHRE specific event handlers for Pigweed's UT Framework.

 #### Running Tests

 Under the current incarnation of the CHRE Unit Test Framework, the following
 steps need to be taken to run the on-device tests:
 * Set to true and export an environment variable called `CHRE_ON_DEVICE_TESTS_ENABLED`
 from your Makefile invocation before CHRE is built.
   * Ensure that this flag is not always set to avoid codesize bloat.
 * Append your test source file to `COMMON_SRCS` either in _test/test.mk_ or in
 your own Makefile.
 * Call `chre::runAllTests` from somewhere in your code.

 ##### Sample code

 In _math_test.cc_
 ```cpp
 #include <gtest/gtest.h>

 TEST(MyMath, Add) {
   int x = 1, y = 2;
   int result = myAdd(x, y);
   EXPECT_EQ(result, 3);
 }
 ```

 In _some_source.cc_
 ```cpp
 #include "chre/test/common/unit_test.h"

 void utEntryFunc() {
   chre::runAllTests();
 }
 ```

 #### Caveats

 Some advanced features of gtest (SCOPED_TRACE, etc.) are unsupported by Pigweed.

 #### Compatibility

 The framework has been tested with Pigweed's git revision ee460238b8a7ec0a6b4f61fe7e67a12231db6d3e.

 ## PAL implementation tests

 PAL implementation tests verify implementations of PAL interfaces adhere to the
 requirements of that interface, and are intended primarily to support
 development of PAL implementations, typically done by a chip vendor partner.
 Additional setup may be required to integrate with the PAL under test and supply
 necessary dependencies. The source code is in the files under `pal/tests/src`
 and follows the naming scheme `*_pal_impl_test.cc`.

 In order to run PAL tests, run: `run_pal_impl_tests.sh`. Note that there are
 also PAL unit tests in the same directory. The unit tests are added to the
 `GOOGLETEST_SRCS` target while PAL tests are added to the
 `GOOGLETEST_PAL_IMPL_SRCS` target.

 ## FeatureWorld nanoapps

 Located under the `apps/` directory, FeatureWorld nanoapps interact with the set
 of CHRE APIs that they are named after, and can be useful during framework
 development for manual verification of a feature area. For example, SensorWorld
 attempts to samples from sensors and outputs to the log. It also offers a
 break-it mode that randomly changes which sensors are being sampled in an
 attempt to point out stress points in the framework or platform implementation.

 These apps are usually built into the CHRE framework binary as static nanoapps
 to facilitate easy development. See the Deploying Nanoapps section for more
 information on static nanoapps.

 ## CHQTS

 The Context Hub Qualification Test Suite (CHQTS) tests perform end-to-end
 validation of a CHRE implementation, by using the Java APIs in Android to load
 and interact with test nanoapps which then exercise the CHRE API. While this
 code is nominally integrated in another test suite, the source code is available
 under `java/test/chqts/` for the Java side code and `apps/test/chqts/` for the
 CHQTS-only nanoapp code and `apps/test/common/` for the nanoapp code shared by
 CHQTS and other test suites.

 [PW_URL]: https://pigweed.dev
 [PW_UT_URL]: https://pigweed.googlesource.com/pigweed/pigweed/+/refs/heads/master/pw_unit_test
 [GT_URL]: https://github.com/google/googletest
	# Testing the CHRE Framework

	[TOC]

	The CHRE framework can be tested at various levels to ensure that
	components/modules components are working correctly and API contracts are being
	met. Below are some examples of what the team currently does to verify new
	changes.

	## Unit tests

	### Tests run on a host machine

	Currently, unit tests exist for various core components and utilities capable
	of running on a Linux host machine. Since
	platform-specific components likely aren’t compilable/available on a host
	machine, only components that are OS independent can be tested via this path.

	In order to write new tests, add a test source file under the test directory in
	the appropriate subdirectory. e.g. `util/tests`. Then, add the file to the
	`GOOGLETEST_SRCS` variable in the appropriate .mk file for that subdir,
	`util/util.mk` for example.

	Unit tests can be built and executed using `run_tests.sh`.


	### On-device unit tests

	#### Background

	The framework aims to provide an environment to test CHRE (and its users) code
	on-device, using [Pigweed's][PW_URL] Unit Test [Framework][PW_UT_URL]. Test
	instantiations are syntactically identical to [Googletest][GT_URL], so
	modifications to on-host unit tests to run on-device are easier.

	CHRE recommends running the same host-side gtests on-device using this
	framework, to catch subtle bugs. For example, the target CPU may raise an
	exception on unaligned access, when the same code would run without any
	problems on a local x86-based machine.

	#### Use Cases

	Example use cases of the framework include:

	* In continuous integration frameworks with device farms
	* As a superior alternative to logging and/or flag based debugging to quickly test a feature
	* As a modular set of tests to test feature or peripheral functioning (eg: a system timer implementation) during device bringup.

	###### Note

	One key difference is to run the tests via a call to `chre::runAllTests` in
	_chre/test/common/unit_test.h_, which basically wraps the gtest `RUN_ALL_TESTS`
	macro, and implements CHRE specific event handlers for Pigweed's UT Framework.

	#### Running Tests

	Under the current incarnation of the CHRE Unit Test Framework, the following
	steps need to be taken to run the on-device tests:
	* Set to true and export an environment variable called `CHRE_ON_DEVICE_TESTS_ENABLED`
	from your Makefile invocation before CHRE is built.
	* Ensure that this flag is not always set to avoid codesize bloat.
	* Append your test source file to `COMMON_SRCS` either in _test/test.mk_ or in
	your own Makefile.
	* Call `chre::runAllTests` from somewhere in your code.

	##### Sample code

	In _math_test.cc_
	```cpp
	#include <gtest/gtest.h>

	TEST(MyMath, Add) {
	int x = 1, y = 2;
	int result = myAdd(x, y);
	EXPECT_EQ(result, 3);
	}
	```

	In _some_source.cc_
	```cpp
	#include "chre/test/common/unit_test.h"

	void utEntryFunc() {
	chre::runAllTests();
	}
	```

	#### Caveats

	Some advanced features of gtest (SCOPED_TRACE, etc.) are unsupported by Pigweed.

	#### Compatibility

	The framework has been tested with Pigweed's git revision ee460238b8a7ec0a6b4f61fe7e67a12231db6d3e.

	## PAL implementation tests

	PAL implementation tests verify implementations of PAL interfaces adhere to the
	requirements of that interface, and are intended primarily to support
	development of PAL implementations, typically done by a chip vendor partner.
	Additional setup may be required to integrate with the PAL under test and supply
	necessary dependencies. The source code is in the files under `pal/tests/src`
	and follows the naming scheme `*_pal_impl_test.cc`.

	In order to run PAL tests, run: `run_pal_impl_tests.sh`. Note that there are
	also PAL unit tests in the same directory. The unit tests are added to the
	`GOOGLETEST_SRCS` target while PAL tests are added to the
	`GOOGLETEST_PAL_IMPL_SRCS` target.

	## FeatureWorld nanoapps

	Located under the `apps/` directory, FeatureWorld nanoapps interact with the set
	of CHRE APIs that they are named after, and can be useful during framework
	development for manual verification of a feature area. For example, SensorWorld
	attempts to samples from sensors and outputs to the log. It also offers a
	break-it mode that randomly changes which sensors are being sampled in an
	attempt to point out stress points in the framework or platform implementation.

	These apps are usually built into the CHRE framework binary as static nanoapps
	to facilitate easy development. See the Deploying Nanoapps section for more
	information on static nanoapps.

	## CHQTS

	The Context Hub Qualification Test Suite (CHQTS) tests perform end-to-end
	validation of a CHRE implementation, by using the Java APIs in Android to load
	and interact with test nanoapps which then exercise the CHRE API. While this
	code is nominally integrated in another test suite, the source code is available
	under `java/test/chqts/` for the Java side code and `apps/test/chqts/` for the
	CHQTS-only nanoapp code and `apps/test/common/` for the nanoapp code shared by
	CHQTS and other test suites.

	[PW_URL]: https://pigweed.dev
	[PW_UT_URL]: https://pigweed.googlesource.com/pigweed/pigweed/+/refs/heads/master/pw_unit_test
	[GT_URL]: https://github.com/google/googletest