From 1b0a8721596f09f97e519a62181c09e83c47b33e Mon Sep 17 00:00:00 2001 From: Daniel Micay Date: Fri, 31 May 2019 00:16:27 -0400 Subject: [PATCH] add section on CTS --- static/build.html | 83 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/static/build.html b/static/build.html index 34e4636b..2073054d 100644 --- a/static/build.html +++ b/static/build.html @@ -426,6 +426,89 @@ cd ../..

The Auditor app is simply built from the latest upstream tag and bundled as an apk into external/ repositories. There are no modifications to it for GrapheneOS.

+ +

+ Testing + +

+

+ Compatibility Test Suite + +

+ +

+ Download + +

+ +

Testing with the Compatibility Test Suite (CTS) can be done by either building the + test suite from source or using the official releases.

+

Official releases of the CTS can be downloaded from + the Compatibility + Suite Downloads page. You should download the CTS for the relevant release + (Android 9) and architecture (ARM). There's a separate zip for the main CTS, the + manual portion (CTS Verifier) and the CTS for Instant Apps. The latest release of the + CTS Media Files also needs to be downloaded from that section.

+ +

+ Setup + +

+

You'll need a device attached to your computer with ADB enabled along with the + Android SDK installed. The build-tools and platform-tools packages need to be + installed and the binaries need to be added to your PATH. For example, with the SDK + located at /home/username:

+ 
export ANDROID_HOME="$HOME/sdk"
+export PATH="$PATH:$HOME/sdk/tools:$HOME/sdk/tools/bin:$HOME/sdk/platform-tools:$HOME/sdk/build-tools/27.0.3:$HOME/sdk/ndk-bundle"
+

Copy media onto the device:

+ 
cd android-cts-media-1.4
+./copy_images.sh
+./copy_media.sh
+ +

You also need to do some basic setup for the device. It's possible for changes from + a baseline install to cause interference, so it can be a good idea to factory reset + the device if assorted changes have been made. The device needs to be running a user + build for the security model to be fully intact in order to pass all the security + tests. A userdebug build is expected to fail some of the tests. GrapheneOS also makes + various changes intentionally deviating from the requirements expected by the CTS, so + there will always be some expected failures. A few of the tests are also known to be + quite flaky or broken even with the stock OS and/or AOSP. These will be documented + here at some point.

+ + + +

+ Run modules + +

+ +

Run the test harness:

+ 
./android-cts/tools/cts-tradefed
+

Note that _JAVA_OPTIONS being set will break the version detection.

+

To obtain a list of CTS modules:

+ 
list modules
+

To run a specific module and avoid wasting time capturing device information:

+ 
run cts --skip-device-info --module CtsModuleName
+

To speed up initialization after running some initial tests:

+ 
run cts --skip-device-info --skip-preconditions --module CtsModuleName
+

It's possible to run the whole standard CTS plan with a single command, but running + specific modules is recommended, especially if you don't have everything set up for + the entire test suite.