941 lines
51 KiB
HTML
941 lines
51 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en" prefix="og: http://ogp.me/ns#">
|
||
<head>
|
||
<meta charset="utf-8"/>
|
||
<title>Build | GrapheneOS</title>
|
||
<meta name="description" content="Building instructions for GrapheneOS, a security and privacy focused mobile OS with Android app compatibility."/>
|
||
<meta name="theme-color" content="#212121"/>
|
||
<meta name="msapplication-TileColor" content="#ffffff"/>
|
||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||
<meta name="twitter:site" content="@GrapheneOS"/>
|
||
<meta name="twitter:creator" content="@GrapheneOS"/>
|
||
<meta property="og:title" content="GrapheneOS build documentation"/>
|
||
<meta property="og:description" content="Building instructions for GrapheneOS, a security and privacy focused mobile OS with Android app compatibility."/>
|
||
<meta property="og:type" content="website"/>
|
||
<meta property="og:image" content="https://grapheneos.org/opengraph.png"/>
|
||
<meta property="og:image:width" content="512"/>
|
||
<meta property="og:image:height" content="512"/>
|
||
<meta property="og:image:alt" content="GrapheneOS logo"/>
|
||
<meta property="og:url" content="https://grapheneos.org/build"/>
|
||
<meta property="og:site_name" content="GrapheneOS"/>
|
||
<link rel="icon" type="image/vnd.microsoft.icon" href="/favicon.ico"/>
|
||
<link rel="mask-icon" href="/mask-icon.svg" color="#1a1a1a"/>
|
||
<link rel="stylesheet" href="/grapheneos.css?17"/>
|
||
<link rel="manifest" href="/manifest.webmanifest"/>
|
||
<link rel="canonical" href="https://grapheneos.org/build"/>
|
||
<script type="module" src="/redirect.js?6"></script>
|
||
</head>
|
||
<body>
|
||
<nav>
|
||
<ul>
|
||
<li><a href="/">GrapheneOS</a></li>
|
||
<li><a href="/install">Install</a></li>
|
||
<li class="active"><a href="/build">Build</a></li>
|
||
<li><a href="/usage">Usage</a></li>
|
||
<li><a href="/faq">FAQ</a></li>
|
||
<li><a href="/releases">Releases</a></li>
|
||
<li><a href="/source">Source</a></li>
|
||
<li><a href="/donate">Donate</a></li>
|
||
<li><a href="/contact">Contact</a></li>
|
||
</ul>
|
||
</nav>
|
||
<div id="content">
|
||
<h1 id="build">
|
||
<a href="#build">Build</a>
|
||
</h1>
|
||
|
||
<p>This is a guide on building, modifying and contributing to GrapheneOS as a
|
||
developer.</p>
|
||
|
||
<h2 id="table-of-contents">
|
||
<a href="#table-of-contents">Table of contents</a>
|
||
</h2>
|
||
<ul>
|
||
<li><a href="#build-targets">Build targets</a></li>
|
||
<li><a href="#build-dependencies">Build dependencies</a></li>
|
||
<li><a href="#downloading-source-code">Downloading source code</a></li>
|
||
<li><a href="#development-branch">Development branch</a></li>
|
||
<li><a href="#stable-release">Stable release</a></li>
|
||
<li><a href="#updating-and-switching-branches-or-tags">Updating and switching branches or tags</a></li>
|
||
<li><a href="#kernel">Kernel</a></li>
|
||
<li><a href="#setting-up-the-os-build-environment">Setting up the OS build environment</a></li>
|
||
<li><a href="#reproducible-builds">Reproducible builds</a></li>
|
||
<li><a href="#extracting-vendor-files-for-pixel-devices">Extracting vendor files for Pixel devices</a></li>
|
||
<li><a href="#building">Building</a></li>
|
||
<li><a href="#faster-builds-for-development-use-only">Faster builds for development use only</a></li>
|
||
<li>
|
||
<a href="#generating-release-signing-keys">Generating release signing keys</a>
|
||
<ul>
|
||
<li><a href="#encrypting-keys">Encrypting keys</a></li>
|
||
<li><a href="#enabling-updatable-apex-components">Enabling updatable APEX components</a></li>
|
||
</ul>
|
||
</li>
|
||
<li>
|
||
<a href="#generating-signed-factory-images-and-full-update-packages">Generating signed factory images and full update packages</a>
|
||
<ul>
|
||
<li><a href="#generating-delta-updates">Generating delta updates</a></li>
|
||
</ul>
|
||
</li>
|
||
<li>
|
||
<a href="#prebuilt-code">Prebuilt code</a>
|
||
<ul>
|
||
<li><a href="#browser-and-webview">Browser and WebView</a></li>
|
||
<li><a href="#prebuilt-apps">Prebuilt apps</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#standalone-sdk">Standalone SDK</a></li>
|
||
<li>
|
||
<a href="#testing">Testing</a>
|
||
<ul>
|
||
<li><a href="#emulator">Emulator</a></li>
|
||
<li>
|
||
<a href="#compatibility-test-suite">Compatibility Test Suite</a>
|
||
<ul>
|
||
<li><a href="#compatibility-test-suite-download">Download</a></li>
|
||
<li><a href="#compatibility-test-suite-setup">Setup</a></li>
|
||
<li><a href="#compatibility-test-suite-run-modules">Run modules</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li>
|
||
<a href="#development-guidelines">Development guidelines</a>
|
||
<ul>
|
||
<li><a href="#programming-languages">Programming languages</a></li>
|
||
<li><a href="#code-style">Code style</a></li>
|
||
<li><a href="#library-usage">Library usage</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
<h2 id="build-targets">
|
||
<a href="#build-targets">Build targets</a>
|
||
</h2>
|
||
|
||
<p>Smartphone targets:</p>
|
||
|
||
<ul>
|
||
<li>aosp_taimen (Pixel 2 XL) - legacy</li>
|
||
<li>aosp_walleye (Pixel 2) - legacy</li>
|
||
<li>aosp_crosshatch (Pixel 3 XL)</li>
|
||
<li>aosp_blueline (Pixel 3)</li>
|
||
<li>aosp_bonito (Pixel 3a XL)</li>
|
||
<li>aosp_sargo (Pixel 3a)</li>
|
||
</ul>
|
||
|
||
<p>These are all fully supported production-ready targets supporting all the baseline
|
||
security features and receiving full monthly security updates covering all firmware,
|
||
kernel drivers, driver libraries / services and other device-specific code. A fully
|
||
signed user build for these devices is a proper GrapheneOS release. Newer generation
|
||
devices have stronger hardware / firmware security and hardware-based OS security
|
||
features and are better development devices for that reason. It's not possible to work
|
||
on everything via past generation devices. The best development devices are the Pixel
|
||
3, Pixel 3 XL, Pixel 3a and Pixel 3a XL.</p>
|
||
|
||
<p>Generic targets:</p>
|
||
|
||
<ul>
|
||
<li>aosp_arm</li>
|
||
<li>aosp_arm64</li>
|
||
<li>aosp_mips</li>
|
||
<li>aosp_mips64</li>
|
||
<li>aosp_x86</li>
|
||
<li>aosp_x86_64</li>
|
||
</ul>
|
||
|
||
<p>These generic targets can be used with the emulator along with many smartphones,
|
||
tablets and other devices. These targets don't receive full monthly security updates,
|
||
don't offer all of the baseline security features and are intended for development
|
||
usage.</p>
|
||
|
||
<p>Providing proper support for a device or generic device family requires providing
|
||
an up-to-date kernel and device support code including driver libraries, firmware and
|
||
device SELinux policy extensions. Other than some special cases like the emulator, the
|
||
generic targets rely on the device support code present on the device. Shipping all of
|
||
this is necessary for full security updates and is tied to enabling verified boot /
|
||
attestation. Pixel targets have a lot of device-specific hardening in the AOSP base
|
||
along with some in GrapheneOS which needs to be ported over too. For example, various
|
||
security features in the kernel including type-based Control Flow Integrity (CFI) and
|
||
the shadow call stack are currently specific to the kernels for these devices.</p>
|
||
|
||
<p>SDK emulator targets:</p>
|
||
|
||
<ul>
|
||
<li>sdk_phone_armv7</li>
|
||
<li>sdk_phone_arm64</li>
|
||
<li>sdk_phone_mips</li>
|
||
<li>sdk_phone_mips64</li>
|
||
<li>sdk_phone_x86</li>
|
||
<li>sdk_phone_x86_64</li>
|
||
</ul>
|
||
|
||
<p>These are extended versions of the generic targets with extra components for the
|
||
SDK. These targets don't receive full monthly security updates, don't provide all of
|
||
the baseline security features and are intended for development usage.</p>
|
||
|
||
<p>Board targets:</p>
|
||
|
||
<ul>
|
||
<li>hikey - legacy</li>
|
||
<li>hikey960</li>
|
||
</ul>
|
||
|
||
<p>The hikey and hikey960 targets are not actively tested and have unresolved upstream
|
||
memory corruption bugs uncovered by GrapheneOS security features. It boots, but there
|
||
are major issues with the graphics drivers among other problems. The intention is to
|
||
support them, but the necessary time has not yet been dedicated to it. These targets
|
||
don't receive full monthly security updates, don't provide all of the baseline
|
||
security features and are intended for development usage.</p>
|
||
|
||
<h2 id="build-dependencies">
|
||
<a href="#build-dependencies">Build dependencies</a>
|
||
</h2>
|
||
<ul>
|
||
<li>x86_64 Linux build environment (macOS is not supported, unlike AOSP which
|
||
partially supports it)</li>
|
||
<li>Android Open Source Project build dependencies</li>
|
||
<li>Linux kernel build dependencies</li>
|
||
<li>16GiB of memory or more</li>
|
||
<li>300GiB of free storage space</li>
|
||
</ul>
|
||
|
||
<h2 id="downloading-source-code">
|
||
<a href="#downloading-source-code">Downloading source code</a>
|
||
</h2>
|
||
|
||
<p>Since this is syncing the sources for the entire operating system and application
|
||
layer, it will use a lot of bandwidth and storage space.</p>
|
||
|
||
<p>You likely want to use the most recent stable tag, not the development branch, even
|
||
for developing a feature. It's easier to port between stable tags that are known to
|
||
work properly than dealing with a moving target.</p>
|
||
|
||
<h2 id="development-branch">
|
||
<a href="#development-branch">Development branch</a>
|
||
</h2>
|
||
|
||
<p>The <code>10</code> branch is the only active development branch for GrapheneOS
|
||
development. Older branches are no longer maintained. It is currently used for all
|
||
officially supported devices and should be used for the basis of ports to other
|
||
devices. Occasionally, some devices may be supported through device support branches
|
||
to avoid impacting other devices with changes needed to support them.</p>
|
||
|
||
<pre>mkdir grapheneos-10
|
||
cd grapheneos-10
|
||
repo init -u https://github.com/GrapheneOS/platform_manifest.git -b 10
|
||
repo sync -j32</pre>
|
||
|
||
<p>If your network is unreliable and <code>repo sync</code> fails, you can run the
|
||
<code>repo sync</code> command again as many times as needed for it to fully
|
||
succeed.</p>
|
||
|
||
<h2 id="stable-release">
|
||
<a href="#stable-release">Stable release</a>
|
||
</h2>
|
||
|
||
<p>Pick a specific build for a device from the <a href="/releases">releases page</a>
|
||
and download the source tree. Note that some devices use different Android Open Source
|
||
Project branches so they can end up with different tags. Make sure to use the correct
|
||
tag for a device. For devices without official support, use the latest tag for the
|
||
Pixel 3.</p>
|
||
|
||
<pre>mkdir grapheneos-TAG_NAME
|
||
cd grapheneos-TAG_NAME
|
||
repo init -u https://github.com/GrapheneOS/platform_manifest.git -b refs/tags/TAG_NAME</pre>
|
||
|
||
<p>Verify the manifest:</p>
|
||
|
||
<pre>gpg --recv-keys 65EEFE022108E2B708CBFCF7F9E712E59AF5F22A
|
||
gpg --recv-keys 4340D13570EF945E83810964E8AD3F819AB10E78
|
||
cd .repo/manifests
|
||
git verify-tag --raw $(git describe)
|
||
cd ../..</pre>
|
||
|
||
<p>Complete the source tree download:</p>
|
||
|
||
<pre>repo sync -j32</pre>
|
||
|
||
<p>Verify the source tree:</p>
|
||
|
||
<pre>repo forall -c 'git verify-tag --raw $(git describe)' || echo Verification failed!</pre>
|
||
|
||
<p>These instructions will be extended in the future to check the verify-tag
|
||
output.</p>
|
||
|
||
<p>Note that the repo command itself takes care of updating itself and uses gpg to
|
||
verify by default.</p>
|
||
|
||
<h2 id="updating-and-switching-branches-or-tags">
|
||
<a href="#updating-and-switching-branches-or-tags">Updating and switching branches or tags</a>
|
||
</h2>
|
||
|
||
<p>To update the source tree, run the <code>repo init</code> command again to select
|
||
the branch or tag and then run <code>repo sync -j32</code> again. You may need to add
|
||
<code>--force-sync</code> if a repository switched from one source to another,
|
||
such as when GrapheneOS forks an additional Android Open Source Project repository.
|
||
You don't need to start over to switch between different branches or tags. You may
|
||
need to run <code>repo init</code> again to continue down the same branch since
|
||
GrapheneOS only provides a stable history via tags.</p>
|
||
|
||
<h2 id="kernel">
|
||
<a href="#kernel">Kernel</a>
|
||
</h2>
|
||
|
||
<p>The kernel needs to be built in advance, since it uses a separate build system.</p>
|
||
|
||
<p>List of kernels corresponding to officially supported devices:</p>
|
||
|
||
<ul>
|
||
<li>Pixel 2, Pixel 2 XL: wahoo - separate taimen and walleye builds due to hardening</li>
|
||
<li>Pixel 3, Pixel 3 XL, Pixel 3a, Pixel 3a XL: crosshatch - separate crosshatch, blueline and bonito builds due to hardening</li>
|
||
</ul>
|
||
|
||
<p>As part of the hardening in GrapheneOS, it uses fully monolithic kernel builds with
|
||
dynamic kernel modules disabled. This improves the effectiveness of mitigations like
|
||
Control Flow Integrity benefiting from whole program analysis. It also reduces attack
|
||
surface and complexity including making the build system simpler. The kernel trees
|
||
marked as using a separate build above need to have the device variant passed to the
|
||
GrapheneOS kernel build script to select the device.</p>
|
||
|
||
<p>For the Pixel 3, Pixel 3 XL, Pixel 3a and Pixel 3a XL, the kernel repository uses
|
||
submodules for building in out-of-tree modules. You need to make sure the submodule
|
||
sources are updated before building. In the future, this should end up being handled
|
||
automatically by <code>repo</code>. There's no harm in running the submodule commands
|
||
for other devices as they will simply not do anything.</p>
|
||
|
||
<p>For example, to build the kernel for blueline:</p>
|
||
|
||
<pre>cd kernel/google/crosshatch
|
||
git submodule sync
|
||
git submodule update --init
|
||
./build.sh blueline</pre>
|
||
|
||
<p>The <code>kernel/google/wahoo</code> repository is for the Pixel 2 and Pixel 2 XL
|
||
and the <code>kernel/google/crosshatch</code> repository is for the Pixel 3, Pixel 3
|
||
XL, Pixel 3a and Pixel 3a XL.</p>
|
||
|
||
<h2 id="setting-up-the-os-build-environment">
|
||
<a href="#setting-up-the-os-build-environment">Setting up the OS build environment</a>
|
||
</h2>
|
||
|
||
<p>The build has to be done from bash as envsetup.sh is not compatible with other
|
||
shells like zsh.</p>
|
||
|
||
<p>Set up the build environment:</p>
|
||
|
||
<pre>source script/envsetup.sh</pre>
|
||
|
||
<p>Select the desired build target (<code>aosp_crosshatch</code> is the Pixel 3 XL):
|
||
|
||
<pre>choosecombo release aosp_crosshatch user</pre>
|
||
|
||
<p>For a development build, you may want to replace <code>user</code> with
|
||
<code>userdebug</code> in order to have better debugging support. Production builds
|
||
should be <code>user</code> builds as they are significantly more secure and don't
|
||
make additional performance sacrifices to improve debugging.</p>
|
||
|
||
<h2 id="reproducible-builds">
|
||
<a href="#reproducible-builds">Reproducible builds</a>
|
||
</h2>
|
||
|
||
<p>To reproduce a past build, you need to export <code>BUILD_DATETIME</code> and
|
||
<code>BUILD_NUMBER</code> to the values set for the past build. These can be obtained
|
||
from <code>out/build_date.txt</code> and <code>out/build_number.txt</code> in a build
|
||
output directory and the <code>ro.build.date.utc</code> and
|
||
<code>ro.build.version.incremental</code> properties which are also included in the
|
||
over-the-air zip metadata rather than just the OS itself.</p>
|
||
|
||
<p>The signing process for release builds is done after completing builds and replaces
|
||
the dm-verity trees, apk signatures, etc. and can only be reproduced with access to
|
||
the same private keys. If you want to compare to production builds signed with
|
||
different keys you need to stick to comparing everything other than the
|
||
signatures.</p>
|
||
|
||
<h2 id="extracting-vendor-files-for-pixel-devices">
|
||
<a href="#extracting-vendor-files-for-pixel-devices">Extracting vendor files for Pixel devices</a>
|
||
</h2>
|
||
|
||
<p>This section does not apply to devices where no extra vendor files are required (HiKey, HiKey 960, emulator, generic targets).</p>
|
||
|
||
<p>Many of these components are already open source, but not everything is set up to
|
||
be built by the Android Open Source Project build system. Switching to building these
|
||
components from source will be an incremental effort. In many cases, the vendor files
|
||
simply need to be ignored and AOSP will already provide them instead. Firmware cannot
|
||
generally be built from source even when sources are available, other than to verify
|
||
that the official builds match the sources, since it has signature verification (which
|
||
is an important part of the verified boot and attestation security model).</p>
|
||
|
||
<p>Extract the vendor files corresponding to the matching release:</p>
|
||
|
||
<pre>vendor/android-prepare-vendor/execute-all.sh -d DEVICE -b BUILD_ID -o vendor/android-prepare-vendor
|
||
mkdir -p vendor/google_devices
|
||
rm -rf vendor/google_devices/DEVICE
|
||
mv vendor/android-prepare-vendor/DEVICE/BUILD_ID/vendor/google_devices/* vendor/google_devices/</pre>
|
||
|
||
<p>Note that android-prepare-vendor is non-deterministic unless a timestamp parameter is
|
||
passed with <code>--timestamp</code> (seconds since Epoch).</p>
|
||
|
||
<h2 id="building">
|
||
<a href="#building">Building</a>
|
||
</h2>
|
||
|
||
<p>Incremental builds (i.e. starting from the old build) usually work for development
|
||
and are the normal way to develop changes. However, there are cases where changes are
|
||
not properly picked up by the build system. For production builds, you should remove
|
||
the remnants of any past builds before starting, particularly if there were
|
||
non-trivial changes:</p>
|
||
|
||
<pre>rm -r out</pre>
|
||
|
||
<p>Start the build process, with -j# used to set the number of parallel jobs to the
|
||
number of CPU threads. You also need 2-4GiB of memory per job, so reduce it based on
|
||
available memory if necessary:</p>
|
||
|
||
<pre>make target-files-package -j20</pre>
|
||
|
||
<p><strong>For an emulator build, always use the development build approach below.</strong></p>
|
||
|
||
<h2 id="faster-builds-for-development-use-only">
|
||
<a href="#faster-builds-for-development-use-only">Faster builds for development use only</a>
|
||
</h2>
|
||
|
||
<p>The normal production build process involves building a target files package to be
|
||
resigned with secure release keys and then converted into factory images and/or an
|
||
update zip via the sections below. If you have a dedicated development device with no
|
||
security requirements, you can save time by using the default make target, leaving the
|
||
bootloader unlocked and flashing the raw images that are signed with the default
|
||
public test keys:</p>
|
||
|
||
<pre>make -j20</pre>
|
||
|
||
<p>Technically, you could generate test key signed update packages. However, there's
|
||
no point of sideloading update packages when the bootloader is unlocked and there's no
|
||
value in a locked bootloader without signing the build using release keys, since
|
||
verified boot will be meaningless and the keys used to verify sideloaded updates are
|
||
also public. The only reason to use update packages or a locked bootloader without
|
||
signing the build with release keys would be testing that functionality and it makes a
|
||
lot more sense to test it with proper signing keys rather than the default public test
|
||
keys.</p>
|
||
|
||
<h2 id="generating-release-signing-keys">
|
||
<a href="#generating-release-signing-keys">Generating release signing keys</a>
|
||
</h2>
|
||
|
||
<p>Keys need to be generated for resigning completed builds from the publicly
|
||
available test keys. The keys must then be reused for subsequent builds and cannot be
|
||
changed without flashing the generated factory images again which will perform a
|
||
factory reset. Note that the keys are used for a lot more than simply verifying
|
||
updates and verified boot.</p>
|
||
|
||
<p>The sample certificate subject (<code>CN=GrapheneOS</code>) should be replaced with
|
||
your own information.</p>
|
||
|
||
<p>You should set a passphrase for the signing keys to keep them at rest until you
|
||
need to sign a release with them. The GrapheneOS scripts (<code>make_key</code> and
|
||
<code>encrypt_keys.sh</code>) encrypt the signing keys using scrypt for key derivation
|
||
and AES256 as the cipher. If you use swap, make sure it's encrypted, ideally with an
|
||
ephemeral key rather a persistent key to support hibernation. Even with an ephemeral
|
||
key, swap will reduce the security gained from encrypting the keys since it breaks the
|
||
guarantee that they become at rest as soon as the signing process is finished.
|
||
Consider disabling swap, at least during the signing process.</p>
|
||
|
||
<p>The encryption passphrase for all the keys generated for a device needs to
|
||
match for compatibility with the GrapheneOS scripts.</p>
|
||
|
||
<p>To generate keys for crosshatch (you should use unique keys per device
|
||
variant):</p>
|
||
|
||
<pre>mkdir -p keys/crosshatch
|
||
cd keys/crosshatch
|
||
../../development/tools/make_key releasekey '/CN=GrapheneOS/'
|
||
../../development/tools/make_key platform '/CN=GrapheneOS/'
|
||
../../development/tools/make_key shared '/CN=GrapheneOS/'
|
||
../../development/tools/make_key media '/CN=GrapheneOS/'
|
||
../../development/tools/make_key networkstack '/CN=GrapheneOS/'
|
||
openssl genrsa 2048 | openssl pkcs8 -topk8 -scrypt -out avb.pem
|
||
../../external/avb/avbtool extract_public_key --key avb.pem --output avb_pkmd.bin
|
||
cd ../..</pre>
|
||
|
||
<p>The <code>avb_pkmd.bin</code> file isn't needed for generating a signed release but
|
||
rather to set the public key used by the device to enforce verified boot.</p>
|
||
|
||
<p>Generate a signify key for signing factory images:</p>
|
||
|
||
<pre>signify -G -n -p keys/crosshatch/factory.pub -s keys/crosshatch/factory.sec</pre>
|
||
|
||
<p>Remove the <code>-n</code> switch to set a passphrase. The <code>signify</code>
|
||
tool doesn't provide a way to change the passphrase without generating a new key, so
|
||
this is currently handled separately from encrypting the other keys and there will be
|
||
a separate prompt for the passphrase. In the future, expect this to be handled by the
|
||
same scripts along with the expectation of it using the same passphrase as the other
|
||
keys.</p>
|
||
|
||
<h3 id="encrypting-keys">
|
||
<a href="#encrypting-keys">Encrypting keys</a>
|
||
</h3>
|
||
|
||
<p>You can (re-)encrypt your signing keys using the <code>encrypt_keys</code> script,
|
||
which will prompt for the old passphrase (if any) and new passphrase:</p>
|
||
|
||
<pre>script/encrypt_keys.sh keys/crosshatch</pre>
|
||
|
||
<p>The <code>script/decrypt_keys.sh</code> script can be used to remove encryption,
|
||
which is not recommended. The script exists primarily for internal usage to decrypt
|
||
the keys in tmpfs to perform signing.</p>
|
||
|
||
<h3 id="enabling-updatable-apex-components">
|
||
<a href="#enabling-updatable-apex-components">Enabling updatable APEX components</a>
|
||
</h3>
|
||
|
||
<p>GrapheneOS disables updatable APEX components for the officially supported devices
|
||
and targets inheriting from the mainline target, so APEX signing keys are not needed
|
||
and this section can be ignored for unmodified builds.</p>
|
||
|
||
<p>GrapheneOS uses the <code>TARGET_FLATTEN_APEX := true</code> format to include APEX
|
||
components as part of the base OS without supporting out-of-band updates.</p>
|
||
|
||
<p><strong>If you don't disable updatable APEX packages, you need to generate an APK and
|
||
AVB key for each APEX component and extend the GrapheneOS release.sh script to pass
|
||
the appropriate parameters to replace the APK and AVB keys for each APEX
|
||
component.</strong></p>
|
||
|
||
<p>APEX components that are not flattened are a signed APK (used to verify updates)
|
||
with an embedded filesystem image signed with an AVB key (for verified boot). Each
|
||
APEX package must have a unique set of keys. GrapheneOS has no use for these
|
||
out-of-band updates at this time and flattening APEX components avoids needing a bunch
|
||
of extra keys and complexity.</p>
|
||
|
||
<p>For now, consult the upstream documentation on generating these keys. It will be
|
||
covered here in the future.</p>
|
||
|
||
<h2 id="generating-signed-factory-images-and-full-update-packages">
|
||
<a href="#generating-signed-factory-images-and-full-update-packages">Generating signed factory images and full update packages</a>
|
||
</h2>
|
||
|
||
<p>Build the tool needed to generate A/B updates:</p>
|
||
|
||
<pre>make -j20 brillo_update_payload</pre>
|
||
|
||
<p>Generate a signed release build with the release.sh script:</p>
|
||
|
||
<pre>script/release.sh crosshatch</pre>
|
||
|
||
<p>The factory images and update package will be in
|
||
<code>out/release-crosshatch-$BUILD_NUMBER</code>. The update zip performs a full OS
|
||
installation so it can be used to update from any previous version. More efficient
|
||
incremental updates are used for official over-the-air GrapheneOS updates and can be
|
||
generated by keeping around past signed <code>target_files</code> zips and generating
|
||
incremental updates from those to the most recent signed <code>target_files</code>
|
||
zip.</p>
|
||
|
||
<h3 id="generating-delta-updates">
|
||
<a href="#generating-delta-updates">Generating delta updates</a>
|
||
</h3>
|
||
|
||
<p>Incremental updates shipping only the changes between two versions can be generated
|
||
as a much more efficient way of shipping updates than a full update package containing
|
||
the entire operating system. The GrapheneOS Updater app will automatically use a delta
|
||
update if one exists for going directly from the currently installed version to the
|
||
latest release. In order to generate a delta update, the original signed target files
|
||
package for both the source version and target version are needed. The
|
||
<code>script/generate_delta.sh</code> script provides a wrapper script for generating
|
||
delta updates by passing the device, source version build number and target version
|
||
build number. For example:</p>
|
||
|
||
<pre>script/generate_delta.sh crosshatch 2019.09.25.00 2019.10.07.21</pre>
|
||
|
||
<p>The script assumes that the releases are organized in the following directory
|
||
structure:</p>
|
||
|
||
<pre>releases
|
||
├── 2019.09.25.00
|
||
│ └── release-crosshatch-2019.09.25.00
|
||
│ ├── crosshatch-factory-2019.09.25.00.zip
|
||
│ ├── crosshatch-factory-2019.09.25.00.zip.sig
|
||
│ ├── crosshatch-img-2019.09.25.00.zip
|
||
│ ├── crosshatch-ota_update-2019.09.25.00.zip
|
||
│ ├── crosshatch-target_files-2019.09.25.00.zip
|
||
│ └── crosshatch-testing
|
||
└── 2019.10.07.21
|
||
└── release-crosshatch-2019.10.07.21
|
||
├── crosshatch-factory-2019.10.07.21.zip
|
||
├── crosshatch-factory-2019.10.07.21.zip.sig
|
||
├── crosshatch-img-2019.10.07.21.zip
|
||
├── crosshatch-ota_update-2019.10.07.21.zip
|
||
├── crosshatch-target_files-2019.10.07.21.zip
|
||
└── crosshatch-testing</pre>
|
||
|
||
<h2 id="prebuilt-code">
|
||
<a href="#prebuilt-code">Prebuilt code</a>
|
||
</h2>
|
||
|
||
Like the Android Open Source Project, GrapheneOS contains some code that's built
|
||
separately and then bundled into the source tree as binaries. This section will be
|
||
gradually expanded to cover building all of it.
|
||
|
||
<h3 id="browser-and-webview">
|
||
<a href="#browser-and-webview">Browser and WebView</a>
|
||
</h3>
|
||
|
||
<p>Vanadium is a hardened fork of Chromium developed by GrapheneOS and used to provide
|
||
the WebView and <em>optionally</em> the standalone browser app. It tracks the Chromium
|
||
release cycles along with having additional updates for downstream changes to the
|
||
privacy and security hardening patches, so it's updated at a different schedule than
|
||
the monthly Android releases.</p>
|
||
|
||
<p>The browser and the WebView are independent applications built from the Chromium
|
||
source tree. The GrapheneOS browser build is located at external/vanadium and the
|
||
WebView is at external/chromium-webview.</p>
|
||
|
||
<p>See <a href="https://chromium.googlesource.com/chromium/src/+/master/docs/android_build_instructions.md">
|
||
Chromium's Android build instructions</a> for details on obtaining the
|
||
prerequisites.</p>
|
||
|
||
<p>You can obtain the proper configuration from the
|
||
<a href="https://github.com/GrapheneOS/Vanadium">
|
||
GrapheneOS Vanadium repository</a> in <code>args.gn</code> including the correct
|
||
version.</p>
|
||
|
||
<pre>git clone https://github.com/GrapheneOS/Vanadium.git
|
||
cd Vanadium
|
||
git checkout $CORRECT_BRANCH_OR_TAG</pre>
|
||
|
||
<p>Fetch the Chromium sources:</p>
|
||
|
||
<pre>fetch --nohooks android</pre>
|
||
|
||
<p>Sync to the latest stable release for Android (replace $VERSION with the correct
|
||
value):</p>
|
||
|
||
<pre>gclient sync -D --with_branch_heads -r $VERSION --jobs 32</pre>
|
||
|
||
<p>Apply the GrapheneOS patches on top of the tagged release:</p>
|
||
|
||
<pre>cd src
|
||
git am --whitespace=nowarn ../patches/*.patch</pre>
|
||
|
||
<p>Generate a signing key for Vanadium if this is the initial build:</p>
|
||
|
||
<pre>keytool -genkey -v -keystore vanadium.keystore -storetype pkcs12 -alias vanadium -keyalg RSA -keysize 4096 -sigalg SHA512withRSA -validity 10000 -dname "cn=GrapheneOS"</pre>
|
||
|
||
<p>You will be prompted to enter a password which will be requested by the script for
|
||
signing releases. You should back this up the generated keystore with your other
|
||
keys.</p>
|
||
|
||
<p>Then, configure the build in the <code>src</code> directory:</p>
|
||
|
||
<pre>gn args out/Default</pre>
|
||
|
||
<p>Copy the GrapheneOS configuration from <code>../args.gn</code> and save/exit the
|
||
editor. Modify <code>target_cpu</code> as needed if the target is not arm64. For
|
||
x86_64, the correct value for <code>target_cpu</code> is <code>x64</code>, but note
|
||
that the Android source tree refers to it as x86_64.</p>
|
||
|
||
<p>You need to set <code>trichrome_certdigest</code> to the correct value for your
|
||
generated signing key. You can obtain this with the following command:</p>
|
||
|
||
<pre>keytool -export-cert -alias vanadium -keystore vanadium.keystore | sha256sum</pre>
|
||
|
||
<p>Build the components:</p>
|
||
|
||
<pre>ninja -C out/Default/ trichrome_webview_apk trichrome_chrome_bundle trichrome_library_apk</pre>
|
||
|
||
<p>Generate TrichromeChrome.apk from the bundle and sign the apks:</p>
|
||
|
||
<pre>../generate_release.sh</pre>
|
||
|
||
<p>The apks needs to be copied from <code>out/Default/apks/release/*.apk</code>
|
||
into the Android source tree at
|
||
<code>external/vanadium/prebuilt/arm64/</code> with arm64
|
||
substituted with the correct value for other architectures (arm, x86, x86_64).</p>
|
||
|
||
<p>WebView provider apps need to be whitelisted in
|
||
<code>frameworks/base/core/res/res/xml/config_webview_packages</code>. By default,
|
||
only the Vanadium WebView is whitelisted.</p>
|
||
|
||
<h3 id="prebuilt-apps">
|
||
<a href="#prebuilt-apps">Prebuilt apps</a>
|
||
</h3>
|
||
|
||
<p>The official releases of the Auditor and PdfViewer apps are bundled as an apk into
|
||
external/ repositories. There are no modifications to these for GrapheneOS. These are
|
||
built and signed with the standard <code>gradle</code> Android plugin build
|
||
system.</p>
|
||
|
||
<p>A build of Seedvault is bundled as an apk into an external/ repository. There are
|
||
no modifications made to it.</p>
|
||
|
||
<h2 id="standalone-sdk">
|
||
<a href="#standalone-sdk">Standalone SDK</a>
|
||
</h2>
|
||
|
||
<p>It can be useful to set up a standalone installation of the SDK separate from
|
||
the Android Open Source Project tree. This is how the prebuilt apps are built, rather
|
||
than using the older branch of the SDK in the OS source tree.</p>
|
||
|
||
<p>Android Studio can also be set up to use an existing SDK and will recognize it and use
|
||
it automatically if Android Studio is installed with an SDK installation already
|
||
available and set up in the environment. You'll also likely want a working
|
||
command-line SDK environment even if you do heavily use Android Studio.</p>
|
||
|
||
<p>Using the official releases of the SDK is recommended for simplicity, although with
|
||
a lot of effort you can build everything yourself. Distribution packages are generally
|
||
quite out-of-date and should be avoided. To set up a minimal SDK installation without
|
||
Android Studio on Linux:</p>
|
||
|
||
<pre>mkdir ~/sdk
|
||
cd ~/sdk
|
||
wget https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip
|
||
unzip sdk-tools-linux-4333796.zip
|
||
rm sdk-tools-linux-4333796.zip</pre>
|
||
|
||
<p>Add the directories to your PATH in your shell profile configuration and do the
|
||
same in your current shell:</p>
|
||
|
||
<pre>export PATH="$HOME/sdk/tools:$HOME/sdk/tools/bin:$HOME/sdk/platform-tools:$HOME/sdk/build-tools/29.0.3:$PATH:$HOME/sdk/ndk-bundle"
|
||
export ANDROID_HOME="$HOME/sdk"</pre>
|
||
|
||
<p>Run an initial update:</p>
|
||
|
||
<pre>sdkmanager --update</pre>
|
||
|
||
<p>Install platform-tools for tools like adb and fastboot:</p>
|
||
|
||
<pre>sdkmanager platform-tools</pre>
|
||
|
||
<p>For running the Compatibility Test Suite you'll also need the build-tools for
|
||
aapt:</p>
|
||
|
||
<pre>sdkmanager 'build-tools;29.0.3'</pre>
|
||
|
||
<p>For working with native code, you need the NDK:</p>
|
||
|
||
<pre>sdkmanager ndk-bundle</pre>
|
||
|
||
<p>You should update the sdk before use from this point onwards:</p>
|
||
|
||
<pre>sdkmanager --update</pre>
|
||
|
||
<h2 id="testing">
|
||
<a href="#testing">Testing</a>
|
||
</h2>
|
||
|
||
<p>This section will be expanded to cover various test suites and testing procedures
|
||
rather than only the current very minimal coverage of the Compatibility Test Suite
|
||
(CTS).</p>
|
||
|
||
<h3 id="emulator">
|
||
<a href="#emulator">Emulator</a>
|
||
</h3>
|
||
|
||
<p>To test a build for the emulator, run <code>emulator</code> within the build
|
||
environment. The emulator will use CPU hardware acceleration via KVM along with
|
||
optional graphics acceleration via the host GPU if these are available.</p>
|
||
|
||
<h3 id="compatibility-test-suite">
|
||
<a href="#compatibility-test-suite">Compatibility Test Suite</a>
|
||
</h3>
|
||
|
||
<h4 id="compatibility-test-suite-download">
|
||
<a href="#compatibility-test-suite-download">Download</a>
|
||
</h4>
|
||
|
||
<p>Testing with the Compatibility Test Suite (CTS) can be done by either building the
|
||
test suite from source or using the official releases.</p>
|
||
<p>Official releases of the CTS can be downloaded from
|
||
<a href="https://source.android.com/compatibility/cts/downloads">the Compatibility
|
||
Suite Downloads page</a>. You should download the CTS for the relevant release
|
||
(Android 10) and architecture (ARM). There's a separate zip for the main CTS, the
|
||
manual portion (CTS Verifier) and the CTS for Instant Apps. The latest release of the
|
||
CTS Media Files also needs to be downloaded from that section.</p>
|
||
|
||
<h4 id="compatibility-test-suite-setup">
|
||
<a href="#compatibility-test-suite-setup">Setup</a>
|
||
</h4>
|
||
<p>You'll need a device attached to your computer with ADB enabled along with the
|
||
Android SDK installed. The build-tools and platform-tools packages need to be
|
||
installed and the binaries need to be added to your PATH. For example, with the SDK
|
||
located at <code>/home/username</code>:</p>
|
||
<pre>export ANDROID_HOME="$HOME/sdk"
|
||
export PATH="$PATH:$HOME/sdk/tools:$HOME/sdk/tools/bin:$HOME/sdk/platform-tools:$HOME/sdk/build-tools/29.0.3:$HOME/sdk/ndk-bundle"</pre>
|
||
<p>Copy media onto the device:</p>
|
||
<pre>cd android-cts-media-1.4
|
||
./copy_images.sh
|
||
./copy_media.sh</pre>
|
||
|
||
<p>You also need to do some basic setup for the device. It's possible for changes from
|
||
a baseline install to cause interference, so it can be a good idea to factory reset
|
||
the device if assorted changes have been made. The device needs to be running a user
|
||
build for the security model to be fully intact in order to pass all the security
|
||
tests. A userdebug build is expected to fail some of the tests. GrapheneOS also makes
|
||
various changes intentionally deviating from the requirements expected by the CTS, so
|
||
there will always be some expected failures. A few of the tests are also known to be
|
||
quite flaky or broken even with the stock OS and/or AOSP. These will be documented
|
||
here at some point.</p>
|
||
|
||
<ul>
|
||
<li>Must be connected to a WiFi network with IPv6 internet access</li>
|
||
<li>Must have a working SIM card with mobile data with IPv6 internet access</li>
|
||
<li>Disable SIM lock</li>
|
||
<li>Enable Bluetooth</li>
|
||
<li>Enable NFC and NDEF (Android Beam)</li>
|
||
<li>Open / close Chromium to deal with initial setup</li>
|
||
<li>Prop up with a good object to focus on and good lighting for Camera tests.
|
||
Both the front and rear cameras will be used, so ensure this is true for both the
|
||
front and the rear cameras.</li>
|
||
<li>Bluetooth beacons for Bluetooth tests</li>
|
||
<li>Must have a great GPS/GNSS signal for location tests</li>
|
||
<li>SIM card with carrier privilege rules</li>
|
||
<li>Secure element applet installed on the embedded secure element or SIM
|
||
card</li>
|
||
<li>At least one Wi-Fi RTT access point powered up but not connected to any
|
||
network</li>
|
||
<li>The screen lock must be disabled.</li>
|
||
</ul>
|
||
|
||
<h4 id="compatibility-test-suite-run-modules">
|
||
<a href="#compatibility-test-suite-run-modules">Run modules</a>
|
||
</h4>
|
||
|
||
<p>Run the test harness:</p>
|
||
<pre>./android-cts/tools/cts-tradefed</pre>
|
||
<p>Note that <code>_JAVA_OPTIONS</code> being set will break the version detection.</p>
|
||
<p>To obtain a list of CTS modules:</p>
|
||
<pre>list modules</pre>
|
||
<p>To run a specific module and avoid wasting time capturing device information:</p>
|
||
<pre>run cts --skip-device-info --module CtsModuleName</pre>
|
||
<p>To speed up initialization after running some initial tests:</p>
|
||
<pre>run cts --skip-device-info --skip-preconditions --module CtsModuleName</pre>
|
||
<p>It's possible to run the whole standard CTS plan with a single command, but running
|
||
specific modules is recommended, especially if you don't have everything set up for
|
||
the entire test suite.</p>
|
||
|
||
<h2 id="development-guidelines">
|
||
<a href="#development-guidelines">Development guidelines</a>
|
||
</h2>
|
||
|
||
<h3 id="programming-languages">
|
||
<a href="#programming-languages">Programming languages</a>
|
||
</h3>
|
||
|
||
<p>The following programming languages are acceptable for <strong>completely
|
||
new</strong> GrapheneOS projects:</p>
|
||
<ul>
|
||
<li>Kotlin for apps and any services closely tied to the apps, now that it's not
|
||
only officially supported by the Android SDK and Android Studio but also the
|
||
default language with Kotlin exclusive enhancements to the APIs</li>
|
||
<li>Web applications must be entirely static HTML/CSS/JavaScript. TypeScript would
|
||
make sense at a larger scale but there are no plans for any large web
|
||
applications.</li>
|
||
<li>Rust with <code>no_std</code> for low-level code used in a hypervisor, kernel,
|
||
daemon, system library, etc. Keep in mind that low-level code is to be avoided
|
||
whenever a higher language language is better suited to the job. In general,
|
||
the project aims to avoid creating more low-level code manually dealing with
|
||
memory ownership and lifetimes in the first place.</li>
|
||
<li>C in rare cases for very small and particularly low-level projects without
|
||
opportunities to reduce the trusted computing base for memory corruption to
|
||
any significant degree with Rust, such as for the hardened_malloc project</li>
|
||
<li>arm64 assembly in extremely rare cases where C or Rust aren't usable with
|
||
compiler intrinsics</li>
|
||
<li>Python 3 for small (less than 500 lines) development-related scripts that are
|
||
not exposed to untrusted input. It's never acceptable to use it for
|
||
client-side code on devices or for servers. It isn't used on the servers even
|
||
for non-application-server code.</li>
|
||
<li>Bash for tiny (less than 200 lines) build scripts without any non-trivial
|
||
logic where Python would be an annoyance.</li>
|
||
</ul>
|
||
|
||
<p>Much of the work is done on existing projects, and the existing languages should be
|
||
used unless there are already clear stable API boundaries where a different language
|
||
could be used without causing a substantial maintenance burden. The following
|
||
languages are typical from most to least common: Java, C++, C, JavaScript, arm64
|
||
assembly, POSIX shell, Bash.</p>
|
||
|
||
<h3 id="code-style">
|
||
<a href="#code-style">Code style</a>
|
||
</h3>
|
||
|
||
<p>For existing projects, use the official upstream code style. Avoid using legacy
|
||
conventions that they're moving away from themselves. Follow the code style they use
|
||
for new additions. Some projects have different code styles for different directories
|
||
or files depending on their sources, in which case respect the per-file style.</p>
|
||
|
||
<p>For new projects, follow the official code style for the language. Treat the
|
||
standard library APIs as defining the naming style for usage of the language, i.e. C
|
||
uses <code>variable_or_function_name</code>, <code>type_name</code>,
|
||
<code>MACRO_NAME</code> while JavaScript uses <code>variable_or_function_name</code>,
|
||
<code>ClassName</code> and <code>CONSTANT_NAME</code>. For Python, follow PEP8 and the
|
||
same goes for other languages with official styles whether defined in a document or by
|
||
the default mode for the official formatting tool like <code>rustfmt</code>.</p>
|
||
|
||
<p>For cases where there isn't an official or prevailing code style for other things,
|
||
avoid tabs, use 4-space indents, <code>function_name</code>,
|
||
<code>variable_name</code>, <code>TypeName</code> and <code>CONSTANT_NAME</code>.
|
||
Prefer single-line comment syntax other than rare cases where it makes sense to add a
|
||
tiny comment within a line of code. In languages with the optional braces misfeature
|
||
(C, C++, Java), always use them. Open braces on the same line as function definitions
|
||
/ statements. Wrap lines at 100 columns except in rare cases where it would be far
|
||
uglier to wrap the line.</p>
|
||
|
||
<p>For JavaScript, put <code>"use strict";</code> at the top of every file, end lines
|
||
with semicolons (since automatic insertion is poorly designed) and always use
|
||
<code>const</code> to declare variables, unless they are reassigned in which case they
|
||
should be declared with <code>let</code> but never use <code>var</code> as it is
|
||
effectively broken. Try to prefer loops with <code>for..of</code>.</p>
|
||
|
||
<p>For web content, use dashes as user-facing word separators rather than underscores.
|
||
Page titles should follow the scheme "Page | Directory | Higher-level directory |
|
||
Site" for usability with a traditional title as the Open Graph title.</p>
|
||
|
||
<p>Avoid designing around class inheritance unless it's a rare case where it's an
|
||
extremely good fit or the language sucks (Java) and it's the least bad approach, but
|
||
still try to avoid it.</p>
|
||
|
||
<p>Use concise but self-explanatory variable names. Prefer communicating information
|
||
via naming rather than using comments whenever possible. Don't name variables
|
||
<code>i</code>, <code>j</code>, <code>k</code>, etc. like C programmers. It's okay to
|
||
use things like <code>x</code> and <code>y</code> for parameters if the function is
|
||
genuinely that generic and operates on arbitrary values. In general, try to scope
|
||
variables into the most limited scope (in C or C++, be careful about this when
|
||
references are taken).</p>
|
||
|
||
<p>Write code that's clean and self-explanatory. Use comments to explain or justify
|
||
non-obvious things, but try to avoid needing them in the first place. In most cases,
|
||
they should just be communicating non-local information such as explaining why an
|
||
invariant is true based on the code elsewhere (consider a runtime check to make sure
|
||
it's true, or an assertion if performance would be an issue). Docstrings at the top of
|
||
top-level functions, modules, etc. are a different story and shouldn't be avoided.</p>
|
||
|
||
<h3 id="library-usage">
|
||
<a href="#library-usage">Library usage</a>
|
||
</h3>
|
||
|
||
<p>Make extensive usage of well designed standard library modules. For apps, treat
|
||
Jetpack (androidx) as part of the standard library and make good use of it. For Java,
|
||
Guava can also be treated as part of the standard library.</p>
|
||
|
||
<p>Libraries outside of the standard library should be used very cautiously. They
|
||
should be well maintained, stable, well tested and widely used. Libraries implemented
|
||
with memory unsafe languages should generally be avoided (one exception: SQLite).</p>
|
||
|
||
<p>Generally, frameworks and libraries existing solely to provide different paradigms
|
||
and coding patterns are to be avoided. They increase barrier to entry for developers,
|
||
generally only increase complexity unless used at very large scales (and may not even
|
||
make things simpler in those cases) and come and go as fads. This is only okay when
|
||
it's part of the standard libraries or libraries that are considered standard
|
||
(androidx, Guava) by GrapheneOS and should still be approached cautiously. Only use it
|
||
if it truly makes the correct approach simpler. Ignore fads and figure out if it
|
||
actually makes sense to use, otherwise just stick to the old fashioned way if the
|
||
fancy alternatives aren't genuinely better.</p>
|
||
</div>
|
||
<footer>
|
||
<a href="/"><img src="/logo.png" width="512" height="512" alt=""/>GrapheneOS</a>
|
||
<ul id="social">
|
||
<li><a href="https://twitter.com/GrapheneOS">Twitter</a></li>
|
||
<li><a href="https://github.com/GrapheneOS">GitHub</a></li>
|
||
<li><a href="https://reddit.com/r/GrapheneOS">Reddit</a></li>
|
||
</ul>
|
||
</footer>
|
||
</body>
|
||
</html>
|