1
0
forked from rosa/hakurei

180 Commits

Author SHA1 Message Date
852f3a9b3d internal/rosa/kernel: 6.12.81 to 6.12.82
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 22:11:13 +09:00
5e02dbdb0d internal/rosa/python: remove pypi helpers
Pypi is disallowed by policy so these helpers are no longer useful.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:37:10 +09:00
6a3248d472 internal/rosa/python: install pyyaml from source
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:35:30 +09:00
67404c98d9 internal/rosa/nss: install buildcatrust from source
Dependencies are now available, so this no longer has to rely on the release.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:09:24 +09:00
b9bf69cfce internal/rosa/python: install mako from source
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:55:23 +09:00
4648f98272 internal/rosa/python: run tests via helper
Despite the lack of standards, pytest seems somewhat widely agreed upon.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:50:57 +09:00
11d99439ac internal/rosa/python: install markupsafe from source
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:26:11 +09:00
39e4c5b8ac internal/rosa/python: optionally install before check
Some test suites require package to be installed globally.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:25:43 +09:00
e8f6db38b6 internal/rosa/python: install pytest from source
Used by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 23:17:38 +09:00
20d5b71575 internal/rosa/python: install iniconfig from source
This also required the setuptools-scm hack.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:53:32 +09:00
e903e7f542 internal/rosa/python: install pygments from source
This finally has its dependencies.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:40:43 +09:00
1caa051f4d internal/rosa/python: hatchling artifact
Required by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:35:18 +09:00
dcdc6f7f6d internal/rosa/python: trove-classifiers artifact
Required by hatchling.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:32:12 +09:00
5ad6f26b46 internal/rosa/python: install packaging from source
This is required by many packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:12:49 +09:00
7ba75a79f4 internal/rosa/python: install pluggy from source
This finally has all its dependencies at this point.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 21:55:55 +09:00
9ef84d3904 internal/rosa/python: setuptools-scm artifact
Awful hack required by many packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 21:38:44 +09:00
3b7b6e51fb internal/rosa/python: pass build dependencies separately
This is cleaner with less duplicate code.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 20:26:41 +09:00
b1b4debb82 internal/rosa/python: pathspec artifact
Required by hatchling, which is required by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 20:13:26 +09:00
021cbbc2a8 cmd/mbf: default daemon socket in cache
This location makes more sense than the current directory.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:50:54 +09:00
a4a54a4a4d cmd/mbf: remove pointless recover
This used to scrub the cache, and was not fully removed when that became nonviable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:49:01 +09:00
04a344aac6 internal/rosa/python: flirt_core artifact
A build system required by a dependency of another build system, which is required by yet another build system.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:25:04 +09:00
6b98156a3d internal/rosa/python: change insane strict_timestamps default
There is no scenario where this is useful, and it breaks builds.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 18:56:22 +09:00
753432cf09 cmd/mbf: optionally wait for cancel
Synchronisation is not needed here during interactive use.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 18:24:11 +09:00
f8902e3679 internal/rosa/python: append to source path
This gets around messy projects with multiple packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 17:51:00 +09:00
8ee53a5164 internal/rosa: use builtin for checksum warning
This avoids having to configure the logger early.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 17:50:12 +09:00
3981d44757 internal/rosa/python: migrate setuptools to wrapper
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:36:43 +09:00
9fd67e47b4 internal/rosa/python: wrap python package
Metadata for this is somewhat boilerplate-heavy, so wrap it to create metadata in one call.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:22:18 +09:00
4dcec40156 cmd/mbf: close on cancel completion
Like the previous change, this enables synchronisation on the client side via epoll.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:03:52 +09:00
9a274c78a3 cmd/mbf: close on abort completion
This enables synchronisation on the client side via epoll.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 14:53:28 +09:00
5647c3a91f internal/rosa/meson: run meson test suite
Tests requiring internet access or unreasonable dependencies are removed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 01:07:20 +09:00
992139c75d internal/rosa/python: extra script after install
This is generally for test suite, due to the lack of standard or widely agreed upon convention.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:35:24 +09:00
57c69b533e internal/rosa/meson: migrate to helper
This also migrates to source from the Microsoft Github release.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:16:22 +09:00
6f0c2a80f2 internal/rosa/python: migrate setuptools to helper
This is much cleaner, and should be functionally equivalent.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:04:19 +09:00
08dfefb28d internal/rosa/python: pip helper
Binary pip releases are not considered acceptable, this more generic helper is required for building from source.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:03:36 +09:00
b081629662 internal/rosa/libxml2: 2.15.2 to 2.15.3
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:05:49 +09:00
fba541f301 internal/rosa/nss: 3.122.1 to 3.123
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:05:23 +09:00
5f0da3d5c2 internal/rosa/gnu: mpc 1.4.0 to 1.4.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:04:33 +09:00
4d5841dd62 internal/rosa: elfutils 0.194 to 0.195
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:03:49 +09:00
9e752b588a internal/pkg: drop cached error on cancel
This avoids disabling the artifact when using the individual cancel method. Unfortunately this makes the method blocking.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 03:24:48 +09:00
27b1aaae38 internal/pkg: pending error alongside done channel
This significantly simplifies synchronisation of access to identErr.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 03:10:37 +09:00
9e18de1dc2 internal/pkg: flush cached errors on abort
This avoids disabling the artifact until cache is reopened. The same has to be implemented for Cancel in a future change.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 02:59:44 +09:00
b80ea91a42 cmd/mbf: abort remote cures
This command arranges for all pending cures to be aborted. It does not wait for cures to complete.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:47:02 +09:00
30a9dfa4b8 internal/pkg: abort all pending cures
This cancels all current pending cures without closing the cache.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:40:35 +09:00
8d657b6fdf cmd/mbf: cancel remote cure
This exposes the new fine-grained cancel API in cmd/mbf.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:00:04 +09:00
ae9b9adfd2 internal/rosa: retry in SIGSEGV test
Munmap is not always immediate.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 20:45:19 +09:00
dd6a480a21 cmd/mbf: handle flags in serve
This enables easier expansion of the protocol.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 20:14:09 +09:00
3942272c30 internal/pkg: fine-grained cancellation
This enables a specific artifact to be targeted for cancellation.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 19:33:21 +09:00
9036986156 cmd/mbf: optionally ignore reply
An acknowledgement is not always required in this use case. This change also adds 64 bits of connection configuration for future expansion.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 16:46:49 +09:00
a394971dd7 cmd/mbf: do not abort cache acquisition during testing
This can sometimes fire during testing due to how short the test is.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 02:06:51 +09:00
9daba60809 cmd/mbf: daemon command
This services internal/pkg artifact IR with Rosa OS extensions originating from another process.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 02:05:59 +09:00
bcd79a22ff cmd/mbf: do not open cache for IR encoding
This can now be allocated independently.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 01:04:39 +09:00
0ff7ab915b internal/pkg: move IR primitives out of cache
These are memory management and caching primitives. Having them as part of Cache is cumbersome and requires a temporary directory that is never used. This change isolates them from Cache to enable independent use.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 01:02:13 +09:00
823575acac cmd/mbf: move info command
This is cleaner with less shared state.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-16 17:43:52 +09:00
136bc0917b cmd/mbf: optionally open cache
Some commands do not require the cache. This change also makes acquisition of locked cache cancelable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-16 15:59:34 +09:00
d6b082dd0b internal/rosa/ninja: bootstrap with verbose output
This otherwise outputs nothing, and appears to hang until the (fully single-threaded) bootstrap completes.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:19:05 +09:00
89d6d9576b internal/rosa/make: optionally format value as is
This enables correct formatting for awkward configure scripts.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:17:58 +09:00
fafce04a5d internal/rosa/kernel: firmware 20260309 to 20260410
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:16:47 +09:00
5d760a1db9 internal/rosa/kernel: 6.12.80 to 6.12.81
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:16:30 +09:00
d197e40b2a internal/rosa/python: mako 1.3.10 to 1.3.11
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:21:54 +09:00
2008902247 internal/rosa/python: packaging 26.0 to 26.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:15:18 +09:00
30ac985fd2 internal/rosa/meson: 1.10.2 to 1.11.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:14:52 +09:00
e9fec368f8 internal/rosa/nss: 3.122 to 3.122.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:13:45 +09:00
46add42f58 internal/rosa/openssl: disable building docs
These take very long and are never used in the Rosa OS environment.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:13:18 +09:00
377b61e342 internal/rosa/openssl: do not double test job count
The test suite is racy, this reduces flakiness.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:12:36 +09:00
520c36db6d internal/rosa: respect preferred job count
This discontinues use of nproc, and also overrides detection behaviour in ninja.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 18:49:36 +09:00
3352bb975b internal/pkg: job count in container environment
This exposes preferred job count to the container initial process.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 15:49:21 +09:00
f7f48d57e9 internal/pkg: pass impure job count
This is cleaner than checking cpu count during cure, it is impossible to avoid impurity in both situations but this is configurable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 15:36:44 +09:00
5c2345128e internal/rosa/llvm: autodetect stage0 target
This is fine, now that stages beyond stage0 have explicit target.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 03:10:26 +09:00
78f9676b1f internal/rosa/llvm: centralise llvm source
This avoids having to sidestep the NewPackage name formatting machinery to take the cache fast path.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 03:03:06 +09:00
5b5b676132 internal/rosa/cmake: remove variant
This has no effect outside formatting of name and is a remnant of the old llvm helpers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:57:47 +09:00
78383fb6e8 internal/rosa/llvm: migrate libclc
This eliminates newLLVMVariant.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:40:13 +09:00
e97f6a393f internal/rosa/llvm: migrate runtimes and clang
This eliminates most newLLVM family of functions.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:07:13 +09:00
eeffefd22b internal/rosa/llvm: migrate compiler-rt helper
This also removes unused dependencies.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 01:12:56 +09:00
ac825640ab internal/rosa/llvm: migrate musl
This removes the pointless special treatment given to musl.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:35:42 +09:00
a7f7ce1795 internal/rosa/llvm: migrate compiler-rt
The newLLVM family of functions predate the package system. This change migrates compiler-rt without changing any resulting artifacts.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:19:33 +09:00
38c639e35c internal/rosa/llvm: remove project/runtime helper
More remnants from early days, these are not reusable at all but that was not known at the time.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:03:23 +09:00
b2cb13e94c internal/rosa/llvm: centralise patches
This enables easier reuse of the patchset.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 23:52:44 +09:00
46f98d12d6 internal/rosa/llvm: remove conditional flags in helper
The llvm helper is a remnant from very early days, and ended up not being very useful, but was never removed. This change begins its removal, without changing the resulting artifacts for now.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 23:38:11 +09:00
503c7f953c internal/rosa/x: libpciaccess artifact
Required by userspace gpu drivers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 19:04:38 +09:00
15c9f6545d internal/rosa/perl: populate anitya identifiers
These are also tracked by Anitya.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 18:44:43 +09:00
83b0e32c55 internal/rosa: helpers for common url formats
This cleans up call site of NewPackage.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 18:02:57 +09:00
eeaf26e7a2 internal/rosa: wrapper around git helper
This results in much cleaner call site for the majority of use cases.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 15:20:51 +09:00
b587caf2e8 internal/rosa: assume file source is xz-compressed
XZ happens to be the only widely-used format that is awful to deal with, everything else is natively supported.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 15:07:30 +09:00
f1c2ca4928 internal/rosa/mesa: libdrm artifact
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:27:09 +09:00
0ca301219f internal/rosa/python: pyyaml artifact
Mesa unfortunately requires this horrible format.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:18:47 +09:00
e2199e1276 internal/rosa/python: mako artifact
This unfortunately pulls in platform-specific package.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:11:38 +09:00
86eacb3208 cmd/mbf: checksum command
This computes and encodes sha384 checksum of data streamed from standard input.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:09:21 +09:00
8541bdd858 internal/rosa: wrap per-arch values
This is cleaner syntax in some specific cases.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:59:55 +09:00
46be0b0dc8 internal/rosa/nss: buildcatrust 0.4.0 to 0.5.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:18:21 +09:00
cbe37e87e7 internal/rosa/python: pytest 9.0.2 to 9.0.3
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:18:02 +09:00
66d741fb07 internal/rosa/python: pygments 2.19.2 to 2.20.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:13:04 +09:00
0d449011f6 internal/rosa/python: use predictable URLs
This is much cleaner and more maintainable than specifying URL prefix manually. This change also populates Anitya project identifiers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:08:22 +09:00
46428ed85d internal/rosa/python: url pip wheel helper
This enables a cleaner higher-level helper.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 01:59:28 +09:00
081d6b463c internal/rosa/llvm: libclc artifact
This is built independently of llvm build system to avoid having to build llvm again.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:57:04 +09:00
11b3171180 internal/rosa/glslang: glslang artifact
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:34:17 +09:00
adbb84c3dd internal/rosa/glslang: spirv-tools artifact
Required by glslang.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:27:49 +09:00
1084e31d95 internal/rosa/glslang: spirv-headers artifact
Required by spirv-tools.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:16:29 +09:00
27a1b8fe0a internal/rosa/mesa: libglvnd artifact
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 21:27:30 +09:00
b2141a41d7 internal/rosa/dbus: xdg-dbus-proxy artifact
This is currently a hakurei runtime dependency, but will eventually be removed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 19:41:49 +09:00
c0dff5bc87 internal/rosa/gnu: gcc set with-multilib-list as needed
This breaks riscv64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 18:03:45 +09:00
04513c0510 internal/rosa/gnu: gmp explicit CC
The configure script is hard coded to use gcc without fallback on riscv64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 17:25:15 +09:00
28ebf973d6 nix: add sharefs supplementary group
This works around vfs inode file attribute race.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 23:28:18 +09:00
41aeb404ec internal/rosa/hakurei: 0.3.7 to 0.4.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 10:53:29 +09:00
0b1009786f release: 0.4.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 10:49:43 +09:00
b390640376 internal/landlock: relocate from package container
This is not possible to use directly, so remove it from the public API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 23:56:45 +09:00
ad2c9f36cd container: unexport PR_SET_NO_NEW_PRIVS wrapper
This is subtle to use correctly. It also does not make sense as part of the container API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 23:45:51 +09:00
67db3fbb8d check: use encoding interfaces
This turned out not to require specific treatment, so the shared interfaces are cleaner.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 22:11:53 +09:00
560cb626a1 hst: remove enablement json adapter
The go116 behaviour of built-in new function makes this cleaner.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 20:47:30 +09:00
c33a6a5b7e hst: optionally reject insecure options
This prevents inadvertent use of insecure compatibility features.

Closes #30.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 19:34:02 +09:00
952082bd9b internal/rosa/python: 3.14.3 to 3.14.4
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:38:22 +09:00
24a9b24823 internal/rosa/openssl: 3.6.1 to 3.6.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:38:02 +09:00
c2e61e7987 internal/rosa/libcap: 2.77 to 2.78
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:37:04 +09:00
86787b3bc5 internal/rosa/tamago: 1.26.1 to 1.26.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:31:57 +09:00
cdfcfe6ce0 internal/rosa/go: 1.26.1 to 1.26.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:18:27 +09:00
68a2f0c240 internal/rosa/llvm: remove unused field
This change also renames confusingly named flags field and corrects its doc comment.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:13:26 +09:00
7319c7adf9 internal/rosa/llvm: use latest version on arm64
This also removes arch-specific patches because they were not useful.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 01:07:25 +09:00
e9c890cbb2 internal/rosa/llvm: enable cross compilation
This now passes the test suite.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 00:59:14 +09:00
6f924336fc internal/rosa/llvm: increase stack size
Some aarch64 regression tests fail intermittently on the default size.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 00:56:51 +09:00
bd88f10524 internal/rosa/llvm: 22.1.2 to 22.1.3
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-09 17:36:23 +09:00
e34e3b917e internal/kobject: process uevent message
This deals with environment variables generally present in every message.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 18:00:04 +09:00
b0ba165107 cmd/sharefs: group-accessible permission bits
This works around the race in vfs via supplementary group.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 16:14:47 +09:00
351d6c5a35 cmd/sharefs: reproduce vfs inode file attribute race
This happens in the vfs permissions check only and stale data appears to never reach userspace.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 15:51:36 +09:00
f23f73701c cmd/mbf: optional host abstract
This works around kernels with Landlock LSM disabled. Does not affect cure outcome.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 18:15:49 +09:00
876917229a internal/rosa/go: enable riscv64 bootstrap path
This is quite expensive, but no other option, unfortunately.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 18:11:42 +09:00
0558032c2d container: do not set static deadline
This usually ends up in the buffer, or completes well before the deadline, however this can still timeout on a very slow system.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 17:00:20 +09:00
c61cdc505f internal/params: relocate from package container
This does not make sense as part of the public API, so make it internal.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 16:37:44 +09:00
062edb3487 container: remove setup pipe helper
The API forces use of finalizer to close the read end of the setup pipe, which is no longer considered acceptable. Exporting this as part of package container also imposes unnecessary maintenance burden.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 16:05:33 +09:00
e4355279a1 all: optionally forbid degrading in tests
This enables transparently degradable tests to be forced on in environments known to support them.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 15:22:52 +09:00
289fdebead container: transparently degrade landlock in tests
Explicitly requiring landlock in tests will be supported in a future change.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 15:03:48 +09:00
9c9e190db9 ldd: remove timeout
The program generally never blocks, and it is more flexible to leave it up to the caller to set a timeout.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:49:20 +09:00
d7d42c69a1 internal/pkg: transparently degrade landlock in tests
This does not test package container, so should transparently cope with Landlock LSM being unavailable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:44:34 +09:00
c758e762bd container: skip landlock on hostnet
This overlaps with net namespace, so can be skipped without degrading security.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:36:44 +09:00
10f8b1c221 internal/pkg: optional landlock LSM
The alpine linux riscv64 kernel does not enable Landlock LSM, and kernel compilation is not yet feasible.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 12:44:07 +09:00
6907700d67 cmd/dist: set hsu tar header mode bits
This has no effect, but is nice to have.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 23:37:38 +09:00
0243f3ffbd internal/rosa/stage0: add riscv64 tarball
This had not yet passed all test suites because emulator is prohibitively slow.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 13:57:43 +09:00
cd0beeaf8e internal/uevent: optionally pass UUID during coldboot
This enables rejection of non-coldboot synthetic events.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 12:42:47 +09:00
a69273ab2a cmd/dist: replace dist/release.sh
This is much more robust than a shell script.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 23:58:08 +09:00
4cd0f57e48 dist: remove redundant cleanup
This breaks on shells that do not evaluate pathnames.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 16:16:37 +09:00
33a0e6c01b hst: conditionally skip root remount
This enables the writable root overlay use case.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 14:04:21 +09:00
d58f5c7590 dist: destroy workdir on exist
This no longer relies on the hermetic build system to clean up.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:32:16 +09:00
1da992e342 dist: prefix from environment
These are baked in, so make them configurable for the build.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:11:44 +09:00
9641805ec2 container/init: ignore finished process
This is not considered an error, if the process finishes while the signal is being delivered.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:38:33 +09:00
0738f4889a internal/rosa/gnu: fetch mpc source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:35:15 +09:00
7de3cfe221 internal/rosa/netfilter: fetch iptables source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:19:40 +09:00
8b0648dd5d internal/rosa/netfilter: fetch libnftnl source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:14:45 +09:00
4667fac76c internal/rosa/libbsd: fetch source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:05:56 +09:00
52e5443b0e internal/rosa/libbsd: fetch libmd source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:02:56 +09:00
130e470b60 internal/rosa/libxslt: fetch source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:05:10 +09:00
ba5ee8e3ee internal/rosa/libxml2: fetch source via git
Eliminates the xz dependency. This also switches to meson to avoid pulling in autotools.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:58:29 +09:00
d1cef30877 internal/rosa/gtk: fetch glib source via git
This eliminates xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:47:10 +09:00
0188a3f0c7 internal/rosa/gnu: gnutls disable arm64 hardware acceleration
Hardware on arm64 is quite messy, this miscompiles.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:37:18 +09:00
04fe3b24ce internal/rosa/gnu: gnutls configure trust store
The test suite is somehow happy on amd64 but fails on arm64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:33:51 +09:00
93ad551054 internal/rosa/git: shallow clone
The .git directory is destroyed anyway, so no point fetching more than the bare minimum.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 20:14:40 +09:00
3d54d1f176 internal/rosa: drop caches
This enables accurate benchmarking of the toolchain abstraction.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:58:23 +09:00
9feac7738f internal/pkg: optionally suppress via assumed checksum
This is quite error-prone and causes cache inconsistency similar to the store inconsistency seen on nix when a similar condition happens. Keep this behind a flag in case it is ever beneficial.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:33:59 +09:00
591a60bac9 internal/pkg: per-cache SCHED_IDLE
This is cleaner than setting it globally, and is impossible to race.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:49:34 +09:00
5093a06026 internal/pkg: cache flags
This is cleaner for extending the API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:16:55 +09:00
50c1d7f880 internal/rosa/kernel: 6.12.78 to 6.12.80
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:14:16 +09:00
9e63633fbc container: remove test timeouts
These timeouts are no longer useful, and causes spurious test failures under load.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:51:37 +09:00
61f981a34a internal/rosa/perl: 5.42.1 to 5.42.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:25 +09:00
d717c41bbe internal/rosa/cmake: 4.3.0 to 4.3.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:07 +09:00
b896eec9b7 internal/rosa/gnu: parallel 20260222 to 20260322
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:48 +09:00
8ab99e5e40 internal/rosa/util-linux: 2.41.3 to 2.42
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:25 +09:00
2b6160ef7d internal/rosa/wayland: wayland-protocols 1.47 to 1.48
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:04 +09:00
4dcac7f133 internal/rosa/xz: 5.8.2 to 5.8.3
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:43:37 +09:00
966fd4df9e internal/rosa: connman artifact
Will be gradually replaced with a native implementation.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:17:10 +09:00
a2cf59b989 internal/rosa/ncurses: also build dynamic library
GNU readline breaks without this.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:13:04 +09:00
e87f59c4e4 internal/rosa/gnu: readline artifact
Nice to have library for command line programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:42:59 +09:00
3b221c3e77 internal/rosa/gnu: gnutls artifact
Incredibly ugly and expensive package, but unfortunately required by some packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:36:28 +09:00
ff3b385b12 internal/rosa: libunistring artifact
Required by GnuTLS.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:00:13 +09:00
c6920e6ab7 cmd/mbf: pick up $TERM
This improves behaviour of some programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:59:15 +09:00
59b25d45fe internal/pkg: pick up $TERM if attaching stdin
This improves behaviour of some programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:50:07 +09:00
9b99650eb1 internal/rosa: libev artifact
Required by gnutls.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:35:01 +09:00
15bff9e1a6 internal/rosa/git: determine reporting name from url
This is generally correct, and is a lot cleaner to call.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:42:16 +09:00
b948525c07 internal/rosa: nettle3 artifact
Removed after all packages upgrade for nettle 4.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:12:39 +09:00
9acbd16e9a internal/rosa/p11: explicitly enable libffi
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:05:07 +09:00
64e5a1068b internal/rosa: libtasn1 artifact
Optional dependency of p11-kit.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 17:20:12 +09:00
b6cbd49d8c internal/rosa: p11-kit artifact
Another package distributed in xz only. This is fetched from the git remote directly to avoid XZ Utils.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 16:34:30 +09:00
6913b9224a internal/rosa/git: recursively clone submodules
There is generally no reason to disable this, so it was not made optional.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 16:21:46 +09:00
9584958ecc internal/rosa/pkg-config: generate build system
This unfortunately pulls automake, libtool and their dependencies into stage2.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 15:31:24 +09:00
182 changed files with 17021 additions and 2298 deletions

31
.gitignore vendored
View File

@@ -1,27 +1,7 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
*.pkg
/hakurei
# Test binary, built with `go test -c`
# produced by tools and text editors
*.qcow2
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
# vendor/
# Go workspace file
go.work
go.work.sum
# env file
.env
.idea
.vscode
@@ -30,8 +10,5 @@ go.work.sum
/internal/pkg/testdata/testtool
/internal/rosa/hakurei_current.tar.gz
# release
/dist/hakurei-*
# interactive nixos vm
nixos.qcow2
# cmd/dist default destination
/dist

6
all.sh Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/sh -e
TOOLCHAIN_VERSION="$(go version)"
cd "$(dirname -- "$0")/"
echo "# Building cmd/dist using ${TOOLCHAIN_VERSION}."
go run -v --tags=dist ./cmd/dist

View File

@@ -2,7 +2,7 @@
package check
import (
"encoding/json"
"encoding"
"errors"
"fmt"
"path/filepath"
@@ -30,6 +30,16 @@ func (e AbsoluteError) Is(target error) bool {
// Absolute holds a pathname checked to be absolute.
type Absolute struct{ pathname unique.Handle[string] }
var (
_ encoding.TextAppender = new(Absolute)
_ encoding.TextMarshaler = new(Absolute)
_ encoding.TextUnmarshaler = new(Absolute)
_ encoding.BinaryAppender = new(Absolute)
_ encoding.BinaryMarshaler = new(Absolute)
_ encoding.BinaryUnmarshaler = new(Absolute)
)
// ok returns whether [Absolute] is not the zero value.
func (a *Absolute) ok() bool { return a != nil && *a != (Absolute{}) }
@@ -84,13 +94,16 @@ func (a *Absolute) Append(elem ...string) *Absolute {
// Dir calls [filepath.Dir] with [Absolute] as its argument.
func (a *Absolute) Dir() *Absolute { return unsafeAbs(filepath.Dir(a.String())) }
// GobEncode returns the checked pathname.
func (a *Absolute) GobEncode() ([]byte, error) {
return []byte(a.String()), nil
// AppendText appends the checked pathname.
func (a *Absolute) AppendText(data []byte) ([]byte, error) {
return append(data, a.String()...), nil
}
// GobDecode stores data if it represents an absolute pathname.
func (a *Absolute) GobDecode(data []byte) error {
// MarshalText returns the checked pathname.
func (a *Absolute) MarshalText() ([]byte, error) { return a.AppendText(nil) }
// UnmarshalText stores data if it represents an absolute pathname.
func (a *Absolute) UnmarshalText(data []byte) error {
pathname := string(data)
if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname)
@@ -99,23 +112,9 @@ func (a *Absolute) GobDecode(data []byte) error {
return nil
}
// MarshalJSON returns a JSON representation of the checked pathname.
func (a *Absolute) MarshalJSON() ([]byte, error) {
return json.Marshal(a.String())
}
// UnmarshalJSON stores data if it represents an absolute pathname.
func (a *Absolute) UnmarshalJSON(data []byte) error {
var pathname string
if err := json.Unmarshal(data, &pathname); err != nil {
return err
}
if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname)
}
a.pathname = unique.Make(pathname)
return nil
}
func (a *Absolute) AppendBinary(data []byte) ([]byte, error) { return a.AppendText(data) }
func (a *Absolute) MarshalBinary() ([]byte, error) { return a.MarshalText() }
func (a *Absolute) UnmarshalBinary(data []byte) error { return a.UnmarshalText(data) }
// SortAbs calls [slices.SortFunc] for a slice of [Absolute].
func SortAbs(x []*Absolute) {

View File

@@ -170,20 +170,20 @@ func TestCodecAbsolute(t *testing.T) {
{"good", MustAbs("/etc"),
nil,
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\b\xff\x80\x00\x04/etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x04/etc\x01\xfc\xc0\xed\x00\x00\x00",
"\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\b\xff\x80\x00\x04/etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x04/etc\x01\xfc\xc0\xed\x00\x00\x00",
`"/etc"`, `{"val":"/etc","magic":3236757504}`},
{"not absolute", nil,
AbsoluteError("etc"),
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\a\xff\x80\x00\x03etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x03etc\x01\xfb\x01\x81\xda\x00\x00\x00",
"\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\a\xff\x80\x00\x03etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x03etc\x01\xfb\x01\x81\xda\x00\x00\x00",
`"etc"`, `{"val":"etc","magic":3236757504}`},
{"zero", nil,
new(AbsoluteError),
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x04\xff\x80\x00\x00",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\f\xff\x84\x01\x00\x01\xfb\x01\x81\xda\x00\x00\x00",
"\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x04\xff\x80\x00\x00",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\f\xff\x84\x01\x00\x01\xfb\x01\x81\xda\x00\x00\x00",
`""`, `{"val":"","magic":3236757504}`},
}
@@ -347,15 +347,6 @@ func TestCodecAbsolute(t *testing.T) {
})
})
}
t.Run("json passthrough", func(t *testing.T) {
t.Parallel()
wantErr := "invalid character ':' looking for beginning of value"
if err := new(Absolute).UnmarshalJSON([]byte(":3")); err == nil || err.Error() != wantErr {
t.Errorf("UnmarshalJSON: error = %v, want %s", err, wantErr)
}
})
}
func TestAbsoluteWrap(t *testing.T) {

237
cmd/dist/main.go vendored Normal file
View File

@@ -0,0 +1,237 @@
//go:build dist
package main
import (
"archive/tar"
"compress/gzip"
"context"
"crypto/sha512"
_ "embed"
"encoding/hex"
"fmt"
"io"
"io/fs"
"log"
"os"
"os/exec"
"os/signal"
"path/filepath"
"runtime"
)
// getenv looks up an environment variable, and returns fallback if it is unset.
func getenv(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok {
return v
}
return fallback
}
// mustRun runs a command with the current process's environment and panics
// on error or non-zero exit code.
func mustRun(ctx context.Context, name string, arg ...string) {
cmd := exec.CommandContext(ctx, name, arg...)
cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
if err := cmd.Run(); err != nil {
panic(err)
}
}
//go:embed comp/_hakurei
var comp []byte
func main() {
fmt.Println()
log.SetFlags(0)
log.SetPrefix("# ")
version := getenv("HAKUREI_VERSION", "untagged")
prefix := getenv("PREFIX", "/usr")
destdir := getenv("DESTDIR", "dist")
if err := os.MkdirAll(destdir, 0755); err != nil {
log.Fatal(err)
}
s, err := os.MkdirTemp(destdir, ".dist.*")
if err != nil {
log.Fatal(err)
}
defer func() {
var code int
if err = os.RemoveAll(s); err != nil {
code = 1
log.Println(err)
}
if r := recover(); r != nil {
code = 1
log.Println(r)
}
os.Exit(code)
}()
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()
log.Println("Building hakurei.")
mustRun(ctx, "go", "generate", "./...")
mustRun(
ctx, "go", "build",
"-trimpath",
"-v", "-o", s,
"-ldflags=-s -w "+
"-buildid= -linkmode external -extldflags=-static "+
"-X hakurei.app/internal/info.buildVersion="+version+" "+
"-X hakurei.app/internal/info.hakureiPath="+prefix+"/bin/hakurei "+
"-X hakurei.app/internal/info.hsuPath="+prefix+"/bin/hsu "+
"-X main.hakureiPath="+prefix+"/bin/hakurei",
"./...",
)
fmt.Println()
log.Println("Testing Hakurei.")
mustRun(
ctx, "go", "test",
"-ldflags=-buildid= -linkmode external -extldflags=-static",
"./...",
)
fmt.Println()
log.Println("Creating distribution.")
const suffix = ".tar.gz"
distName := "hakurei-" + version + "-" + runtime.GOARCH
var f *os.File
if f, err = os.OpenFile(
filepath.Join(s, distName+suffix),
os.O_CREATE|os.O_EXCL|os.O_WRONLY,
0644,
); err != nil {
panic(err)
}
defer func() {
if f == nil {
return
}
if err = f.Close(); err != nil {
log.Println(err)
}
}()
h := sha512.New()
gw := gzip.NewWriter(io.MultiWriter(f, h))
tw := tar.NewWriter(gw)
mustWriteHeader := func(name string, size int64, mode os.FileMode) {
header := tar.Header{
Name: filepath.Join(distName, name),
Size: size,
Mode: int64(mode),
Uname: "root",
Gname: "root",
}
if mode&os.ModeDir != 0 {
header.Typeflag = tar.TypeDir
fmt.Printf("%s %s\n", mode, name)
} else {
header.Typeflag = tar.TypeReg
fmt.Printf("%s %s (%d bytes)\n", mode, name, size)
}
if err = tw.WriteHeader(&header); err != nil {
panic(err)
}
}
mustWriteFile := func(name string, data []byte, mode os.FileMode) {
mustWriteHeader(name, int64(len(data)), mode)
if mode&os.ModeDir != 0 {
return
}
if _, err = tw.Write(data); err != nil {
panic(err)
}
}
mustWriteFromPath := func(dst, src string, mode os.FileMode) {
var r *os.File
if r, err = os.Open(src); err != nil {
panic(err)
}
var fi os.FileInfo
if fi, err = r.Stat(); err != nil {
_ = r.Close()
panic(err)
}
if mode == 0 {
mode = fi.Mode()
}
mustWriteHeader(dst, fi.Size(), mode)
if _, err = io.Copy(tw, r); err != nil {
_ = r.Close()
panic(err)
} else if err = r.Close(); err != nil {
panic(err)
}
}
mustWriteFile(".", nil, fs.ModeDir|0755)
mustWriteFile("comp/", nil, os.ModeDir|0755)
mustWriteFile("comp/_hakurei", comp, 0644)
mustWriteFile("install.sh", []byte(`#!/bin/sh -e
cd "$(dirname -- "$0")" || exit 1
install -vDm0755 "bin/hakurei" "${DESTDIR}`+prefix+`/bin/hakurei"
install -vDm0755 "bin/sharefs" "${DESTDIR}`+prefix+`/bin/sharefs"
install -vDm4511 "bin/hsu" "${DESTDIR}`+prefix+`/bin/hsu"
if [ ! -f "${DESTDIR}/etc/hsurc" ]; then
install -vDm0400 "hsurc.default" "${DESTDIR}/etc/hsurc"
fi
install -vDm0644 "comp/_hakurei" "${DESTDIR}`+prefix+`/share/zsh/site-functions/_hakurei"
`), 0755)
mustWriteFromPath("README.md", "README.md", 0)
mustWriteFile("hsurc.default", []byte("1000 0"), 0400)
mustWriteFromPath("bin/hsu", filepath.Join(s, "hsu"), 04511)
for _, name := range []string{
"hakurei",
"sharefs",
} {
mustWriteFromPath(
filepath.Join("bin", name),
filepath.Join(s, name),
0,
)
}
if err = tw.Close(); err != nil {
panic(err)
} else if err = gw.Close(); err != nil {
panic(err)
} else if err = f.Close(); err != nil {
panic(err)
}
f = nil
if err = os.WriteFile(
filepath.Join(destdir, distName+suffix+".sha512"),
append(hex.AppendEncode(nil, h.Sum(nil)), " "+distName+suffix+"\n"...),
0644,
); err != nil {
panic(err)
}
if err = os.Rename(
filepath.Join(s, distName+suffix),
filepath.Join(destdir, distName+suffix),
); err != nil {
panic(err)
}
}

View File

@@ -38,8 +38,9 @@ var errSuccess = errors.New("success")
func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErrs, out io.Writer) command.Command {
var (
flagVerbose bool
flagJSON bool
flagVerbose bool
flagInsecure bool
flagJSON bool
)
c := command.New(out, log.Printf, "hakurei", func([]string) error {
msg.SwapVerbose(flagVerbose)
@@ -57,6 +58,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
return nil
}).
Flag(&flagVerbose, "v", command.BoolFlag(false), "Increase log verbosity").
Flag(&flagInsecure, "insecure", command.BoolFlag(false), "Allow use of insecure compatibility options").
Flag(&flagJSON, "json", command.BoolFlag(false), "Serialise output in JSON when applicable")
c.Command("shim", command.UsageInternal, func([]string) error { outcome.Shim(msg); return errSuccess })
@@ -75,7 +77,12 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
config.Container.Args = append(config.Container.Args, args[1:]...)
}
outcome.Main(ctx, msg, config, flagIdentifierFile)
var flags int
if flagInsecure {
flags |= hst.VAllowInsecure
}
outcome.Main(ctx, msg, config, flags, flagIdentifierFile)
panic("unreachable")
}).
Flag(&flagIdentifierFile, "identifier-fd", command.IntFlag(-1),
@@ -145,7 +152,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
}
}
var et hst.Enablement
var et hst.Enablements
if flagWayland {
et |= hst.EWayland
}
@@ -163,7 +170,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
ID: flagID,
Identity: flagIdentity,
Groups: flagGroups,
Enablements: hst.NewEnablements(et),
Enablements: &et,
Container: &hst.ContainerConfig{
Filesystem: []hst.FilesystemConfigJSON{
@@ -282,7 +289,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
}
}
outcome.Main(ctx, msg, &config, -1)
outcome.Main(ctx, msg, &config, 0, -1)
panic("unreachable")
}).
Flag(&flagDBusConfigSession, "dbus-config", command.StringFlag("builtin"),

View File

@@ -20,7 +20,7 @@ func TestHelp(t *testing.T) {
}{
{
"main", []string{}, `
Usage: hakurei [-h | --help] [-v] [--json] COMMAND [OPTIONS]
Usage: hakurei [-h | --help] [-v] [--insecure] [--json] COMMAND [OPTIONS]
Commands:
run Load and start container from configuration file

View File

@@ -56,7 +56,7 @@ func printShowInstance(
t := newPrinter(output)
defer t.MustFlush()
if err := config.Validate(); err != nil {
if err := config.Validate(hst.VAllowInsecure); err != nil {
valid = false
if m, ok := message.GetMessage(err); ok {
mustPrint(output, "Error: "+m+"!\n\n")

View File

@@ -32,7 +32,7 @@ var (
PID: 0xbeef,
ShimPID: 0xcafe,
Config: &hst.Config{
Enablements: hst.NewEnablements(hst.EWayland | hst.EPipeWire),
Enablements: new(hst.EWayland | hst.EPipeWire),
Identity: 1,
Container: &hst.ContainerConfig{
Shell: check.MustAbs("/bin/sh"),

91
cmd/mbf/cache.go Normal file
View File

@@ -0,0 +1,91 @@
package main
import (
"context"
"os"
"path/filepath"
"testing"
"hakurei.app/check"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
// cache refers to an instance of [pkg.Cache] that might be open.
type cache struct {
ctx context.Context
msg message.Msg
// Should generally not be used directly.
c *pkg.Cache
cures, jobs int
hostAbstract, idle bool
base string
}
// open opens the underlying [pkg.Cache].
func (cache *cache) open() (err error) {
if cache.c != nil {
return os.ErrInvalid
}
var base *check.Absolute
if cache.base, err = filepath.Abs(cache.base); err != nil {
return
} else if base, err = check.NewAbs(cache.base); err != nil {
return
}
var flags int
if cache.idle {
flags |= pkg.CSchedIdle
}
if cache.hostAbstract {
flags |= pkg.CHostAbstract
}
done := make(chan struct{})
defer close(done)
go func() {
select {
case <-cache.ctx.Done():
if testing.Testing() {
return
}
os.Exit(2)
case <-done:
return
}
}()
cache.msg.Verbosef("opening cache at %s", base)
cache.c, err = pkg.Open(
cache.ctx,
cache.msg,
flags,
cache.cures,
cache.jobs,
base,
)
return
}
// Close closes the underlying [pkg.Cache] if it is open.
func (cache *cache) Close() {
if cache.c != nil {
cache.c.Close()
}
}
// Do calls f on the underlying cache and returns its error value.
func (cache *cache) Do(f func(cache *pkg.Cache) error) error {
if cache.c == nil {
if err := cache.open(); err != nil {
return err
}
}
return f(cache.c)
}

37
cmd/mbf/cache_test.go Normal file
View File

@@ -0,0 +1,37 @@
package main
import (
"log"
"os"
"testing"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
func TestCache(t *testing.T) {
t.Parallel()
cm := cache{
ctx: t.Context(),
msg: message.New(log.New(os.Stderr, "check: ", 0)),
base: t.TempDir(),
hostAbstract: true, idle: true,
}
defer cm.Close()
cm.Close()
if err := cm.open(); err != nil {
t.Fatalf("open: error = %v", err)
}
if err := cm.open(); err != os.ErrInvalid {
t.Errorf("(duplicate) open: error = %v", err)
}
if err := cm.Do(func(cache *pkg.Cache) error {
return cache.Scrub(0)
}); err != nil {
t.Errorf("Scrub: error = %v", err)
}
}

354
cmd/mbf/daemon.go Normal file
View File

@@ -0,0 +1,354 @@
package main
import (
"context"
"encoding/binary"
"errors"
"io"
"log"
"math"
"net"
"os"
"sync"
"syscall"
"testing"
"time"
"unique"
"hakurei.app/check"
"hakurei.app/internal/pkg"
)
// daemonTimeout is the maximum amount of time cureFromIR will wait on I/O.
const daemonTimeout = 30 * time.Second
// daemonDeadline returns the deadline corresponding to daemonTimeout, or the
// zero value when running in a test.
func daemonDeadline() time.Time {
if testing.Testing() {
return time.Time{}
}
return time.Now().Add(daemonTimeout)
}
const (
// remoteNoReply notifies that the client will not receive a cure reply.
remoteNoReply = 1 << iota
)
// cureFromIR services an IR curing request.
func cureFromIR(
cache *pkg.Cache,
conn net.Conn,
flags uint64,
) (pkg.Artifact, error) {
a, decodeErr := cache.NewDecoder(conn).Decode()
if decodeErr != nil {
_, err := conn.Write([]byte("\x00" + decodeErr.Error()))
return nil, errors.Join(decodeErr, err, conn.Close())
}
pathname, _, cureErr := cache.Cure(a)
if flags&remoteNoReply != 0 {
return a, errors.Join(cureErr, conn.Close())
}
if err := conn.SetWriteDeadline(daemonDeadline()); err != nil {
return a, errors.Join(cureErr, err, conn.Close())
}
if cureErr != nil {
_, err := conn.Write([]byte("\x00" + cureErr.Error()))
return a, errors.Join(cureErr, err, conn.Close())
}
_, err := conn.Write([]byte(pathname.String()))
if testing.Testing() && errors.Is(err, io.ErrClosedPipe) {
return a, nil
}
return a, errors.Join(err, conn.Close())
}
const (
// specialCancel is a message consisting of a single identifier referring
// to a curing artifact to be cancelled.
specialCancel = iota
// specialAbort requests for all pending cures to be aborted. It has no
// message body.
specialAbort
// remoteSpecial denotes a special message with custom layout.
remoteSpecial = math.MaxUint64
)
// writeSpecialHeader writes the header of a remoteSpecial message.
func writeSpecialHeader(conn net.Conn, kind uint64) error {
var sh [16]byte
binary.LittleEndian.PutUint64(sh[:], remoteSpecial)
binary.LittleEndian.PutUint64(sh[8:], kind)
if n, err := conn.Write(sh[:]); err != nil {
return err
} else if n != len(sh) {
return io.ErrShortWrite
}
return nil
}
// cancelIdent reads an identifier from conn and cancels the corresponding cure.
func cancelIdent(
cache *pkg.Cache,
conn net.Conn,
) (*pkg.ID, bool, error) {
var ident pkg.ID
if _, err := io.ReadFull(conn, ident[:]); err != nil {
return nil, false, errors.Join(err, conn.Close())
}
ok := cache.Cancel(unique.Make(ident))
return &ident, ok, conn.Close()
}
// serve services connections from a [net.UnixListener].
func serve(
ctx context.Context,
log *log.Logger,
cm *cache,
ul *net.UnixListener,
) error {
ul.SetUnlinkOnClose(true)
if cm.c == nil {
if err := cm.open(); err != nil {
return errors.Join(err, ul.Close())
}
}
var wg sync.WaitGroup
defer wg.Wait()
wg.Go(func() {
for {
if ctx.Err() != nil {
break
}
conn, err := ul.AcceptUnix()
if err != nil {
if !errors.Is(err, os.ErrDeadlineExceeded) {
log.Println(err)
}
continue
}
wg.Go(func() {
done := make(chan struct{})
defer close(done)
go func() {
select {
case <-ctx.Done():
_ = conn.SetDeadline(time.Now())
case <-done:
return
}
}()
if _err := conn.SetReadDeadline(daemonDeadline()); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
var word [8]byte
if _, _err := io.ReadFull(conn, word[:]); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
flags := binary.LittleEndian.Uint64(word[:])
if flags == remoteSpecial {
if _, _err := io.ReadFull(conn, word[:]); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
switch special := binary.LittleEndian.Uint64(word[:]); special {
default:
log.Printf("invalid special %d", special)
case specialCancel:
if id, ok, _err := cancelIdent(cm.c, conn); _err != nil {
log.Println(_err)
} else if !ok {
log.Println(
"attempting to cancel invalid artifact",
pkg.Encode(*id),
)
} else {
log.Println(
"cancelled artifact",
pkg.Encode(*id),
)
}
case specialAbort:
log.Println("aborting all pending cures")
cm.c.Abort()
if _err := conn.Close(); _err != nil {
log.Println(_err)
}
}
return
}
if a, _err := cureFromIR(cm.c, conn, flags); _err != nil {
log.Println(_err)
} else {
log.Printf(
"fulfilled artifact %s",
pkg.Encode(cm.c.Ident(a).Value()),
)
}
})
}
})
<-ctx.Done()
if err := ul.SetDeadline(time.Now()); err != nil {
return errors.Join(err, ul.Close())
}
wg.Wait()
return ul.Close()
}
// dial wraps [net.DialUnix] with a context.
func dial(ctx context.Context, addr *net.UnixAddr) (
done chan<- struct{},
conn *net.UnixConn,
err error,
) {
conn, err = net.DialUnix("unix", nil, addr)
if err != nil {
return
}
d := make(chan struct{})
done = d
go func() {
select {
case <-ctx.Done():
_ = conn.SetDeadline(time.Now())
case <-d:
return
}
}()
return
}
// cureRemote cures a [pkg.Artifact] on a daemon.
func cureRemote(
ctx context.Context,
addr *net.UnixAddr,
a pkg.Artifact,
flags uint64,
) (*check.Absolute, error) {
if flags == remoteSpecial {
return nil, syscall.EINVAL
}
done, conn, err := dial(ctx, addr)
if err != nil {
return nil, err
}
defer close(done)
if n, flagErr := conn.Write(binary.LittleEndian.AppendUint64(nil, flags)); flagErr != nil {
return nil, errors.Join(flagErr, conn.Close())
} else if n != 8 {
return nil, errors.Join(io.ErrShortWrite, conn.Close())
}
if err = pkg.NewIR().EncodeAll(conn, a); err != nil {
return nil, errors.Join(err, conn.Close())
} else if err = conn.CloseWrite(); err != nil {
return nil, errors.Join(err, conn.Close())
}
if flags&remoteNoReply != 0 {
return nil, conn.Close()
}
payload, recvErr := io.ReadAll(conn)
if err = errors.Join(recvErr, conn.Close()); err != nil {
if errors.Is(err, os.ErrDeadlineExceeded) {
if cancelErr := ctx.Err(); cancelErr != nil {
err = cancelErr
}
}
return nil, err
}
if len(payload) > 0 && payload[0] == 0 {
return nil, errors.New(string(payload[1:]))
}
var p *check.Absolute
p, err = check.NewAbs(string(payload))
return p, err
}
// cancelRemote cancels a [pkg.Artifact] curing on a daemon.
func cancelRemote(
ctx context.Context,
addr *net.UnixAddr,
a pkg.Artifact,
wait bool,
) error {
done, conn, err := dial(ctx, addr)
if err != nil {
return err
}
defer close(done)
if err = writeSpecialHeader(conn, specialCancel); err != nil {
return errors.Join(err, conn.Close())
}
var n int
id := pkg.NewIR().Ident(a).Value()
if n, err = conn.Write(id[:]); err != nil {
return errors.Join(err, conn.Close())
} else if n != len(id) {
return errors.Join(io.ErrShortWrite, conn.Close())
}
if wait {
if _, err = conn.Read(make([]byte, 1)); err == io.EOF {
err = nil
}
}
return errors.Join(err, conn.Close())
}
// abortRemote aborts all [pkg.Artifact] curing on a daemon.
func abortRemote(
ctx context.Context,
addr *net.UnixAddr,
wait bool,
) error {
done, conn, err := dial(ctx, addr)
if err != nil {
return err
}
defer close(done)
err = writeSpecialHeader(conn, specialAbort)
if wait && err == nil {
if _, err = conn.Read(make([]byte, 1)); err == io.EOF {
err = nil
}
}
return errors.Join(err, conn.Close())
}

146
cmd/mbf/daemon_test.go Normal file
View File

@@ -0,0 +1,146 @@
package main
import (
"bytes"
"context"
"errors"
"io"
"log"
"net"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"hakurei.app/check"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
func TestNoReply(t *testing.T) {
t.Parallel()
if !daemonDeadline().IsZero() {
t.Fatal("daemonDeadline did not return the zero value")
}
c, err := pkg.Open(
t.Context(),
message.New(log.New(os.Stderr, "cir: ", 0)),
0, 0, 0,
check.MustAbs(t.TempDir()),
)
if err != nil {
t.Fatalf("Open: error = %v", err)
}
defer c.Close()
client, server := net.Pipe()
done := make(chan struct{})
go func() {
defer close(done)
go func() {
<-t.Context().Done()
if _err := client.SetDeadline(time.Now()); _err != nil && !errors.Is(_err, io.ErrClosedPipe) {
panic(_err)
}
}()
if _err := c.EncodeAll(
client,
pkg.NewFile("check", []byte{0}),
); _err != nil {
panic(_err)
} else if _err = client.Close(); _err != nil {
panic(_err)
}
}()
a, cureErr := cureFromIR(c, server, remoteNoReply)
if cureErr != nil {
t.Fatalf("cureFromIR: error = %v", cureErr)
}
<-done
wantIdent := pkg.MustDecode("fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG")
if gotIdent := c.Ident(a).Value(); gotIdent != wantIdent {
t.Errorf(
"cureFromIR: %s, want %s",
pkg.Encode(gotIdent), pkg.Encode(wantIdent),
)
}
}
func TestDaemon(t *testing.T) {
t.Parallel()
var buf bytes.Buffer
logger := log.New(&buf, "daemon: ", 0)
addr := net.UnixAddr{
Name: filepath.Join(t.TempDir(), "daemon"),
Net: "unix",
}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
cm := cache{
ctx: ctx,
msg: message.New(logger),
base: t.TempDir(),
}
defer cm.Close()
ul, err := net.ListenUnix("unix", &addr)
if err != nil {
t.Fatalf("ListenUnix: error = %v", err)
}
done := make(chan struct{})
go func() {
defer close(done)
if _err := serve(ctx, logger, &cm, ul); _err != nil {
panic(_err)
}
}()
if err = cancelRemote(ctx, &addr, pkg.NewFile("nonexistent", nil), true); err != nil {
t.Fatalf("cancelRemote: error = %v", err)
}
if err = abortRemote(ctx, &addr, true); err != nil {
t.Fatalf("abortRemote: error = %v", err)
}
// keep this last for synchronisation
var p *check.Absolute
p, err = cureRemote(ctx, &addr, pkg.NewFile("check", []byte{0}), 0)
if err != nil {
t.Fatalf("cureRemote: error = %v", err)
}
cancel()
<-done
const want = "fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG"
if got := filepath.Base(p.String()); got != want {
t.Errorf("cureRemote: %s, want %s", got, want)
}
wantLog := []string{
"",
"daemon: aborting all pending cures",
"daemon: attempting to cancel invalid artifact kQm9fmnCmXST1-MMmxzcau2oKZCXXrlZydo4PkeV5hO_2PKfeC8t98hrbV_ZZx_j",
"daemon: fulfilled artifact fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG",
}
gotLog := strings.Split(buf.String(), "\n")
slices.Sort(gotLog)
if !slices.Equal(gotLog, wantLog) {
t.Errorf(
"serve: logged\n%s\nwant\n%s",
strings.Join(gotLog, "\n"), strings.Join(wantLog, "\n"),
)
}
}

127
cmd/mbf/info.go Normal file
View File

@@ -0,0 +1,127 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"strings"
"hakurei.app/internal/pkg"
"hakurei.app/internal/rosa"
)
// commandInfo implements the info subcommand.
func commandInfo(
cm *cache,
args []string,
w io.Writer,
writeStatus bool,
reportPath string,
) (err error) {
if len(args) == 0 {
return errors.New("info requires at least 1 argument")
}
var r *rosa.Report
if reportPath != "" {
if r, err = rosa.OpenReport(reportPath); err != nil {
return err
}
defer func() {
if closeErr := r.Close(); err == nil {
err = closeErr
}
}()
defer r.HandleAccess(&err)()
}
// recovered by HandleAccess
mustPrintln := func(a ...any) {
if _, _err := fmt.Fprintln(w, a...); _err != nil {
panic(_err)
}
}
mustPrint := func(a ...any) {
if _, _err := fmt.Fprint(w, a...); _err != nil {
panic(_err)
}
}
for i, name := range args {
if p, ok := rosa.ResolveName(name); !ok {
return fmt.Errorf("unknown artifact %q", name)
} else {
var suffix string
if version := rosa.Std.Version(p); version != rosa.Unversioned {
suffix += "-" + version
}
mustPrintln("name : " + name + suffix)
meta := rosa.GetMetadata(p)
mustPrintln("description : " + meta.Description)
if meta.Website != "" {
mustPrintln("website : " +
strings.TrimSuffix(meta.Website, "/"))
}
if len(meta.Dependencies) > 0 {
mustPrint("depends on :")
for _, d := range meta.Dependencies {
s := rosa.GetMetadata(d).Name
if version := rosa.Std.Version(d); version != rosa.Unversioned {
s += "-" + version
}
mustPrint(" " + s)
}
mustPrintln()
}
const statusPrefix = "status : "
if writeStatus {
if r == nil {
var f io.ReadSeekCloser
err = cm.Do(func(cache *pkg.Cache) (err error) {
f, err = cache.OpenStatus(rosa.Std.Load(p))
return
})
if err != nil {
if errors.Is(err, os.ErrNotExist) {
mustPrintln(
statusPrefix + "not yet cured",
)
} else {
return
}
} else {
mustPrint(statusPrefix)
_, err = io.Copy(w, f)
if err = errors.Join(err, f.Close()); err != nil {
return
}
}
} else if err = cm.Do(func(cache *pkg.Cache) (err error) {
status, n := r.ArtifactOf(cache.Ident(rosa.Std.Load(p)))
if status == nil {
mustPrintln(
statusPrefix + "not in report",
)
} else {
mustPrintln("size :", n)
mustPrint(statusPrefix)
if _, err = w.Write(status); err != nil {
return
}
}
return
}); err != nil {
return
}
}
if i != len(args)-1 {
mustPrintln()
}
}
}
return nil
}

170
cmd/mbf/info_test.go Normal file
View File

@@ -0,0 +1,170 @@
package main
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"reflect"
"strings"
"syscall"
"testing"
"unsafe"
"hakurei.app/internal/pkg"
"hakurei.app/internal/rosa"
"hakurei.app/message"
)
func TestInfo(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
args []string
status map[string]string
report string
want string
wantErr any
}{
{"qemu", []string{"qemu"}, nil, "", `
name : qemu-` + rosa.Std.Version(rosa.QEMU) + `
description : a generic and open source machine emulator and virtualizer
website : https://www.qemu.org
depends on : glib-` + rosa.Std.Version(rosa.GLib) + ` zstd-` + rosa.Std.Version(rosa.Zstd) + `
`, nil},
{"multi", []string{"hakurei", "hakurei-dist"}, nil, "", `
name : hakurei-` + rosa.Std.Version(rosa.Hakurei) + `
description : low-level userspace tooling for Rosa OS
website : https://hakurei.app
name : hakurei-dist-` + rosa.Std.Version(rosa.HakureiDist) + `
description : low-level userspace tooling for Rosa OS (distribution tarball)
website : https://hakurei.app
`, nil},
{"nonexistent", []string{"zlib", "\x00"}, nil, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
`, fmt.Errorf("unknown artifact %q", "\x00")},
{"status cache", []string{"zlib", "zstd"}, map[string]string{
"zstd": "internal/pkg (amd64) on satori\n",
"hakurei": "internal/pkg (amd64) on satori\n\n",
}, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
status : not yet cured
name : zstd-` + rosa.Std.Version(rosa.Zstd) + `
description : a fast compression algorithm
website : https://facebook.github.io/zstd
status : internal/pkg (amd64) on satori
`, nil},
{"status cache perm", []string{"zlib"}, map[string]string{
"zlib": "\x00",
}, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
`, func(cm *cache) error {
return &os.PathError{
Op: "open",
Path: filepath.Join(cm.base, "status", pkg.Encode(cm.c.Ident(rosa.Std.Load(rosa.Zlib)).Value())),
Err: syscall.EACCES,
}
}},
{"status report", []string{"zlib"}, nil, strings.Repeat("\x00", len(pkg.Checksum{})+8), `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
status : not in report
`, nil},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var (
cm *cache
buf strings.Builder
rp string
)
if tc.status != nil || tc.report != "" {
cm = &cache{
ctx: context.Background(),
msg: message.New(log.New(os.Stderr, "info: ", 0)),
base: t.TempDir(),
}
defer cm.Close()
}
if tc.report != "" {
rp = filepath.Join(t.TempDir(), "report")
if err := os.WriteFile(
rp,
unsafe.Slice(unsafe.StringData(tc.report), len(tc.report)),
0400,
); err != nil {
t.Fatal(err)
}
}
if tc.status != nil {
for name, status := range tc.status {
p, ok := rosa.ResolveName(name)
if !ok {
t.Fatalf("invalid name %q", name)
}
perm := os.FileMode(0400)
if status == "\x00" {
perm = 0
}
if err := cm.Do(func(cache *pkg.Cache) error {
return os.WriteFile(filepath.Join(
cm.base,
"status",
pkg.Encode(cache.Ident(rosa.Std.Load(p)).Value()),
), unsafe.Slice(unsafe.StringData(status), len(status)), perm)
}); err != nil {
t.Fatalf("Do: error = %v", err)
}
}
}
var wantErr error
switch c := tc.wantErr.(type) {
case error:
wantErr = c
case func(cm *cache) error:
wantErr = c(cm)
default:
if tc.wantErr != nil {
t.Fatalf("invalid wantErr %#v", tc.wantErr)
}
}
if err := commandInfo(
cm,
tc.args,
&buf,
cm != nil,
rp,
); !reflect.DeepEqual(err, wantErr) {
t.Fatalf("commandInfo: error = %v, want %v", err, wantErr)
}
if got := buf.String(); got != strings.TrimPrefix(tc.want, "\n") {
t.Errorf("commandInfo:\n%s\nwant\n%s", got, tc.want)
}
})
}
}

View File

@@ -14,16 +14,17 @@ package main
import (
"context"
"crypto/sha512"
"errors"
"fmt"
"io"
"log"
"net"
"os"
"os/signal"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
"syscall"
@@ -53,77 +54,77 @@ func main() {
log.Fatal("this program must not run as root")
}
var cache *pkg.Cache
ctx, stop := signal.NotifyContext(context.Background(),
syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
defer stop()
defer func() {
if cache != nil {
cache.Close()
}
if r := recover(); r != nil {
fmt.Println(r)
log.Fatal("consider scrubbing the on-disk cache")
}
}()
var cm cache
defer func() { cm.Close() }()
var (
flagQuiet bool
flagCures int
flagBase string
flagTShift int
flagIdle bool
flagQuiet bool
addr net.UnixAddr
)
c := command.New(os.Stderr, log.Printf, "mbf", func([]string) (err error) {
c := command.New(os.Stderr, log.Printf, "mbf", func([]string) error {
msg.SwapVerbose(!flagQuiet)
flagBase = os.ExpandEnv(flagBase)
if flagBase == "" {
flagBase = "cache"
cm.ctx, cm.msg = ctx, msg
cm.base = os.ExpandEnv(cm.base)
if cm.base == "" {
cm.base = "cache"
}
var base *check.Absolute
if flagBase, err = filepath.Abs(flagBase); err != nil {
return
} else if base, err = check.NewAbs(flagBase); err != nil {
return
}
if cache, err = pkg.Open(ctx, msg, flagCures, base); err == nil {
if flagTShift < 0 {
cache.SetThreshold(0)
} else if flagTShift > 31 {
cache.SetThreshold(1 << 31)
} else {
cache.SetThreshold(1 << flagTShift)
}
addr.Net = "unix"
addr.Name = os.ExpandEnv(addr.Name)
if addr.Name == "" {
addr.Name = filepath.Join(cm.base, "daemon")
}
if flagIdle {
pkg.SetSchedIdle = true
}
return
return nil
}).Flag(
&flagQuiet,
"q", command.BoolFlag(false),
"Do not print cure messages",
).Flag(
&flagCures,
&cm.cures,
"cures", command.IntFlag(0),
"Maximum number of dependencies to cure at any given time",
).Flag(
&flagBase,
&cm.jobs,
"jobs", command.IntFlag(0),
"Preferred number of jobs to run, when applicable",
).Flag(
&cm.base,
"d", command.StringFlag("$MBF_CACHE_DIR"),
"Directory to store cured artifacts",
).Flag(
&flagTShift,
"tshift", command.IntFlag(-1),
"Dependency graph size exponent, to the power of 2",
).Flag(
&flagIdle,
&cm.idle,
"sched-idle", command.BoolFlag(false),
"Set SCHED_IDLE scheduling policy",
).Flag(
&cm.hostAbstract,
"host-abstract", command.BoolFlag(
os.Getenv("MBF_HOST_ABSTRACT") != "",
),
"Do not restrict networked cure containers from connecting to host "+
"abstract UNIX sockets",
).Flag(
&addr.Name,
"socket", command.StringFlag("$MBF_DAEMON_SOCKET"),
"Pathname of socket to bind to",
)
c.NewCommand(
"checksum", "Compute checksum of data read from standard input",
func([]string) error {
go func() { <-ctx.Done(); os.Exit(1) }()
h := sha512.New384()
if _, err := io.Copy(h, os.Stdin); err != nil {
return err
}
log.Println(pkg.Encode(pkg.Checksum(h.Sum(nil))))
return nil
},
)
{
@@ -137,7 +138,9 @@ func main() {
if flagShifts < 0 || flagShifts > 31 {
flagShifts = 12
}
return cache.Scrub(runtime.NumCPU() << flagShifts)
return cm.Do(func(cache *pkg.Cache) error {
return cache.Scrub(runtime.NumCPU() << flagShifts)
})
},
).Flag(
&flagShifts,
@@ -155,105 +158,17 @@ func main() {
"info",
"Display out-of-band metadata of an artifact",
func(args []string) (err error) {
if len(args) == 0 {
return errors.New("info requires at least 1 argument")
}
var r *rosa.Report
if flagReport != "" {
if r, err = rosa.OpenReport(flagReport); err != nil {
return err
}
defer func() {
if closeErr := r.Close(); err == nil {
err = closeErr
}
}()
defer r.HandleAccess(&err)()
}
for i, name := range args {
if p, ok := rosa.ResolveName(name); !ok {
return fmt.Errorf("unknown artifact %q", name)
} else {
var suffix string
if version := rosa.Std.Version(p); version != rosa.Unversioned {
suffix += "-" + version
}
fmt.Println("name : " + name + suffix)
meta := rosa.GetMetadata(p)
fmt.Println("description : " + meta.Description)
if meta.Website != "" {
fmt.Println("website : " +
strings.TrimSuffix(meta.Website, "/"))
}
if len(meta.Dependencies) > 0 {
fmt.Print("depends on :")
for _, d := range meta.Dependencies {
s := rosa.GetMetadata(d).Name
if version := rosa.Std.Version(d); version != rosa.Unversioned {
s += "-" + version
}
fmt.Print(" " + s)
}
fmt.Println()
}
const statusPrefix = "status : "
if flagStatus {
if r == nil {
var f io.ReadSeekCloser
f, err = cache.OpenStatus(rosa.Std.Load(p))
if err != nil {
if errors.Is(err, os.ErrNotExist) {
fmt.Println(
statusPrefix + "not yet cured",
)
} else {
return
}
} else {
fmt.Print(statusPrefix)
_, err = io.Copy(os.Stdout, f)
if err = errors.Join(err, f.Close()); err != nil {
return
}
}
} else {
status, n := r.ArtifactOf(cache.Ident(rosa.Std.Load(p)))
if status == nil {
fmt.Println(
statusPrefix + "not in report",
)
} else {
fmt.Println("size :", n)
fmt.Print(statusPrefix)
if _, err = os.Stdout.Write(status); err != nil {
return
}
}
}
}
if i != len(args)-1 {
fmt.Println()
}
}
}
return nil
return commandInfo(&cm, args, os.Stdout, flagStatus, flagReport)
},
).
Flag(
&flagStatus,
"status", command.BoolFlag(false),
"Display cure status if available",
).
Flag(
&flagReport,
"report", command.StringFlag(""),
"Load cure status from this report file instead of cache",
)
).Flag(
&flagStatus,
"status", command.BoolFlag(false),
"Display cure status if available",
).Flag(
&flagReport,
"report", command.StringFlag(""),
"Load cure status from this report file instead of cache",
)
}
c.NewCommand(
@@ -287,7 +202,9 @@ func main() {
if ext.Isatty(int(w.Fd())) {
return errors.New("output appears to be a terminal")
}
return rosa.WriteReport(msg, w, cache)
return cm.Do(func(cache *pkg.Cache) error {
return rosa.WriteReport(msg, w, cache)
})
},
)
@@ -350,14 +267,26 @@ func main() {
" package(s) are out of date"))
}
return errors.Join(errs...)
}).
Flag(
&flagJobs,
"j", command.IntFlag(32),
"Maximum number of simultaneous connections",
)
}).Flag(
&flagJobs,
"j", command.IntFlag(32),
"Maximum number of simultaneous connections",
)
}
c.NewCommand(
"daemon",
"Service artifact IR with Rosa OS extensions",
func(args []string) error {
ul, err := net.ListenUnix("unix", &addr)
if err != nil {
return err
}
log.Printf("listening on pathname socket at %s", addr.Name)
return serve(ctx, log.Default(), &cm, ul)
},
)
{
var (
flagGentoo string
@@ -382,25 +311,37 @@ func main() {
rosa.SetGentooStage3(flagGentoo, checksum)
}
_, _, _, stage1 := (t - 2).NewLLVM()
_, _, _, stage2 := (t - 1).NewLLVM()
_, _, _, stage3 := t.NewLLVM()
var (
pathname *check.Absolute
checksum [2]unique.Handle[pkg.Checksum]
)
if pathname, _, err = cache.Cure(stage1); err != nil {
return err
if err = cm.Do(func(cache *pkg.Cache) (err error) {
pathname, _, err = cache.Cure(
(t - 2).Load(rosa.Clang),
)
return
}); err != nil {
return
}
log.Println("stage1:", pathname)
if pathname, checksum[0], err = cache.Cure(stage2); err != nil {
return err
if err = cm.Do(func(cache *pkg.Cache) (err error) {
pathname, checksum[0], err = cache.Cure(
(t - 1).Load(rosa.Clang),
)
return
}); err != nil {
return
}
log.Println("stage2:", pathname)
if pathname, checksum[1], err = cache.Cure(stage3); err != nil {
return err
if err = cm.Do(func(cache *pkg.Cache) (err error) {
pathname, checksum[1], err = cache.Cure(
t.Load(rosa.Clang),
)
return
}); err != nil {
return
}
log.Println("stage3:", pathname)
@@ -417,39 +358,41 @@ func main() {
}
if flagStage0 {
if pathname, _, err = cache.Cure(
t.Load(rosa.Stage0),
); err != nil {
return err
if err = cm.Do(func(cache *pkg.Cache) (err error) {
pathname, _, err = cache.Cure(
t.Load(rosa.Stage0),
)
return
}); err != nil {
return
}
log.Println(pathname)
}
return
},
).
Flag(
&flagGentoo,
"gentoo", command.StringFlag(""),
"Bootstrap from a Gentoo stage3 tarball",
).
Flag(
&flagChecksum,
"checksum", command.StringFlag(""),
"Checksum of Gentoo stage3 tarball",
).
Flag(
&flagStage0,
"stage0", command.BoolFlag(false),
"Create bootstrap stage0 tarball",
)
).Flag(
&flagGentoo,
"gentoo", command.StringFlag(""),
"Bootstrap from a Gentoo stage3 tarball",
).Flag(
&flagChecksum,
"checksum", command.StringFlag(""),
"Checksum of Gentoo stage3 tarball",
).Flag(
&flagStage0,
"stage0", command.BoolFlag(false),
"Create bootstrap stage0 tarball",
)
}
{
var (
flagDump string
flagEnter bool
flagExport string
flagDump string
flagEnter bool
flagExport string
flagRemote bool
flagNoReply bool
)
c.NewCommand(
"cure",
@@ -465,7 +408,11 @@ func main() {
switch {
default:
pathname, _, err := cache.Cure(rosa.Std.Load(p))
var pathname *check.Absolute
err := cm.Do(func(cache *pkg.Cache) (err error) {
pathname, _, err = cache.Cure(rosa.Std.Load(p))
return
})
if err != nil {
return err
}
@@ -505,7 +452,7 @@ func main() {
return err
}
if err = cache.EncodeAll(f, rosa.Std.Load(p)); err != nil {
if err = pkg.NewIR().EncodeAll(f, rosa.Std.Load(p)); err != nil {
_ = f.Close()
return err
}
@@ -513,33 +460,68 @@ func main() {
return f.Close()
case flagEnter:
return cache.EnterExec(
ctx,
rosa.Std.Load(p),
true, os.Stdin, os.Stdout, os.Stderr,
rosa.AbsSystem.Append("bin", "mksh"),
"sh",
)
return cm.Do(func(cache *pkg.Cache) error {
return cache.EnterExec(
ctx,
rosa.Std.Load(p),
true, os.Stdin, os.Stdout, os.Stderr,
rosa.AbsSystem.Append("bin", "mksh"),
"sh",
)
})
case flagRemote:
var flags uint64
if flagNoReply {
flags |= remoteNoReply
}
a := rosa.Std.Load(p)
pathname, err := cureRemote(ctx, &addr, a, flags)
if !flagNoReply && err == nil {
log.Println(pathname)
}
if errors.Is(err, context.Canceled) {
cc, cancel := context.WithDeadline(context.Background(), daemonDeadline())
defer cancel()
if _err := cancelRemote(cc, &addr, a, false); _err != nil {
log.Println(err)
}
}
return err
}
},
).
Flag(
&flagDump,
"dump", command.StringFlag(""),
"Write IR to specified pathname and terminate",
).
Flag(
&flagExport,
"export", command.StringFlag(""),
"Export cured artifact to specified pathname",
).
Flag(
&flagEnter,
"enter", command.BoolFlag(false),
"Enter cure container with an interactive shell",
)
).Flag(
&flagDump,
"dump", command.StringFlag(""),
"Write IR to specified pathname and terminate",
).Flag(
&flagExport,
"export", command.StringFlag(""),
"Export cured artifact to specified pathname",
).Flag(
&flagEnter,
"enter", command.BoolFlag(false),
"Enter cure container with an interactive shell",
).Flag(
&flagRemote,
"daemon", command.BoolFlag(false),
"Cure artifact on the daemon",
).Flag(
&flagNoReply,
"no-reply", command.BoolFlag(false),
"Do not receive a reply from the daemon",
)
}
c.NewCommand(
"abort",
"Abort all pending cures on the daemon",
func([]string) error { return abortRemote(ctx, &addr, false) },
)
{
var (
flagNet bool
@@ -551,7 +533,7 @@ func main() {
"shell",
"Interactive shell in the specified Rosa OS environment",
func(args []string) error {
presets := make([]rosa.PArtifact, len(args))
presets := make([]rosa.PArtifact, len(args)+3)
for i, arg := range args {
p, ok := rosa.ResolveName(arg)
if !ok {
@@ -559,21 +541,24 @@ func main() {
}
presets[i] = p
}
base := rosa.Clang
if !flagWithToolchain {
base = rosa.Musl
}
presets = append(presets,
base,
rosa.Mksh,
rosa.Toybox,
)
root := make(pkg.Collect, 0, 6+len(args))
root = rosa.Std.AppendPresets(root, presets...)
if flagWithToolchain {
musl, compilerRT, runtimes, clang := (rosa.Std - 1).NewLLVM()
root = append(root, musl, compilerRT, runtimes, clang)
} else {
root = append(root, rosa.Std.Load(rosa.Musl))
}
root = append(root,
rosa.Std.Load(rosa.Mksh),
rosa.Std.Load(rosa.Toybox),
)
if _, _, err := cache.Cure(&root); err == nil {
if err := cm.Do(func(cache *pkg.Cache) error {
_, _, err := cache.Cure(&root)
return err
}); err == nil {
return errors.New("unreachable")
} else if !pkg.IsCollected(err) {
return err
@@ -585,11 +570,22 @@ func main() {
}
cured := make(map[pkg.Artifact]cureRes)
for _, a := range root {
pathname, checksum, err := cache.Cure(a)
if err != nil {
if err := cm.Do(func(cache *pkg.Cache) error {
pathname, checksum, err := cache.Cure(a)
if err == nil {
cured[a] = cureRes{pathname, checksum}
}
return err
}); err != nil {
return err
}
}
// explicitly open for direct error-free use from this point
if cm.c == nil {
if err := cm.open(); err != nil {
return err
}
cured[a] = cureRes{pathname, checksum}
}
layers := pkg.PromoteLayers(root, func(a pkg.Artifact) (
@@ -599,7 +595,7 @@ func main() {
res := cured[a]
return res.pathname, res.checksum
}, func(i int, d pkg.Artifact) {
r := pkg.Encode(cache.Ident(d).Value())
r := pkg.Encode(cm.c.Ident(d).Value())
if s, ok := d.(fmt.Stringer); ok {
if name := s.String(); name != "" {
r += "-" + name
@@ -618,6 +614,9 @@ func main() {
z.Hostname = "localhost"
z.Uid, z.Gid = (1<<10)-1, (1<<10)-1
z.Stdin, z.Stdout, z.Stderr = os.Stdin, os.Stdout, os.Stderr
if s, ok := os.LookupEnv("TERM"); ok {
z.Env = append(z.Env, "TERM="+s)
}
var tempdir *check.Absolute
if s, err := filepath.Abs(os.TempDir()); err != nil {
@@ -660,22 +659,19 @@ func main() {
}
return z.Wait()
},
).
Flag(
&flagNet,
"net", command.BoolFlag(false),
"Share host net namespace",
).
Flag(
&flagSession,
"session", command.BoolFlag(true),
"Retain session",
).
Flag(
&flagWithToolchain,
"with-toolchain", command.BoolFlag(false),
"Include the stage2 LLVM toolchain",
)
).Flag(
&flagNet,
"net", command.BoolFlag(false),
"Share host net namespace",
).Flag(
&flagSession,
"session", command.BoolFlag(true),
"Retain session",
).Flag(
&flagWithToolchain,
"with-toolchain", command.BoolFlag(false),
"Include the stage2 LLVM toolchain",
)
}
@@ -686,9 +682,7 @@ func main() {
)
c.MustParse(os.Args[1:], func(err error) {
if cache != nil {
cache.Close()
}
cm.Close()
if w, ok := err.(interface{ Unwrap() []error }); !ok {
log.Fatal(err)
} else {

View File

@@ -7,8 +7,8 @@
#endif
#define SHAREFS_MEDIA_RW_ID (1 << 10) - 1 /* owning gid presented to userspace */
#define SHAREFS_PERM_DIR 0700 /* permission bits for directories presented to userspace */
#define SHAREFS_PERM_REG 0600 /* permission bits for regular files presented to userspace */
#define SHAREFS_PERM_DIR 0770 /* permission bits for directories presented to userspace */
#define SHAREFS_PERM_REG 0660 /* permission bits for regular files presented to userspace */
#define SHAREFS_FORBIDDEN_FLAGS O_DIRECT /* these open flags are cleared unconditionally */
/* sharefs_private is populated by sharefs_init and contains process-wide context */

View File

@@ -19,7 +19,6 @@ import (
"encoding/gob"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
@@ -470,13 +469,14 @@ func _main(s ...string) (exitCode int) {
os.NewFile(uintptr(C.fuse_session_fd(se)), "fuse"),
))
var setupWriter io.WriteCloser
if fd, w, err := container.Setup(&z.ExtraFiles); err != nil {
var setupPipe [2]*os.File
if r, w, err := os.Pipe(); err != nil {
log.Println(err)
return 5
} else {
z.Args = append(z.Args, "-osetup="+strconv.Itoa(fd))
setupWriter = w
z.Args = append(z.Args, "-osetup="+strconv.Itoa(3+len(z.ExtraFiles)))
z.ExtraFiles = append(z.ExtraFiles, r)
setupPipe[0], setupPipe[1] = r, w
}
if err := z.Start(); err != nil {
@@ -487,6 +487,9 @@ func _main(s ...string) (exitCode int) {
}
return 5
}
if err := setupPipe[0].Close(); err != nil {
log.Println(err)
}
if err := z.Serve(); err != nil {
if m, ok := message.GetMessage(err); ok {
log.Println(m)
@@ -496,10 +499,10 @@ func _main(s ...string) (exitCode int) {
return 5
}
if err := gob.NewEncoder(setupWriter).Encode(&setup); err != nil {
if err := gob.NewEncoder(setupPipe[1]).Encode(&setup); err != nil {
log.Println(err)
return 5
} else if err = setupWriter.Close(); err != nil {
} else if err = setupPipe[1].Close(); err != nil {
log.Println(err)
}

View File

@@ -0,0 +1,122 @@
//go:build raceattr
// The raceattr program reproduces vfs inode file attribute race.
//
// Even though libfuse high-level API presents the address of a struct stat
// alongside struct fuse_context, file attributes are actually inherent to the
// inode, instead of the specific call from userspace. The kernel implementation
// in fs/fuse/xattr.c appears to make stale data in the inode (set by a previous
// call) impossible or very unlikely to reach userspace via the stat family of
// syscalls. However, when using default_permissions to have the VFS check
// permissions, this race still happens, despite the resulting struct stat being
// correct when overriding the check via capabilities otherwise.
//
// This program reproduces the failure, but because of its continuous nature, it
// is provided independent of the vm integration test suite.
package main
import (
"context"
"flag"
"log"
"os"
"os/signal"
"runtime"
"sync"
"sync/atomic"
"syscall"
)
func newStatAs(
ctx context.Context, cancel context.CancelFunc,
n *atomic.Uint64, ok *atomic.Bool,
uid uint32, pathname string,
continuous bool,
) func() {
return func() {
runtime.LockOSThread()
defer cancel()
if _, _, errno := syscall.Syscall(
syscall.SYS_SETUID, uintptr(uid),
0, 0,
); errno != 0 {
cancel()
log.Printf("cannot set uid to %d: %s", uid, errno)
}
var stat syscall.Stat_t
for {
if ctx.Err() != nil {
return
}
if err := syscall.Lstat(pathname, &stat); err != nil {
// SHAREFS_PERM_DIR not world executable, or
// SHAREFS_PERM_REG not world readable
if !continuous {
cancel()
}
ok.Store(true)
log.Printf("uid %d: %v", uid, err)
} else if stat.Uid != uid {
// appears to be unreachable
if !continuous {
cancel()
}
ok.Store(true)
log.Printf("got uid %d instead of %d", stat.Uid, uid)
}
n.Add(1)
}
}
}
func main() {
log.SetFlags(0)
log.SetPrefix("raceattr: ")
p := flag.String("target", "/sdcard/raceattr", "pathname of test file")
u0 := flag.Int("uid0", 1<<10-1, "first uid")
u1 := flag.Int("uid1", 1<<10-2, "second uid")
count := flag.Int("count", 1, "threads per uid")
continuous := flag.Bool("continuous", false, "keep running even after reproduce")
flag.Parse()
if os.Geteuid() != 0 {
log.Fatal("this program must run as root")
}
ctx, cancel := signal.NotifyContext(
context.Background(),
syscall.SIGINT,
syscall.SIGTERM,
syscall.SIGHUP,
)
if err := os.WriteFile(*p, nil, 0); err != nil {
log.Fatal(err)
}
var (
wg sync.WaitGroup
n atomic.Uint64
ok atomic.Bool
)
if *count < 1 {
*count = 1
}
for range *count {
wg.Go(newStatAs(ctx, cancel, &n, &ok, uint32(*u0), *p, *continuous))
if *u1 >= 0 {
wg.Go(newStatAs(ctx, cancel, &n, &ok, uint32(*u1), *p, *continuous))
}
}
wg.Wait()
if !*continuous && ok.Load() {
log.Printf("reproduced after %d calls", n.Load())
}
}

View File

@@ -21,6 +21,7 @@ import (
"hakurei.app/container/std"
"hakurei.app/ext"
"hakurei.app/fhs"
"hakurei.app/internal/landlock"
"hakurei.app/message"
)
@@ -28,9 +29,6 @@ const (
// CancelSignal is the signal expected by container init on context cancel.
// A custom [Container.Cancel] function must eventually deliver this signal.
CancelSignal = SIGUSR2
// Timeout for writing initParams to Container.setup.
initSetupTimeout = 5 * time.Second
)
type (
@@ -53,7 +51,7 @@ type (
ExtraFiles []*os.File
// Write end of a pipe connected to the init to deliver [Params].
setup *os.File
setup [2]*os.File
// Cancels the context passed to the underlying cmd.
cancel context.CancelFunc
// Closed after Wait returns. Keeps the spawning thread alive.
@@ -287,14 +285,16 @@ func (p *Container) Start() error {
}
// place setup pipe before user supplied extra files, this is later restored by init
if fd, f, err := Setup(&p.cmd.ExtraFiles); err != nil {
if r, w, err := os.Pipe(); err != nil {
return &StartError{
Fatal: true,
Step: "set up params stream",
Err: err,
}
} else {
p.setup = f
fd := 3 + len(p.cmd.ExtraFiles)
p.cmd.ExtraFiles = append(p.cmd.ExtraFiles, r)
p.setup[0], p.setup[1] = r, w
p.cmd.Env = []string{setupEnv + "=" + strconv.Itoa(fd)}
}
p.cmd.ExtraFiles = append(p.cmd.ExtraFiles, p.ExtraFiles...)
@@ -308,7 +308,7 @@ func (p *Container) Start() error {
done <- func() error {
// PR_SET_NO_NEW_PRIVS: thread-directed but acts on all processes
// created from the calling thread
if err := SetNoNewPrivs(); err != nil {
if err := setNoNewPrivs(); err != nil {
return &StartError{
Fatal: true,
Step: "prctl(PR_SET_NO_NEW_PRIVS)",
@@ -318,15 +318,17 @@ func (p *Container) Start() error {
// landlock: depends on per-thread state but acts on a process group
{
rulesetAttr := &RulesetAttr{Scoped: LANDLOCK_SCOPE_SIGNAL}
rulesetAttr := &landlock.RulesetAttr{
Scoped: landlock.LANDLOCK_SCOPE_SIGNAL,
}
if !p.HostAbstract {
rulesetAttr.Scoped |= LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET
rulesetAttr.Scoped |= landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET
}
if abi, err := LandlockGetABI(); err != nil {
if p.HostAbstract {
if abi, err := landlock.GetABI(); err != nil {
if p.HostAbstract || !p.HostNet {
// landlock can be skipped here as it restricts access
// to resources already covered by namespaces (pid)
// to resources already covered by namespaces (pid, net)
goto landlockOut
}
return &StartError{Step: "get landlock ABI", Err: err}
@@ -352,7 +354,7 @@ func (p *Container) Start() error {
}
} else {
p.msg.Verbosef("enforcing landlock ruleset %s", rulesetAttr)
if err = LandlockRestrictSelf(rulesetFd, 0); err != nil {
if err = landlock.RestrictSelf(rulesetFd, 0); err != nil {
_ = Close(rulesetFd)
return &StartError{
Fatal: true,
@@ -428,24 +430,33 @@ func (p *Container) Start() error {
// Serve serves [Container.Params] to the container init.
//
// Serve must only be called once.
func (p *Container) Serve() error {
if p.setup == nil {
func (p *Container) Serve() (err error) {
if p.setup[0] == nil || p.setup[1] == nil {
panic("invalid serve")
}
setup := p.setup
p.setup = nil
if err := setup.SetDeadline(time.Now().Add(initSetupTimeout)); err != nil {
done := make(chan struct{})
defer func() {
if closeErr := p.setup[1].Close(); err == nil {
err = closeErr
}
if err != nil {
p.cancel()
}
close(done)
p.setup[0], p.setup[1] = nil, nil
}()
if err = p.setup[0].Close(); err != nil {
return &StartError{
Fatal: true,
Step: "set init pipe deadline",
Step: "close read end of init pipe",
Err: err,
Passthrough: true,
}
}
if p.Path == nil {
p.cancel()
return &StartError{
Step: "invalid executable pathname",
Err: EINVAL,
@@ -461,18 +472,27 @@ func (p *Container) Serve() error {
p.SeccompRules = make([]std.NativeRule, 0)
}
err := gob.NewEncoder(setup).Encode(&initParams{
t := time.Now().UTC()
go func(f *os.File) {
select {
case <-p.ctx.Done():
if cancelErr := f.SetWriteDeadline(t); cancelErr != nil {
p.msg.Verbose(err)
}
case <-done:
p.msg.Verbose("setup payload took", time.Since(t))
return
}
}(p.setup[1])
return gob.NewEncoder(p.setup[1]).Encode(&initParams{
p.Params,
Getuid(),
Getgid(),
len(p.ExtraFiles),
p.msg.IsVerbose(),
})
_ = setup.Close()
if err != nil {
p.cancel()
}
return err
}
// Wait blocks until the container init process to exit and releases any

View File

@@ -16,7 +16,6 @@ import (
"strings"
"syscall"
"testing"
"time"
"hakurei.app/check"
"hakurei.app/command"
@@ -26,6 +25,9 @@ import (
"hakurei.app/ext"
"hakurei.app/fhs"
"hakurei.app/hst"
"hakurei.app/internal/info"
"hakurei.app/internal/landlock"
"hakurei.app/internal/params"
"hakurei.app/ldd"
"hakurei.app/message"
"hakurei.app/vfs"
@@ -84,9 +86,9 @@ func TestStartError(t *testing.T) {
{"params env", &container.StartError{
Fatal: true,
Step: "set up params stream",
Err: container.ErrReceiveEnv,
Err: params.ErrReceiveEnv,
}, "set up params stream: environment variable not set",
container.ErrReceiveEnv, syscall.EBADF,
params.ErrReceiveEnv, syscall.EBADF,
"cannot set up params stream: environment variable not set"},
{"params", &container.StartError{
@@ -436,11 +438,8 @@ func TestContainer(t *testing.T) {
wantOps, wantOpsCtx := tc.ops(t)
wantMnt := tc.mnt(t, wantOpsCtx)
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout)
defer cancel()
var libPaths []*check.Absolute
c := helperNewContainerLibPaths(ctx, &libPaths, "container", strconv.Itoa(i))
c := helperNewContainerLibPaths(t.Context(), &libPaths, "container", strconv.Itoa(i))
c.Uid = tc.uid
c.Gid = tc.gid
c.Hostname = hostnameFromTestCase(tc.name)
@@ -450,7 +449,6 @@ func TestContainer(t *testing.T) {
} else {
c.Stdout, c.Stderr = os.Stdout, os.Stderr
}
c.WaitDelay = helperDefaultTimeout
*c.Ops = append(*c.Ops, *wantOps...)
c.SeccompRules = tc.rules
c.SeccompFlags = tc.flags | seccomp.AllowMultiarch
@@ -458,6 +456,15 @@ func TestContainer(t *testing.T) {
c.SeccompDisable = !tc.filter
c.RetainSession = tc.session
c.HostNet = tc.net
if info.CanDegrade {
if _, err := landlock.GetABI(); err != nil {
if !errors.Is(err, syscall.ENOSYS) {
t.Fatalf("LandlockGetABI: error = %v", err)
}
c.HostAbstract = true
t.Log("Landlock LSM is unavailable, enabling HostAbstract")
}
}
c.
Readonly(check.MustAbs(pathReadonly), 0755).
@@ -553,11 +560,10 @@ func testContainerCancel(
) func(t *testing.T) {
return func(t *testing.T) {
t.Parallel()
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout)
ctx, cancel := context.WithCancel(t.Context())
c := helperNewContainer(ctx, "block")
c.Stdout, c.Stderr = os.Stdout, os.Stderr
c.WaitDelay = helperDefaultTimeout
if containerExtra != nil {
containerExtra(c)
}
@@ -738,8 +744,7 @@ func init() {
const (
envDoCheck = "HAKUREI_TEST_DO_CHECK"
helperDefaultTimeout = 5 * time.Second
helperInnerPath = "/usr/bin/helper"
helperInnerPath = "/usr/bin/helper"
)
var (

View File

@@ -16,6 +16,7 @@ import (
"hakurei.app/container/std"
"hakurei.app/ext"
"hakurei.app/internal/netlink"
"hakurei.app/internal/params"
"hakurei.app/message"
)
@@ -56,7 +57,7 @@ type syscallDispatcher interface {
// isatty provides [Isatty].
isatty(fd int) bool
// receive provides [Receive].
receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error)
receive(key string, e any, fdp *int) (closeFunc func() error, err error)
// bindMount provides procPaths.bindMount.
bindMount(msg message.Msg, source, target string, flags uintptr) error
@@ -147,7 +148,7 @@ func (direct) lockOSThread() { runtime.LockOSThread() }
func (direct) setPtracer(pid uintptr) error { return ext.SetPtracer(pid) }
func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) }
func (direct) setNoNewPrivs() error { return SetNoNewPrivs() }
func (direct) setNoNewPrivs() error { return setNoNewPrivs() }
func (direct) lastcap(msg message.Msg) uintptr { return LastCap(msg) }
func (direct) capset(hdrp *capHeader, datap *[2]capData) error { return capset(hdrp, datap) }
@@ -155,8 +156,8 @@ func (direct) capBoundingSetDrop(cap uintptr) error { return capBound
func (direct) capAmbientClearAll() error { return capAmbientClearAll() }
func (direct) capAmbientRaise(cap uintptr) error { return capAmbientRaise(cap) }
func (direct) isatty(fd int) bool { return ext.Isatty(fd) }
func (direct) receive(key string, e any, fdp *uintptr) (func() error, error) {
return Receive(key, e, fdp)
func (direct) receive(key string, e any, fdp *int) (func() error, error) {
return params.Receive(key, e, fdp)
}
func (direct) bindMount(msg message.Msg, source, target string, flags uintptr) error {

View File

@@ -390,7 +390,7 @@ func (k *kstub) isatty(fd int) bool {
return expect.Ret.(bool)
}
func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) {
func (k *kstub) receive(key string, e any, fdp *int) (closeFunc func() error, err error) {
k.Helper()
expect := k.Expects("receive")
@@ -408,10 +408,17 @@ func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error
}
return nil
}
// avoid changing test cases
var fdpComp *uintptr
if fdp != nil {
fdpComp = new(uintptr(*fdp))
}
err = expect.Error(
stub.CheckArg(k.Stub, "key", key, 0),
stub.CheckArgReflect(k.Stub, "e", e, 1),
stub.CheckArgReflect(k.Stub, "fdp", fdp, 2))
stub.CheckArgReflect(k.Stub, "fdp", fdpComp, 2))
// 3 is unused so stores params
if expect.Args[3] != nil {
@@ -426,7 +433,7 @@ func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error
if expect.Args[4] != nil {
if v, ok := expect.Args[4].(uintptr); ok && v >= 3 {
if fdp != nil {
*fdp = v
*fdp = int(v)
}
}
}

View File

@@ -19,6 +19,7 @@ import (
"hakurei.app/container/seccomp"
"hakurei.app/ext"
"hakurei.app/fhs"
"hakurei.app/internal/params"
"hakurei.app/message"
)
@@ -147,35 +148,33 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
}
var (
params initParams
closeSetup func() error
setupFd uintptr
offsetSetup int
param initParams
closeSetup func() error
setupFd int
)
if f, err := k.receive(setupEnv, &params, &setupFd); err != nil {
if f, err := k.receive(setupEnv, &param, &setupFd); err != nil {
if errors.Is(err, EBADF) {
k.fatal(msg, "invalid setup descriptor")
}
if errors.Is(err, ErrReceiveEnv) {
if errors.Is(err, params.ErrReceiveEnv) {
k.fatal(msg, setupEnv+" not set")
}
k.fatalf(msg, "cannot decode init setup payload: %v", err)
} else {
if params.Ops == nil {
if param.Ops == nil {
k.fatal(msg, "invalid setup parameters")
}
if params.ParentPerm == 0 {
params.ParentPerm = 0755
if param.ParentPerm == 0 {
param.ParentPerm = 0755
}
msg.SwapVerbose(params.Verbose)
msg.SwapVerbose(param.Verbose)
msg.Verbose("received setup parameters")
closeSetup = f
offsetSetup = int(setupFd + 1)
}
if !params.HostNet {
if !param.HostNet {
ctx, cancel := signal.NotifyContext(context.Background(), CancelSignal,
os.Interrupt, SIGTERM, SIGQUIT)
defer cancel() // for panics
@@ -188,7 +187,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot set SUID_DUMP_USER: %v", err)
}
if err := k.writeFile(fhs.Proc+"self/uid_map",
append([]byte{}, strconv.Itoa(params.Uid)+" "+strconv.Itoa(params.HostUid)+" 1\n"...),
append([]byte{}, strconv.Itoa(param.Uid)+" "+strconv.Itoa(param.HostUid)+" 1\n"...),
0); err != nil {
k.fatalf(msg, "%v", err)
}
@@ -198,7 +197,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "%v", err)
}
if err := k.writeFile(fhs.Proc+"self/gid_map",
append([]byte{}, strconv.Itoa(params.Gid)+" "+strconv.Itoa(params.HostGid)+" 1\n"...),
append([]byte{}, strconv.Itoa(param.Gid)+" "+strconv.Itoa(param.HostGid)+" 1\n"...),
0); err != nil {
k.fatalf(msg, "%v", err)
}
@@ -207,8 +206,8 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
}
oldmask := k.umask(0)
if params.Hostname != "" {
if err := k.sethostname([]byte(params.Hostname)); err != nil {
if param.Hostname != "" {
if err := k.sethostname([]byte(param.Hostname)); err != nil {
k.fatalf(msg, "cannot set hostname: %v", err)
}
}
@@ -221,7 +220,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
}
ctx, cancel := context.WithCancel(context.Background())
state := &setupState{process: make(map[int]WaitStatus), Params: &params.Params, Msg: msg, Context: ctx}
state := &setupState{process: make(map[int]WaitStatus), Params: &param.Params, Msg: msg, Context: ctx}
defer cancel()
/* early is called right before pivot_root into intermediate root;
@@ -229,7 +228,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
difficult to obtain via library functions after pivot_root, and
implementations are expected to avoid changing the state of the mount
namespace */
for i, op := range *params.Ops {
for i, op := range *param.Ops {
if op == nil || !op.Valid() {
k.fatalf(msg, "invalid op at index %d", i)
}
@@ -272,7 +271,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
step sets up the container filesystem, and implementations are expected to
keep the host root and sysroot mount points intact but otherwise can do
whatever they need to. Calling chdir is allowed but discouraged. */
for i, op := range *params.Ops {
for i, op := range *param.Ops {
// ops already checked during early setup
if prefix, ok := op.prefix(); ok {
msg.Verbosef("%s %s", prefix, op)
@@ -328,7 +327,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot clear the ambient capability set: %v", err)
}
for i := uintptr(0); i <= lastcap; i++ {
if params.Privileged && i == CAP_SYS_ADMIN {
if param.Privileged && i == CAP_SYS_ADMIN {
continue
}
if err := k.capBoundingSetDrop(i); err != nil {
@@ -337,7 +336,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
}
var keep [2]uint32
if params.Privileged {
if param.Privileged {
keep[capToIndex(CAP_SYS_ADMIN)] |= capToMask(CAP_SYS_ADMIN)
if err := k.capAmbientRaise(CAP_SYS_ADMIN); err != nil {
@@ -351,13 +350,13 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot capset: %v", err)
}
if !params.SeccompDisable {
rules := params.SeccompRules
if !param.SeccompDisable {
rules := param.SeccompRules
if len(rules) == 0 { // non-empty rules slice always overrides presets
msg.Verbosef("resolving presets %#x", params.SeccompPresets)
rules = seccomp.Preset(params.SeccompPresets, params.SeccompFlags)
msg.Verbosef("resolving presets %#x", param.SeccompPresets)
rules = seccomp.Preset(param.SeccompPresets, param.SeccompFlags)
}
if err := k.seccompLoad(rules, params.SeccompFlags); err != nil {
if err := k.seccompLoad(rules, param.SeccompFlags); err != nil {
// this also indirectly asserts PR_SET_NO_NEW_PRIVS
k.fatalf(msg, "cannot load syscall filter: %v", err)
}
@@ -366,10 +365,10 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
msg.Verbose("syscall filter not configured")
}
extraFiles := make([]*os.File, params.Count)
extraFiles := make([]*os.File, param.Count)
for i := range extraFiles {
// setup fd is placed before all extra files
extraFiles[i] = k.newFile(uintptr(offsetSetup+i), "extra file "+strconv.Itoa(i))
extraFiles[i] = k.newFile(uintptr(setupFd+1+i), "extra file "+strconv.Itoa(i))
}
k.umask(oldmask)
@@ -447,7 +446,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
// called right before startup of initial process, all state changes to the
// current process is prohibited during late
for i, op := range *params.Ops {
for i, op := range *param.Ops {
// ops already checked during early setup
if err := op.late(state, k); err != nil {
if m, ok := messageFromError(err); ok {
@@ -468,14 +467,14 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot close setup pipe: %v", err)
}
cmd := exec.Command(params.Path.String())
cmd := exec.Command(param.Path.String())
cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
cmd.Args = params.Args
cmd.Env = params.Env
cmd.Args = param.Args
cmd.Env = param.Env
cmd.ExtraFiles = extraFiles
cmd.Dir = params.Dir.String()
cmd.Dir = param.Dir.String()
msg.Verbosef("starting initial process %s", params.Path)
msg.Verbosef("starting initial process %s", param.Path)
if err := k.start(cmd); err != nil {
k.fatalf(msg, "%v", err)
}
@@ -493,9 +492,9 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
for {
select {
case s := <-sig:
if s == CancelSignal && params.ForwardCancel && cmd.Process != nil {
if s == CancelSignal && param.ForwardCancel && cmd.Process != nil {
msg.Verbose("forwarding context cancellation")
if err := k.signal(cmd, os.Interrupt); err != nil {
if err := k.signal(cmd, os.Interrupt); err != nil && !errors.Is(err, os.ErrProcessDone) {
k.printf(msg, "cannot forward cancellation: %v", err)
}
continue
@@ -525,7 +524,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
cancel()
// start timeout early
go func() { time.Sleep(params.AdoptWaitDelay); close(timeout) }()
go func() { time.Sleep(param.AdoptWaitDelay); close(timeout) }()
// close initial process files; this also keeps them alive
for _, f := range extraFiles {

View File

@@ -10,6 +10,7 @@ import (
"hakurei.app/check"
"hakurei.app/container/seccomp"
"hakurei.app/container/std"
"hakurei.app/internal/params"
"hakurei.app/internal/stub"
)
@@ -40,7 +41,7 @@ func TestInitEntrypoint(t *testing.T) {
call("lockOSThread", stub.ExpectArgs{}, nil, nil),
call("getpid", stub.ExpectArgs{}, 1, nil),
call("setPtracer", stub.ExpectArgs{uintptr(0)}, nil, nil),
call("receive", stub.ExpectArgs{"HAKUREI_SETUP", new(initParams), new(uintptr)}, nil, ErrReceiveEnv),
call("receive", stub.ExpectArgs{"HAKUREI_SETUP", new(initParams), new(uintptr)}, nil, params.ErrReceiveEnv),
call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SETUP not set"}}, nil, nil),
},
}, nil},

View File

@@ -1,65 +0,0 @@
package container_test
import (
"testing"
"unsafe"
"hakurei.app/container"
)
func TestLandlockString(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
rulesetAttr *container.RulesetAttr
want string
}{
{"nil", nil, "NULL"},
{"zero", new(container.RulesetAttr), "0"},
{"some", &container.RulesetAttr{Scoped: container.LANDLOCK_SCOPE_SIGNAL}, "scoped: signal"},
{"set", &container.RulesetAttr{
HandledAccessFS: container.LANDLOCK_ACCESS_FS_MAKE_SYM | container.LANDLOCK_ACCESS_FS_IOCTL_DEV | container.LANDLOCK_ACCESS_FS_WRITE_FILE,
HandledAccessNet: container.LANDLOCK_ACCESS_NET_BIND_TCP,
Scoped: container.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET | container.LANDLOCK_SCOPE_SIGNAL,
}, "fs: write_file make_sym fs_ioctl_dev, net: bind_tcp, scoped: abstract_unix_socket signal"},
{"all", &container.RulesetAttr{
HandledAccessFS: container.LANDLOCK_ACCESS_FS_EXECUTE |
container.LANDLOCK_ACCESS_FS_WRITE_FILE |
container.LANDLOCK_ACCESS_FS_READ_FILE |
container.LANDLOCK_ACCESS_FS_READ_DIR |
container.LANDLOCK_ACCESS_FS_REMOVE_DIR |
container.LANDLOCK_ACCESS_FS_REMOVE_FILE |
container.LANDLOCK_ACCESS_FS_MAKE_CHAR |
container.LANDLOCK_ACCESS_FS_MAKE_DIR |
container.LANDLOCK_ACCESS_FS_MAKE_REG |
container.LANDLOCK_ACCESS_FS_MAKE_SOCK |
container.LANDLOCK_ACCESS_FS_MAKE_FIFO |
container.LANDLOCK_ACCESS_FS_MAKE_BLOCK |
container.LANDLOCK_ACCESS_FS_MAKE_SYM |
container.LANDLOCK_ACCESS_FS_REFER |
container.LANDLOCK_ACCESS_FS_TRUNCATE |
container.LANDLOCK_ACCESS_FS_IOCTL_DEV,
HandledAccessNet: container.LANDLOCK_ACCESS_NET_BIND_TCP |
container.LANDLOCK_ACCESS_NET_CONNECT_TCP,
Scoped: container.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET |
container.LANDLOCK_SCOPE_SIGNAL,
}, "fs: execute write_file read_file read_dir remove_dir remove_file make_char make_dir make_reg make_sock make_fifo make_block make_sym fs_refer fs_truncate fs_ioctl_dev, net: bind_tcp connect_tcp, scoped: abstract_unix_socket signal"},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if got := tc.rulesetAttr.String(); got != tc.want {
t.Errorf("String: %s, want %s", got, tc.want)
}
})
}
}
func TestLandlockAttrSize(t *testing.T) {
t.Parallel()
want := 24
if got := unsafe.Sizeof(container.RulesetAttr{}); got != uintptr(want) {
t.Errorf("Sizeof: %d, want %d", got, want)
}
}

View File

@@ -1,47 +0,0 @@
package container
import (
"encoding/gob"
"errors"
"os"
"strconv"
"syscall"
)
// Setup appends the read end of a pipe for setup params transmission and returns its fd.
func Setup(extraFiles *[]*os.File) (int, *os.File, error) {
if r, w, err := os.Pipe(); err != nil {
return -1, nil, err
} else {
fd := 3 + len(*extraFiles)
*extraFiles = append(*extraFiles, r)
return fd, w, nil
}
}
var (
ErrReceiveEnv = errors.New("environment variable not set")
)
// Receive retrieves setup fd from the environment and receives params.
func Receive(key string, e any, fdp *uintptr) (func() error, error) {
var setup *os.File
if s, ok := os.LookupEnv(key); !ok {
return nil, ErrReceiveEnv
} else {
if fd, err := strconv.Atoi(s); err != nil {
return nil, optionalErrorUnwrap(err)
} else {
setup = os.NewFile(uintptr(fd), "setup")
if setup == nil {
return nil, syscall.EDOM
}
if fdp != nil {
*fdp = setup.Fd()
}
}
}
return setup.Close, gob.NewDecoder(setup).Decode(e)
}

View File

@@ -7,8 +7,8 @@ import (
"hakurei.app/ext"
)
// SetNoNewPrivs sets the calling thread's no_new_privs attribute.
func SetNoNewPrivs() error {
// setNoNewPrivs sets the calling thread's no_new_privs attribute.
func setNoNewPrivs() error {
return ext.Prctl(PR_SET_NO_NEW_PRIVS, 1, 0)
}

1
dist/hsurc.default vendored
View File

@@ -1 +0,0 @@
1000 0

12
dist/install.sh vendored
View File

@@ -1,12 +0,0 @@
#!/bin/sh
cd "$(dirname -- "$0")" || exit 1
install -vDm0755 "bin/hakurei" "${DESTDIR}/usr/bin/hakurei"
install -vDm0755 "bin/sharefs" "${DESTDIR}/usr/bin/sharefs"
install -vDm4511 "bin/hsu" "${DESTDIR}/usr/bin/hsu"
if [ ! -f "${DESTDIR}/etc/hsurc" ]; then
install -vDm0400 "hsurc.default" "${DESTDIR}/etc/hsurc"
fi
install -vDm0644 "comp/_hakurei" "${DESTDIR}/usr/share/zsh/site-functions/_hakurei"

31
dist/release.sh vendored
View File

@@ -1,31 +0,0 @@
#!/bin/sh -e
cd "$(dirname -- "$0")/.."
VERSION="${HAKUREI_VERSION:-untagged}"
pname="hakurei-${VERSION}-$(go env GOARCH)"
out="${DESTDIR:-dist}/${pname}"
echo '# Preparing distribution files.'
mkdir -p "${out}"
cp -v "README.md" "dist/hsurc.default" "dist/install.sh" "${out}"
cp -rv "dist/comp" "${out}"
echo
echo '# Building hakurei.'
go generate ./...
go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w
-buildid= -linkmode external -extldflags=-static
-X hakurei.app/internal/info.buildVersion=${VERSION}
-X hakurei.app/internal/info.hakureiPath=/usr/bin/hakurei
-X hakurei.app/internal/info.hsuPath=/usr/bin/hsu
-X main.hakureiPath=/usr/bin/hakurei" ./...
echo
echo '# Testing hakurei.'
go test -ldflags='-buildid= -linkmode external -extldflags=-static' ./...
echo
echo '# Creating distribution.'
rm -f "${out}.tar.gz" && tar -C "${out}/.." -vczf "${out}.tar.gz" "${pname}"
rm -rf "${out}"
(cd "${out}/.." && sha512sum "${pname}.tar.gz" > "${pname}.tar.gz.sha512")
echo

View File

@@ -137,11 +137,10 @@
CC="musl-clang -O3 -Werror -Qunused-arguments" \
GOCACHE="$(mktemp -d)" \
HAKUREI_TEST_SKIP_ACL=1 \
PATH="${pkgs.pkgsStatic.musl.bin}/bin:$PATH" \
DESTDIR="$out" \
HAKUREI_VERSION="v${hakurei.version}" \
./dist/release.sh
./all.sh
'';
}
);
@@ -196,6 +195,7 @@
./test/interactive/vm.nix
./test/interactive/hakurei.nix
./test/interactive/trace.nix
./test/interactive/raceattr.nix
self.nixosModules.hakurei
home-manager.nixosModules.home-manager

View File

@@ -140,21 +140,29 @@ var (
ErrInsecure = errors.New("configuration is insecure")
)
const (
// VAllowInsecure allows use of compatibility options considered insecure
// under any configuration, to work around ecosystem-wide flaws.
VAllowInsecure = 1 << iota
)
// Validate checks [Config] and returns [AppError] if an invalid value is encountered.
func (config *Config) Validate() error {
func (config *Config) Validate(flags int) error {
const step = "validate configuration"
if config == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull,
return &AppError{Step: step, Err: ErrConfigNull,
Msg: "invalid configuration"}
}
// this is checked again in hsu
if config.Identity < IdentityStart || config.Identity > IdentityEnd {
return &AppError{Step: "validate configuration", Err: ErrIdentityBounds,
return &AppError{Step: step, Err: ErrIdentityBounds,
Msg: "identity " + strconv.Itoa(config.Identity) + " out of range"}
}
if config.SchedPolicy < 0 || config.SchedPolicy > ext.SCHED_LAST {
return &AppError{Step: "validate configuration", Err: ErrSchedPolicyBounds,
return &AppError{Step: step, Err: ErrSchedPolicyBounds,
Msg: "scheduling policy " +
strconv.Itoa(int(config.SchedPolicy)) +
" out of range"}
@@ -168,34 +176,51 @@ func (config *Config) Validate() error {
}
if config.Container == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull,
return &AppError{Step: step, Err: ErrConfigNull,
Msg: "configuration missing container state"}
}
if config.Container.Home == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull,
return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to home directory"}
}
if config.Container.Shell == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull,
return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to shell"}
}
if config.Container.Path == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull,
return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to initial program"}
}
for key := range config.Container.Env {
if strings.IndexByte(key, '=') != -1 || strings.IndexByte(key, 0) != -1 {
return &AppError{Step: "validate configuration", Err: ErrEnviron,
return &AppError{Step: step, Err: ErrEnviron,
Msg: "invalid environment variable " + strconv.Quote(key)}
}
}
if et := config.Enablements.Unwrap(); !config.DirectPulse && et&EPulse != 0 {
return &AppError{Step: "validate configuration", Err: ErrInsecure,
et := config.Enablements.Unwrap()
if !config.DirectPulse && et&EPulse != 0 {
return &AppError{Step: step, Err: ErrInsecure,
Msg: "enablement PulseAudio is insecure and no longer supported"}
}
if flags&VAllowInsecure == 0 {
switch {
case et&EWayland != 0 && config.DirectWayland:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_wayland is insecure and no longer supported"}
case et&EPipeWire != 0 && config.DirectPipeWire:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_pipewire is insecure and no longer supported"}
case et&EPulse != 0 && config.DirectPulse:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_pulse is insecure and no longer supported"}
}
}
return nil
}

View File

@@ -14,65 +14,109 @@ func TestConfigValidate(t *testing.T) {
testCases := []struct {
name string
config *hst.Config
flags int
wantErr error
}{
{"nil", nil, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
{"nil", nil, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "invalid configuration"}},
{"identity lower", &hst.Config{Identity: -1}, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
{"identity lower", &hst.Config{Identity: -1}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
Msg: "identity -1 out of range"}},
{"identity upper", &hst.Config{Identity: 10000}, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
{"identity upper", &hst.Config{Identity: 10000}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
Msg: "identity 10000 out of range"}},
{"sched lower", &hst.Config{SchedPolicy: -1}, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
{"sched lower", &hst.Config{SchedPolicy: -1}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
Msg: "scheduling policy -1 out of range"}},
{"sched upper", &hst.Config{SchedPolicy: 0xcafe}, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
{"sched upper", &hst.Config{SchedPolicy: 0xcafe}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
Msg: "scheduling policy 51966 out of range"}},
{"dbus session", &hst.Config{SessionBus: &hst.BusConfig{See: []string{""}}},
{"dbus session", &hst.Config{SessionBus: &hst.BusConfig{See: []string{""}}}, 0,
&hst.BadInterfaceError{Interface: "", Segment: "session"}},
{"dbus system", &hst.Config{SystemBus: &hst.BusConfig{See: []string{""}}},
{"dbus system", &hst.Config{SystemBus: &hst.BusConfig{See: []string{""}}}, 0,
&hst.BadInterfaceError{Interface: "", Segment: "system"}},
{"container", &hst.Config{}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
{"container", &hst.Config{}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "configuration missing container state"}},
{"home", &hst.Config{Container: &hst.ContainerConfig{}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
{"home", &hst.Config{Container: &hst.ContainerConfig{}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to home directory"}},
{"shell", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to shell"}},
{"path", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to initial program"}},
{"env equals", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
Env: map[string]string{"TERM=": ""},
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
Msg: `invalid environment variable "TERM="`}},
{"env NUL", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
Env: map[string]string{"TERM\x00": ""},
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
Msg: `invalid environment variable "TERM\x00"`}},
{"insecure pulse", &hst.Config{Enablements: hst.NewEnablements(hst.EPulse), Container: &hst.ContainerConfig{
{"insecure pulse", &hst.Config{Enablements: new(hst.EPulse), Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "enablement PulseAudio is insecure and no longer supported"}},
{"direct wayland", &hst.Config{Enablements: new(hst.EWayland), DirectWayland: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_wayland is insecure and no longer supported"}},
{"direct wayland allow", &hst.Config{Enablements: new(hst.EWayland), DirectWayland: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"direct pipewire", &hst.Config{Enablements: new(hst.EPipeWire), DirectPipeWire: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_pipewire is insecure and no longer supported"}},
{"direct pipewire allow", &hst.Config{Enablements: new(hst.EPipeWire), DirectPipeWire: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"direct pulse", &hst.Config{Enablements: new(hst.EPulse), DirectPulse: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_pulse is insecure and no longer supported"}},
{"direct pulse allow", &hst.Config{Enablements: new(hst.EPulse), DirectPulse: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"valid", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, nil},
}}, 0, nil},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if err := tc.config.Validate(); !reflect.DeepEqual(err, tc.wantErr) {
if err := tc.config.Validate(tc.flags); !reflect.DeepEqual(err, tc.wantErr) {
t.Errorf("Validate: error = %#v, want %#v", err, tc.wantErr)
}
})

View File

@@ -7,12 +7,12 @@ import (
"syscall"
)
// Enablement represents an optional host service to export to the target user.
type Enablement byte
// Enablements denotes optional host service to export to the target user.
type Enablements byte
const (
// EWayland exposes a Wayland pathname socket via security-context-v1.
EWayland Enablement = 1 << iota
EWayland Enablements = 1 << iota
// EX11 adds the target user via X11 ChangeHosts and exposes the X11
// pathname socket.
EX11
@@ -28,8 +28,8 @@ const (
EM
)
// String returns a string representation of the flags set on [Enablement].
func (e Enablement) String() string {
// String returns a string representation of the flags set on [Enablements].
func (e Enablements) String() string {
switch e {
case 0:
return "(no enablements)"
@@ -47,7 +47,7 @@ func (e Enablement) String() string {
buf := new(strings.Builder)
buf.Grow(32)
for i := Enablement(1); i < EM; i <<= 1 {
for i := Enablements(1); i < EM; i <<= 1 {
if e&i != 0 {
buf.WriteString(", " + i.String())
}
@@ -60,12 +60,6 @@ func (e Enablement) String() string {
}
}
// NewEnablements returns the address of [Enablement] as [Enablements].
func NewEnablements(e Enablement) *Enablements { return (*Enablements)(&e) }
// Enablements is the [json] adapter for [Enablement].
type Enablements Enablement
// enablementsJSON is the [json] representation of [Enablements].
type enablementsJSON = struct {
Wayland bool `json:"wayland,omitempty"`
@@ -75,24 +69,21 @@ type enablementsJSON = struct {
Pulse bool `json:"pulse,omitempty"`
}
// Unwrap returns the underlying [Enablement].
func (e *Enablements) Unwrap() Enablement {
// Unwrap returns the value pointed to by e.
func (e *Enablements) Unwrap() Enablements {
if e == nil {
return 0
}
return Enablement(*e)
return *e
}
func (e *Enablements) MarshalJSON() ([]byte, error) {
if e == nil {
return nil, syscall.EINVAL
}
func (e Enablements) MarshalJSON() ([]byte, error) {
return json.Marshal(&enablementsJSON{
Wayland: Enablement(*e)&EWayland != 0,
X11: Enablement(*e)&EX11 != 0,
DBus: Enablement(*e)&EDBus != 0,
PipeWire: Enablement(*e)&EPipeWire != 0,
Pulse: Enablement(*e)&EPulse != 0,
Wayland: e&EWayland != 0,
X11: e&EX11 != 0,
DBus: e&EDBus != 0,
PipeWire: e&EPipeWire != 0,
Pulse: e&EPulse != 0,
})
}
@@ -106,22 +97,21 @@ func (e *Enablements) UnmarshalJSON(data []byte) error {
return err
}
var ve Enablement
*e = 0
if v.Wayland {
ve |= EWayland
*e |= EWayland
}
if v.X11 {
ve |= EX11
*e |= EX11
}
if v.DBus {
ve |= EDBus
*e |= EDBus
}
if v.PipeWire {
ve |= EPipeWire
*e |= EPipeWire
}
if v.Pulse {
ve |= EPulse
*e |= EPulse
}
*e = Enablements(ve)
return nil
}

View File

@@ -13,7 +13,7 @@ func TestEnablementString(t *testing.T) {
t.Parallel()
testCases := []struct {
flags hst.Enablement
flags hst.Enablements
want string
}{
{0, "(no enablements)"},
@@ -59,13 +59,13 @@ func TestEnablements(t *testing.T) {
sData string
}{
{"nil", nil, "null", `{"value":null,"magic":3236757504}`},
{"zero", hst.NewEnablements(0), `{}`, `{"value":{},"magic":3236757504}`},
{"wayland", hst.NewEnablements(hst.EWayland), `{"wayland":true}`, `{"value":{"wayland":true},"magic":3236757504}`},
{"x11", hst.NewEnablements(hst.EX11), `{"x11":true}`, `{"value":{"x11":true},"magic":3236757504}`},
{"dbus", hst.NewEnablements(hst.EDBus), `{"dbus":true}`, `{"value":{"dbus":true},"magic":3236757504}`},
{"pipewire", hst.NewEnablements(hst.EPipeWire), `{"pipewire":true}`, `{"value":{"pipewire":true},"magic":3236757504}`},
{"pulse", hst.NewEnablements(hst.EPulse), `{"pulse":true}`, `{"value":{"pulse":true},"magic":3236757504}`},
{"all", hst.NewEnablements(hst.EM - 1), `{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true}`, `{"value":{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true},"magic":3236757504}`},
{"zero", new(hst.Enablements(0)), `{}`, `{"value":{},"magic":3236757504}`},
{"wayland", new(hst.EWayland), `{"wayland":true}`, `{"value":{"wayland":true},"magic":3236757504}`},
{"x11", new(hst.EX11), `{"x11":true}`, `{"value":{"x11":true},"magic":3236757504}`},
{"dbus", new(hst.EDBus), `{"dbus":true}`, `{"value":{"dbus":true},"magic":3236757504}`},
{"pipewire", new(hst.EPipeWire), `{"pipewire":true}`, `{"value":{"pipewire":true},"magic":3236757504}`},
{"pulse", new(hst.EPulse), `{"pulse":true}`, `{"value":{"pulse":true},"magic":3236757504}`},
{"all", new(hst.EM - 1), `{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true}`, `{"value":{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true},"magic":3236757504}`},
}
for _, tc := range testCases {
@@ -137,7 +137,7 @@ func TestEnablements(t *testing.T) {
})
t.Run("val", func(t *testing.T) {
if got := hst.NewEnablements(hst.EWayland | hst.EPulse).Unwrap(); got != hst.EWayland|hst.EPulse {
if got := new(hst.EWayland | hst.EPulse).Unwrap(); got != hst.EWayland|hst.EPulse {
t.Errorf("Unwrap: %v", got)
}
})
@@ -146,9 +146,6 @@ func TestEnablements(t *testing.T) {
t.Run("passthrough", func(t *testing.T) {
t.Parallel()
if _, err := (*hst.Enablements)(nil).MarshalJSON(); !errors.Is(err, syscall.EINVAL) {
t.Errorf("MarshalJSON: error = %v", err)
}
if err := (*hst.Enablements)(nil).UnmarshalJSON(nil); !errors.Is(err, syscall.EINVAL) {
t.Errorf("UnmarshalJSON: error = %v", err)
}

View File

@@ -56,8 +56,10 @@ type Ops interface {
// ApplyState holds the address of [Ops] and any relevant application state.
type ApplyState struct {
// AutoEtcPrefix is the prefix for [FSBind] in autoetc [FSBind.Special] condition.
// Prefix for [FSBind] in autoetc [FSBind.Special] condition.
AutoEtcPrefix string
// Whether to skip remounting root.
NoRemountRoot bool
Ops
}

View File

@@ -5,6 +5,7 @@ import (
"strings"
"hakurei.app/check"
"hakurei.app/fhs"
)
func init() { gob.Register(new(FSOverlay)) }
@@ -69,9 +70,12 @@ func (o *FSOverlay) Apply(z *ApplyState) {
return
}
if o.Upper != nil && o.Work != nil { // rw
if o.Upper != nil && o.Work != nil {
z.Overlay(o.Target, o.Upper, o.Work, o.Lower...)
} else { // ro
if o.Target.Is(fhs.AbsRoot) {
z.NoRemountRoot = true
}
} else {
z.OverlayReadonly(o.Target, o.Lower...)
}
}

View File

@@ -49,5 +49,18 @@ func TestFSOverlay(t *testing.T) {
Lower: ms("/tmp/.src0", "/tmp/.src1"),
}}, m("/mnt/src"), ms("/tmp/.src0", "/tmp/.src1"),
"*/mnt/src:/tmp/.src0:/tmp/.src1"},
{"no remount root", &hst.FSOverlay{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}, true, container.Ops{&container.MountOverlayOp{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}}, m("/"), ms("/tmp/upper", "/tmp/work", "/tmp/.src0", "/tmp/.src1"),
"w*/:/tmp/upper:/tmp/work:/tmp/.src0:/tmp/.src1"},
})
}

View File

@@ -72,7 +72,7 @@ func Template() *Config {
return &Config{
ID: "org.chromium.Chromium",
Enablements: NewEnablements(EWayland | EDBus | EPipeWire),
Enablements: new(EWayland | EDBus | EPipeWire),
SessionBus: &BusConfig{
See: nil,

View File

@@ -11,9 +11,11 @@ import (
"path/filepath"
"reflect"
"strconv"
"syscall"
"testing"
"hakurei.app/internal/acl"
"hakurei.app/internal/info"
)
const testFileName = "acl.test"
@@ -24,8 +26,14 @@ var (
)
func TestUpdate(t *testing.T) {
if os.Getenv("HAKUREI_TEST_SKIP_ACL") == "1" {
t.Skip("acl test skipped")
if info.CanDegrade {
name := filepath.Join(t.TempDir(), "check-degrade")
if err := os.WriteFile(name, nil, 0); err != nil {
t.Fatal(err)
}
if err := acl.Update(name, os.Geteuid()); errors.Is(err, syscall.ENOTSUP) {
t.Skip(err)
}
}
testFilePath := filepath.Join(t.TempDir(), testFileName)

View File

@@ -0,0 +1,7 @@
//go:build !noskip
package info
// CanDegrade is whether tests are allowed to transparently degrade or skip due
// to required system features being denied or unavailable.
const CanDegrade = true

View File

@@ -0,0 +1,5 @@
//go:build noskip
package info
const CanDegrade = false

90
internal/kobject/event.go Normal file
View File

@@ -0,0 +1,90 @@
package kobject
import (
"errors"
"strconv"
"strings"
"unsafe"
"hakurei.app/internal/uevent"
)
// Event is a [uevent.Message] with known environment variables processed.
type Event struct {
// alloc_uevent_skb: action_string
Action uevent.KobjectAction `json:"action"`
// alloc_uevent_skb: devpath
DevPath string `json:"devpath"`
// Uninterpreted environment variable pairs. An entry missing a separator
// gains the value "\x00".
Env map[string]string `json:"env"`
// SEQNUM value set by the kernel.
Sequence uint64 `json:"seqnum"`
// SYNTH_UUID value set on trigger, nil denotes a non-synthetic event.
Synth *uevent.UUID `json:"synth_uuid,omitempty"`
// SUBSYSTEM value set by the kernel.
Subsystem string `json:"subsystem"`
}
// Populate populates e with the contents of a [uevent.Message].
//
// The ACTION and DEVPATH environment variables are ignored and assumed to be
// consistent with the header.
func (e *Event) Populate(reportErr func(error), m *uevent.Message) {
if reportErr == nil {
reportErr = func(error) {}
}
*e = Event{
Action: m.Action,
DevPath: m.DevPath,
Env: make(map[string]string),
}
for _, s := range m.Env {
k, v, ok := strings.Cut(s, "=")
if !ok {
if _, ok = e.Env[s]; !ok {
e.Env[s] = "\x00"
}
continue
}
switch k {
case "ACTION", "DEVPATH":
continue
case "SEQNUM":
seq, err := strconv.ParseUint(v, 10, 64)
if err != nil {
if _e := errors.Unwrap(err); _e != nil {
err = _e
}
reportErr(err)
e.Env[k] = v
continue
}
e.Sequence = seq
case "SYNTH_UUID":
var uuid uevent.UUID
err := uuid.UnmarshalText(unsafe.Slice(unsafe.StringData(v), len(v)))
if err != nil {
reportErr(err)
e.Env[k] = v
continue
}
e.Synth = &uuid
case "SUBSYSTEM":
e.Subsystem = v
default:
e.Env[k] = v
}
}
}

View File

@@ -0,0 +1,92 @@
package kobject_test
import (
"reflect"
"strconv"
"testing"
"hakurei.app/internal/kobject"
"hakurei.app/internal/uevent"
)
func TestEvent(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
msg uevent.Message
want kobject.Event
errs []error
}{
{"sample coldboot qemu", uevent.Message{
Action: uevent.KOBJ_ADD,
DevPath: "/devices/LNXSYSTM:00/LNXPWRBN:00",
Env: []string{
"ACTION=add",
"DEVPATH=/devices/LNXSYSTM:00/LNXPWRBN:00",
"SUBSYSTEM=acpi",
"SYNTH_UUID=fe4d7c9d-b8c6-4a70-9ef1-3d8a58d18eed",
"MODALIAS=acpi:LNXPWRBN:",
"SEQNUM=777",
}}, kobject.Event{
Action: uevent.KOBJ_ADD,
DevPath: "/devices/LNXSYSTM:00/LNXPWRBN:00",
Env: map[string]string{
"MODALIAS": "acpi:LNXPWRBN:",
},
Sequence: 777,
Synth: &uevent.UUID{
0xfe, 0x4d, 0x7c, 0x9d,
0xb8, 0xc6,
0x4a, 0x70,
0x9e, 0xf1,
0x3d, 0x8a, 0x58, 0xd1, 0x8e, 0xed,
},
Subsystem: "acpi",
}, []error{}},
{"nil reportErr", uevent.Message{Env: []string{
"SEQNUM=\x00",
}}, kobject.Event{Env: map[string]string{
"SEQNUM": "\x00",
}}, nil},
{"bad SEQNUM SYNTH_UUID", uevent.Message{Env: []string{
"SEQNUM=\x00",
"SYNTH_UUID=\x00",
"SUBSYSTEM=\x00",
}}, kobject.Event{Subsystem: "\x00", Env: map[string]string{
"SEQNUM": "\x00",
"SYNTH_UUID": "\x00",
}}, []error{strconv.ErrSyntax, uevent.UUIDSizeError(1)}},
{"bad sep", uevent.Message{Env: []string{
"SYNTH_UUID",
}}, kobject.Event{Env: map[string]string{
"SYNTH_UUID": "\x00",
}}, []error{}},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var f func(error)
gotErrs := make([]error, 0)
if tc.errs != nil {
f = func(err error) {
gotErrs = append(gotErrs, err)
}
}
var got kobject.Event
got.Populate(f, &tc.msg)
if !reflect.DeepEqual(&got, &tc.want) {
t.Errorf("Populate: %#v, want %#v", got, tc.want)
}
if tc.errs != nil && !reflect.DeepEqual(gotErrs, tc.errs) {
t.Errorf("Populate: errs = %v, want %v", gotErrs, tc.errs)
}
})
}
}

View File

@@ -1,4 +1,4 @@
package container
package landlock
import (
"strings"
@@ -14,11 +14,11 @@ const (
LANDLOCK_CREATE_RULESET_VERSION = 1 << iota
)
// LandlockAccessFS is bitmask of handled filesystem actions.
type LandlockAccessFS uint64
// AccessFS is bitmask of handled filesystem actions.
type AccessFS uint64
const (
LANDLOCK_ACCESS_FS_EXECUTE LandlockAccessFS = 1 << iota
LANDLOCK_ACCESS_FS_EXECUTE AccessFS = 1 << iota
LANDLOCK_ACCESS_FS_WRITE_FILE
LANDLOCK_ACCESS_FS_READ_FILE
LANDLOCK_ACCESS_FS_READ_DIR
@@ -38,8 +38,8 @@ const (
_LANDLOCK_ACCESS_FS_DELIM
)
// String returns a space-separated string of [LandlockAccessFS] flags.
func (f LandlockAccessFS) String() string {
// String returns a space-separated string of [AccessFS] flags.
func (f AccessFS) String() string {
switch f {
case LANDLOCK_ACCESS_FS_EXECUTE:
return "execute"
@@ -90,8 +90,8 @@ func (f LandlockAccessFS) String() string {
return "fs_ioctl_dev"
default:
var c []LandlockAccessFS
for i := LandlockAccessFS(1); i < _LANDLOCK_ACCESS_FS_DELIM; i <<= 1 {
var c []AccessFS
for i := AccessFS(1); i < _LANDLOCK_ACCESS_FS_DELIM; i <<= 1 {
if f&i != 0 {
c = append(c, i)
}
@@ -107,18 +107,18 @@ func (f LandlockAccessFS) String() string {
}
}
// LandlockAccessNet is bitmask of handled network actions.
type LandlockAccessNet uint64
// AccessNet is bitmask of handled network actions.
type AccessNet uint64
const (
LANDLOCK_ACCESS_NET_BIND_TCP LandlockAccessNet = 1 << iota
LANDLOCK_ACCESS_NET_BIND_TCP AccessNet = 1 << iota
LANDLOCK_ACCESS_NET_CONNECT_TCP
_LANDLOCK_ACCESS_NET_DELIM
)
// String returns a space-separated string of [LandlockAccessNet] flags.
func (f LandlockAccessNet) String() string {
// String returns a space-separated string of [AccessNet] flags.
func (f AccessNet) String() string {
switch f {
case LANDLOCK_ACCESS_NET_BIND_TCP:
return "bind_tcp"
@@ -127,8 +127,8 @@ func (f LandlockAccessNet) String() string {
return "connect_tcp"
default:
var c []LandlockAccessNet
for i := LandlockAccessNet(1); i < _LANDLOCK_ACCESS_NET_DELIM; i <<= 1 {
var c []AccessNet
for i := AccessNet(1); i < _LANDLOCK_ACCESS_NET_DELIM; i <<= 1 {
if f&i != 0 {
c = append(c, i)
}
@@ -144,18 +144,18 @@ func (f LandlockAccessNet) String() string {
}
}
// LandlockScope is bitmask of scopes restricting a Landlock domain from accessing outside resources.
type LandlockScope uint64
// Scope is bitmask of scopes restricting a Landlock domain from accessing outside resources.
type Scope uint64
const (
LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET LandlockScope = 1 << iota
LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET Scope = 1 << iota
LANDLOCK_SCOPE_SIGNAL
_LANDLOCK_SCOPE_DELIM
)
// String returns a space-separated string of [LandlockScope] flags.
func (f LandlockScope) String() string {
// String returns a space-separated string of [Scope] flags.
func (f Scope) String() string {
switch f {
case LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET:
return "abstract_unix_socket"
@@ -164,8 +164,8 @@ func (f LandlockScope) String() string {
return "signal"
default:
var c []LandlockScope
for i := LandlockScope(1); i < _LANDLOCK_SCOPE_DELIM; i <<= 1 {
var c []Scope
for i := Scope(1); i < _LANDLOCK_SCOPE_DELIM; i <<= 1 {
if f&i != 0 {
c = append(c, i)
}
@@ -184,12 +184,12 @@ func (f LandlockScope) String() string {
// RulesetAttr is equivalent to struct landlock_ruleset_attr.
type RulesetAttr struct {
// Bitmask of handled filesystem actions.
HandledAccessFS LandlockAccessFS
HandledAccessFS AccessFS
// Bitmask of handled network actions.
HandledAccessNet LandlockAccessNet
HandledAccessNet AccessNet
// Bitmask of scopes restricting a Landlock domain from accessing outside
// resources (e.g. IPCs).
Scoped LandlockScope
Scoped Scope
}
// String returns a user-facing description of [RulesetAttr].
@@ -239,13 +239,13 @@ func (rulesetAttr *RulesetAttr) Create(flags uintptr) (fd int, err error) {
return fd, nil
}
// LandlockGetABI returns the ABI version supported by the kernel.
func LandlockGetABI() (int, error) {
// GetABI returns the ABI version supported by the kernel.
func GetABI() (int, error) {
return (*RulesetAttr)(nil).Create(LANDLOCK_CREATE_RULESET_VERSION)
}
// LandlockRestrictSelf applies a loaded ruleset to the calling thread.
func LandlockRestrictSelf(rulesetFd int, flags uintptr) error {
// RestrictSelf applies a loaded ruleset to the calling thread.
func RestrictSelf(rulesetFd int, flags uintptr) error {
r, _, errno := syscall.Syscall(
ext.SYS_LANDLOCK_RESTRICT_SELF,
uintptr(rulesetFd),

View File

@@ -0,0 +1,65 @@
package landlock_test
import (
"testing"
"unsafe"
"hakurei.app/internal/landlock"
)
func TestLandlockString(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
rulesetAttr *landlock.RulesetAttr
want string
}{
{"nil", nil, "NULL"},
{"zero", new(landlock.RulesetAttr), "0"},
{"some", &landlock.RulesetAttr{Scoped: landlock.LANDLOCK_SCOPE_SIGNAL}, "scoped: signal"},
{"set", &landlock.RulesetAttr{
HandledAccessFS: landlock.LANDLOCK_ACCESS_FS_MAKE_SYM | landlock.LANDLOCK_ACCESS_FS_IOCTL_DEV | landlock.LANDLOCK_ACCESS_FS_WRITE_FILE,
HandledAccessNet: landlock.LANDLOCK_ACCESS_NET_BIND_TCP,
Scoped: landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET | landlock.LANDLOCK_SCOPE_SIGNAL,
}, "fs: write_file make_sym fs_ioctl_dev, net: bind_tcp, scoped: abstract_unix_socket signal"},
{"all", &landlock.RulesetAttr{
HandledAccessFS: landlock.LANDLOCK_ACCESS_FS_EXECUTE |
landlock.LANDLOCK_ACCESS_FS_WRITE_FILE |
landlock.LANDLOCK_ACCESS_FS_READ_FILE |
landlock.LANDLOCK_ACCESS_FS_READ_DIR |
landlock.LANDLOCK_ACCESS_FS_REMOVE_DIR |
landlock.LANDLOCK_ACCESS_FS_REMOVE_FILE |
landlock.LANDLOCK_ACCESS_FS_MAKE_CHAR |
landlock.LANDLOCK_ACCESS_FS_MAKE_DIR |
landlock.LANDLOCK_ACCESS_FS_MAKE_REG |
landlock.LANDLOCK_ACCESS_FS_MAKE_SOCK |
landlock.LANDLOCK_ACCESS_FS_MAKE_FIFO |
landlock.LANDLOCK_ACCESS_FS_MAKE_BLOCK |
landlock.LANDLOCK_ACCESS_FS_MAKE_SYM |
landlock.LANDLOCK_ACCESS_FS_REFER |
landlock.LANDLOCK_ACCESS_FS_TRUNCATE |
landlock.LANDLOCK_ACCESS_FS_IOCTL_DEV,
HandledAccessNet: landlock.LANDLOCK_ACCESS_NET_BIND_TCP |
landlock.LANDLOCK_ACCESS_NET_CONNECT_TCP,
Scoped: landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET |
landlock.LANDLOCK_SCOPE_SIGNAL,
}, "fs: execute write_file read_file read_dir remove_dir remove_file make_char make_dir make_reg make_sock make_fifo make_block make_sym fs_refer fs_truncate fs_ioctl_dev, net: bind_tcp connect_tcp, scoped: abstract_unix_socket signal"},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if got := tc.rulesetAttr.String(); got != tc.want {
t.Errorf("String: %s, want %s", got, tc.want)
}
})
}
}
func TestLandlockAttrSize(t *testing.T) {
t.Parallel()
want := 24
if got := unsafe.Sizeof(landlock.RulesetAttr{}); got != uintptr(want) {
t.Errorf("Sizeof: %d, want %d", got, want)
}
}

View File

@@ -17,6 +17,7 @@ import (
"hakurei.app/ext"
"hakurei.app/internal/dbus"
"hakurei.app/internal/info"
"hakurei.app/internal/params"
"hakurei.app/message"
)
@@ -84,7 +85,7 @@ type syscallDispatcher interface {
// setDumpable provides [container.SetDumpable].
setDumpable(dumpable uintptr) error
// receive provides [container.Receive].
receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error)
receive(key string, e any, fdp *int) (closeFunc func() error, err error)
// containerStart provides the Start method of [container.Container].
containerStart(z *container.Container) error
@@ -154,8 +155,8 @@ func (direct) prctl(op, arg2, arg3 uintptr) error { return ext.Prctl(op, arg2, a
func (direct) overflowUid(msg message.Msg) int { return container.OverflowUid(msg) }
func (direct) overflowGid(msg message.Msg) int { return container.OverflowGid(msg) }
func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) }
func (direct) receive(key string, e any, fdp *uintptr) (func() error, error) {
return container.Receive(key, e, fdp)
func (direct) receive(key string, e any, fdp *int) (func() error, error) {
return params.Receive(key, e, fdp)
}
func (direct) containerStart(z *container.Container) error { return z.Start() }

View File

@@ -401,12 +401,12 @@ func (k *kstub) setDumpable(dumpable uintptr) error {
stub.CheckArg(k.Stub, "dumpable", dumpable, 0))
}
func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) {
func (k *kstub) receive(key string, e any, fdp *int) (closeFunc func() error, err error) {
k.Helper()
expect := k.Expects("receive")
reflect.ValueOf(e).Elem().Set(reflect.ValueOf(expect.Args[1]))
if expect.Args[2] != nil {
*fdp = expect.Args[2].(uintptr)
*fdp = int(expect.Args[2].(uintptr))
}
return func() error { return k.Expects("closeReceive").Err }, expect.Error(
stub.CheckArg(k.Stub, "key", key, 0))
@@ -690,38 +690,38 @@ func (panicMsgContext) Value(any) any { panic("unreachable") }
// This type is meant to be embedded in partial syscallDispatcher implementations.
type panicDispatcher struct{}
func (panicDispatcher) new(func(k syscallDispatcher, msg message.Msg)) { panic("unreachable") }
func (panicDispatcher) getppid() int { panic("unreachable") }
func (panicDispatcher) getpid() int { panic("unreachable") }
func (panicDispatcher) getuid() int { panic("unreachable") }
func (panicDispatcher) getgid() int { panic("unreachable") }
func (panicDispatcher) lookupEnv(string) (string, bool) { panic("unreachable") }
func (panicDispatcher) pipe() (*os.File, *os.File, error) { panic("unreachable") }
func (panicDispatcher) stat(string) (os.FileInfo, error) { panic("unreachable") }
func (panicDispatcher) open(string) (osFile, error) { panic("unreachable") }
func (panicDispatcher) readdir(string) ([]os.DirEntry, error) { panic("unreachable") }
func (panicDispatcher) tempdir() string { panic("unreachable") }
func (panicDispatcher) mkdir(string, os.FileMode) error { panic("unreachable") }
func (panicDispatcher) removeAll(string) error { panic("unreachable") }
func (panicDispatcher) exit(int) { panic("unreachable") }
func (panicDispatcher) evalSymlinks(string) (string, error) { panic("unreachable") }
func (panicDispatcher) prctl(uintptr, uintptr, uintptr) error { panic("unreachable") }
func (panicDispatcher) lookupGroupId(string) (string, error) { panic("unreachable") }
func (panicDispatcher) lookPath(string) (string, error) { panic("unreachable") }
func (panicDispatcher) cmdOutput(*exec.Cmd) ([]byte, error) { panic("unreachable") }
func (panicDispatcher) overflowUid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) overflowGid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) setDumpable(uintptr) error { panic("unreachable") }
func (panicDispatcher) receive(string, any, *uintptr) (func() error, error) { panic("unreachable") }
func (panicDispatcher) containerStart(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerServe(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerWait(*container.Container) error { panic("unreachable") }
func (panicDispatcher) mustHsuPath() *check.Absolute { panic("unreachable") }
func (panicDispatcher) dbusAddress() (string, string) { panic("unreachable") }
func (panicDispatcher) setupContSignal(int) (io.ReadCloser, func(), error) { panic("unreachable") }
func (panicDispatcher) getMsg() message.Msg { panic("unreachable") }
func (panicDispatcher) fatal(...any) { panic("unreachable") }
func (panicDispatcher) fatalf(string, ...any) { panic("unreachable") }
func (panicDispatcher) new(func(k syscallDispatcher, msg message.Msg)) { panic("unreachable") }
func (panicDispatcher) getppid() int { panic("unreachable") }
func (panicDispatcher) getpid() int { panic("unreachable") }
func (panicDispatcher) getuid() int { panic("unreachable") }
func (panicDispatcher) getgid() int { panic("unreachable") }
func (panicDispatcher) lookupEnv(string) (string, bool) { panic("unreachable") }
func (panicDispatcher) pipe() (*os.File, *os.File, error) { panic("unreachable") }
func (panicDispatcher) stat(string) (os.FileInfo, error) { panic("unreachable") }
func (panicDispatcher) open(string) (osFile, error) { panic("unreachable") }
func (panicDispatcher) readdir(string) ([]os.DirEntry, error) { panic("unreachable") }
func (panicDispatcher) tempdir() string { panic("unreachable") }
func (panicDispatcher) mkdir(string, os.FileMode) error { panic("unreachable") }
func (panicDispatcher) removeAll(string) error { panic("unreachable") }
func (panicDispatcher) exit(int) { panic("unreachable") }
func (panicDispatcher) evalSymlinks(string) (string, error) { panic("unreachable") }
func (panicDispatcher) prctl(uintptr, uintptr, uintptr) error { panic("unreachable") }
func (panicDispatcher) lookupGroupId(string) (string, error) { panic("unreachable") }
func (panicDispatcher) lookPath(string) (string, error) { panic("unreachable") }
func (panicDispatcher) cmdOutput(*exec.Cmd) ([]byte, error) { panic("unreachable") }
func (panicDispatcher) overflowUid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) overflowGid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) setDumpable(uintptr) error { panic("unreachable") }
func (panicDispatcher) receive(string, any, *int) (func() error, error) { panic("unreachable") }
func (panicDispatcher) containerStart(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerServe(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerWait(*container.Container) error { panic("unreachable") }
func (panicDispatcher) mustHsuPath() *check.Absolute { panic("unreachable") }
func (panicDispatcher) dbusAddress() (string, string) { panic("unreachable") }
func (panicDispatcher) setupContSignal(int) (io.ReadCloser, func(), error) { panic("unreachable") }
func (panicDispatcher) getMsg() message.Msg { panic("unreachable") }
func (panicDispatcher) fatal(...any) { panic("unreachable") }
func (panicDispatcher) fatalf(string, ...any) { panic("unreachable") }
func (panicDispatcher) notifyContext(context.Context, ...os.Signal) (context.Context, context.CancelFunc) {
panic("unreachable")

View File

@@ -32,7 +32,14 @@ type outcome struct {
syscallDispatcher
}
func (k *outcome) finalise(ctx context.Context, msg message.Msg, id *hst.ID, config *hst.Config) error {
// finalise prepares an outcome for main.
func (k *outcome) finalise(
ctx context.Context,
msg message.Msg,
id *hst.ID,
config *hst.Config,
flags int,
) error {
if ctx == nil || id == nil {
// unreachable
panic("invalid call to finalise")
@@ -43,7 +50,7 @@ func (k *outcome) finalise(ctx context.Context, msg message.Msg, id *hst.ID, con
}
k.ctx = ctx
if err := config.Validate(); err != nil {
if err := config.Validate(flags); err != nil {
return err
}

View File

@@ -194,7 +194,7 @@ type outcomeStateSys struct {
// Copied from [hst.Config]. Safe for read by outcomeOp.toSystem.
appId string
// Copied from [hst.Config]. Safe for read by outcomeOp.toSystem.
et hst.Enablement
et hst.Enablements
// Copied from [hst.Config]. Safe for read by spWaylandOp.toSystem only.
directWayland bool

View File

@@ -13,7 +13,6 @@ import (
"time"
"hakurei.app/check"
"hakurei.app/container"
"hakurei.app/fhs"
"hakurei.app/hst"
"hakurei.app/internal/info"
@@ -298,12 +297,12 @@ func (k *outcome) main(msg message.Msg, identifierFd int) {
// accumulate enablements of remaining instances
var (
// alive enablement bits
rt hst.Enablement
rt hst.Enablements
// alive instance count
n int
)
for eh := range entries {
var et hst.Enablement
var et hst.Enablements
if et, err = eh.Load(nil); err != nil {
perror(err, "read state header of instance "+eh.ID.String())
} else {
@@ -372,17 +371,18 @@ func (k *outcome) start(ctx context.Context, msg message.Msg,
// shim runs in the same session as monitor; see shim.go for behaviour
cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGCONT) }
var shimPipe *os.File
if fd, w, err := container.Setup(&cmd.ExtraFiles); err != nil {
var shimPipe [2]*os.File
if r, w, err := os.Pipe(); err != nil {
return cmd, nil, &hst.AppError{Step: "create shim setup pipe", Err: err}
} else {
shimPipe = w
cmd.Env = []string{
// passed through to shim by hsu
shimEnv + "=" + strconv.Itoa(fd),
shimEnv + "=" + strconv.Itoa(3+len(cmd.ExtraFiles)),
// interpreted by hsu
"HAKUREI_IDENTITY=" + k.state.identity.String(),
}
cmd.ExtraFiles = append(cmd.ExtraFiles, r)
shimPipe[0], shimPipe[1] = r, w
}
if len(k.supp) > 0 {
@@ -393,12 +393,16 @@ func (k *outcome) start(ctx context.Context, msg message.Msg,
msg.Verbosef("setuid helper at %s", hsuPath)
if err := cmd.Start(); err != nil {
_, _ = shimPipe[0].Close(), shimPipe[1].Close()
msg.Resume()
return cmd, shimPipe, &hst.AppError{Step: "start setuid wrapper", Err: err}
return cmd, nil, &hst.AppError{Step: "start setuid wrapper", Err: err}
}
if err := shimPipe[0].Close(); err != nil {
msg.Verbose(err)
}
*startTime = time.Now().UTC()
return cmd, shimPipe, nil
return cmd, shimPipe[1], nil
}
// serveShim serves outcomeState through the shim setup pipe.
@@ -411,11 +415,11 @@ func serveShim(msg message.Msg, shimPipe *os.File, state *outcomeState) error {
msg.Verbose(err.Error())
}
if err := gob.NewEncoder(shimPipe).Encode(state); err != nil {
_ = shimPipe.Close()
msg.Resume()
return &hst.AppError{Step: "transmit shim config", Err: err}
}
_ = shimPipe.Close()
return nil
return shimPipe.Close()
}
// printMessageError prints the error message according to [message.GetMessage],

View File

@@ -18,7 +18,13 @@ import (
func IsPollDescriptor(fd uintptr) bool
// Main runs an app according to [hst.Config] and terminates. Main does not return.
func Main(ctx context.Context, msg message.Msg, config *hst.Config, fd int) {
func Main(
ctx context.Context,
msg message.Msg,
config *hst.Config,
flags int,
fd int,
) {
// avoids runtime internals or standard streams
if fd >= 0 {
if IsPollDescriptor(uintptr(fd)) || fd < 3 {
@@ -34,7 +40,7 @@ func Main(ctx context.Context, msg message.Msg, config *hst.Config, fd int) {
k := outcome{syscallDispatcher: direct{msg}}
finaliseTime := time.Now()
if err := k.finalise(ctx, msg, &id, config); err != nil {
if err := k.finalise(ctx, msg, &id, config, flags); err != nil {
printMessageError(msg.GetLogger().Fatalln, "cannot seal app:", err)
panic("unreachable")
}

View File

@@ -288,7 +288,7 @@ func TestOutcomeRun(t *testing.T) {
},
Filter: true,
},
Enablements: hst.NewEnablements(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Enablements: new(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Container: &hst.ContainerConfig{
Filesystem: []hst.FilesystemConfigJSON{
@@ -427,7 +427,7 @@ func TestOutcomeRun(t *testing.T) {
DirectPipeWire: true,
ID: "org.chromium.Chromium",
Enablements: hst.NewEnablements(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Enablements: new(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Container: &hst.ContainerConfig{
Env: nil,
Filesystem: []hst.FilesystemConfigJSON{

View File

@@ -20,6 +20,7 @@ import (
"hakurei.app/ext"
"hakurei.app/fhs"
"hakurei.app/hst"
"hakurei.app/internal/params"
"hakurei.app/internal/pipewire"
"hakurei.app/message"
)
@@ -197,7 +198,7 @@ func shimEntrypoint(k syscallDispatcher) {
if errors.Is(err, syscall.EBADF) {
k.fatal("invalid config descriptor")
}
if errors.Is(err, container.ErrReceiveEnv) {
if errors.Is(err, params.ErrReceiveEnv) {
k.fatal(shimEnv + " not set")
}

View File

@@ -16,6 +16,7 @@ import (
"hakurei.app/fhs"
"hakurei.app/hst"
"hakurei.app/internal/env"
"hakurei.app/internal/params"
"hakurei.app/internal/stub"
)
@@ -172,7 +173,7 @@ func TestShimEntrypoint(t *testing.T) {
call("setDumpable", stub.ExpectArgs{uintptr(ext.SUID_DUMP_DISABLE)}, nil, nil),
call("getppid", stub.ExpectArgs{}, 0xbad, nil),
call("setupContSignal", stub.ExpectArgs{0xbad}, 0, nil),
call("receive", stub.ExpectArgs{"HAKUREI_SHIM", outcomeState{}, nil}, nil, container.ErrReceiveEnv),
call("receive", stub.ExpectArgs{"HAKUREI_SHIM", outcomeState{}, nil}, nil, params.ErrReceiveEnv),
call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SHIM not set"}}, nil, nil),
// deferred

View File

@@ -290,7 +290,9 @@ func (s *spFilesystemOp) toContainer(state *outcomeStateParams) error {
if state.Container.Flags&hst.FDevice == 0 {
state.params.Remount(fhs.AbsDev, syscall.MS_RDONLY)
}
state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY)
if !state.as.NoRemountRoot {
state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY)
}
state.params.Env = make([]string, 0, len(state.env))
for key, value := range state.env {

View File

@@ -21,7 +21,7 @@ func TestSpPulseOp(t *testing.T) {
newConfig := func() *hst.Config {
config := hst.Template()
config.DirectPulse = true
config.Enablements = hst.NewEnablements(hst.EPulse)
config.Enablements = new(hst.EPulse)
return config
}

42
internal/params/params.go Normal file
View File

@@ -0,0 +1,42 @@
// Package params provides helpers for receiving setup payload from parent.
package params
import (
"encoding/gob"
"errors"
"os"
"strconv"
"syscall"
)
// ErrReceiveEnv is returned by [Receive] if setup fd is not present in environment.
var ErrReceiveEnv = errors.New("environment variable not set")
// Receive retrieves setup fd from the environment and receives params.
//
// The file descriptor written to the value pointed to by fdp must not be passed
// to any system calls. It is made available for ordering file descriptor only.
func Receive(key string, v any, fdp *int) (func() error, error) {
var setup *os.File
if s, ok := os.LookupEnv(key); !ok {
return nil, ErrReceiveEnv
} else {
if fd, err := strconv.Atoi(s); err != nil {
if _err := errors.Unwrap(err); _err != nil {
err = _err
}
return nil, err
} else {
setup = os.NewFile(uintptr(fd), "setup")
if setup == nil {
return nil, syscall.EDOM
}
if fdp != nil {
*fdp = fd
}
}
}
return setup.Close, gob.NewDecoder(setup).Decode(v)
}

View File

@@ -1,4 +1,4 @@
package container_test
package params_test
import (
"encoding/gob"
@@ -9,7 +9,7 @@ import (
"syscall"
"testing"
"hakurei.app/container"
"hakurei.app/internal/params"
)
func TestSetupReceive(t *testing.T) {
@@ -30,8 +30,8 @@ func TestSetupReceive(t *testing.T) {
})
}
if _, err := container.Receive(key, nil, nil); !errors.Is(err, container.ErrReceiveEnv) {
t.Errorf("Receive: error = %v, want %v", err, container.ErrReceiveEnv)
if _, err := params.Receive(key, nil, nil); !errors.Is(err, params.ErrReceiveEnv) {
t.Errorf("Receive: error = %v, want %v", err, params.ErrReceiveEnv)
}
})
@@ -39,7 +39,7 @@ func TestSetupReceive(t *testing.T) {
const key = "TEST_ENV_FORMAT"
t.Setenv(key, "")
if _, err := container.Receive(key, nil, nil); !errors.Is(err, strconv.ErrSyntax) {
if _, err := params.Receive(key, nil, nil); !errors.Is(err, strconv.ErrSyntax) {
t.Errorf("Receive: error = %v, want %v", err, strconv.ErrSyntax)
}
})
@@ -48,7 +48,7 @@ func TestSetupReceive(t *testing.T) {
const key = "TEST_ENV_RANGE"
t.Setenv(key, "-1")
if _, err := container.Receive(key, nil, nil); !errors.Is(err, syscall.EDOM) {
if _, err := params.Receive(key, nil, nil); !errors.Is(err, syscall.EDOM) {
t.Errorf("Receive: error = %v, want %v", err, syscall.EDOM)
}
})
@@ -60,16 +60,22 @@ func TestSetupReceive(t *testing.T) {
encoderDone := make(chan error, 1)
extraFiles := make([]*os.File, 0, 1)
deadline, _ := t.Deadline()
if fd, f, err := container.Setup(&extraFiles); err != nil {
if r, w, err := os.Pipe(); err != nil {
t.Fatalf("Setup: error = %v", err)
} else if fd != 3 {
t.Fatalf("Setup: fd = %d, want 3", fd)
} else {
if err = f.SetDeadline(deadline); err != nil {
t.Fatal(err.Error())
t.Cleanup(func() {
if err = errors.Join(r.Close(), w.Close()); err != nil {
t.Fatal(err)
}
})
extraFiles = append(extraFiles, r)
if deadline, ok := t.Deadline(); ok {
if err = w.SetDeadline(deadline); err != nil {
t.Fatal(err)
}
}
go func() { encoderDone <- gob.NewEncoder(f).Encode(payload) }()
go func() { encoderDone <- gob.NewEncoder(w).Encode(payload) }()
}
if len(extraFiles) != 1 {
@@ -87,13 +93,13 @@ func TestSetupReceive(t *testing.T) {
var (
gotPayload []uint64
fdp *uintptr
fdp *int
)
if !useNilFdp {
fdp = new(uintptr)
fdp = new(int)
}
var closeFile func() error
if f, err := container.Receive(key, &gotPayload, fdp); err != nil {
if f, err := params.Receive(key, &gotPayload, fdp); err != nil {
t.Fatalf("Receive: error = %v", err)
} else {
closeFile = f
@@ -103,7 +109,7 @@ func TestSetupReceive(t *testing.T) {
}
}
if !useNilFdp {
if int(*fdp) != dupFd {
if *fdp != dupFd {
t.Errorf("Fd: %d, want %d", *fdp, dupFd)
}
}

View File

@@ -184,6 +184,32 @@ func TestFlatten(t *testing.T) {
{Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d"), nil},
{"sample no assume checksum", fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},
"checksum": {Mode: fs.ModeDir | 0700},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M": {Mode: fs.ModeDir | 0500},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check": {Mode: 0400, Data: []byte{}},
"identifier": {Mode: fs.ModeDir | 0700},
"identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"work": {Mode: fs.ModeDir | 0700},
}, []pkg.FlatEntry{
{Mode: fs.ModeDir | 0700, Path: "."},
{Mode: fs.ModeDir | 0700, Path: "checksum"},
{Mode: fs.ModeDir | 0500, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M"},
{Mode: 0400, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check", Data: []byte{}},
{Mode: fs.ModeDir | 0700, Path: "identifier"},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w"), nil},
{"sample tar step unpack", fstest.MapFS{
".": {Mode: fs.ModeDir | 0500},

View File

@@ -27,6 +27,11 @@ import (
// AbsWork is the container pathname [TContext.GetWorkDir] is mounted on.
var AbsWork = fhs.AbsRoot.Append("work/")
// EnvJobs is the name of the environment variable holding a decimal
// representation of the preferred job count. Its value must not affect cure
// outcome.
const EnvJobs = "CURE_JOBS"
// ExecPath is a slice of [Artifact] and the [check.Absolute] pathname to make
// it available at under in the container.
type ExecPath struct {
@@ -40,9 +45,6 @@ type ExecPath struct {
W bool
}
// SetSchedIdle is whether to set [ext.SCHED_IDLE] scheduling priority.
var SetSchedIdle bool
// GetArtifactFunc is the function signature of [FContext.GetArtifact].
type GetArtifactFunc func(Artifact) (*check.Absolute, unique.Handle[Checksum])
@@ -400,6 +402,7 @@ const SeccompPresets = std.PresetStrict &
func (a *execArtifact) makeContainer(
ctx context.Context,
msg message.Msg,
flags, jobs int,
hostNet bool,
temp, work *check.Absolute,
getArtifact GetArtifactFunc,
@@ -426,15 +429,16 @@ func (a *execArtifact) makeContainer(
z.SeccompFlags |= seccomp.AllowMultiarch
z.ParentPerm = 0700
z.HostNet = hostNet
z.HostAbstract = flags&CHostAbstract != 0
z.Hostname = "cure"
z.SetScheduler = SetSchedIdle
z.SetScheduler = flags&CSchedIdle != 0
z.SchedPolicy = ext.SCHED_IDLE
if z.HostNet {
z.Hostname = "cure-net"
}
z.Uid, z.Gid = (1<<10)-1, (1<<10)-1
z.Dir, z.Env, z.Path, z.Args = a.dir, a.env, a.path, a.args
z.Dir, z.Path, z.Args = a.dir, a.path, a.args
z.Env = slices.Concat(a.env, []string{EnvJobs + "=" + strconv.Itoa(jobs)})
z.Grow(len(a.paths) + 4)
for i, b := range a.paths {
@@ -563,6 +567,8 @@ func (c *Cache) EnterExec(
var z *container.Container
z, err = e.makeContainer(
ctx, c.msg,
c.flags,
c.jobs,
hostNet,
temp, work,
func(a Artifact) (*check.Absolute, unique.Handle[Checksum]) {
@@ -579,6 +585,11 @@ func (c *Cache) EnterExec(
z.Stdin, z.Stdout, z.Stderr = stdin, stdout, stderr
z.Path, z.Args = path, args
z.RetainSession = retainSession
if stdin == os.Stdin {
if s, ok := os.LookupEnv("TERM"); ok {
z.Env = append(z.Env, "TERM="+s)
}
}
if err = z.Start(); err != nil {
return
@@ -597,7 +608,7 @@ func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
msg := f.GetMessage()
var z *container.Container
if z, err = a.makeContainer(
ctx, msg, hostNet,
ctx, msg, f.cache.flags, f.GetJobs(), hostNet,
f.GetTempDir(), f.GetWorkDir(),
f.GetArtifact,
f.cache.Ident,

View File

@@ -33,8 +33,7 @@ func TestExec(t *testing.T) {
)
checkWithCache(t, []cacheTestCase{
{"offline", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"offline", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{
@@ -111,8 +110,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c)
}, pkg.MustDecode("Q5DluWQCAeohLoiGRImurwFp3vdz9IfQCoj7Fuhh73s4KQPRHpEQEnHTdNHmB8Fx")},
{"net", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"net", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
wantChecksum := pkg.MustDecode(
@@ -146,8 +144,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c)
}, pkg.MustDecode("bPYvvqxpfV7xcC1EptqyKNK1klLJgYHMDkzBcoOyK6j_Aj5hb0mXNPwTwPSK5F6Z")},
{"overlay root", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"overlay root", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{
@@ -172,8 +169,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c)
}, pkg.MustDecode("PO2DSSCa4yoSgEYRcCSZfQfwow1yRigL3Ry-hI0RDI4aGuFBha-EfXeSJnG_5_Rl")},
{"overlay work", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"overlay work", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{
@@ -203,8 +199,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c)
}, pkg.MustDecode("iaRt6l_Wm2n-h5UsDewZxQkCmjZjyL8r7wv32QT2kyV55-Lx09Dq4gfg9BiwPnKs")},
{"multiple layers", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"multiple layers", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{
@@ -256,8 +251,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c)
}, pkg.MustDecode("O2YzyR7IUGU5J2CADy0hUZ3A5NkP_Vwzs4UadEdn2oMZZVWRtH0xZGJ3HXiimTnZ")},
{"overlay layer promotion", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"overlay layer promotion", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{

View File

@@ -11,9 +11,7 @@ func TestFile(t *testing.T) {
t.Parallel()
checkWithCache(t, []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"file", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
cureMany(t, c, []cureStep{
{"short", pkg.NewFile("null", []byte{0}), base.Append(
"identifier",

View File

@@ -3,7 +3,6 @@ package pkg
import (
"bufio"
"bytes"
"context"
"crypto/sha512"
"encoding/binary"
"errors"
@@ -11,6 +10,7 @@ import (
"io"
"slices"
"strconv"
"sync"
"syscall"
"unique"
"unsafe"
@@ -39,22 +39,45 @@ func panicToError(errP *error) {
}
}
// irCache implements [IRCache].
type irCache struct {
// Artifact to [unique.Handle] of identifier cache.
artifact sync.Map
// Identifier free list, must not be accessed directly.
identPool sync.Pool
}
// zeroIRCache returns the initialised value of irCache.
func zeroIRCache() irCache {
return irCache{
identPool: sync.Pool{New: func() any { return new(extIdent) }},
}
}
// IRCache provides memory management and caching primitives for IR and
// identifier operations against [Artifact] implementations.
//
// The zero value is not safe for use.
type IRCache struct{ irCache }
// NewIR returns the address of a new [IRCache].
func NewIR() *IRCache {
return &IRCache{zeroIRCache()}
}
// IContext is passed to [Artifact.Params] and provides methods for writing
// values to the IR writer. It does not expose the underlying [io.Writer].
//
// IContext is valid until [Artifact.Params] returns.
type IContext struct {
// Address of underlying [Cache], should be zeroed or made unusable after
// Address of underlying irCache, should be zeroed or made unusable after
// [Artifact.Params] returns and must not be exposed directly.
cache *Cache
ic *irCache
// Written to by various methods, should be zeroed after [Artifact.Params]
// returns and must not be exposed directly.
w io.Writer
}
// Unwrap returns the underlying [context.Context].
func (i *IContext) Unwrap() context.Context { return i.cache.ctx }
// irZero is a zero IR word.
var irZero [wordSize]byte
@@ -136,11 +159,11 @@ func (i *IContext) mustWrite(p []byte) {
// WriteIdent is not defined for an [Artifact] not part of the slice returned by
// [Artifact.Dependencies].
func (i *IContext) WriteIdent(a Artifact) {
buf := i.cache.getIdentBuf()
defer i.cache.putIdentBuf(buf)
buf := i.ic.getIdentBuf()
defer i.ic.putIdentBuf(buf)
IRKindIdent.encodeHeader(0).put(buf[:])
*(*ID)(buf[wordSize:]) = i.cache.Ident(a).Value()
*(*ID)(buf[wordSize:]) = i.ic.Ident(a).Value()
i.mustWrite(buf[:])
}
@@ -183,19 +206,19 @@ func (i *IContext) WriteString(s string) {
// Encode writes a deterministic, efficient representation of a to w and returns
// the first non-nil error encountered while writing to w.
func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
func (ic *irCache) Encode(w io.Writer, a Artifact) (err error) {
deps := a.Dependencies()
idents := make([]*extIdent, len(deps))
for i, d := range deps {
dbuf, did := c.unsafeIdent(d, true)
dbuf, did := ic.unsafeIdent(d, true)
if dbuf == nil {
dbuf = c.getIdentBuf()
dbuf = ic.getIdentBuf()
binary.LittleEndian.PutUint64(dbuf[:], uint64(d.Kind()))
*(*ID)(dbuf[wordSize:]) = did.Value()
} else {
c.storeIdent(d, dbuf)
ic.storeIdent(d, dbuf)
}
defer c.putIdentBuf(dbuf)
defer ic.putIdentBuf(dbuf)
idents[i] = dbuf
}
slices.SortFunc(idents, func(a, b *extIdent) int {
@@ -221,10 +244,10 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
}
func() {
i := IContext{c, w}
i := IContext{ic, w}
defer panicToError(&err)
defer func() { i.cache, i.w = nil, nil }()
defer func() { i.ic, i.w = nil, nil }()
a.Params(&i)
}()
@@ -233,7 +256,7 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
}
var f IREndFlag
kcBuf := c.getIdentBuf()
kcBuf := ic.getIdentBuf()
sz := wordSize
if kc, ok := a.(KnownChecksum); ok {
f |= IREndKnownChecksum
@@ -243,13 +266,13 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
IRKindEnd.encodeHeader(uint32(f)).put(kcBuf[:])
_, err = w.Write(kcBuf[:sz])
c.putIdentBuf(kcBuf)
ic.putIdentBuf(kcBuf)
return
}
// encodeAll implements EncodeAll by recursively encoding dependencies and
// performs deduplication by value via the encoded map.
func (c *Cache) encodeAll(
func (ic *irCache) encodeAll(
w io.Writer,
a Artifact,
encoded map[Artifact]struct{},
@@ -259,13 +282,13 @@ func (c *Cache) encodeAll(
}
for _, d := range a.Dependencies() {
if err = c.encodeAll(w, d, encoded); err != nil {
if err = ic.encodeAll(w, d, encoded); err != nil {
return
}
}
encoded[a] = struct{}{}
return c.Encode(w, a)
return ic.Encode(w, a)
}
// EncodeAll writes a self-describing IR stream of a to w and returns the first
@@ -283,8 +306,8 @@ func (c *Cache) encodeAll(
// the ident cache, nor does it contribute identifiers it computes back to the
// ident cache. Because of this, multiple invocations of EncodeAll will have
// similar cost and does not amortise when combined with a call to Cure.
func (c *Cache) EncodeAll(w io.Writer, a Artifact) error {
return c.encodeAll(w, a, make(map[Artifact]struct{}))
func (ic *irCache) EncodeAll(w io.Writer, a Artifact) error {
return ic.encodeAll(w, a, make(map[Artifact]struct{}))
}
// ErrRemainingIR is returned for a [IRReadFunc] that failed to call

View File

@@ -85,7 +85,7 @@ func TestIRRoundtrip(t *testing.T) {
testCasesCache := make([]cacheTestCase, len(testCases))
for i, tc := range testCases {
want := tc.a
testCasesCache[i] = cacheTestCase{tc.name, nil,
testCasesCache[i] = cacheTestCase{tc.name, 0, nil,
func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
r, w := io.Pipe()

View File

@@ -8,7 +8,6 @@ import (
"testing"
"testing/fstest"
"unique"
"unsafe"
"hakurei.app/check"
"hakurei.app/internal/pkg"
@@ -32,21 +31,15 @@ func TestHTTPGet(t *testing.T) {
}))
checkWithCache(t, []cacheTestCase{
{"direct", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
{"direct", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
r := newRContext(t, c)
f := pkg.NewHTTPGet(
&client,
"file:///testdata",
testdataChecksum.Value(),
)
var got []byte
if rc, err := f.Cure(&r); err != nil {
if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err)
@@ -65,7 +58,7 @@ func TestHTTPGet(t *testing.T) {
wantErrMismatch := &pkg.ChecksumMismatchError{
Got: testdataChecksum.Value(),
}
if rc, err := f.Cure(&r); err != nil {
if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err)
@@ -76,7 +69,7 @@ func TestHTTPGet(t *testing.T) {
}
// check fallback validation
if rc, err := f.Cure(&r); err != nil {
if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err)
} else if err = rc.Close(); !reflect.DeepEqual(err, wantErrMismatch) {
t.Fatalf("Close: error = %#v, want %#v", err, wantErrMismatch)
@@ -89,18 +82,13 @@ func TestHTTPGet(t *testing.T) {
pkg.Checksum{},
)
wantErrNotFound := pkg.ResponseStatusError(http.StatusNotFound)
if _, err := f.Cure(&r); !reflect.DeepEqual(err, wantErrNotFound) {
if _, err := f.Cure(r); !reflect.DeepEqual(err, wantErrNotFound) {
t.Fatalf("Cure: error = %#v, want %#v", err, wantErrNotFound)
}
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"cure", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
{"cure", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
r := newRContext(t, c)
f := pkg.NewHTTPGet(
&client,
@@ -120,7 +108,7 @@ func TestHTTPGet(t *testing.T) {
}
var got []byte
if rc, err := f.Cure(&r); err != nil {
if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err)
@@ -136,7 +124,7 @@ func TestHTTPGet(t *testing.T) {
"file:///testdata",
testdataChecksum.Value(),
)
if rc, err := f.Cure(&r); err != nil {
if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err)

View File

@@ -13,7 +13,6 @@ import (
"hash"
"io"
"io/fs"
"iter"
"maps"
"os"
"path/filepath"
@@ -73,6 +72,10 @@ func MustDecode(s string) (checksum Checksum) {
// common holds elements and receives methods shared between different contexts.
type common struct {
// Context specific to this [Artifact]. The toplevel context in [Cache] must
// not be exposed directly.
ctx context.Context
// Address of underlying [Cache], should be zeroed or made unusable after
// Cure returns and must not be exposed directly.
cache *Cache
@@ -184,11 +187,15 @@ func (t *TContext) destroy(errP *error) {
}
// Unwrap returns the underlying [context.Context].
func (c *common) Unwrap() context.Context { return c.cache.ctx }
func (c *common) Unwrap() context.Context { return c.ctx }
// GetMessage returns [message.Msg] held by the underlying [Cache].
func (c *common) GetMessage() message.Msg { return c.cache.msg }
// GetJobs returns the preferred number of jobs to run, when applicable. Its
// value must not affect cure outcome.
func (c *common) GetJobs() int { return c.cache.jobs }
// GetWorkDir returns a pathname to a directory which [Artifact] is expected to
// write its output to. This is not the final resting place of the [Artifact]
// and this pathname should not be directly referred to in the final contents.
@@ -208,11 +215,11 @@ func (t *TContext) GetTempDir() *check.Absolute { return t.temp }
// [ChecksumMismatchError], or the underlying implementation may block on Close.
func (c *common) Open(a Artifact) (r io.ReadCloser, err error) {
if f, ok := a.(FileArtifact); ok {
return c.cache.openFile(f)
return c.cache.openFile(c.ctx, f)
}
var pathname *check.Absolute
if pathname, _, err = c.cache.Cure(a); err != nil {
if pathname, _, err = c.cache.cure(a, true); err != nil {
return
}
@@ -331,23 +338,6 @@ type FloodArtifact interface {
Artifact
}
// Flood returns an iterator over the dependency tree of an [Artifact].
func Flood(a Artifact) iter.Seq[Artifact] {
return func(yield func(Artifact) bool) {
for _, d := range a.Dependencies() {
if !yield(d) {
return
}
for d0 := range Flood(d) {
if !yield(d0) {
return
}
}
}
}
}
// TrivialArtifact refers to an [Artifact] that cures without requiring that
// any other [Artifact] is cured before it. Its dependency tree is ignored after
// computing its identifier.
@@ -390,6 +380,9 @@ type KnownChecksum interface {
}
// FileArtifact refers to an [Artifact] backed by a single file.
//
// FileArtifact does not support fine-grained cancellation. Its context is
// inherited from the first [TrivialArtifact] or [FloodArtifact] that opens it.
type FileArtifact interface {
// Cure returns [io.ReadCloser] of the full contents of [FileArtifact]. If
// [FileArtifact] implements [KnownChecksum], Cure is responsible for
@@ -506,6 +499,73 @@ type pendingArtifactDep struct {
*sync.WaitGroup
}
const (
// CValidateKnown arranges for [KnownChecksum] outcomes to be validated to
// match its intended checksum.
//
// A correct implementation of [KnownChecksum] does not successfully cure
// with output not matching its intended checksum. When an implementation
// fails to perform this validation correctly, the on-disk format enters
// an inconsistent state (correctable by [Cache.Scrub]).
//
// This flag causes [Cache.Cure] to always compute the checksum, and reject
// a cure if it does not match the intended checksum.
//
// This behaviour significantly reduces performance and is not recommended
// outside of testing a custom [Artifact] implementation.
CValidateKnown = 1 << iota
// CSchedIdle arranges for the [ext.SCHED_IDLE] scheduling priority to be
// set for [KindExec] and [KindExecNet] containers.
CSchedIdle
// CAssumeChecksum enables the use of [KnownChecksum] for duplicate function
// call suppression via the on-disk cache.
//
// This may cause incorrect cure outcome if an impossible checksum is
// specified that matches an output already present in the on-disk cache.
// This may be avoided by purposefully specifying a statistically
// unattainable checksum, like the zero value.
//
// While this optimisation might seem appealing, it is almost never
// applicable in real world use. Almost every time this path was taken, it
// was caused by an incorrect checksum accidentally left behind while
// bumping a package. Only enable this if you are really sure you need it.
CAssumeChecksum
// CHostAbstract disables restriction of sandboxed processes from connecting
// to an abstract UNIX socket created by a host process.
//
// This is considered less secure in some systems, but does not introduce
// impurity due to [KindExecNet] being [KnownChecksum]. This flag exists
// to support kernels without Landlock LSM enabled.
CHostAbstract
)
// toplevel holds [context.WithCancel] over caller-supplied context, where all
// [Artifact] context are derived from.
type toplevel struct {
ctx context.Context
cancel context.CancelFunc
}
// newToplevel returns the address of a new toplevel via ctx.
func newToplevel(ctx context.Context) *toplevel {
var t toplevel
t.ctx, t.cancel = context.WithCancel(ctx)
return &t
}
// pendingCure provides synchronisation and cancellation for pending cures.
type pendingCure struct {
// Closed on cure completion.
done <-chan struct{}
// Error outcome, safe to access after done is closed.
err error
// Cancels the corresponding cure.
cancel context.CancelFunc
}
// Cache is a support layer that implementations of [Artifact] can use to store
// cured [Artifact] data in a content addressed fashion.
type Cache struct {
@@ -513,11 +573,10 @@ type Cache struct {
// implementation and receives an equal amount of elements after.
cures chan struct{}
// [context.WithCancel] over caller-supplied context, used by [Artifact] and
// all dependency curing goroutines.
ctx context.Context
// Cancels ctx.
cancel context.CancelFunc
// Parent context which toplevel was derived from.
parent context.Context
// For deriving curing context, must not be accessed directly.
toplevel atomic.Pointer[toplevel]
// For waiting on dependency curing goroutines.
wg sync.WaitGroup
// Reports new cures and passed to [Artifact].
@@ -525,17 +584,13 @@ type Cache struct {
// Directory where all [Cache] related files are placed.
base *check.Absolute
// Immutable cure options set by [Open].
flags int
// Immutable job count, when applicable.
jobs int
// Whether to validate [FileArtifact.Cure] for a [KnownChecksum] file. This
// significantly reduces performance.
strict bool
// Maximum size of a dependency graph.
threshold uintptr
// Artifact to [unique.Handle] of identifier cache.
artifact sync.Map
// Identifier free list, must not be accessed directly.
identPool sync.Pool
// Must not be exposed directly.
irCache
// Synchronises access to dirChecksum.
checksumMu sync.RWMutex
@@ -545,9 +600,11 @@ type Cache struct {
// Identifier to error pair for unrecoverably faulted [Artifact].
identErr map[unique.Handle[ID]]error
// Pending identifiers, accessed through Cure for entries not in ident.
identPending map[unique.Handle[ID]]<-chan struct{}
identPending map[unique.Handle[ID]]*pendingCure
// Synchronises access to ident and corresponding filesystem entries.
identMu sync.RWMutex
// Synchronises entry into Abort and Cure.
abortMu sync.RWMutex
// Synchronises entry into exclusive artifacts for the cure method.
exclMu sync.Mutex
@@ -556,51 +613,37 @@ type Cache struct {
// Unlocks the on-filesystem cache. Must only be called from Close.
unlock func()
// Synchronises calls to Close.
closeOnce sync.Once
// Whether [Cache] is considered closed.
closed bool
// Synchronises calls to Abort and Close.
closeMu sync.Mutex
// Whether EnterExec has not yet returned.
inExec atomic.Bool
}
// IsStrict returns whether the [Cache] strictly verifies checksums.
func (c *Cache) IsStrict() bool { return c.strict }
// SetStrict sets whether the [Cache] strictly verifies checksums, even when
// the implementation promises to validate them internally. This significantly
// reduces performance and is not recommended outside of testing.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetStrict(strict bool) { c.strict = strict }
// SetThreshold imposes a maximum size on the dependency graph, checked on every
// call to Cure. The zero value disables this check entirely.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetThreshold(threshold uintptr) { c.threshold = threshold }
// extIdent is a [Kind] concatenated with [ID].
type extIdent [wordSize + len(ID{})]byte
// getIdentBuf returns the address of an extIdent for Ident.
func (c *Cache) getIdentBuf() *extIdent { return c.identPool.Get().(*extIdent) }
func (ic *irCache) getIdentBuf() *extIdent { return ic.identPool.Get().(*extIdent) }
// putIdentBuf adds buf to identPool.
func (c *Cache) putIdentBuf(buf *extIdent) { c.identPool.Put(buf) }
func (ic *irCache) putIdentBuf(buf *extIdent) { ic.identPool.Put(buf) }
// storeIdent adds an [Artifact] to the artifact cache.
func (c *Cache) storeIdent(a Artifact, buf *extIdent) unique.Handle[ID] {
func (ic *irCache) storeIdent(a Artifact, buf *extIdent) unique.Handle[ID] {
idu := unique.Make(ID(buf[wordSize:]))
c.artifact.Store(a, idu)
ic.artifact.Store(a, idu)
return idu
}
// Ident returns the identifier of an [Artifact].
func (c *Cache) Ident(a Artifact) unique.Handle[ID] {
buf, idu := c.unsafeIdent(a, false)
func (ic *irCache) Ident(a Artifact) unique.Handle[ID] {
buf, idu := ic.unsafeIdent(a, false)
if buf != nil {
idu = c.storeIdent(a, buf)
c.putIdentBuf(buf)
idu = ic.storeIdent(a, buf)
ic.putIdentBuf(buf)
}
return idu
}
@@ -608,17 +651,17 @@ func (c *Cache) Ident(a Artifact) unique.Handle[ID] {
// unsafeIdent implements Ident but returns the underlying buffer for a newly
// computed identifier. Callers must return this buffer to identPool. encodeKind
// is only a hint, kind may still be encoded in the buffer.
func (c *Cache) unsafeIdent(a Artifact, encodeKind bool) (
func (ic *irCache) unsafeIdent(a Artifact, encodeKind bool) (
buf *extIdent,
idu unique.Handle[ID],
) {
if id, ok := c.artifact.Load(a); ok {
if id, ok := ic.artifact.Load(a); ok {
idu = id.(unique.Handle[ID])
return
}
if ki, ok := a.(KnownIdent); ok {
buf = c.getIdentBuf()
buf = ic.getIdentBuf()
if encodeKind {
binary.LittleEndian.PutUint64(buf[:], uint64(a.Kind()))
}
@@ -626,9 +669,9 @@ func (c *Cache) unsafeIdent(a Artifact, encodeKind bool) (
return
}
buf = c.getIdentBuf()
buf = ic.getIdentBuf()
h := sha512.New384()
if err := c.Encode(h, a); err != nil {
if err := ic.Encode(h, a); err != nil {
// unreachable
panic(err)
}
@@ -997,7 +1040,11 @@ func (c *Cache) Scrub(checks int) error {
// loadOrStoreIdent attempts to load a cached [Artifact] by its identifier or
// wait for a pending [Artifact] to cure. If neither is possible, the current
// identifier is stored in identPending and a non-nil channel is returned.
//
// Since identErr is treated as grow-only, loadOrStoreIdent must not be entered
// without holding a read lock on abortMu.
func (c *Cache) loadOrStoreIdent(id unique.Handle[ID]) (
ctx context.Context,
done chan<- struct{},
checksum unique.Handle[Checksum],
err error,
@@ -1014,20 +1061,23 @@ func (c *Cache) loadOrStoreIdent(id unique.Handle[ID]) (
return
}
var notify <-chan struct{}
if notify, ok = c.identPending[id]; ok {
var pending *pendingCure
if pending, ok = c.identPending[id]; ok {
c.identMu.Unlock()
<-notify
<-pending.done
c.identMu.RLock()
if checksum, ok = c.ident[id]; !ok {
err = c.identErr[id]
err = pending.err
}
c.identMu.RUnlock()
return
}
d := make(chan struct{})
c.identPending[id] = d
pending = &pendingCure{done: d}
ctx, pending.cancel = context.WithCancel(c.toplevel.Load().ctx)
c.wg.Add(1)
c.identPending[id] = pending
c.identMu.Unlock()
done = d
return
@@ -1043,22 +1093,63 @@ func (c *Cache) finaliseIdent(
) {
c.identMu.Lock()
if err != nil {
c.identPending[id].err = err
c.identErr[id] = err
} else {
c.ident[id] = checksum
}
delete(c.identPending, id)
c.identMu.Unlock()
c.wg.Done()
close(done)
}
// Done returns a channel that is closed when the ongoing cure of an [Artifact]
// referred to by the specified identifier completes. Done may return nil if
// no ongoing cure of the specified identifier exists.
func (c *Cache) Done(id unique.Handle[ID]) <-chan struct{} {
c.identMu.RLock()
pending, ok := c.identPending[id]
c.identMu.RUnlock()
if !ok || pending == nil {
return nil
}
return pending.done
}
// Cancel cancels the ongoing cure of an [Artifact] referred to by the specified
// identifier. Cancel returns whether the [context.CancelFunc] has been killed.
// Cancel returns after the cure is complete.
func (c *Cache) Cancel(id unique.Handle[ID]) bool {
c.identMu.RLock()
pending, ok := c.identPending[id]
c.identMu.RUnlock()
if !ok || pending == nil || pending.cancel == nil {
return false
}
pending.cancel()
<-pending.done
c.abortMu.Lock()
c.identMu.Lock()
delete(c.identErr, id)
c.identMu.Unlock()
c.abortMu.Unlock()
return true
}
// openFile tries to load [FileArtifact] from [Cache], and if that fails,
// obtains it via [FileArtifact.Cure] instead. Notably, it does not cure
// [FileArtifact] to the filesystem. If err is nil, the caller is responsible
// for closing the resulting [io.ReadCloser].
func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) {
if kc, ok := f.(KnownChecksum); ok {
//
// The context must originate from loadOrStoreIdent to enable cancellation.
func (c *Cache) openFile(
ctx context.Context,
f FileArtifact,
) (r io.ReadCloser, err error) {
if kc, ok := f.(KnownChecksum); c.flags&CAssumeChecksum != 0 && ok {
c.checksumMu.RLock()
r, err = os.Open(c.base.Append(
dirChecksum,
@@ -1088,7 +1179,7 @@ func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) {
}
}()
}
return f.Cure(&RContext{common{c}})
return f.Cure(&RContext{common{ctx, c}})
}
return
}
@@ -1229,14 +1320,6 @@ func (e InvalidArtifactError) Error() string {
return "artifact " + Encode(e) + " cannot be cured"
}
// DependencyError refers to an artifact with a dependency tree larger than the
// threshold specified by a previous call to [Cache.SetThreshold].
type DependencyError struct{ A Artifact }
func (e DependencyError) Error() string {
return "artifact has too many dependencies"
}
// Cure cures the [Artifact] and returns its pathname and [Checksum]. Direct
// calls to Cure are not subject to the cures limit.
func (c *Cache) Cure(a Artifact) (
@@ -1244,24 +1327,11 @@ func (c *Cache) Cure(a Artifact) (
checksum unique.Handle[Checksum],
err error,
) {
select {
case <-c.ctx.Done():
err = c.ctx.Err()
c.abortMu.RLock()
defer c.abortMu.RUnlock()
if err = c.toplevel.Load().ctx.Err(); err != nil {
return
default:
}
if c.threshold > 0 {
var n uintptr
for range Flood(a) {
if n == c.threshold {
err = DependencyError{a}
return
}
n++
}
c.msg.Verbosef("visited %d artifacts", n)
}
return c.cure(a, true)
@@ -1347,15 +1417,16 @@ func (c *Cache) enterCure(a Artifact, curesExempt bool) error {
return nil
}
ctx := c.toplevel.Load().ctx
select {
case c.cures <- struct{}{}:
return nil
case <-c.ctx.Done():
case <-ctx.Done():
if a.IsExclusive() {
c.exclMu.Unlock()
}
return c.ctx.Err()
return ctx.Err()
}
}
@@ -1453,7 +1524,8 @@ func (r *RContext) NewMeasuredReader(
return r.cache.newMeasuredReader(rc, checksum)
}
// cure implements Cure without checking the full dependency graph.
// cure implements Cure without acquiring a read lock on abortMu. cure must not
// be entered during Abort.
func (c *Cache) cure(a Artifact, curesExempt bool) (
pathname *check.Absolute,
checksum unique.Handle[Checksum],
@@ -1472,8 +1544,11 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
}
}()
var done chan<- struct{}
done, checksum, err = c.loadOrStoreIdent(id)
var (
ctx context.Context
done chan<- struct{}
)
ctx, done, checksum, err = c.loadOrStoreIdent(id)
if done == nil {
return
} else {
@@ -1521,16 +1596,18 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
checksums,
)
c.checksumMu.RLock()
checksumFi, err = os.Stat(checksumPathname.String())
c.checksumMu.RUnlock()
if c.flags&CAssumeChecksum != 0 {
c.checksumMu.RLock()
checksumFi, err = os.Stat(checksumPathname.String())
c.checksumMu.RUnlock()
if err != nil {
if !errors.Is(err, os.ErrNotExist) {
return
if err != nil {
if !errors.Is(err, os.ErrNotExist) {
return
}
checksumFi, err = nil, nil
}
checksumFi, err = nil, nil
}
}
@@ -1584,9 +1661,9 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
if err = c.enterCure(a, curesExempt); err != nil {
return
}
r, err = f.Cure(&RContext{common{c}})
r, err = f.Cure(&RContext{common{ctx, c}})
if err == nil {
if checksumPathname == nil || c.IsStrict() {
if checksumPathname == nil || c.flags&CValidateKnown != 0 {
h := sha512.New384()
hbw := c.getWriter(h)
_, err = io.Copy(w, io.TeeReader(r, hbw))
@@ -1603,7 +1680,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
if checksumPathname == nil {
checksum = unique.Make(Checksum(buf[:]))
checksums = Encode(Checksum(buf[:]))
} else if c.IsStrict() {
} else if c.flags&CValidateKnown != 0 {
if got := Checksum(buf[:]); got != checksum.Value() {
err = &ChecksumMismatchError{
Got: got,
@@ -1664,7 +1741,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
c.base.Append(dirWork, ids),
c.base.Append(dirTemp, ids),
ids, nil, nil, nil,
common{c},
common{ctx, c},
}
switch ca := a.(type) {
case TrivialArtifact:
@@ -1815,14 +1892,42 @@ func (c *Cache) OpenStatus(a Artifact) (r io.ReadSeekCloser, err error) {
return
}
// Abort cancels all pending cures and waits for them to clean up, but does not
// close the cache.
func (c *Cache) Abort() {
c.closeMu.Lock()
defer c.closeMu.Unlock()
if c.closed {
return
}
c.toplevel.Load().cancel()
c.abortMu.Lock()
defer c.abortMu.Unlock()
// holding abortMu, identPending stays empty
c.wg.Wait()
c.identMu.Lock()
c.toplevel.Store(newToplevel(c.parent))
clear(c.identErr)
c.identMu.Unlock()
}
// Close cancels all pending cures and waits for them to clean up.
func (c *Cache) Close() {
c.closeOnce.Do(func() {
c.cancel()
c.wg.Wait()
close(c.cures)
c.unlock()
})
c.closeMu.Lock()
defer c.closeMu.Unlock()
if c.closed {
return
}
c.closed = true
c.toplevel.Load().cancel()
c.wg.Wait()
close(c.cures)
c.unlock()
}
// Open returns the address of a newly opened instance of [Cache].
@@ -1831,7 +1936,7 @@ func (c *Cache) Close() {
// caller-supplied value, however direct calls to [Cache.Cure] is not subject
// to this limitation.
//
// A cures value of 0 or lower is equivalent to the value returned by
// A cures or jobs value of 0 or lower is equivalent to the value returned by
// [runtime.NumCPU].
//
// A successful call to Open guarantees exclusive access to the on-filesystem
@@ -1841,10 +1946,10 @@ func (c *Cache) Close() {
func Open(
ctx context.Context,
msg message.Msg,
cures int,
flags, cures, jobs int,
base *check.Absolute,
) (*Cache, error) {
return open(ctx, msg, cures, base, true)
return open(ctx, msg, flags, cures, jobs, base, true)
}
// open implements Open but allows omitting the [lockedfile] lock when called
@@ -1852,13 +1957,16 @@ func Open(
func open(
ctx context.Context,
msg message.Msg,
cures int,
flags, cures, jobs int,
base *check.Absolute,
lock bool,
) (*Cache, error) {
if cures < 1 {
cures = runtime.NumCPU()
}
if jobs < 1 {
jobs = runtime.NumCPU()
}
for _, name := range []string{
dirIdentifier,
@@ -1873,21 +1981,25 @@ func open(
}
c := Cache{
parent: ctx,
cures: make(chan struct{}, cures),
flags: flags,
jobs: jobs,
msg: msg,
base: base,
identPool: sync.Pool{New: func() any { return new(extIdent) }},
irCache: zeroIRCache(),
ident: make(map[unique.Handle[ID]]unique.Handle[Checksum]),
identErr: make(map[unique.Handle[ID]]error),
identPending: make(map[unique.Handle[ID]]<-chan struct{}),
identPending: make(map[unique.Handle[ID]]*pendingCure),
brPool: sync.Pool{New: func() any { return new(bufio.Reader) }},
bwPool: sync.Pool{New: func() any { return new(bufio.Writer) }},
}
c.ctx, c.cancel = context.WithCancel(ctx)
c.toplevel.Store(newToplevel(ctx))
if lock || !testing.Testing() {
if unlock, err := lockedfile.MutexAt(

View File

@@ -16,6 +16,7 @@ import (
"path/filepath"
"reflect"
"strconv"
"sync"
"syscall"
"testing"
"unique"
@@ -24,6 +25,8 @@ import (
"hakurei.app/check"
"hakurei.app/container"
"hakurei.app/fhs"
"hakurei.app/internal/info"
"hakurei.app/internal/landlock"
"hakurei.app/internal/pkg"
"hakurei.app/internal/stub"
"hakurei.app/message"
@@ -33,11 +36,28 @@ import (
func unsafeOpen(
ctx context.Context,
msg message.Msg,
cures int,
flags, cures, jobs int,
base *check.Absolute,
lock bool,
) (*pkg.Cache, error)
// newRContext returns the address of a new [pkg.RContext] unsafely created for
// the specified [testing.TB].
func newRContext(tb testing.TB, c *pkg.Cache) *pkg.RContext {
var r pkg.RContext
rContextVal := reflect.ValueOf(&r).Elem().FieldByName("ctx")
reflect.NewAt(
rContextVal.Type(),
unsafe.Pointer(rContextVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(tb.Context()))
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
return &r
}
func TestMain(m *testing.M) { container.TryArgv0(nil); os.Exit(m.Run()) }
// overrideIdent overrides the ID method of [Artifact].
@@ -228,7 +248,7 @@ func TestIdent(t *testing.T) {
var cache *pkg.Cache
if a, err := check.NewAbs(t.TempDir()); err != nil {
t.Fatal(err)
} else if cache, err = pkg.Open(t.Context(), msg, 0, a); err != nil {
} else if cache, err = pkg.Open(t.Context(), msg, 0, 0, 0, a); err != nil {
t.Fatal(err)
}
t.Cleanup(cache.Close)
@@ -252,6 +272,7 @@ func TestIdent(t *testing.T) {
// on test completion.
type cacheTestCase struct {
name string
flags int
early func(t *testing.T, base *check.Absolute)
f func(t *testing.T, base *check.Absolute, c *pkg.Cache)
want pkg.Checksum
@@ -288,8 +309,20 @@ func checkWithCache(t *testing.T, testCases []cacheTestCase) {
msg := message.New(log.New(os.Stderr, "cache: ", 0))
msg.SwapVerbose(testing.Verbose())
flags := tc.flags
if info.CanDegrade {
if _, err := landlock.GetABI(); err != nil {
if !errors.Is(err, syscall.ENOSYS) {
t.Fatalf("LandlockGetABI: error = %v", err)
}
flags |= pkg.CHostAbstract
t.Log("Landlock LSM is unavailable, setting CHostAbstract")
}
}
var scrubFunc func() error // scrub after hashing
if c, err := pkg.Open(t.Context(), msg, 1<<4, base); err != nil {
if c, err := pkg.Open(t.Context(), msg, flags, 1<<4, 0, base); err != nil {
t.Fatalf("Open: error = %v", err)
} else {
t.Cleanup(c.Close)
@@ -468,9 +501,7 @@ func TestCache(t *testing.T) {
}()
testCases := []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"file", pkg.CValidateKnown | pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
identifier := (pkg.ID)(bytes.Repeat([]byte{
0x75, 0xe6, 0x9d, 0x6d, 0xe7, 0x9f,
}, 8))
@@ -593,7 +624,7 @@ func TestCache(t *testing.T) {
if c0, err := unsafeOpen(
t.Context(),
message.New(nil),
0, base, false,
0, 0, 0, base, false,
); err != nil {
t.Fatalf("open: error = %v", err)
} else {
@@ -627,7 +658,7 @@ func TestCache(t *testing.T) {
}
}, pkg.MustDecode("St9rlE-mGZ5gXwiv_hzQ_B8bZP-UUvSNmf4nHUZzCMOumb6hKnheZSe0dmnuc4Q2")},
{"directory", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
{"directory", pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
id := pkg.MustDecode(
"HnySzeLQvSBZuTUcvfmLEX_OmH4yJWWH788NxuLuv7kVn8_uPM6Ks4rqFWM2NZJY",
)
@@ -804,9 +835,7 @@ func TestCache(t *testing.T) {
})
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d")},
{"pending", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
{"pending", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
wantErr := stub.UniqueError(0xcafe)
n, ready := make(chan struct{}), make(chan struct{})
go func() {
@@ -865,18 +894,117 @@ func TestCache(t *testing.T) {
t.Fatalf("Scrub: error = %#v, want %#v", err, wantErrScrub)
}
identPendingVal := reflect.ValueOf(c).Elem().FieldByName("identPending")
identPending := reflect.NewAt(
identPendingVal.Type(),
unsafe.Pointer(identPendingVal.UnsafeAddr()),
).Elem().Interface().(map[unique.Handle[pkg.ID]]<-chan struct{})
notify := identPending[unique.Make(pkg.ID{0xff})]
notify := c.Done(unique.Make(pkg.ID{0xff}))
go close(n)
<-notify
if notify != nil {
<-notify
}
for c.Done(unique.Make(pkg.ID{0xff})) != nil {
}
<-wCureDone
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"scrub", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
{"cancel abort block", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var wg sync.WaitGroup
defer wg.Wait()
var started sync.WaitGroup
defer started.Wait()
blockCures := func(d byte, e stub.UniqueError, n int) {
started.Add(n)
for i := range n {
wg.Go(func() {
if _, _, err := c.Cure(overrideIdent{pkg.ID{d, byte(i)}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
started.Done()
<-t.Unwrap().Done()
return e + stub.UniqueError(i)
},
}}); !reflect.DeepEqual(err, e+stub.UniqueError(i)) {
panic(err)
}
})
}
started.Wait()
}
blockCures(0xfd, 0xbad, 16)
c.Abort()
wg.Wait()
blockCures(0xfd, 0xcafe, 16)
c.Abort()
wg.Wait()
blockCures(0xff, 0xbad, 1)
if !c.Cancel(unique.Make(pkg.ID{0xff})) {
t.Fatal("missed cancellation")
}
wg.Wait()
blockCures(0xff, 0xcafe, 1)
if !c.Cancel(unique.Make(pkg.ID{0xff})) {
t.Fatal("missed cancellation")
}
wg.Wait()
for c.Cancel(unique.Make(pkg.ID{0xff})) {
}
c.Close()
c.Abort()
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"no assume checksum", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
makeGarbage := func(work *check.Absolute, wantErr error) error {
if err := os.Mkdir(work.String(), 0700); err != nil {
return err
}
if err := os.WriteFile(work.Append(
"check",
).String(), nil, 0400); err != nil {
return err
}
return wantErr
}
wantChecksum := pkg.MustDecode("Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")
cureMany(t, c, []cureStep{
{"create", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 0}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 0}),
), wantChecksum, nil},
{"reject", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xfe, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), stub.UniqueError(0xbad))
},
}}}, nil, pkg.Checksum{}, stub.UniqueError(0xbad)},
{"match", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 1}),
), wantChecksum, nil},
})
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w")},
{"scrub", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
cureMany(t, c, []cureStep{
{"bad measured file", newStubFile(
pkg.KindHTTPGet,
@@ -1182,7 +1310,7 @@ func (a earlyFailureF) Cure(*pkg.FContext) error {
func TestDependencyCureErrorEarly(t *testing.T) {
checkWithCache(t, []cacheTestCase{
{"early", nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) {
{"early", 0, nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) {
_, _, err := c.Cure(earlyFailureF(8))
if !errors.Is(err, stub.UniqueError(0xcafe)) {
t.Fatalf("Cure: error = %v", err)
@@ -1205,7 +1333,7 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open(
t.Context(),
message.New(nil),
0, check.MustAbs(container.Nonexistent),
0, 0, 0, check.MustAbs(container.Nonexistent),
); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr)
}
@@ -1233,7 +1361,7 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open(
t.Context(),
message.New(nil),
0, tempDir.Append("cache"),
0, 0, 0, tempDir.Append("cache"),
); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr)
}

View File

@@ -43,8 +43,7 @@ var _ fmt.Stringer = new(tarArtifactNamed)
func (a *tarArtifactNamed) String() string { return a.name + "-unpack" }
// NewTar returns a new [Artifact] backed by the supplied [Artifact] and
// compression method. The source [Artifact] must be compatible with
// [TContext.Open].
// compression method. The source [Artifact] must be a [FileArtifact].
func NewTar(a Artifact, compression uint32) Artifact {
ta := tarArtifact{a, compression}
if s, ok := a.(fmt.Stringer); ok {

View File

@@ -21,7 +21,7 @@ func TestTar(t *testing.T) {
t.Parallel()
checkWithCache(t, []cacheTestCase{
{"http", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
{"http", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},
@@ -42,7 +42,7 @@ func TestTar(t *testing.T) {
))
}, pkg.MustDecode("NQTlc466JmSVLIyWklm_u8_g95jEEb98PxJU-kjwxLpfdjwMWJq0G8ze9R4Vo1Vu")},
{"http expand", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
{"http expand", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},

View File

@@ -9,7 +9,9 @@ import (
"os"
"path/filepath"
"reflect"
"runtime"
"slices"
"strconv"
"strings"
"hakurei.app/check"
@@ -21,6 +23,10 @@ func main() {
log.SetFlags(0)
log.SetPrefix("testtool: ")
environ := slices.DeleteFunc(slices.Clone(os.Environ()), func(s string) bool {
return s == "CURE_JOBS="+strconv.Itoa(runtime.NumCPU())
})
var hostNet, layers, promote bool
if len(os.Args) == 2 && os.Args[0] == "testtool" {
switch os.Args[1] {
@@ -48,15 +54,15 @@ func main() {
var overlayRoot bool
wantEnv := []string{"HAKUREI_TEST=1"}
if len(os.Environ()) == 2 {
if len(environ) == 2 {
overlayRoot = true
if !layers && !promote {
log.SetPrefix("testtool(overlay root): ")
}
wantEnv = []string{"HAKUREI_TEST=1", "HAKUREI_ROOT=1"}
}
if !slices.Equal(wantEnv, os.Environ()) {
log.Fatalf("Environ: %q, want %q", os.Environ(), wantEnv)
if !slices.Equal(wantEnv, environ) {
log.Fatalf("Environ: %q, want %q", environ, wantEnv)
}
var overlayWork bool

View File

@@ -7,10 +7,10 @@ func (t Toolchain) newAttr() (pkg.Artifact, string) {
version = "2.5.2"
checksum = "YWEphrz6vg1sUMmHHVr1CRo53pFXRhq_pjN-AlG8UgwZK1y6m7zuDhxqJhD0SV0l"
)
return t.NewPackage("attr", version, pkg.NewHTTPGetTar(
nil, "https://download.savannah.nongnu.org/releases/attr/"+
return t.NewPackage("attr", version, newTar(
"https://download.savannah.nongnu.org/releases/attr/"+
"attr-"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), &PackageAttr{
Patches: []KV{
@@ -81,10 +81,10 @@ func (t Toolchain) newACL() (pkg.Artifact, string) {
version = "2.3.2"
checksum = "-fY5nwH4K8ZHBCRXrzLdguPkqjKI6WIiGu4dBtrZ1o0t6AIU73w8wwJz_UyjIS0P"
)
return t.NewPackage("acl", version, pkg.NewHTTPGetTar(
nil, "https://download.savannah.nongnu.org/releases/acl/"+
return t.NewPackage("acl", version, newTar(
"https://download.savannah.nongnu.org/releases/acl/"+
"acl-"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
// makes assumptions about uid_map/gid_map

View File

@@ -16,9 +16,9 @@ import (
type PArtifact int
const (
LLVMCompilerRT PArtifact = iota
CompilerRT PArtifact = iota
LLVMRuntimes
LLVMClang
Clang
// EarlyInit is the Rosa OS init program.
EarlyInit
@@ -47,6 +47,7 @@ const (
Bison
Bzip2
CMake
Connman
Coreutils
Curl
DBus
@@ -63,6 +64,8 @@ const (
GenInitCPIO
Gettext
Git
Glslang
GnuTLS
Go
Gperf
Grep
@@ -74,17 +77,24 @@ const (
LibXau
Libbsd
Libcap
Libclc
Libdrm
Libev
Libexpat
Libiconv
Libpsl
Libffi
Libgd
Libglvnd
Libiconv
Libmd
Libmnl
Libpciaccess
Libnftnl
Libtool
Libpsl
Libseccomp
Libtasn1
Libtool
Libucontext
Libunistring
Libxml2
Libxslt
M4
@@ -101,6 +111,7 @@ const (
Nettle
Ninja
OpenSSL
P11Kit
PCRE2
Parallel
Patch
@@ -113,21 +124,33 @@ const (
PerlTermReadKey
PerlTextCharWidth
PerlTextWrapI18N
PerlUnicodeGCString
PerlUnicodeLineBreak
PerlYAMLTiny
PkgConfig
Procps
Python
PythonFlitCore
PythonHatchling
PythonIniConfig
PythonMako
PythonMarkupSafe
PythonPackaging
PythonPathspec
PythonPluggy
PythonPyTest
PythonPyYAML
PythonPygments
PythonSetuptools
PythonSetuptoolsSCM
PythonTroveClassifiers
PythonVCSVersioning
QEMU
Rdfind
Readline
Rsync
Sed
Setuptools
SPIRVHeaders
SPIRVTools
SquashfsTools
Strace
TamaGo
@@ -141,15 +164,17 @@ const (
WaylandProtocols
XCB
XCBProto
Xproto
XDGDBusProxy
XZ
Xproto
Zlib
Zstd
// PresetUnexportedStart is the first unexported preset.
PresetUnexportedStart
buildcatrust = iota - 1
llvmSource = iota - 1
buildcatrust
utilMacros
// Musl is a standalone libc that does not depend on the toolchain.
@@ -159,6 +184,9 @@ const (
// stages only. This preset and its direct output must never be exposed.
gcc
// nettle3 is an older version of [Nettle].
nettle3
// Stage0 is a tarball containing all compile-time dependencies of artifacts
// part of the [Std] toolchain.
Stage0
@@ -297,6 +325,17 @@ var (
artifactsOnce [_toolchainEnd][len(artifactsM)]sync.Once
)
// zero zeros the value pointed to by p.
func zero[T any](p *T) { var v T; *p = v }
// DropCaches arranges for all cached [pkg.Artifact] to be freed some time after
// it returns. Must not be used concurrently with any other function from this
// package.
func DropCaches() {
zero(&artifacts)
zero(&artifactsOnce)
}
// GetMetadata returns [Metadata] of a [PArtifact].
func GetMetadata(p PArtifact) *Metadata { return &artifactsM[p] }

View File

@@ -19,6 +19,18 @@ func TestLoad(t *testing.T) {
}
}
func BenchmarkAll(b *testing.B) {
for b.Loop() {
for i := range rosa.PresetEnd {
rosa.Std.Load(rosa.PArtifact(i))
}
b.StopTimer()
rosa.DropCaches()
b.StartTimer()
}
}
func TestResolveName(t *testing.T) {
t.Parallel()

View File

@@ -7,10 +7,10 @@ func (t Toolchain) newArgpStandalone() (pkg.Artifact, string) {
version = "1.3"
checksum = "vtW0VyO2pJ-hPyYmDI2zwSLS8QL0sPAUKC1t3zNYbwN2TmsaE-fADhaVtNd3eNFl"
)
return t.NewPackage("argp-standalone", version, pkg.NewHTTPGetTar(
nil, "http://www.lysator.liu.se/~nisse/misc/"+
return t.NewPackage("argp-standalone", version, newTar(
"http://www.lysator.liu.se/~nisse/misc/"+
"argp-standalone-"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), &PackageAttr{
Env: []string{

View File

@@ -7,9 +7,9 @@ func (t Toolchain) newBzip2() (pkg.Artifact, string) {
version = "1.0.8"
checksum = "cTLykcco7boom-s05H1JVsQi1AtChYL84nXkg_92Dm1Xt94Ob_qlMg_-NSguIK-c"
)
return t.NewPackage("bzip2", version, pkg.NewHTTPGetTar(
nil, "https://sourceware.org/pub/bzip2/bzip2-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("bzip2", version, newTar(
"https://sourceware.org/pub/bzip2/bzip2-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,

View File

@@ -10,13 +10,14 @@ import (
func (t Toolchain) newCMake() (pkg.Artifact, string) {
const (
version = "4.3.0"
checksum = "amBtnY2eGsEdlrB-cTRuOESBTsIqtyaxWlEKNlnp2EWLwAKWINjssilo4KXE6El9"
version = "4.3.1"
checksum = "RHpzZiM1kJ5bwLjo9CpXSeHJJg3hTtV9QxBYpQoYwKFtRh5YhGWpShrqZCSOzQN6"
)
return t.NewPackage("cmake", version, pkg.NewHTTPGetTar(
nil, "https://github.com/Kitware/CMake/releases/download/"+
"v"+version+"/cmake-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("cmake", version, newFromGitHubRelease(
"Kitware/CMake",
"v"+version,
"cmake-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
// test suite expects writable source tree
@@ -90,7 +91,7 @@ index 2ead810437..f85cbb8b1c 100644
ConfigureName: "/usr/src/cmake/bootstrap",
Configure: []KV{
{"prefix", "/system"},
{"parallel", `"$(nproc)"`},
{"parallel", jobsE},
{"--"},
{"-DCMAKE_USE_OPENSSL", "OFF"},
{"-DCMake_TEST_NO_NETWORK", "ON"},
@@ -118,9 +119,6 @@ func init() {
// CMakeHelper is the [CMake] build system helper.
type CMakeHelper struct {
// Joined with name with a dash if non-empty.
Variant string
// Path elements joined with source.
Append []string
@@ -135,14 +133,6 @@ type CMakeHelper struct {
var _ Helper = new(CMakeHelper)
// name returns its arguments and an optional variant string joined with '-'.
func (attr *CMakeHelper) name(name, version string) string {
if attr != nil && attr.Variant != "" {
name += "-" + attr.Variant
}
return name + "-" + version
}
// extra returns a hardcoded slice of [CMake] and [Ninja].
func (attr *CMakeHelper) extra(int) P {
if attr != nil && attr.Make {
@@ -180,10 +170,8 @@ func (attr *CMakeHelper) script(name string) string {
}
generate := "Ninja"
jobs := ""
if attr.Make {
generate = "'Unix Makefiles'"
jobs += ` "--parallel=$(nproc)"`
}
return `
@@ -201,7 +189,7 @@ cmake -G ` + generate + ` \
}), " \\\n\t") + ` \
-DCMAKE_INSTALL_PREFIX=/system \
'/usr/src/` + name + `/` + filepath.Join(attr.Append...) + `'
cmake --build .` + jobs + `
cmake --build . --parallel=` + jobsE + `
cmake --install . --prefix=/work/system
` + attr.Script
}

109
internal/rosa/connman.go Normal file
View File

@@ -0,0 +1,109 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newConnman() (pkg.Artifact, string) {
const (
version = "2.0"
checksum = "MhVTdJOhndnZn2SWd8URKo_Pj7Zvc14tntEbrVOf9L3yVWJvpb3v3Q6104tWJgtW"
)
return t.NewPackage("connman", version, newTar(
"https://git.kernel.org/pub/scm/network/connman/connman.git/"+
"snapshot/connman-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Patches: []KV{
{"alpine-musl-res", `musl does not implement res_ninit
--- a/gweb/gresolv.c
+++ b/gweb/gresolv.c
@@ -877,8 +877,6 @@
resolv->index = index;
resolv->nameserver_list = NULL;
- res_ninit(&resolv->res);
-
return resolv;
}
@@ -918,8 +916,6 @@
flush_nameservers(resolv);
- res_nclose(&resolv->res);
-
g_free(resolv);
}
@@ -1022,24 +1018,19 @@
debug(resolv, "hostname %s", hostname);
if (!resolv->nameserver_list) {
- int i;
-
- for (i = 0; i < resolv->res.nscount; i++) {
- char buf[100];
- int family = resolv->res.nsaddr_list[i].sin_family;
- void *sa_addr = &resolv->res.nsaddr_list[i].sin_addr;
-
- if (family != AF_INET &&
- resolv->res._u._ext.nsaddrs[i]) {
- family = AF_INET6;
- sa_addr = &resolv->res._u._ext.nsaddrs[i]->sin6_addr;
+ FILE *f = fopen("/etc/resolv.conf", "r");
+ if (f) {
+ char line[256], *s;
+ int i;
+ while (fgets(line, sizeof(line), f)) {
+ if (strncmp(line, "nameserver", 10) || !isspace(line[10]))
+ continue;
+ for (s = &line[11]; isspace(s[0]); s++);
+ for (i = 0; s[i] && !isspace(s[i]); i++);
+ s[i] = 0;
+ g_resolv_add_nameserver(resolv, s, 53, 0);
}
-
- if (family != AF_INET && family != AF_INET6)
- continue;
-
- if (inet_ntop(family, sa_addr, buf, sizeof(buf)))
- g_resolv_add_nameserver(resolv, buf, 53, 0);
+ fclose(f);
}
if (!resolv->nameserver_list)
`},
},
}, &MakeHelper{
Generate: "./bootstrap",
},
Automake,
Libtool,
PkgConfig,
DBus,
IPTables,
GnuTLS,
Readline,
KernelHeaders,
), version
}
func init() {
artifactsM[Connman] = Metadata{
f: Toolchain.newConnman,
Name: "connman",
Description: "a daemon for managing Internet connections",
Website: "https://git.kernel.org/pub/scm/network/connman/connman.git/",
Dependencies: P{
DBus,
IPTables,
GnuTLS,
Readline,
},
ID: 337,
}
}

View File

@@ -7,15 +7,15 @@ func (t Toolchain) newCurl() (pkg.Artifact, string) {
version = "8.19.0"
checksum = "YHuVLVVp8q_Y7-JWpID5ReNjq2Zk6t7ArHB6ngQXilp_R5l3cubdxu3UKo-xDByv"
)
return t.NewPackage("curl", version, pkg.NewHTTPGetTar(
nil, "https://curl.se/download/curl-"+version+".tar.bz2",
mustDecode(checksum),
return t.NewPackage("curl", version, newTar(
"https://curl.se/download/curl-"+version+".tar.bz2",
checksum,
pkg.TarBzip2,
), &PackageAttr{
// remove broken test
Writable: true,
ScriptEarly: `
chmod +w tests/data && rm tests/data/test459
chmod +w tests/data && rm -f tests/data/test459
`,
}, &MakeHelper{
Configure: []KV{
@@ -25,7 +25,7 @@ chmod +w tests/data && rm tests/data/test459
{"disable-smb"},
},
Check: []string{
`TFLAGS="-j$(expr "$(nproc)" '*' 2)"`,
"TFLAGS=" + jobsLFlagE,
"test-nonflaky",
},
},

View File

@@ -7,11 +7,11 @@ func (t Toolchain) newDBus() (pkg.Artifact, string) {
version = "1.16.2"
checksum = "INwOuNdrDG7XW5ilW_vn8JSxEa444rRNc5ho97i84I1CNF09OmcFcV-gzbF4uCyg"
)
return t.NewPackage("dbus", version, pkg.NewHTTPGetTar(
nil, "https://gitlab.freedesktop.org/dbus/dbus/-/archive/"+
"dbus-"+version+"/dbus-dbus-"+version+".tar.bz2",
mustDecode(checksum),
pkg.TarBzip2,
return t.NewPackage("dbus", version, newFromGitLab(
"gitlab.freedesktop.org",
"dbus/dbus",
"dbus-"+version,
checksum,
), &PackageAttr{
// OSError: [Errno 30] Read-only file system: '/usr/src/dbus/subprojects/packagecache'
Writable: true,
@@ -44,3 +44,38 @@ func init() {
ID: 5356,
}
}
func (t Toolchain) newXDGDBusProxy() (pkg.Artifact, string) {
const (
version = "0.1.7"
checksum = "UW5Pe-TP-XAaN-kTbxrkOQ7eYdmlAQlr2pdreLtPT0uwdAz-7rzDP8V_8PWuZBup"
)
return t.NewPackage("xdg-dbus-proxy", version, newFromGitHub(
"flatpak/xdg-dbus-proxy",
version,
checksum,
), nil, &MesonHelper{
Setup: []KV{
{"Dman", "disabled"},
},
},
DBus,
GLib,
), version
}
func init() {
artifactsM[XDGDBusProxy] = Metadata{
f: Toolchain.newXDGDBusProxy,
Name: "xdg-dbus-proxy",
Description: "a filtering proxy for D-Bus connections",
Website: "https://github.com/flatpak/xdg-dbus-proxy",
Dependencies: P{
GLib,
},
ID: 58434,
}
}

View File

@@ -7,10 +7,10 @@ func (t Toolchain) newDTC() (pkg.Artifact, string) {
version = "1.7.2"
checksum = "vUoiRynPyYRexTpS6USweT5p4SVHvvVJs8uqFkkVD-YnFjwf6v3elQ0-Etrh00Dt"
)
return t.NewPackage("dtc", version, pkg.NewHTTPGetTar(
nil, "https://git.kernel.org/pub/scm/utils/dtc/dtc.git/snapshot/"+
return t.NewPackage("dtc", version, newTar(
"https://git.kernel.org/pub/scm/utils/dtc/dtc.git/snapshot/"+
"dtc-v"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), &PackageAttr{
// works around buggy test:

View File

@@ -4,13 +4,13 @@ import "hakurei.app/internal/pkg"
func (t Toolchain) newElfutils() (pkg.Artifact, string) {
const (
version = "0.194"
checksum = "Q3XUygUPv9vR1TkWucwUsQ8Kb1_F6gzk-KMPELr3cC_4AcTrprhVPMvN0CKkiYRa"
version = "0.195"
checksum = "JrGnBD38w8Mj0ZxDw3fKlRBFcLvRKu8rcYnX35R9yTlUSYnzTazyLboG-a2CsJlu"
)
return t.NewPackage("elfutils", version, pkg.NewHTTPGetTar(
nil, "https://sourceware.org/elfutils/ftp/"+
return t.NewPackage("elfutils", version, newTar(
"https://sourceware.org/elfutils/ftp/"+
version+"/elfutils-"+version+".tar.bz2",
mustDecode(checksum),
checksum,
pkg.TarBzip2,
), &PackageAttr{
Env: []string{

View File

@@ -135,10 +135,11 @@ func newIANAEtc() pkg.Artifact {
version = "20251215"
checksum = "kvKz0gW_rGG5QaNK9ZWmWu1IEgYAdmhj_wR7DYrh3axDfIql_clGRHmelP7525NJ"
)
return pkg.NewHTTPGetTar(
nil, "https://github.com/Mic92/iana-etc/releases/download/"+
version+"/iana-etc-"+version+".tar.gz",
mustDecode(checksum),
return newFromGitHubRelease(
"Mic92/iana-etc",
version,
"iana-etc-"+version+".tar.gz",
checksum,
pkg.TarGzip,
)
}

View File

@@ -7,11 +7,11 @@ func (t Toolchain) newFakeroot() (pkg.Artifact, string) {
version = "1.37.2"
checksum = "4ve-eDqVspzQ6VWDhPS0NjW3aSenBJcPAJq_BFT7OOFgUdrQzoTBxZWipDAGWxF8"
)
return t.NewPackage("fakeroot", version, pkg.NewHTTPGetTar(
nil, "https://salsa.debian.org/clint/fakeroot/-/archive/upstream/"+
version+"/fakeroot-upstream-"+version+".tar.bz2",
mustDecode(checksum),
pkg.TarBzip2,
return t.NewPackage("fakeroot", version, newFromGitLab(
"salsa.debian.org",
"clint/fakeroot",
"upstream/"+version,
checksum,
), &PackageAttr{
Patches: []KV{
{"remove-broken-docs", `diff --git a/doc/Makefile.am b/doc/Makefile.am

View File

@@ -9,10 +9,11 @@ func (t Toolchain) newFlex() (pkg.Artifact, string) {
version = "2.6.4"
checksum = "p9POjQU7VhgOf3x5iFro8fjhy0NOanvA7CTeuWS_veSNgCixIJshTrWVkc5XLZkB"
)
return t.NewPackage("flex", version, pkg.NewHTTPGetTar(
nil, "https://github.com/westes/flex/releases/download/"+
"v"+version+"/flex-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("flex", version, newFromGitHubRelease(
"westes/flex",
"v"+version,
"flex-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil),
M4,

View File

@@ -7,10 +7,11 @@ func (t Toolchain) newFuse() (pkg.Artifact, string) {
version = "3.18.2"
checksum = "iL-7b7eUtmlVSf5cSq0dzow3UiqSjBmzV3cI_ENPs1tXcHdktkG45j1V12h-4jZe"
)
return t.NewPackage("fuse", version, pkg.NewHTTPGetTar(
nil, "https://github.com/libfuse/libfuse/releases/download/"+
"fuse-"+version+"/fuse-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("fuse", version, newFromGitHubRelease(
"libfuse/libfuse",
"fuse-"+version,
"fuse-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MesonHelper{
Setup: []KV{

View File

@@ -1,16 +1,21 @@
package rosa
import "hakurei.app/internal/pkg"
import (
"path"
"strings"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newGit() (pkg.Artifact, string) {
const (
version = "2.53.0"
checksum = "rlqSTeNgSeVKJA7nvzGqddFH8q3eFEPB4qRZft-4zth8wTHnbTbm7J90kp_obHGm"
)
return t.NewPackage("git", version, pkg.NewHTTPGetTar(
nil, "https://www.kernel.org/pub/software/scm/git/"+
return t.NewPackage("git", version, newTar(
"https://www.kernel.org/pub/software/scm/git/"+
"git-"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), &PackageAttr{
ScriptEarly: `
@@ -53,7 +58,7 @@ disable_test t2200-add-update
"prove",
},
Install: `make \
"-j$(nproc)" \
` + jobsFlagE + ` \
DESTDIR=/work \
NO_INSTALL_HARDLINKS=1 \
install`,
@@ -87,19 +92,30 @@ func init() {
// NewViaGit returns a [pkg.Artifact] for cloning a git repository.
func (t Toolchain) NewViaGit(
name, url, rev string,
url, rev string,
checksum pkg.Checksum,
) pkg.Artifact {
return t.New(name+"-"+rev, 0, t.AppendPresets(nil,
return t.New(strings.TrimSuffix(
path.Base(url),
".git",
)+"-src-"+path.Base(rev), 0, t.AppendPresets(nil,
NSSCACert,
Git,
), &checksum, nil, `
git \
-c advice.detachedHead=false \
clone \
--depth=1 \
--revision=`+rev+` \
--shallow-submodules \
--recurse-submodules \
`+url+` \
/work
rm -rf /work/.git
`, resolvconf())
}
// newTagRemote is a helper around NewViaGit for a tag on a git remote.
func (t Toolchain) newTagRemote(url, tag, checksum string) pkg.Artifact {
return t.NewViaGit(url, "refs/tags/"+tag, mustDecode(checksum))
}

130
internal/rosa/glslang.go Normal file
View File

@@ -0,0 +1,130 @@
package rosa
import (
"slices"
"strings"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newSPIRVHeaders() (pkg.Artifact, string) {
const (
version = "1.4.341.0"
checksum = "0PL43-19Iaw4k7_D8J8BvoJ-iLgCVSYZ2ThgDPGfAJwIJFtre7l0cnQtLjcY-JvD"
)
return t.NewPackage("spirv-headers", version, newFromGitHub(
"KhronosGroup/SPIRV-Headers",
"vulkan-sdk-"+version,
checksum,
), nil, &CMakeHelper{
Cache: []KV{
{"CMAKE_BUILD_TYPE", "Release"},
},
}), version
}
func init() {
artifactsM[SPIRVHeaders] = Metadata{
f: Toolchain.newSPIRVHeaders,
Name: "spirv-headers",
Description: "machine-readable files for the SPIR-V Registry",
Website: "https://github.com/KhronosGroup/SPIRV-Headers",
ID: 230542,
// upstream changed version scheme, anitya incapable of filtering them
latest: func(v *Versions) string {
for _, s := range v.Stable {
fields := strings.SplitN(s, ".", 4)
if len(fields) != 4 {
continue
}
if slices.ContainsFunc(fields, func(f string) bool {
return slices.ContainsFunc([]byte(f), func(d byte) bool {
return d < '0' || d > '9'
})
}) {
continue
}
return s
}
return v.Latest
},
}
}
func (t Toolchain) newSPIRVTools() (pkg.Artifact, string) {
const (
version = "2026.1"
checksum = "ZSQPQx8NltCDzQLk4qlaVxyWRWeI_JtsjEpeFt3kezTanl9DTHfLixSUCezMFBjv"
)
return t.NewPackage("spirv-tools", version, newFromGitHub(
"KhronosGroup/SPIRV-Tools",
"v"+version,
checksum,
), nil, &CMakeHelper{
Cache: []KV{
{"CMAKE_BUILD_TYPE", "Release"},
{"SPIRV-Headers_SOURCE_DIR", "/system"},
},
},
Python,
SPIRVHeaders,
), version
}
func init() {
artifactsM[SPIRVTools] = Metadata{
f: Toolchain.newSPIRVTools,
Name: "spirv-tools",
Description: "an API and commands for processing SPIR-V modules",
Website: "https://github.com/KhronosGroup/SPIRV-Tools",
Dependencies: P{
SPIRVHeaders,
},
ID: 14894,
}
}
func (t Toolchain) newGlslang() (pkg.Artifact, string) {
const (
version = "16.2.0"
checksum = "6_UuF9reLRDaVkgO-9IfB3kMwme3lQZM8LL8YsJwPdUFkrjzxJtf2A9X3w9nFxj2"
)
return t.NewPackage("glslang", version, newFromGitHub(
"KhronosGroup/glslang",
version,
checksum,
), &PackageAttr{
// test suite writes to source
Writable: true,
Chmod: true,
}, &CMakeHelper{
Cache: []KV{
{"CMAKE_BUILD_TYPE", "Release"},
{"BUILD_SHARED_LIBS", "ON"},
{"ALLOW_EXTERNAL_SPIRV_TOOLS", "ON"},
},
Script: "ctest",
},
Python,
Bash,
Diffutils,
SPIRVTools,
), version
}
func init() {
artifactsM[Glslang] = Metadata{
f: Toolchain.newGlslang,
Name: "glslang",
Description: "reference front end for GLSL/ESSL",
Website: "https://github.com/KhronosGroup/glslang",
ID: 205796,
}
}

View File

@@ -1,15 +1,19 @@
package rosa
import "hakurei.app/internal/pkg"
import (
"runtime"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newM4() (pkg.Artifact, string) {
const (
version = "1.4.21"
checksum = "pPa6YOo722Jw80l1OsH1tnUaklnPFjFT-bxGw5iAVrZTm1P8FQaWao_NXop46-pm"
)
return t.NewPackage("m4", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/m4/m4-"+version+".tar.bz2",
mustDecode(checksum),
return t.NewPackage("m4", version, newTar(
"https://ftpmirror.gnu.org/gnu/m4/m4-"+version+".tar.bz2",
checksum,
pkg.TarBzip2,
), &PackageAttr{
Writable: true,
@@ -39,9 +43,9 @@ func (t Toolchain) newBison() (pkg.Artifact, string) {
version = "3.8.2"
checksum = "BhRM6K7URj1LNOkIDCFDctSErLS-Xo5d9ba9seg10o6ACrgC1uNhED7CQPgIY29Y"
)
return t.NewPackage("bison", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/bison/bison-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("bison", version, newTar(
"https://ftpmirror.gnu.org/gnu/bison/bison-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil),
M4,
@@ -66,9 +70,9 @@ func (t Toolchain) newSed() (pkg.Artifact, string) {
version = "4.9"
checksum = "pe7HWH4PHNYrazOTlUoE1fXmhn2GOPFN_xE62i0llOr3kYGrH1g2_orDz0UtZ9Nt"
)
return t.NewPackage("sed", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/sed/sed-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("sed", version, newTar(
"https://ftpmirror.gnu.org/gnu/sed/sed-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil),
Diffutils,
@@ -91,15 +95,15 @@ func (t Toolchain) newAutoconf() (pkg.Artifact, string) {
version = "2.73"
checksum = "yGabDTeOfaCUB0JX-h3REYLYzMzvpDwFmFFzHNR7QilChCUNE4hR6q7nma4viDYg"
)
return t.NewPackage("autoconf", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/autoconf/autoconf-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("autoconf", version, newTar(
"https://ftpmirror.gnu.org/gnu/autoconf/autoconf-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Flag: TExclusive,
}, &MakeHelper{
Check: []string{
`TESTSUITEFLAGS="-j$(nproc)"`,
"TESTSUITEFLAGS=" + jobsFlagE,
"check",
},
},
@@ -131,9 +135,9 @@ func (t Toolchain) newAutomake() (pkg.Artifact, string) {
version = "1.18.1"
checksum = "FjvLG_GdQP7cThTZJLDMxYpRcKdpAVG-YDs1Fj1yaHlSdh_Kx6nRGN14E0r_BjcG"
)
return t.NewPackage("automake", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/automake/automake-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("automake", version, newTar(
"https://ftpmirror.gnu.org/gnu/automake/automake-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -175,13 +179,13 @@ func (t Toolchain) newLibtool() (pkg.Artifact, string) {
version = "2.5.4"
checksum = "pa6LSrQggh8mSJHQfwGjysAApmZlGJt8wif2cCLzqAAa2jpsTY0jZ-6stS3BWZ2Q"
)
return t.NewPackage("libtool", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/libtool/libtool-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("libtool", version, newTar(
"https://ftpmirror.gnu.org/gnu/libtool/libtool-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
Check: []string{
`TESTSUITEFLAGS="-j$(nproc)"`,
"TESTSUITEFLAGS=" + jobsFlagE,
"check",
},
},
@@ -206,9 +210,9 @@ func (t Toolchain) newGzip() (pkg.Artifact, string) {
version = "1.14"
checksum = "NWhjUavnNfTDFkZJyAUonL9aCOak8GVajWX2OMlzpFnuI0ErpBFyj88mz2xSjz0q"
)
return t.NewPackage("gzip", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/gzip/gzip-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("gzip", version, newTar(
"https://ftpmirror.gnu.org/gnu/gzip/gzip-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
// dependency loop
@@ -232,9 +236,9 @@ func (t Toolchain) newGettext() (pkg.Artifact, string) {
version = "1.0"
checksum = "3MasKeEdPeFEgWgzsBKk7JqWqql1wEMbgPmzAfs-mluyokoW0N8oQVxPQoOnSdgC"
)
return t.NewPackage("gettext", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/gettext/gettext-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("gettext", version, newTar(
"https://ftpmirror.gnu.org/gnu/gettext/gettext-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -278,9 +282,9 @@ func (t Toolchain) newDiffutils() (pkg.Artifact, string) {
version = "3.12"
checksum = "9J5VAq5oA7eqwzS1Yvw-l3G5o-TccUrNQR3PvyB_lgdryOFAfxtvQfKfhdpquE44"
)
return t.NewPackage("diffutils", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/diffutils/diffutils-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("diffutils", version, newTar(
"https://ftpmirror.gnu.org/gnu/diffutils/diffutils-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -311,9 +315,9 @@ func (t Toolchain) newPatch() (pkg.Artifact, string) {
version = "2.8"
checksum = "MA0BQc662i8QYBD-DdGgyyfTwaeALZ1K0yusV9rAmNiIsQdX-69YC4t9JEGXZkeR"
)
return t.NewPackage("patch", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/patch/patch-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("patch", version, newTar(
"https://ftpmirror.gnu.org/gnu/patch/patch-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -343,9 +347,9 @@ func (t Toolchain) newBash() (pkg.Artifact, string) {
version = "5.3"
checksum = "4LQ_GRoB_ko-Ih8QPf_xRKA02xAm_TOxQgcJLmFDT6udUPxTAWrsj-ZNeuTusyDq"
)
return t.NewPackage("bash", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/bash/bash-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("bash", version, newTar(
"https://ftpmirror.gnu.org/gnu/bash/bash-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Flag: TEarly,
@@ -373,9 +377,9 @@ func (t Toolchain) newCoreutils() (pkg.Artifact, string) {
version = "9.10"
checksum = "o-B9wssRnZySzJUI1ZJAgw-bZtj1RC67R9po2AcM2OjjS8FQIl16IRHpC6IwO30i"
)
return t.NewPackage("coreutils", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/coreutils/coreutils-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("coreutils", version, newTar(
"https://ftpmirror.gnu.org/gnu/coreutils/coreutils-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -512,9 +516,9 @@ func (t Toolchain) newTexinfo() (pkg.Artifact, string) {
version = "7.3"
checksum = "RRmC8Xwdof7JuZJeWGAQ_GeASIHAuJFQMbNONXBz5InooKIQGmqmWRjGNGEr5n4-"
)
return t.NewPackage("texinfo", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/texinfo/texinfo-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("texinfo", version, newTar(
"https://ftpmirror.gnu.org/gnu/texinfo/texinfo-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
// nonstandard glibc extension
@@ -545,9 +549,9 @@ func (t Toolchain) newGperf() (pkg.Artifact, string) {
version = "3.3"
checksum = "RtIy9pPb_Bb8-31J2Nw-rRGso2JlS-lDlVhuNYhqR7Nt4xM_nObznxAlBMnarJv7"
)
return t.NewPackage("gperf", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gperf/gperf-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("gperf", version, newTar(
"https://ftpmirror.gnu.org/gperf/gperf-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil),
Diffutils,
@@ -570,9 +574,9 @@ func (t Toolchain) newGawk() (pkg.Artifact, string) {
version = "5.4.0"
checksum = "m0RkIolC-PI7EY5q8pcx5Y-0twlIW0Yp3wXXmV-QaHorSdf8BhZ7kW9F8iWomz0C"
)
return t.NewPackage("gawk", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/gawk/gawk-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("gawk", version, newTar(
"https://ftpmirror.gnu.org/gnu/gawk/gawk-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Flag: TEarly,
@@ -598,9 +602,9 @@ func (t Toolchain) newGrep() (pkg.Artifact, string) {
version = "3.12"
checksum = "qMB4RjaPNRRYsxix6YOrjE8gyAT1zVSTy4nW4wKW9fqa0CHYAuWgPwDTirENzm_1"
)
return t.NewPackage("grep", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/grep/grep-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("grep", version, newTar(
"https://ftpmirror.gnu.org/gnu/grep/grep-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
@@ -635,7 +639,6 @@ func (t Toolchain) newFindutils() (pkg.Artifact, string) {
nil, "https://ftpmirror.gnu.org/gnu/findutils/findutils-"+version+".tar.xz",
mustDecode(checksum),
), &PackageAttr{
SourceKind: SourceKindTarXZ,
ScriptEarly: `
echo '#!/bin/sh' > gnulib-tests/test-c32ispunct.sh
echo 'int main(){return 0;}' > tests/xargs/test-sigusr.c
@@ -663,9 +666,9 @@ func (t Toolchain) newBC() (pkg.Artifact, string) {
version = "1.08.2"
checksum = "8h6f3hjV80XiFs6v9HOPF2KEyg1kuOgn5eeFdVspV05ODBVQss-ey5glc8AmneLy"
)
return t.NewPackage("bc", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/bc/bc-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("bc", version, newTar(
"https://ftpmirror.gnu.org/gnu/bc/bc-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
// source expected to be writable
@@ -692,9 +695,9 @@ func (t Toolchain) newLibiconv() (pkg.Artifact, string) {
version = "1.19"
checksum = "UibB6E23y4MksNqYmCCrA3zTFO6vJugD1DEDqqWYFZNuBsUWMVMcncb_5pPAr88x"
)
return t.NewPackage("libiconv", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/libiconv/libiconv-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("libiconv", version, newTar(
"https://ftpmirror.gnu.org/gnu/libiconv/libiconv-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil)), version
}
@@ -715,9 +718,9 @@ func (t Toolchain) newTar() (pkg.Artifact, string) {
version = "1.35"
checksum = "zSaoSlVUDW0dSfm4sbL4FrXLFR8U40Fh3zY5DWhR5NCIJ6GjU6Kc4VZo2-ZqpBRA"
)
return t.NewPackage("tar", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/tar/tar-"+version+".tar.gz",
mustDecode(checksum),
return t.NewPackage("tar", version, newTar(
"https://ftpmirror.gnu.org/gnu/tar/tar-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
Configure: []KV{
@@ -729,7 +732,7 @@ func (t Toolchain) newTar() (pkg.Artifact, string) {
// very expensive
"TARTEST_SKIP_LARGE_FILES=1",
`TESTSUITEFLAGS="-j$(nproc)"`,
"TESTSUITEFLAGS=" + jobsFlagE,
"check",
},
},
@@ -754,12 +757,12 @@ func init() {
func (t Toolchain) newParallel() (pkg.Artifact, string) {
const (
version = "20260222"
checksum = "4wxjMi3G2zMxr9hvLcIn6D7_12A3e5UNObeTPhzn7mDAYwsZApmmkxfGPyllQQ7E"
version = "20260322"
checksum = "gHoPmFkOO62ev4xW59HqyMlodhjp8LvTsBOwsVKHUUdfrt7KwB8koXmSVqQ4VOrB"
)
return t.NewPackage("parallel", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/parallel/parallel-"+version+".tar.bz2",
mustDecode(checksum),
return t.NewPackage("parallel", version, newTar(
"https://ftpmirror.gnu.org/gnu/parallel/parallel-"+version+".tar.bz2",
checksum,
pkg.TarBzip2,
), nil, (*MakeHelper)(nil),
Perl,
@@ -781,14 +784,285 @@ func init() {
}
}
func (t Toolchain) newLibunistring() (pkg.Artifact, string) {
const (
version = "1.4.2"
checksum = "iW9BbfLoVlXjWoLTZ4AekQSu4cFBnLcZ4W8OHWbv0AhJNgD3j65_zqaLMzFKylg2"
)
return t.NewPackage("libunistring", version, newTar(
"https://ftpmirror.gnu.org/gnu/libunistring/libunistring-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Writable: true,
ScriptEarly: `
test_disable() { chmod +w "$2" && echo "$1" > "$2"; }
test_disable '#!/bin/sh' tests/test-c32ispunct.sh
test_disable 'int main(){return 0;}' tests/test-c32ispunct.c
`,
}, (*MakeHelper)(nil),
Diffutils,
), version
}
func init() {
artifactsM[Libunistring] = Metadata{
f: Toolchain.newLibunistring,
Name: "libunistring",
Description: "provides functions for manipulating Unicode strings",
Website: "https://www.gnu.org/software/libunistring/",
ID: 1747,
}
}
func (t Toolchain) newLibtasn1() (pkg.Artifact, string) {
const (
version = "4.21.0"
checksum = "9DYI3UYbfYLy8JsKUcY6f0irskbfL0fHZA91Q-JEOA3kiUwpodyjemRsYRjUpjuq"
)
return t.NewPackage("libtasn1", version, newTar(
"https://ftpmirror.gnu.org/gnu/libtasn1/libtasn1-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, (*MakeHelper)(nil)), version
}
func init() {
artifactsM[Libtasn1] = Metadata{
f: Toolchain.newLibtasn1,
Name: "libtasn1",
Description: "the ASN.1 library used by GnuTLS, p11-kit and some other packages",
Website: "https://www.gnu.org/software/libtasn1/",
ID: 1734,
}
}
func (t Toolchain) newReadline() (pkg.Artifact, string) {
const (
version = "8.3"
checksum = "r-lcGRJq_MvvBpOq47Z2Y1OI2iqrmtcqhTLVXR0xWo37ZpC2uT_md7gKq5o_qTMV"
)
return t.NewPackage("readline", version, newTar(
"https://ftpmirror.gnu.org/gnu/readline/readline-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), nil, &MakeHelper{
Configure: []KV{
{"with-curses"},
{"with-shared-termcap-library"},
},
},
Ncurses,
), version
}
func init() {
artifactsM[Readline] = Metadata{
f: Toolchain.newReadline,
Name: "readline",
Description: "provides a set of functions for use by applications that allow users to edit command lines as they are typed in",
Website: "https://tiswww.cwru.edu/php/chet/readline/rltop.html",
Dependencies: P{
Ncurses,
},
ID: 4173,
}
}
func (t Toolchain) newGnuTLS() (pkg.Artifact, string) {
const (
version = "3.8.12"
checksum = "VPdP-nRydQQRJcnma-YA7CJYA_kzTJ2rb3QFeP6D27emSyInJ8sQ-Wzn518I38dl"
)
var configureExtra []KV
switch runtime.GOARCH {
case "arm64":
configureExtra = []KV{
{"disable-hardware-acceleration"},
}
}
return t.NewPackage("gnutls", version, t.newTagRemote(
"https://gitlab.com/gnutls/gnutls.git",
version, checksum,
), &PackageAttr{
Patches: []KV{
{"bootstrap-remove-gtk-doc", `diff --git a/bootstrap.conf b/bootstrap.conf
index 1c3cc61e6..32bae9387 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -50,7 +50,6 @@ bison 2.4
gettext 0.17
git 1.4.4
gperf -
-gtkdocize -
perl 5.5
wget -
"
diff --git a/configure.ac b/configure.ac
index 5057536e5..731558a15 100644
--- a/configure.ac
+++ b/configure.ac
@@ -403,11 +403,6 @@ if test "$enable_fuzzer_target" != "no";then
AC_DEFINE([FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION], 1, [Enable fuzzer target -not for production])
fi
-dnl
-dnl check for gtk-doc
-dnl
-GTK_DOC_CHECK([1.14],[--flavour no-tmpl])
-
AM_GNU_GETTEXT([external])
AM_GNU_GETTEXT_VERSION([0.19])
m4_ifdef([AM_GNU_GET][TEXT_REQUIRE_VERSION],[
diff --git a/doc/Makefile.am b/doc/Makefile.am
index fb1390d70..52f0ad9af 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -33,9 +33,6 @@ IMAGES = \
pkcs11-vision.png
SUBDIRS = examples scripts credentials latex
-if ENABLE_GTK_DOC
-SUBDIRS += reference
-endif
-include $(top_srcdir)/doc/doc.mk
diff --git a/doc/reference/Makefile.am b/doc/reference/Makefile.am
index f10c8ed3c..b711b58ec 100644
--- a/doc/reference/Makefile.am
+++ b/doc/reference/Makefile.am
@@ -82,13 +82,4 @@ include $(top_srcdir)/gtk-doc.make
# e.g. EXTRA_DIST += version.xml.in
EXTRA_DIST += version.xml.in
-# Comment this out if you want 'make check' to test you doc status
-# and run some sanity checks
-if ENABLE_GTK_DOC
-TESTS_ENVIRONMENT = \
- DOC_MODULE=$(DOC_MODULE) DOC_MAIN_SGML_FILE=$(DOC_MAIN_SGML_FILE) \
- SRCDIR=$(abs_srcdir) BUILDDIR=$(abs_builddir)
-#TESTS = $(GTKDOC_CHECK)
-endif
-
-include $(top_srcdir)/git.mk
`},
{"alpine-tests-certtool", `I think this tests is simply wrong.
When a PIN is given, the program should run in batch mode.
So the question for "Enter password" should _not_ be present.
DO NOT REMOVE UNLESS VERIFIED IT'S NOT ACTUALLY NECESSARY ANYMORE.
--- a/tests/cert-tests/certtool.sh 2019-02-07 07:33:45.960887338 +0000
+++ b/tests/cert-tests/certtool.sh 2019-02-07 07:36:14.550955051 +0000
@@ -49,7 +49,7 @@
#check whether password is being honoured
#some CI runners need GNUTLS_PIN (GNUTLS_PIN=${PASS})
- ${SETSID} "${CERTTOOL}" --generate-self-signed --load-privkey ${TMPFILE1} --template ${srcdir}/templates/template-test.tmpl --ask-pass >${TMPFILE2} 2>&1 <<EOF
+ GNUTLS_PIN=${PASS} ${SETSID} "${CERTTOOL}" --generate-self-signed --load-privkey ${TMPFILE1} --template ${srcdir}/templates/template-test.tmpl --ask-pass >${TMPFILE2} 2>&1 <<EOF
$PASS
EOF
if test $? != 0;then
@@ -59,7 +59,7 @@
fi
grep "Enter password" ${TMPFILE2} >/dev/null 2>&1
- if test $? != 0;then
+ if test $? != 1; then
cat ${TMPFILE2}
echo "No password was asked"
exit 1
`},
{"test-kernel-version-ksh", `diff --git a/tests/scripts/common.sh b/tests/scripts/common.sh
index 1b78b8cf1..350156a86 100644
--- a/tests/scripts/common.sh
+++ b/tests/scripts/common.sh
@@ -279,10 +279,6 @@ kernel_version_check() {
kernel_major=$(echo $kernel_version | cut -d. -f1 2>/dev/null)
kernel_minor=$(echo $kernel_version | cut -d. -f2 2>/dev/null)
- if ! [[ "$kernel_major" =~ ^[0-9]+$ ]] || ! [[ "$kernel_minor" =~ ^[0-9]+$ ]]; then
- return 1
- fi
-
if [ "$kernel_major" -lt "$required_major" ]; then
return 1
fi
`},
},
}, &MakeHelper{
Generate: "./bootstrap --skip-po --no-git --gnulib-srcdir=gnulib",
Configure: append([]KV{
{"disable-doc"},
{"disable-openssl-compatibility"},
{"with-default-trust-store-file", "/system/etc/ssl/certs/ca-bundle.crt"},
{"with-default-trust-store-pkcs11", "pkcs11:"},
{"with-zlib", "link"},
{"with-zstd", "link"},
}, configureExtra...),
},
Gzip,
Automake,
Libtool,
Bison,
Gettext,
Gperf,
PkgConfig,
Python,
Texinfo,
Diffutils,
NSSCACert,
Libev,
Zlib,
Zstd,
P11Kit,
nettle3,
Libunistring,
), version
}
func init() {
artifactsM[GnuTLS] = Metadata{
f: Toolchain.newGnuTLS,
Name: "gnutls",
Description: "a secure communications library implementing the SSL, TLS and DTLS protocols",
Website: "https://gnutls.org",
Dependencies: P{
Zlib,
Zstd,
P11Kit,
nettle3,
Libunistring,
},
ID: 1221,
}
}
func (t Toolchain) newBinutils() (pkg.Artifact, string) {
const (
version = "2.46.0"
checksum = "4kK1_EXQipxSqqyvwD4LbiMLFKCUApjq6PeG4XJP4dzxYGqDeqXfh8zLuTyOuOVR"
)
return t.NewPackage("binutils", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/binutils/binutils-"+version+".tar.bz2",
mustDecode(checksum),
return t.NewPackage("binutils", version, newTar(
"https://ftpmirror.gnu.org/gnu/binutils/binutils-"+version+".tar.bz2",
checksum,
pkg.TarBzip2,
), nil, (*MakeHelper)(nil),
Bash,
@@ -811,12 +1085,16 @@ func (t Toolchain) newGMP() (pkg.Artifact, string) {
version = "6.3.0"
checksum = "yrgbgEDWKDdMWVHh7gPbVl56-sRtVVhfvv0M_LX7xMUUk_mvZ1QOJEAnt7g4i3k5"
)
return t.NewPackage("gmp", version, pkg.NewHTTPGetTar(
nil, "https://gcc.gnu.org/pub/gcc/infrastructure/"+
return t.NewPackage("gmp", version, newTar(
"https://gcc.gnu.org/pub/gcc/infrastructure/"+
"gmp-"+version+".tar.bz2",
mustDecode(checksum),
checksum,
pkg.TarBzip2,
), nil, (*MakeHelper)(nil),
), &PackageAttr{
Env: []string{
"CC=cc",
},
}, (*MakeHelper)(nil),
M4,
), version
}
@@ -837,10 +1115,10 @@ func (t Toolchain) newMPFR() (pkg.Artifact, string) {
version = "4.2.2"
checksum = "wN3gx0zfIuCn9r3VAn_9bmfvAYILwrRfgBjYSD1IjLqyLrLojNN5vKyQuTE9kA-B"
)
return t.NewPackage("mpfr", version, pkg.NewHTTPGetTar(
nil, "https://gcc.gnu.org/pub/gcc/infrastructure/"+
return t.NewPackage("mpfr", version, newTar(
"https://gcc.gnu.org/pub/gcc/infrastructure/"+
"mpfr-"+version+".tar.bz2",
mustDecode(checksum),
checksum,
pkg.TarBzip2,
), nil, (*MakeHelper)(nil),
GMP,
@@ -864,16 +1142,22 @@ func init() {
func (t Toolchain) newMPC() (pkg.Artifact, string) {
const (
version = "1.4.0"
checksum = "75Sgr2hcDTltHYgFaHsRGsFgW74i2jqAUS0oXaBdJYKjMj_CvEeJ1zwGbNYjEl1H"
version = "1.4.1"
checksum = "wdXAhplnS89FjVp20m2nC2CmLFQeyQqLpQAfViTy4vPxFdv2WYOTtfBKeIk5_Rec"
)
return t.NewPackage("mpc", version, pkg.NewHTTPGet(
nil, "https://ftpmirror.gnu.org/gnu/mpc/mpc-"+version+".tar.xz",
mustDecode(checksum),
return t.NewPackage("mpc", version, t.newTagRemote(
"https://gitlab.inria.fr/mpc/mpc.git",
version, checksum,
), &PackageAttr{
SourceKind: SourceKindTarXZ,
}, (*MakeHelper)(nil),
XZ,
// does not find mpc-impl.h otherwise
EnterSource: true,
}, &MakeHelper{
InPlace: true,
Generate: "autoreconf -vfi",
},
Automake,
Libtool,
Texinfo,
MPFR,
), version
@@ -899,10 +1183,17 @@ func (t Toolchain) newGCC() (pkg.Artifact, string) {
version = "15.2.0"
checksum = "TXJ5WrbXlGLzy1swghQTr4qxgDCyIZFgJry51XEPTBZ8QYbVmFeB4lZbSMtPJ-a1"
)
return t.NewPackage("gcc", version, pkg.NewHTTPGetTar(
nil, "https://ftp.tsukuba.wide.ad.jp/software/gcc/releases/"+
var configureExtra []KV
switch runtime.GOARCH {
case "amd64", "arm64":
configureExtra = append(configureExtra, KV{"with-multilib-list", "''"})
}
return t.NewPackage("gcc", version, newTar(
"https://ftp.tsukuba.wide.ad.jp/software/gcc/releases/"+
"gcc-"+version+"/gcc-"+version+".tar.gz",
mustDecode(checksum),
checksum,
pkg.TarGzip,
), &PackageAttr{
Patches: []KV{
@@ -1064,9 +1355,8 @@ ln -s system/lib /work/
// it also saturates the CPU for a consequential amount of time.
Flag: TExclusive,
}, &MakeHelper{
Configure: []KV{
Configure: append([]KV{
{"disable-multilib"},
{"with-multilib-list", `""`},
{"enable-default-pie"},
{"disable-nls"},
{"with-gnu-as"},
@@ -1074,7 +1364,7 @@ ln -s system/lib /work/
{"with-system-zlib"},
{"enable-languages", "c,c++,go"},
{"with-native-system-header-dir", "/system/include"},
},
}, configureExtra...),
Make: []string{
"BOOT_CFLAGS='-O2 -g'",
"bootstrap",

View File

@@ -21,9 +21,9 @@ cd /work/system/go/src
chmod -R +w ..
./make.bash
`, pkg.Path(AbsUsrSrc.Append("go"), false, pkg.NewHTTPGetTar(
nil, "https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz",
mustDecode(checksum),
`, pkg.Path(AbsUsrSrc.Append("go"), false, newTar(
"https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz",
checksum,
pkg.TarGzip,
)))
}
@@ -55,9 +55,9 @@ ln -s \
../go/bin/go \
../go/bin/gofmt \
/work/system/bin
`, pkg.Path(AbsUsrSrc.Append("go"), false, pkg.NewHTTPGetTar(
nil, "https://go.dev/dl/go"+version+".src.tar.gz",
mustDecode(checksum),
`, pkg.Path(AbsUsrSrc.Append("go"), false, newTar(
"https://go.dev/dl/go"+version+".src.tar.gz",
checksum,
pkg.TarGzip,
)))
}
@@ -73,7 +73,7 @@ func (t Toolchain) newGoLatest() (pkg.Artifact, string) {
case "amd64":
bootstrapExtra = append(bootstrapExtra, t.newGoBootstrap())
case "arm64":
case "arm64", "riscv64":
bootstrapEnv = append(bootstrapEnv, "GOROOT_BOOTSTRAP=/system")
bootstrapExtra = t.AppendPresets(bootstrapExtra, gcc)
finalEnv = append(finalEnv, "CGO_ENABLED=0")
@@ -141,8 +141,8 @@ rm \
)
const (
version = "1.26.1"
checksum = "DdC5Ea-aCYPUHNObQh_09uWU0vn4e-8Ben850Vq-5OoamDRrXhuYI4YQ_BOFgaT0"
version = "1.26.2"
checksum = "v-6BE89_1g3xYf-9oIYpJKFXlo3xKHYJj2_VGkaUq8ZVkIVQmLwrto-xGG03OISH"
)
return t.newGo(
version,

View File

@@ -1,8 +1,6 @@
package rosa
import (
"strings"
"hakurei.app/fhs"
"hakurei.app/internal/pkg"
)
@@ -10,16 +8,12 @@ import (
func (t Toolchain) newGLib() (pkg.Artifact, string) {
const (
version = "2.88.0"
checksum = "bCLkAmp1o_Po4cXDbC06AyjLyxkBxyNJnflwBpSdf4W8K6dc9xKj6Pm3JYbHPdDf"
checksum = "T79Cg4z6j-sDZ2yIwvbY4ccRv2-fbwbqgcw59F5NQ6qJT6z4v261vbYp3dHO6Ma3"
)
return t.NewPackage("glib", version, pkg.NewHTTPGet(
nil, "https://download.gnome.org/sources/glib/"+
strings.Join(strings.SplitN(version, ".", 3)[:2], ".")+
"/glib-"+version+".tar.xz",
mustDecode(checksum),
return t.NewPackage("glib", version, t.newTagRemote(
"https://gitlab.gnome.org/GNOME/glib.git",
version, checksum,
), &PackageAttr{
SourceKind: SourceKindTarXZ,
Paths: []pkg.ExecPath{
pkg.Path(fhs.AbsEtc.Append(
"machine-id",
@@ -39,7 +33,6 @@ func (t Toolchain) newGLib() (pkg.Artifact, string) {
{"Ddefault_library", "both"},
},
},
XZ,
PythonPackaging,
Bash,

View File

@@ -99,7 +99,7 @@ mkdir -p /work/system/bin/
f: func(t Toolchain) (pkg.Artifact, string) {
return t.newHakurei("-dist", `
export HAKUREI_VERSION
DESTDIR=/work /usr/src/hakurei/dist/release.sh
DESTDIR=/work /usr/src/hakurei/all.sh
`, true), hakureiVersion
},

View File

@@ -4,13 +4,13 @@ package rosa
import "hakurei.app/internal/pkg"
const hakureiVersion = "0.3.7"
const hakureiVersion = "0.4.0"
// hakureiSource is the source code of a hakurei release.
var hakureiSource = pkg.NewHTTPGetTar(
nil, "https://git.gensokyo.uk/rosa/hakurei/archive/"+
var hakureiSource = newTar(
"https://git.gensokyo.uk/rosa/hakurei/archive/"+
"v"+hakureiVersion+".tar.gz",
mustDecode("Xh_sdITOATEAQN5_UuaOyrWsgboxorqRO9bml3dGm8GAxF8NFpB7MqhSZgjJxAl2"),
"wfQ9DqCW0Fw9o91wj-I55waoqzB-UqzzuC0_2h-P-1M78SgZ1WHSPCDJMth6EyC2",
pkg.TarGzip,
)

View File

@@ -2,12 +2,12 @@ package rosa
import "hakurei.app/internal/pkg"
const kernelVersion = "6.12.78"
const kernelVersion = "6.12.82"
var kernelSource = pkg.NewHTTPGetTar(
nil, "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/"+
var kernelSource = newTar(
"https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/"+
"snapshot/linux-"+kernelVersion+".tar.gz",
mustDecode("iUlZA-nv04TUOL0TmgDGBjaOe0sIaXTqLvuR4owYgHMZM8vecusnMMqbeuuZP4_G"),
"pYr4Wlk0fagbnlws1SPUFaWtJtqxm9GtSilW0V5A7f_WRap9IMCZV22kZpV_G-zk",
pkg.TarGzip,
)
@@ -1221,7 +1221,7 @@ install -Dm0500 \
/sbin/depmod
make \
"-j$(nproc)" \
` + jobsFlagE + ` \
-f /usr/src/kernel/Makefile \
O=/tmp/kbuild \
LLVM=1 \
@@ -1282,14 +1282,14 @@ func init() {
func (t Toolchain) newFirmware() (pkg.Artifact, string) {
const (
version = "20260309"
checksum = "M1az8BxSiOEH3LA11Trc5VAlakwAHhP7-_LKWg6k-SVIzU3xclMDO4Tiujw1gQrC"
version = "20260410"
checksum = "J8PdQlGqwrivpskPzbL6xacqR6mlKtXpe5RpzFfVzKPAgG81ZRXsc3qrxwdGJbil"
)
return t.NewPackage("firmware", version, pkg.NewHTTPGetTar(
nil, "https://gitlab.com/kernel-firmware/linux-firmware/-/"+
"archive/"+version+"/linux-firmware-"+version+".tar.bz2",
mustDecode(checksum),
pkg.TarBzip2,
return t.NewPackage("firmware", version, newFromGitLab(
"gitlab.com",
"kernel-firmware/linux-firmware",
version,
checksum,
), &PackageAttr{
// dedup creates temporary file
Writable: true,
@@ -1309,7 +1309,7 @@ func (t Toolchain) newFirmware() (pkg.Artifact, string) {
"install-zst",
},
SkipCheck: true, // requires pre-commit
Install: `make "-j$(nproc)" DESTDIR=/work/system dedup`,
Install: "make " + jobsFlagE + " DESTDIR=/work/system dedup",
},
Parallel,
Rdfind,

View File

@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86 6.12.78 Kernel Configuration
# Linux/x86 6.12.80 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="clang version 22.1.2"
CONFIG_GCC_VERSION=0

View File

@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 6.12.78 Kernel Configuration
# Linux/arm64 6.12.80 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="clang version 21.1.8"
CONFIG_GCC_VERSION=0

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,8 @@
package rosa
import _ "embed"
//go:embed kernel_riscv64.config
var kernelConfig []byte
const kernelName = "bzImage"

Some files were not shown because too many files have changed in this diff Show More