237 Commits

Author SHA1 Message Date
cat 337bf20f50 release: 0.4.1
Test / Create distribution (push) Successful in 38s
Test / ShareFS (push) Successful in 43s
Test / Sandbox (push) Successful in 48s
Test / Sandbox (race detector) (push) Successful in 48s
Test / Hakurei (race detector) (push) Successful in 55s
Release / Create release (push) Successful in 1m8s
Test / Hakurei (push) Successful in 2m14s
Test / Flake checks (push) Successful in 1m25s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-04 05:04:00 +09:00
cat 1cb792cf6e cmd/dist: increase gzip level
Test / Sandbox (push) Successful in 2m47s
Test / Create distribution (push) Successful in 34s
Test / Hakurei (push) Successful in 2m6s
Test / ShareFS (push) Successful in 3m59s
Test / Sandbox (race detector) (push) Successful in 3m30s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m22s
Performance does not matter in this case.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-04 04:04:18 +09:00
cat b2b40b07e8 cmd/dist: optional verbosity
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m54s
Test / Hakurei (push) Successful in 3m50s
Test / ShareFS (push) Successful in 3m59s
Test / Sandbox (race detector) (push) Successful in 5m21s
Test / Hakurei (race detector) (push) Successful in 6m29s
Test / Flake checks (push) Successful in 1m30s
This makes output less noisy. The build is fast enough not to require progress indication.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-04 04:02:02 +09:00
cat da11b26ec1 container/initoverlay: configure via fsconfig
Test / ShareFS (push) Successful in 40s
Test / Hakurei (push) Successful in 53s
Test / Hakurei (race detector) (push) Successful in 51s
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 1m48s
Test / Sandbox (race detector) (push) Successful in 2m37s
Test / Flake checks (push) Successful in 1m20s
This works around the page size limit at the cost of negligible performance regressions.

Closes #34.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-04 02:29:56 +09:00
cat 024489e800 ext: wrap file-descriptor-based mount facilities
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 2m47s
Test / Hakurei (push) Successful in 3m50s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 5m12s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m20s
This only implements what is required by package container for now.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-04 01:54:35 +09:00
cat 0f795712b0 internal/rosa/llvm: enable LLVM_BUILD_TESTS
Test / Create distribution (push) Successful in 4m31s
Test / Sandbox (push) Successful in 8m7s
Test / Hakurei (push) Successful in 9m12s
Test / ShareFS (push) Successful in 9m15s
Test / Sandbox (race detector) (push) Successful in 10m40s
Test / Hakurei (race detector) (push) Successful in 12m15s
Test / Flake checks (push) Successful in 3m6s
This arranges for tests to be built early, and is more efficient towards the end of the build.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 20:05:30 +09:00
cat 7e2210ff71 internal/rosa/llvm: provide runtimes early in stage0
Test / Create distribution (push) Successful in 1m9s
Test / Sandbox (push) Successful in 2m58s
Test / Hakurei (push) Successful in 4m16s
Test / ShareFS (push) Successful in 4m11s
Test / Sandbox (race detector) (push) Successful in 6m13s
Test / Hakurei (race detector) (push) Successful in 7m59s
Test / Flake checks (push) Successful in 1m21s
The LLVM build system fails to handle a dynamically linked toolchain correctly, and leaks the system installation during builds.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 19:48:49 +09:00
cat a71a008f3c cmd/mbf: optionally build on early stages
Test / Create distribution (push) Successful in 1m8s
Test / Sandbox (push) Successful in 2m56s
Test / ShareFS (push) Successful in 3m57s
Test / Sandbox (race detector) (push) Successful in 5m21s
Test / Hakurei (race detector) (push) Successful in 6m32s
Test / Hakurei (push) Successful in 2m49s
Test / Flake checks (push) Successful in 1m36s
This makes debugging the bootstrap process much less cumbersome.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 18:46:47 +09:00
cat 162265b47e container: reject strings larger than a page
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m50s
Test / Hakurei (push) Successful in 3m50s
Test / ShareFS (push) Successful in 3m54s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 6m29s
Test / Flake checks (push) Successful in 1m22s
The vfs stores these values in a page obtained via GFP, and silently stops copying once the page is filled. This check prevents confusing behaviour in such cases.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 17:30:25 +09:00
cat 3fa7ac04e4 internal/rosa/x: combine with xcb
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m54s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m13s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m45s
Separating them no longer makes sense.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 04:38:00 +09:00
cat bf2867d653 internal/rosa/x: libxshmfence artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m55s
Test / Hakurei (push) Successful in 3m53s
Test / ShareFS (push) Successful in 3m53s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m23s
Test / Flake checks (push) Successful in 1m30s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 04:35:39 +09:00
cat ec0f0f6507 internal/rosa/x: libXext artifact
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m54s
Test / ShareFS (push) Successful in 3m58s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m30s
Test / Flake checks (push) Successful in 1m20s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 04:23:20 +09:00
cat a77a802955 internal/rosa/x: xlib artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m50s
Test / Hakurei (push) Successful in 3m54s
Test / ShareFS (push) Successful in 3m56s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 3m35s
Test / Flake checks (push) Successful in 1m24s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 04:15:21 +09:00
cat 4407e14dfc internal/rosa/x: migrate to xorgproto
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m50s
Test / Hakurei (push) Successful in 3m54s
Test / ShareFS (push) Successful in 3m52s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m23s
Test / Flake checks (push) Successful in 1m31s
This is much cleaner than the many protocol packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 04:09:36 +09:00
cat e024d3184a internal/rosa/clang: install cpp symlink
Test / Create distribution (push) Successful in 1m13s
Test / Sandbox (push) Successful in 3m8s
Test / Hakurei (push) Successful in 4m19s
Test / ShareFS (push) Successful in 4m24s
Test / Sandbox (race detector) (push) Successful in 5m32s
Test / Hakurei (race detector) (push) Successful in 6m42s
Test / Flake checks (push) Successful in 1m31s
Required by some buggy autotools scripts.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-03 00:41:23 +09:00
cat 8e1bf00c2d internal/rosa/stage0: add arm64 tarball
Test / Create distribution (push) Successful in 1m37s
Test / Sandbox (push) Successful in 5m2s
Test / Hakurei (push) Successful in 8m21s
Test / ShareFS (push) Successful in 8m54s
Test / Hakurei (race detector) (push) Successful in 11m26s
Test / Sandbox (race detector) (push) Successful in 5m44s
Test / Flake checks (push) Successful in 10m12s
This replaces the previous, much larger stage0 distribution.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 23:53:08 +09:00
cat b111e22050 internal/rosa/x: libxtrans artifact
Test / Create distribution (push) Successful in 1m27s
Test / Sandbox (push) Successful in 4m26s
Test / Hakurei (push) Successful in 7m36s
Test / ShareFS (push) Successful in 8m22s
Test / Hakurei (race detector) (push) Successful in 1m24s
Test / Sandbox (race detector) (push) Successful in 3m48s
Test / Flake checks (push) Successful in 3m39s
Required by many X libraries.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 23:42:00 +09:00
cat 1fa458c0be internal/rosa/glslang: SPIRV-LLVM-Translator artifact
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 4m36s
Test / Hakurei (push) Successful in 7m33s
Test / ShareFS (push) Successful in 8m18s
Test / Sandbox (race detector) (push) Successful in 8m31s
Test / Hakurei (race detector) (push) Successful in 10m58s
Test / Flake checks (push) Successful in 1m51s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 22:47:51 +09:00
cat 2c7ae67a67 internal/rosa/llvm: LIT args helper
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 4m14s
Test / Hakurei (push) Successful in 7m20s
Test / ShareFS (push) Successful in 7m43s
Test / Sandbox (race detector) (push) Successful in 7m57s
Test / Hakurei (race detector) (push) Successful in 4m55s
Test / Flake checks (push) Successful in 2m23s
This is useful for other projects using LIT.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 22:17:57 +09:00
cat 3826621b21 internal/rosa/python: lit artifact
Test / Create distribution (push) Successful in 1m32s
Test / Sandbox (push) Successful in 5m49s
Test / Hakurei (push) Successful in 9m10s
Test / Sandbox (race detector) (push) Successful in 9m20s
Test / ShareFS (push) Successful in 9m52s
Test / Hakurei (race detector) (push) Successful in 12m10s
Test / Flake checks (push) Successful in 1m52s
Used by LLVM-related projects.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 22:15:37 +09:00
cat 041b505c2e internal/rosa/cmake: implicit CMAKE_BUILD_TYPE
Test / Create distribution (push) Successful in 1m33s
Test / Sandbox (push) Successful in 4m42s
Test / ShareFS (push) Successful in 8m24s
Test / Sandbox (race detector) (push) Successful in 8m35s
Test / Hakurei (race detector) (push) Successful in 11m41s
Test / Hakurei (push) Successful in 6m27s
Test / Flake checks (push) Successful in 3m35s
Lack of this behaviour is a holdover from when the helper was first split from the (now removed) LLVM helper.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 21:53:38 +09:00
cat e6debce649 internal/rosa/llvm: make source independently available
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 4m36s
Test / Hakurei (push) Successful in 7m41s
Test / ShareFS (push) Successful in 8m18s
Test / Sandbox (race detector) (push) Successful in 8m26s
Test / Hakurei (race detector) (push) Successful in 11m9s
Test / Flake checks (push) Successful in 2m31s
This is unfortunately still required, due to the monorepo nature of LLVM.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 21:47:01 +09:00
cat aa26b86fce internal/rosa/llvm: skip multiple-compile-threads-basic on arm64
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 5m11s
Test / Hakurei (push) Successful in 7m16s
Test / Sandbox (race detector) (push) Successful in 7m49s
Test / ShareFS (push) Successful in 8m16s
Test / Hakurei (race detector) (push) Successful in 11m3s
Test / Flake checks (push) Successful in 2m24s
This intermittently crashes.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 12:39:46 +09:00
cat a57a8fd5d8 internal/rosa/llvm: skip unwind_leaffunction on arm64
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 5m52s
Test / Sandbox (race detector) (push) Successful in 9m5s
Test / Hakurei (push) Successful in 9m11s
Test / ShareFS (push) Successful in 9m23s
Test / Hakurei (race detector) (push) Successful in 5m5s
Test / Flake checks (push) Successful in 1m53s
This unexpectedly passes.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 05:53:00 +09:00
maemachinebroke 1d5d063d6a cmd/mbf: package status dashboard
Test / Create distribution (push) Successful in 1m36s
Test / Sandbox (push) Successful in 5m44s
Test / Sandbox (race detector) (push) Successful in 8m14s
Test / ShareFS (push) Successful in 8m38s
Test / Hakurei (race detector) (push) Successful in 10m57s
Test / Hakurei (push) Successful in 6m38s
Test / Flake checks (push) Successful in 2m56s
This displays package metadata with optional status from a report.
2026-05-02 05:05:56 +09:00
cat e61628a34e cmd/mbf: test cure all via daemon
Test / Create distribution (push) Successful in 1m32s
Test / Sandbox (race detector) (push) Successful in 11m28s
Test / ShareFS (push) Successful in 13m35s
Test / Sandbox (push) Successful in 2m33s
Test / Hakurei (push) Successful in 6m54s
Test / Hakurei (race detector) (push) Successful in 8m27s
Test / Flake checks (push) Successful in 2m20s
This is the daemon equivalent of CureAll in internal/rosa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 02:39:12 +09:00
cat 5a18f14929 internal/rosa/gnu: bison disable broken test
Test / Create distribution (push) Successful in 2m4s
Test / ShareFS (push) Successful in 13m56s
Test / Hakurei (race detector) (push) Successful in 15m51s
Test / Sandbox (push) Successful in 4m53s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (push) Successful in 7m4s
Test / Flake checks (push) Successful in 2m57s
This is miscompiled by the current toolchain.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 02:23:51 +09:00
cat f12880688d internal/rosa/gnu: test skip helper
Test / Create distribution (push) Successful in 2m3s
Test / Sandbox (race detector) (push) Successful in 11m26s
Test / ShareFS (push) Successful in 13m58s
Test / Hakurei (race detector) (push) Successful in 15m42s
Test / Sandbox (push) Successful in 2m46s
Test / Hakurei (push) Successful in 6m19s
Test / Flake checks (push) Successful in 2m48s
The terribleness of GNU invites interesting helpers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 05:19:54 +09:00
cat bb5bbfe16a internal/rosa/go: disable tsan test
Test / Hakurei (push) Successful in 2m7s
Test / Sandbox (push) Successful in 1m48s
Test / Hakurei (race detector) (push) Successful in 2m4s
Test / Create distribution (push) Successful in 1m22s
Test / ShareFS (push) Successful in 1m40s
Test / Sandbox (race detector) (push) Successful in 1m58s
Test / Flake checks (push) Successful in 4m30s
The newly enabled tsan does not play well with go126, so disable for now.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-02 00:12:41 +09:00
cat 427e1ca37c internal/rosa/go: bootstrap 1.25.7 to 1.25.9
Test / Create distribution (push) Successful in 1m27s
Test / Sandbox (push) Successful in 4m21s
Test / Hakurei (push) Successful in 7m38s
Test / ShareFS (push) Successful in 8m10s
Test / Sandbox (race detector) (push) Successful in 8m22s
Test / Hakurei (race detector) (push) Successful in 10m55s
Test / Flake checks (push) Successful in 1m49s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 23:24:07 +09:00
cat 96fdd9ecc5 internal/rosa: disable LTO in tests
Test / Create distribution (push) Successful in 1m27s
Test / Sandbox (push) Successful in 4m15s
Test / ShareFS (push) Successful in 7m44s
Test / Sandbox (race detector) (push) Successful in 8m2s
Test / Hakurei (race detector) (push) Successful in 10m25s
Test / Hakurei (push) Successful in 3m51s
Test / Flake checks (push) Successful in 1m46s
This is too expensive and not feasible for development.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 20:08:26 +09:00
cat 02771b655b internal/rosa/stage0: replace amd64 tarball
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 4m17s
Test / Hakurei (push) Successful in 7m37s
Test / ShareFS (push) Successful in 8m10s
Test / Sandbox (race detector) (push) Successful in 8m19s
Test / Hakurei (race detector) (push) Successful in 10m51s
Test / Flake checks (push) Successful in 1m52s
This is a non-LTO distribution with the new layer configuration.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 18:57:28 +09:00
cat d1c8d2c39b internal/rosa/gnu: skip libtool tests in stage0
Test / Create distribution (push) Successful in 1m33s
Test / Sandbox (push) Successful in 5m19s
Test / Hakurei (push) Successful in 7m48s
Test / Sandbox (race detector) (push) Successful in 8m17s
Test / ShareFS (push) Successful in 8m43s
Test / Hakurei (race detector) (push) Successful in 11m45s
Test / Flake checks (push) Successful in 2m54s
This upsets the linker in stage0.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 05:26:40 +09:00
cat 0efd742e8a internal/rosa/llvm: enable libclc as a runtime
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 5m20s
Test / Hakurei (push) Successful in 7m57s
Test / Sandbox (race detector) (push) Successful in 8m5s
Test / ShareFS (push) Successful in 8m39s
Test / Hakurei (race detector) (push) Successful in 11m53s
Test / Flake checks (push) Successful in 2m57s
Enabling this as a project is deprecated.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 05:17:02 +09:00
cat ae1fe638d5 internal/rosa/stage0: remove unused layers
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 5m8s
Test / Hakurei (push) Successful in 7m30s
Test / Sandbox (race detector) (push) Successful in 8m7s
Test / ShareFS (push) Successful in 8m25s
Test / Hakurei (race detector) (push) Successful in 11m31s
Test / Flake checks (push) Successful in 2m51s
The stage0 toolchain no longer requires bundled dependencies other than the bare toolchain and environment itself.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 03:52:41 +09:00
cat 445d95023b internal/rosa: global preset flags
Test / Create distribution (push) Successful in 1m31s
Test / Sandbox (push) Successful in 5m8s
Test / Hakurei (push) Successful in 7m31s
Test / Sandbox (race detector) (push) Successful in 8m6s
Test / ShareFS (push) Successful in 8m28s
Test / Hakurei (race detector) (push) Successful in 11m8s
Test / Flake checks (push) Successful in 2m41s
These changes preset behaviour globally. Useful for ad hoc workarounds for development or bootstrapping on resource-constrained systems.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 03:42:48 +09:00
cat fc66f0bb47 internal/rosa/llvm: use llvm build system
Test / Create distribution (push) Successful in 1m32s
Test / Sandbox (push) Successful in 5m9s
Test / Hakurei (push) Successful in 7m32s
Test / Sandbox (race detector) (push) Successful in 8m0s
Test / ShareFS (push) Successful in 8m22s
Test / Hakurei (race detector) (push) Successful in 11m20s
Test / Flake checks (push) Successful in 2m48s
This removes the multistep bootstrap hack. Stage0 exceptions are also eliminated for a later change to bring the stage0 distribution down to just a bare toolchain, toybox and shell. This change also enables dynamic linking and ThinLTO.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 03:36:58 +09:00
cat 2cd6b35bee internal/rosa/cmake: run tests
Test / Create distribution (push) Successful in 1m27s
Test / Sandbox (push) Successful in 5m19s
Test / Hakurei (push) Successful in 7m24s
Test / Sandbox (race detector) (push) Successful in 8m4s
Test / ShareFS (push) Successful in 8m17s
Test / Hakurei (race detector) (push) Successful in 11m10s
Test / Flake checks (push) Successful in 2m43s
This uses the standard CMake test target.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-05-01 03:04:59 +09:00
cat 09a216c6ec internal/rosa/perl: make /system/bin writable
Test / Create distribution (push) Successful in 1m25s
Test / Sandbox (push) Successful in 5m3s
Test / Hakurei (push) Successful in 7m4s
Test / Sandbox (race detector) (push) Successful in 8m3s
Test / ShareFS (push) Successful in 8m10s
Test / Hakurei (race detector) (push) Successful in 10m41s
Test / Flake checks (push) Successful in 2m9s
This enables cure in stage0 where /system/bin is read-only.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-30 19:25:46 +09:00
cat 44d17325c2 internal/rosa: raise stage0 extra layers
Test / Create distribution (push) Successful in 1m26s
Test / Sandbox (push) Successful in 4m57s
Test / Hakurei (push) Successful in 7m4s
Test / Sandbox (race detector) (push) Successful in 7m49s
Test / ShareFS (push) Successful in 8m17s
Test / Hakurei (race detector) (push) Successful in 10m46s
Test / Flake checks (push) Successful in 1m55s
This enables extras to override stage0 tarball.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-30 18:58:42 +09:00
cat 544ce77cbc internal/rosa/make: do not attempt check
Test / Create distribution (push) Successful in 2m39s
Test / Sandbox (push) Successful in 6m27s
Test / Hakurei (push) Successful in 8m25s
Test / Sandbox (race detector) (push) Successful in 9m5s
Test / ShareFS (push) Successful in 9m23s
Test / Hakurei (race detector) (push) Successful in 11m58s
Test / Flake checks (push) Successful in 2m18s
This is circular during bootstrap, and tests are silently skipped without perl, so disable them explicitly.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-30 17:36:46 +09:00
cat 63c3c30b23 internal/rosa/zlib: compile with -fPIC
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 4m49s
Test / Hakurei (push) Successful in 6m44s
Test / Sandbox (race detector) (push) Successful in 7m47s
Test / ShareFS (push) Successful in 7m54s
Test / Hakurei (race detector) (push) Successful in 10m42s
Test / Flake checks (push) Successful in 1m59s
For static linking into shared libraries. This was missed when migrating to CMake.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-30 15:55:46 +09:00
cat d23c4ecc7c internal/rosa/llvm: use correct triple for rpath
Test / Create distribution (push) Successful in 1m22s
Test / Sandbox (push) Successful in 4m54s
Test / Hakurei (push) Successful in 7m1s
Test / Sandbox (race detector) (push) Successful in 7m52s
Test / ShareFS (push) Successful in 8m10s
Test / Hakurei (race detector) (push) Successful in 10m43s
Test / Flake checks (push) Successful in 1m57s
MultiarchTriple produces a generic glibc triple string.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-30 00:39:13 +09:00
cat a46656dff8 internal/rosa/python: mako 1.3.11 to 1.3.12
Test / Create distribution (push) Successful in 1m39s
Test / Sandbox (push) Successful in 5m2s
Test / Hakurei (push) Successful in 6m52s
Test / ShareFS (push) Successful in 7m32s
Test / Sandbox (race detector) (push) Successful in 7m47s
Test / Hakurei (race detector) (push) Successful in 10m39s
Test / Flake checks (push) Successful in 1m56s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-29 14:25:26 +09:00
cat 77db153ff5 internal/rosa/python: trove-classifiers 2026.1.14.14 to 2026.4.28.13
Test / Create distribution (push) Successful in 1m34s
Test / Sandbox (push) Successful in 5m8s
Test / Hakurei (push) Successful in 6m42s
Test / ShareFS (push) Successful in 7m29s
Test / Sandbox (race detector) (push) Successful in 7m41s
Test / Hakurei (race detector) (push) Successful in 10m41s
Test / Flake checks (push) Successful in 1m55s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-29 14:25:07 +09:00
cat 520d95bc07 internal/rosa/libxslt: fetch source tarball
Test / Create distribution (push) Successful in 1m43s
Test / Sandbox (push) Successful in 5m8s
Test / Hakurei (push) Successful in 7m11s
Test / ShareFS (push) Successful in 7m38s
Test / Sandbox (race detector) (push) Successful in 8m1s
Test / Hakurei (race detector) (push) Successful in 10m58s
Test / Flake checks (push) Successful in 2m2s
This does not have submodules, so the overhead of git is unnecessary.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 18:31:44 +09:00
cat 451df3f4e7 internal/rosa/libxml2: fetch source tarball
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 4m53s
Test / Hakurei (push) Successful in 6m54s
Test / ShareFS (push) Successful in 7m34s
Test / Sandbox (race detector) (push) Successful in 7m55s
Test / Hakurei (race detector) (push) Successful in 10m50s
Test / Flake checks (push) Successful in 2m6s
This does not have submodules, so the overhead of git is unnecessary.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 18:31:28 +09:00
cat 011fac15ed internal/rosa/git: 2.53.0 to 2.54.0
Test / Create distribution (push) Successful in 1m30s
Test / Sandbox (push) Successful in 5m0s
Test / Hakurei (push) Successful in 7m2s
Test / Sandbox (race detector) (push) Successful in 7m59s
Test / ShareFS (push) Successful in 8m17s
Test / Hakurei (race detector) (push) Successful in 10m45s
Test / Flake checks (push) Successful in 2m49s
This release broke httpd detection and job control on mksh.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 18:23:20 +09:00
cat 347682ad0b internal/rosa/kernel: 6.12.83 to 6.12.84
Test / Create distribution (push) Successful in 1m27s
Test / Sandbox (push) Successful in 4m57s
Test / Hakurei (push) Successful in 6m56s
Test / Sandbox (race detector) (push) Successful in 8m0s
Test / ShareFS (push) Successful in 8m10s
Test / Hakurei (race detector) (push) Successful in 10m50s
Test / Flake checks (push) Successful in 2m43s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 17:44:20 +09:00
cat 1a2b979add internal/rosa/rsync: 3.4.1 to 3.4.2
Test / Create distribution (push) Successful in 1m41s
Test / Hakurei (push) Successful in 10m25s
Test / ShareFS (push) Successful in 11m25s
Test / Sandbox (push) Successful in 2m52s
Test / Sandbox (race detector) (push) Successful in 3m51s
Test / Hakurei (race detector) (push) Successful in 7m21s
Test / Flake checks (push) Successful in 2m41s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:37:47 +09:00
cat b1c90cc380 internal/rosa/libexpat: 2.7.5 to 2.8.0
Test / Create distribution (push) Successful in 1m39s
Test / Sandbox (push) Successful in 6m17s
Test / Hakurei (push) Successful in 10m19s
Test / ShareFS (push) Successful in 11m21s
Test / Hakurei (race detector) (push) Successful in 14m4s
Test / Sandbox (race detector) (push) Successful in 3m47s
Test / Flake checks (push) Successful in 2m46s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:37:16 +09:00
cat 3a66b8143a internal/rosa/nss: 3.123 to 3.123.1
Test / Create distribution (push) Successful in 1m42s
Test / ShareFS (push) Successful in 11m22s
Test / Sandbox (push) Successful in 2m53s
Test / Sandbox (race detector) (push) Successful in 4m13s
Test / Hakurei (push) Successful in 6m43s
Test / Hakurei (race detector) (push) Successful in 9m24s
Test / Flake checks (push) Successful in 1m54s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:15:14 +09:00
cat 64bbd3aabd internal/rosa/mesa: libdrm 2.4.131 to 2.4.133
Test / Create distribution (push) Successful in 43s
Test / ShareFS (push) Successful in 7m51s
Test / Sandbox (race detector) (push) Successful in 8m20s
Test / Hakurei (race detector) (push) Successful in 11m53s
Test / Hakurei (push) Successful in 7m36s
Test / Sandbox (push) Successful in 2m49s
Test / Flake checks (push) Successful in 2m52s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:03:49 +09:00
cat 08799a13d0 internal/rosa/glslang: spirv-tools check stable versions
Test / Create distribution (push) Successful in 44s
Test / ShareFS (push) Successful in 8m21s
Test / Hakurei (push) Successful in 7m52s
Test / Hakurei (race detector) (push) Successful in 9m37s
Test / Sandbox (push) Successful in 2m52s
Test / Sandbox (race detector) (push) Successful in 3m54s
Test / Flake checks (push) Successful in 2m47s
This hides release candidates.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:03:29 +09:00
cat 1aef9c3bbb internal/rosa/python: pathspec 1.0.4 to 1.1.1
Test / Create distribution (push) Successful in 1m45s
Test / Sandbox (push) Successful in 5m18s
Test / Hakurei (push) Successful in 10m58s
Test / ShareFS (push) Successful in 12m18s
Test / Sandbox (race detector) (push) Successful in 5m26s
Test / Hakurei (race detector) (push) Successful in 10m54s
Test / Flake checks (push) Successful in 2m49s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:02:19 +09:00
cat 1f38303747 internal/rosa/python: packaging 26.1 to 26.2
Test / Create distribution (push) Successful in 59s
Test / Hakurei (push) Successful in 8m13s
Test / ShareFS (push) Successful in 8m55s
Test / Hakurei (race detector) (push) Successful in 10m57s
Test / Sandbox (push) Successful in 3m3s
Test / Sandbox (race detector) (push) Successful in 4m12s
Test / Flake checks (push) Successful in 2m41s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 16:01:56 +09:00
cat 640777b00c internal/rosa/gnu: parallel 20260322 to 20260422
Test / Create distribution (push) Successful in 1m30s
Test / Sandbox (push) Successful in 4m59s
Test / Hakurei (push) Successful in 7m3s
Test / ShareFS (push) Successful in 7m55s
Test / Sandbox (race detector) (push) Successful in 7m59s
Test / Hakurei (race detector) (push) Successful in 10m59s
Test / Flake checks (push) Successful in 2m56s
This pulls in bash with nonstandard hardcoded path.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 15:58:59 +09:00
cat 1d657193cf internal/rosa/kernel: disable md
Test / Create distribution (push) Successful in 2m31s
Test / Sandbox (push) Successful in 6m33s
Test / Hakurei (push) Successful in 8m44s
Test / Sandbox (race detector) (push) Successful in 9m22s
Test / ShareFS (push) Successful in 9m32s
Test / Hakurei (race detector) (push) Successful in 7m37s
Test / Flake checks (push) Successful in 3m22s
This is entirely unused and is a somewhat large attack surface, so disable it.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 15:48:20 +09:00
cat bab5406295 internal/rosa/go: require popcnt for x86
Test / Sandbox (push) Successful in 6m33s
Test / Hakurei (push) Successful in 8m16s
Test / ShareFS (push) Successful in 8m12s
Test / Sandbox (race detector) (push) Successful in 8m58s
Test / Hakurei (race detector) (push) Successful in 10m31s
Test / Flake checks (push) Successful in 1m51s
Test / Create distribution (push) Successful in 2m2s
This backports https://go.dev/cl/746640.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-28 14:36:59 +09:00
cat 725ae7d64d nix: remove all explicit timeouts
Test / ShareFS (push) Successful in 2m18s
Test / Create distribution (push) Successful in 2m43s
Test / Sandbox (push) Successful in 3m21s
Test / Sandbox (race detector) (push) Successful in 4m11s
Test / Hakurei (push) Successful in 4m26s
Test / Hakurei (race detector) (push) Successful in 5m14s
Test / Flake checks (push) Successful in 1m54s
These were useful during development because timing out is often the only indication of failure due to the terrible design of nixos vm test harness. This has become a nuisance however especially when the system is under load, so remove explicit values and fall back to the ludicrously high default.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 13:07:22 +09:00
cat 37a0c3967e internal/rosa/gnu: mpc fetch source tarball
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 3m3s
Test / Hakurei (push) Successful in 4m22s
Test / ShareFS (push) Successful in 4m27s
Test / Sandbox (race detector) (push) Successful in 6m7s
Test / Hakurei (race detector) (push) Successful in 3m35s
Test / Flake checks (push) Successful in 3m10s
This does not have submodules, so the overhead of git is unnecessary.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:57:11 +09:00
cat ea0692548f internal/rosa/gnu: coreutils 9.10 to 9.11
Test / Create distribution (push) Successful in 1m15s
Test / Sandbox (push) Successful in 3m31s
Test / ShareFS (push) Successful in 5m13s
Test / Sandbox (race detector) (push) Successful in 6m16s
Test / Hakurei (race detector) (push) Successful in 8m20s
Test / Hakurei (push) Successful in 3m2s
Test / Flake checks (push) Successful in 1m34s
Test regression was fixed, dropping patch.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:30:46 +09:00
cat 48ea23e648 internal/rosa/gnu: sed 4.9 to 4.10
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m52s
Test / Hakurei (push) Successful in 4m33s
Test / ShareFS (push) Successful in 3m14s
Test / Sandbox (race detector) (push) Successful in 5m12s
Test / Hakurei (race detector) (push) Successful in 7m1s
Test / Flake checks (push) Successful in 1m26s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:30:06 +09:00
cat 40320e4920 internal/rosa/meson: 1.11.0 to 1.11.1
Test / Create distribution (push) Successful in 37s
Test / Sandbox (push) Successful in 3m10s
Test / ShareFS (push) Successful in 3m20s
Test / Sandbox (race detector) (push) Successful in 5m44s
Test / Hakurei (race detector) (push) Successful in 7m25s
Test / Hakurei (push) Successful in 2m40s
Test / Flake checks (push) Successful in 1m21s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:29:17 +09:00
cat 3ca0f61632 internal/rosa/llvm: 22.1.3 to 22.1.4
Test / Sandbox (push) Successful in 2m36s
Test / Create distribution (push) Successful in 1m32s
Test / Hakurei (push) Successful in 6m1s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m47s
Test / Hakurei (race detector) (push) Successful in 7m32s
Test / Flake checks (push) Successful in 1m37s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:28:55 +09:00
cat 6ffaac96e3 internal/rosa/cmake: 4.3.1 to 4.3.2
Test / Sandbox (push) Successful in 3m34s
Test / Hakurei (push) Successful in 5m22s
Test / ShareFS (push) Successful in 4m49s
Test / Sandbox (race detector) (push) Successful in 6m35s
Test / Hakurei (race detector) (push) Successful in 8m16s
Test / Flake checks (push) Successful in 1m45s
Test / Create distribution (push) Successful in 57s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:28:34 +09:00
cat 13c7713d0c internal/rosa/kernel: 6.12.82 to 6.12.83
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m27s
Test / ShareFS (push) Successful in 5m19s
Test / Sandbox (race detector) (push) Successful in 6m46s
Test / Hakurei (race detector) (push) Successful in 8m17s
Test / Hakurei (push) Successful in 2m58s
Test / Flake checks (push) Successful in 1m28s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 12:28:14 +09:00
cat 42389f7ec5 internal/rosa/qemu: 10.2.2 to 11.0.0
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m49s
Test / Hakurei (push) Successful in 3m55s
Test / ShareFS (push) Successful in 4m3s
Test / Sandbox (race detector) (push) Successful in 5m31s
Test / Hakurei (race detector) (push) Successful in 6m28s
Test / Flake checks (push) Successful in 1m23s
This pulls in some python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 01:15:13 +09:00
cat 30f130c691 internal/rosa/python: wheel artifact
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m32s
No idea why this ended up as a package.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 01:07:14 +09:00
cat ceb4d26087 internal/pkg: record cache variant on-disk
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m47s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m23s
Test / Hakurei (push) Successful in 2m39s
Test / Flake checks (push) Successful in 1m25s
This makes custom artifacts much less error-prone to use.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-23 00:53:21 +09:00
cat 852f3a9b3d internal/rosa/kernel: 6.12.81 to 6.12.82
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m53s
Test / ShareFS (push) Successful in 4m3s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (race detector) (push) Successful in 6m27s
Test / Hakurei (push) Successful in 2m37s
Test / Flake checks (push) Successful in 1m18s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 22:11:13 +09:00
cat 5e02dbdb0d internal/rosa/python: remove pypi helpers
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m15s
Test / Hakurei (push) Successful in 4m22s
Test / ShareFS (push) Successful in 4m28s
Test / Sandbox (race detector) (push) Successful in 5m39s
Test / Hakurei (race detector) (push) Successful in 6m48s
Test / Flake checks (push) Successful in 1m25s
Pypi is disallowed by policy so these helpers are no longer useful.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:37:10 +09:00
cat 6a3248d472 internal/rosa/python: install pyyaml from source
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m57s
Test / ShareFS (push) Successful in 3m57s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 6m26s
Test / Hakurei (push) Successful in 3m12s
Test / Flake checks (push) Successful in 1m27s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:35:30 +09:00
cat 67404c98d9 internal/rosa/nss: install buildcatrust from source
Test / Create distribution (push) Successful in 2m0s
Test / Sandbox (push) Successful in 3m49s
Test / Hakurei (push) Successful in 4m56s
Test / ShareFS (push) Successful in 5m0s
Test / Sandbox (race detector) (push) Successful in 6m32s
Test / Hakurei (race detector) (push) Successful in 7m30s
Test / Flake checks (push) Successful in 2m55s
Dependencies are now available, so this no longer has to rely on the release.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 02:09:24 +09:00
cat b9bf69cfce internal/rosa/python: install mako from source
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m59s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m30s
Test / Flake checks (push) Successful in 2m28s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:55:23 +09:00
cat 4648f98272 internal/rosa/python: run tests via helper
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m52s
Test / ShareFS (push) Successful in 3m58s
Test / Sandbox (race detector) (push) Successful in 5m22s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Hakurei (push) Successful in 2m39s
Test / Flake checks (push) Successful in 1m25s
Despite the lack of standards, pytest seems somewhat widely agreed upon.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:50:57 +09:00
cat 11d99439ac internal/rosa/python: install markupsafe from source
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m9s
Test / ShareFS (push) Successful in 2m52s
Test / Hakurei (race detector) (push) Successful in 3m51s
Test / Flake checks (push) Successful in 1m24s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:26:11 +09:00
cat 39e4c5b8ac internal/rosa/python: optionally install before check
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 2m52s
Test / ShareFS (push) Successful in 3m27s
Test / Sandbox (race detector) (push) Successful in 4m50s
Test / Hakurei (race detector) (push) Successful in 6m25s
Test / Hakurei (push) Successful in 2m43s
Test / Flake checks (push) Successful in 1m23s
Some test suites require package to be installed globally.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-20 01:25:43 +09:00
cat e8f6db38b6 internal/rosa/python: install pytest from source
Test / Create distribution (push) Successful in 1m2s
Test / ShareFS (push) Successful in 5m1s
Test / Hakurei (push) Successful in 5m5s
Test / Hakurei (race detector) (push) Successful in 10m55s
Test / Sandbox (push) Successful in 3m29s
Test / Sandbox (race detector) (push) Successful in 4m35s
Test / Flake checks (push) Successful in 3m1s
Used by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 23:17:38 +09:00
cat 20d5b71575 internal/rosa/python: install iniconfig from source
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m48s
Test / Hakurei (race detector) (push) Successful in 6m23s
Test / Sandbox (race detector) (push) Successful in 2m40s
Test / ShareFS (push) Successful in 2m50s
Test / Flake checks (push) Successful in 1m21s
This also required the setuptools-scm hack.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:53:32 +09:00
cat e903e7f542 internal/rosa/python: install pygments from source
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m51s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m59s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m26s
Test / Flake checks (push) Successful in 1m22s
This finally has its dependencies.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:40:43 +09:00
cat 1caa051f4d internal/rosa/python: hatchling artifact
Test / Sandbox (push) Successful in 2m48s
Test / ShareFS (push) Successful in 3m53s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Create distribution (push) Successful in 33s
Test / Hakurei (push) Successful in 2m37s
Test / Flake checks (push) Successful in 1m21s
Required by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:35:18 +09:00
cat dcdc6f7f6d internal/rosa/python: trove-classifiers artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 4m55s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m27s
Test / Flake checks (push) Successful in 1m31s
Required by hatchling.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:32:12 +09:00
cat 5ad6f26b46 internal/rosa/python: install packaging from source
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m50s
Test / Hakurei (push) Successful in 3m50s
Test / ShareFS (push) Successful in 3m54s
Test / Sandbox (race detector) (push) Successful in 5m13s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m22s
This is required by many packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 22:12:49 +09:00
cat 7ba75a79f4 internal/rosa/python: install pluggy from source
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m44s
Test / ShareFS (push) Successful in 3m52s
Test / Sandbox (race detector) (push) Successful in 5m13s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m23s
This finally has all its dependencies at this point.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 21:55:55 +09:00
cat 9ef84d3904 internal/rosa/python: setuptools-scm artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m44s
Test / Hakurei (push) Successful in 3m49s
Test / ShareFS (push) Successful in 3m54s
Test / Sandbox (race detector) (push) Successful in 5m19s
Test / Hakurei (race detector) (push) Successful in 6m20s
Test / Flake checks (push) Successful in 1m21s
Awful hack required by many packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 21:38:44 +09:00
cat 3b7b6e51fb internal/rosa/python: pass build dependencies separately
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 3m41s
Test / ShareFS (push) Successful in 3m53s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m22s
This is cleaner with less duplicate code.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 20:26:41 +09:00
cat b1b4debb82 internal/rosa/python: pathspec artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m52s
Test / Hakurei (push) Successful in 3m53s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m28s
Test / Flake checks (push) Successful in 1m22s
Required by hatchling, which is required by many python packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 20:13:26 +09:00
cat 021cbbc2a8 cmd/mbf: default daemon socket in cache
Test / Create distribution (push) Successful in 1m7s
Test / Sandbox (push) Successful in 2m52s
Test / Hakurei (push) Successful in 3m53s
Test / ShareFS (push) Successful in 4m1s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 6m26s
Test / Flake checks (push) Successful in 1m23s
This location makes more sense than the current directory.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:50:54 +09:00
cat a4a54a4a4d cmd/mbf: remove pointless recover
Test / Create distribution (push) Successful in 37s
Test / Sandbox (push) Successful in 1m43s
Test / Hakurei (push) Successful in 2m46s
Test / ShareFS (push) Successful in 2m44s
Test / Sandbox (race detector) (push) Successful in 5m25s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m20s
This used to scrub the cache, and was not fully removed when that became nonviable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:49:01 +09:00
cat 04a344aac6 internal/rosa/python: flirt_core artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 3m42s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m26s
Test / Flake checks (push) Successful in 1m23s
A build system required by a dependency of another build system, which is required by yet another build system.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 19:25:04 +09:00
cat 6b98156a3d internal/rosa/python: change insane strict_timestamps default
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m46s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m25s
Test / Flake checks (push) Successful in 1m20s
There is no scenario where this is useful, and it breaks builds.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 18:56:22 +09:00
cat 753432cf09 cmd/mbf: optionally wait for cancel
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m57s
Test / ShareFS (push) Successful in 3m57s
Test / Sandbox (race detector) (push) Successful in 5m36s
Test / Hakurei (race detector) (push) Successful in 9m26s
Test / Flake checks (push) Successful in 2m12s
Synchronisation is not needed here during interactive use.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 18:24:11 +09:00
cat f8902e3679 internal/rosa/python: append to source path
Test / Create distribution (push) Successful in 1m10s
Test / Sandbox (push) Successful in 2m55s
Test / Hakurei (push) Successful in 4m1s
Test / ShareFS (push) Successful in 4m5s
Test / Sandbox (race detector) (push) Successful in 5m26s
Test / Hakurei (race detector) (push) Successful in 6m34s
Test / Flake checks (push) Successful in 1m22s
This gets around messy projects with multiple packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 17:51:00 +09:00
cat 8ee53a5164 internal/rosa: use builtin for checksum warning
Test / Create distribution (push) Successful in 40s
Test / Sandbox (push) Successful in 2m49s
Test / Hakurei (push) Successful in 3m57s
Test / ShareFS (push) Successful in 3m59s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (race detector) (push) Successful in 6m30s
Test / Flake checks (push) Successful in 1m21s
This avoids having to configure the logger early.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 17:50:12 +09:00
cat 3981d44757 internal/rosa/python: migrate setuptools to wrapper
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m47s
Test / Hakurei (push) Successful in 3m56s
Test / ShareFS (push) Successful in 4m4s
Test / Sandbox (race detector) (push) Successful in 6m2s
Test / Hakurei (race detector) (push) Successful in 7m54s
Test / Flake checks (push) Successful in 2m9s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:36:43 +09:00
cat 9fd67e47b4 internal/rosa/python: wrap python package
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m47s
Test / Hakurei (push) Successful in 3m49s
Test / ShareFS (push) Successful in 3m52s
Test / Sandbox (race detector) (push) Successful in 5m13s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Flake checks (push) Successful in 1m22s
Metadata for this is somewhat boilerplate-heavy, so wrap it to create metadata in one call.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:22:18 +09:00
cat 4dcec40156 cmd/mbf: close on cancel completion
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m55s
Test / Hakurei (push) Successful in 3m52s
Test / ShareFS (push) Successful in 3m57s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m29s
Test / Flake checks (push) Successful in 1m21s
Like the previous change, this enables synchronisation on the client side via epoll.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 15:03:52 +09:00
cat 9a274c78a3 cmd/mbf: close on abort completion
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m49s
Test / Hakurei (push) Successful in 3m46s
Test / ShareFS (push) Successful in 3m50s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Flake checks (push) Successful in 1m19s
This enables synchronisation on the client side via epoll.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 14:53:28 +09:00
cat 5647c3a91f internal/rosa/meson: run meson test suite
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 3m35s
Test / Hakurei (push) Successful in 4m38s
Test / ShareFS (push) Successful in 4m49s
Test / Sandbox (race detector) (push) Successful in 5m53s
Test / Hakurei (race detector) (push) Successful in 6m56s
Test / Flake checks (push) Successful in 1m16s
Tests requiring internet access or unreasonable dependencies are removed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 01:07:20 +09:00
cat 992139c75d internal/rosa/python: extra script after install
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m49s
Test / Hakurei (push) Successful in 3m47s
Test / ShareFS (push) Successful in 3m52s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Flake checks (push) Successful in 1m21s
This is generally for test suite, due to the lack of standard or widely agreed upon convention.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:35:24 +09:00
cat 57c69b533e internal/rosa/meson: migrate to helper
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m44s
Test / ShareFS (push) Successful in 3m47s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 6m25s
Test / Hakurei (push) Successful in 2m51s
Test / Flake checks (push) Successful in 1m21s
This also migrates to source from the Microsoft Github release.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:16:22 +09:00
cat 6f0c2a80f2 internal/rosa/python: migrate setuptools to helper
Test / Create distribution (push) Successful in 1m33s
Test / Sandbox (push) Successful in 3m18s
Test / Hakurei (push) Successful in 4m25s
Test / ShareFS (push) Successful in 4m29s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 6m54s
Test / Flake checks (push) Successful in 1m24s
This is much cleaner, and should be functionally equivalent.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:04:19 +09:00
cat 08dfefb28d internal/rosa/python: pip helper
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 3m12s
Test / ShareFS (push) Successful in 4m25s
Test / Sandbox (race detector) (push) Successful in 5m44s
Test / Hakurei (race detector) (push) Successful in 6m55s
Test / Hakurei (push) Successful in 2m39s
Test / Flake checks (push) Successful in 1m23s
Binary pip releases are not considered acceptable, this more generic helper is required for building from source.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-19 00:03:36 +09:00
cat b081629662 internal/rosa/libxml2: 2.15.2 to 2.15.3
Test / Create distribution (push) Successful in 1m11s
Test / Sandbox (push) Successful in 3m3s
Test / ShareFS (push) Successful in 4m34s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 6m58s
Test / Hakurei (push) Successful in 2m27s
Test / Flake checks (push) Successful in 1m15s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:05:49 +09:00
cat fba541f301 internal/rosa/nss: 3.122.1 to 3.123
Test / Sandbox (push) Successful in 2m1s
Test / Create distribution (push) Successful in 1m1s
Test / Hakurei (push) Successful in 4m36s
Test / ShareFS (push) Successful in 3m3s
Test / Sandbox (race detector) (push) Successful in 5m19s
Test / Hakurei (race detector) (push) Successful in 6m43s
Test / Flake checks (push) Successful in 1m22s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:05:23 +09:00
cat 5f0da3d5c2 internal/rosa/gnu: mpc 1.4.0 to 1.4.1
Test / Create distribution (push) Successful in 36s
Test / Sandbox (push) Successful in 3m5s
Test / Hakurei (push) Successful in 4m22s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 7m2s
Test / Flake checks (push) Successful in 1m24s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:04:33 +09:00
cat 4d5841dd62 internal/rosa: elfutils 0.194 to 0.195
Test / Create distribution (push) Successful in 44s
Test / Sandbox (push) Successful in 3m4s
Test / ShareFS (push) Successful in 4m32s
Test / Sandbox (race detector) (push) Successful in 5m41s
Test / Hakurei (race detector) (push) Successful in 6m53s
Test / Hakurei (push) Successful in 2m31s
Test / Flake checks (push) Successful in 1m20s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 09:03:49 +09:00
cat 9e752b588a internal/pkg: drop cached error on cancel
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m47s
Test / ShareFS (push) Successful in 3m54s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m23s
This avoids disabling the artifact when using the individual cancel method. Unfortunately this makes the method blocking.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 03:24:48 +09:00
cat 27b1aaae38 internal/pkg: pending error alongside done channel
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m44s
Test / ShareFS (push) Successful in 3m48s
Test / Hakurei (push) Successful in 3m52s
Test / Sandbox (race detector) (push) Successful in 5m21s
Test / Hakurei (race detector) (push) Successful in 6m28s
Test / Flake checks (push) Successful in 1m20s
This significantly simplifies synchronisation of access to identErr.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 03:10:37 +09:00
cat 9e18de1dc2 internal/pkg: flush cached errors on abort
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m43s
Test / Hakurei (push) Successful in 3m49s
Test / ShareFS (push) Successful in 3m48s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m17s
Test / Flake checks (push) Successful in 1m17s
This avoids disabling the artifact until cache is reopened. The same has to be implemented for Cancel in a future change.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-18 02:59:44 +09:00
cat b80ea91a42 cmd/mbf: abort remote cures
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m47s
Test / ShareFS (push) Successful in 3m50s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m15s
This command arranges for all pending cures to be aborted. It does not wait for cures to complete.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:47:02 +09:00
cat 30a9dfa4b8 internal/pkg: abort all pending cures
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m53s
Test / Sandbox (race detector) (push) Successful in 5m27s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m31s
This cancels all current pending cures without closing the cache.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:40:35 +09:00
cat 8d657b6fdf cmd/mbf: cancel remote cure
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m43s
Test / Hakurei (push) Successful in 3m49s
Test / ShareFS (push) Successful in 3m50s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m19s
Test / Flake checks (push) Successful in 1m22s
This exposes the new fine-grained cancel API in cmd/mbf.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 22:00:04 +09:00
cat ae9b9adfd2 internal/rosa: retry in SIGSEGV test
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m47s
Test / Hakurei (push) Successful in 3m46s
Test / ShareFS (push) Successful in 3m48s
Test / Sandbox (race detector) (push) Successful in 5m10s
Test / Hakurei (race detector) (push) Successful in 6m19s
Test / Flake checks (push) Successful in 1m21s
Munmap is not always immediate.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 20:45:19 +09:00
cat dd6a480a21 cmd/mbf: handle flags in serve
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m47s
Test / ShareFS (push) Successful in 3m50s
Test / Sandbox (race detector) (push) Successful in 5m13s
Test / Hakurei (race detector) (push) Successful in 6m27s
Test / Hakurei (push) Successful in 2m38s
Test / Flake checks (push) Successful in 1m21s
This enables easier expansion of the protocol.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 20:14:09 +09:00
cat 3942272c30 internal/pkg: fine-grained cancellation
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m46s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m16s
Test / Hakurei (race detector) (push) Successful in 6m34s
Test / Flake checks (push) Successful in 1m21s
This enables a specific artifact to be targeted for cancellation.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 19:33:21 +09:00
cat 9036986156 cmd/mbf: optionally ignore reply
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m57s
Test / Hakurei (push) Successful in 3m52s
Test / ShareFS (push) Successful in 4m58s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m29s
Test / Flake checks (push) Successful in 1m47s
An acknowledgement is not always required in this use case. This change also adds 64 bits of connection configuration for future expansion.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 16:46:49 +09:00
cat a394971dd7 cmd/mbf: do not abort cache acquisition during testing
Test / Create distribution (push) Successful in 1m15s
Test / Sandbox (push) Successful in 2m59s
Test / Hakurei (push) Successful in 3m58s
Test / ShareFS (push) Successful in 4m7s
Test / Sandbox (race detector) (push) Successful in 5m29s
Test / Hakurei (race detector) (push) Successful in 6m32s
Test / Flake checks (push) Successful in 1m22s
This can sometimes fire during testing due to how short the test is.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 02:06:51 +09:00
cat 9daba60809 cmd/mbf: daemon command
Test / Create distribution (push) Successful in 1m9s
Test / Sandbox (push) Successful in 2m49s
Test / ShareFS (push) Successful in 4m0s
Test / Sandbox (race detector) (push) Successful in 5m25s
Test / Hakurei (race detector) (push) Successful in 6m27s
Test / Hakurei (push) Successful in 3m53s
Test / Flake checks (push) Successful in 1m25s
This services internal/pkg artifact IR with Rosa OS extensions originating from another process.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 02:05:59 +09:00
cat bcd79a22ff cmd/mbf: do not open cache for IR encoding
Test / Create distribution (push) Successful in 1m7s
Test / Sandbox (push) Successful in 2m52s
Test / Hakurei (push) Successful in 3m55s
Test / ShareFS (push) Successful in 3m56s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m28s
Test / Flake checks (push) Successful in 1m24s
This can now be allocated independently.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 01:04:39 +09:00
cat 0ff7ab915b internal/pkg: move IR primitives out of cache
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 51s
Test / Hakurei (push) Successful in 2m43s
Test / ShareFS (push) Successful in 2m39s
Test / Sandbox (race detector) (push) Successful in 5m17s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m24s
These are memory management and caching primitives. Having them as part of Cache is cumbersome and requires a temporary directory that is never used. This change isolates them from Cache to enable independent use.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-17 01:02:13 +09:00
cat 823575acac cmd/mbf: move info command
Test / Create distribution (push) Successful in 1m0s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m47s
Test / ShareFS (push) Successful in 3m44s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m19s
This is cleaner with less shared state.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-16 17:43:52 +09:00
cat 136bc0917b cmd/mbf: optionally open cache
Test / Sandbox (push) Successful in 3m13s
Test / Create distribution (push) Successful in 1m1s
Test / ShareFS (push) Successful in 2m42s
Test / Hakurei (push) Successful in 2m49s
Test / Sandbox (race detector) (push) Successful in 5m6s
Test / Hakurei (race detector) (push) Successful in 6m35s
Test / Flake checks (push) Successful in 1m27s
Some commands do not require the cache. This change also makes acquisition of locked cache cancelable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-16 15:59:34 +09:00
cat d6b082dd0b internal/rosa/ninja: bootstrap with verbose output
Test / ShareFS (push) Successful in 41s
Test / Sandbox (push) Successful in 47s
Test / Hakurei (push) Successful in 51s
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (race detector) (push) Successful in 2m33s
Test / Hakurei (race detector) (push) Successful in 3m45s
Test / Flake checks (push) Successful in 2m28s
This otherwise outputs nothing, and appears to hang until the (fully single-threaded) bootstrap completes.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:19:05 +09:00
cat 89d6d9576b internal/rosa/make: optionally format value as is
Test / Create distribution (push) Successful in 1m22s
Test / ShareFS (push) Successful in 4m37s
Test / Sandbox (race detector) (push) Successful in 6m27s
Test / Hakurei (race detector) (push) Successful in 11m35s
Test / Sandbox (push) Successful in 3m51s
Test / Hakurei (push) Successful in 6m59s
Test / Flake checks (push) Successful in 3m5s
This enables correct formatting for awkward configure scripts.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:17:58 +09:00
cat fafce04a5d internal/rosa/kernel: firmware 20260309 to 20260410
Test / Create distribution (push) Successful in 1m42s
Test / Sandbox (push) Successful in 4m43s
Test / Hakurei (push) Successful in 9m0s
Test / ShareFS (push) Successful in 6m37s
Test / Sandbox (race detector) (push) Successful in 7m41s
Test / Hakurei (race detector) (push) Successful in 11m49s
Test / Flake checks (push) Successful in 3m23s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:16:47 +09:00
cat 5d760a1db9 internal/rosa/kernel: 6.12.80 to 6.12.81
Test / Create distribution (push) Successful in 3m56s
Test / Sandbox (push) Successful in 6m45s
Test / Sandbox (race detector) (push) Successful in 8m8s
Test / Hakurei (push) Successful in 9m17s
Test / ShareFS (push) Successful in 7m49s
Test / Hakurei (race detector) (push) Successful in 11m30s
Test / Flake checks (push) Successful in 2m26s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:16:30 +09:00
cat d197e40b2a internal/rosa/python: mako 1.3.10 to 1.3.11
Test / Create distribution (push) Successful in 1m9s
Test / ShareFS (push) Successful in 8m17s
Test / Sandbox (race detector) (push) Successful in 8m23s
Test / Hakurei (race detector) (push) Successful in 11m2s
Test / Sandbox (push) Successful in 3m54s
Test / Hakurei (push) Successful in 6m57s
Test / Flake checks (push) Successful in 3m15s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:21:54 +09:00
cat 2008902247 internal/rosa/python: packaging 26.0 to 26.1
Test / Create distribution (push) Successful in 1m7s
Test / Sandbox (push) Successful in 2m58s
Test / Hakurei (push) Successful in 5m25s
Test / ShareFS (push) Successful in 6m57s
Test / Sandbox (race detector) (push) Successful in 3m28s
Test / Hakurei (race detector) (push) Successful in 6m50s
Test / Flake checks (push) Successful in 3m20s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:15:18 +09:00
cat 30ac985fd2 internal/rosa/meson: 1.10.2 to 1.11.0
Test / Create distribution (push) Successful in 1m7s
Test / Sandbox (push) Successful in 3m0s
Test / Hakurei (push) Successful in 4m52s
Test / ShareFS (push) Successful in 3m42s
Test / Sandbox (race detector) (push) Successful in 5m35s
Test / Hakurei (race detector) (push) Successful in 11m4s
Test / Flake checks (push) Successful in 2m25s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:14:52 +09:00
cat e9fec368f8 internal/rosa/nss: 3.122 to 3.122.1
Test / Create distribution (push) Successful in 37s
Test / Sandbox (push) Successful in 2m55s
Test / ShareFS (push) Successful in 3m58s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 7m5s
Test / Hakurei (push) Successful in 4m35s
Test / Flake checks (push) Successful in 3m29s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:13:45 +09:00
cat 46add42f58 internal/rosa/openssl: disable building docs
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 3m8s
Test / Hakurei (push) Successful in 4m28s
Test / ShareFS (push) Successful in 4m48s
Test / Sandbox (race detector) (push) Successful in 5m58s
Test / Hakurei (race detector) (push) Successful in 7m12s
Test / Flake checks (push) Successful in 2m25s
These take very long and are never used in the Rosa OS environment.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:13:18 +09:00
cat 377b61e342 internal/rosa/openssl: do not double test job count
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 4m16s
Test / ShareFS (push) Successful in 4m25s
Test / Sandbox (race detector) (push) Successful in 5m40s
Test / Hakurei (race detector) (push) Successful in 7m3s
Test / Flake checks (push) Successful in 2m23s
The test suite is racy, this reduces flakiness.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 22:12:36 +09:00
cat 520c36db6d internal/rosa: respect preferred job count
Test / Create distribution (push) Successful in 3m52s
Test / Sandbox (push) Successful in 9m15s
Test / ShareFS (push) Successful in 14m28s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (push) Successful in 8m11s
Test / Hakurei (race detector) (push) Successful in 9m23s
Test / Flake checks (push) Successful in 3m23s
This discontinues use of nproc, and also overrides detection behaviour in ninja.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 18:49:36 +09:00
cat 3352bb975b internal/pkg: job count in container environment
Test / Create distribution (push) Successful in 2m6s
Test / Sandbox (push) Successful in 4m51s
Test / Hakurei (push) Successful in 6m5s
Test / ShareFS (push) Successful in 6m16s
Test / Sandbox (race detector) (push) Successful in 7m15s
Test / Hakurei (race detector) (push) Successful in 8m19s
Test / Flake checks (push) Successful in 2m7s
This exposes preferred job count to the container initial process.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 15:49:21 +09:00
cat f7f48d57e9 internal/pkg: pass impure job count
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m58s
Test / Hakurei (push) Successful in 3m56s
Test / ShareFS (push) Successful in 4m6s
Test / Sandbox (race detector) (push) Successful in 5m24s
Test / Hakurei (race detector) (push) Successful in 6m49s
Test / Flake checks (push) Successful in 1m27s
This is cleaner than checking cpu count during cure, it is impossible to avoid impurity in both situations but this is configurable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-15 15:36:44 +09:00
cat 5c2345128e internal/rosa/llvm: autodetect stage0 target
Test / Create distribution (push) Successful in 1m34s
Test / Sandbox (push) Successful in 4m11s
Test / Hakurei (push) Successful in 5m26s
Test / ShareFS (push) Successful in 5m35s
Test / Sandbox (race detector) (push) Successful in 6m37s
Test / Hakurei (race detector) (push) Successful in 7m49s
Test / Flake checks (push) Successful in 2m26s
This is fine, now that stages beyond stage0 have explicit target.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 03:10:26 +09:00
cat 78f9676b1f internal/rosa/llvm: centralise llvm source
Test / Create distribution (push) Successful in 2m25s
Test / Sandbox (push) Successful in 4m36s
Test / ShareFS (push) Successful in 6m11s
Test / Sandbox (race detector) (push) Successful in 7m20s
Test / Hakurei (race detector) (push) Successful in 11m45s
Test / Hakurei (push) Successful in 4m5s
Test / Flake checks (push) Successful in 1m41s
This avoids having to sidestep the NewPackage name formatting machinery to take the cache fast path.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 03:03:06 +09:00
cat 5b5b676132 internal/rosa/cmake: remove variant
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (race detector) (push) Successful in 9m30s
Test / ShareFS (push) Successful in 9m35s
Test / Sandbox (push) Successful in 3m6s
Test / Hakurei (push) Successful in 4m24s
Test / Hakurei (race detector) (push) Successful in 5m23s
Test / Flake checks (push) Successful in 1m35s
This has no effect outside formatting of name and is a remnant of the old llvm helpers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:57:47 +09:00
cat 78383fb6e8 internal/rosa/llvm: migrate libclc
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m20s
Test / Hakurei (push) Successful in 4m32s
Test / ShareFS (push) Successful in 4m42s
Test / Sandbox (race detector) (push) Successful in 5m55s
Test / Hakurei (race detector) (push) Successful in 7m3s
Test / Flake checks (push) Successful in 1m27s
This eliminates newLLVMVariant.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:40:13 +09:00
cat e97f6a393f internal/rosa/llvm: migrate runtimes and clang
Test / Create distribution (push) Successful in 1m56s
Test / ShareFS (push) Successful in 8m1s
Test / Sandbox (race detector) (push) Successful in 8m31s
Test / Hakurei (race detector) (push) Successful in 9m57s
Test / Sandbox (push) Successful in 2m13s
Test / Hakurei (push) Successful in 3m17s
Test / Flake checks (push) Successful in 1m38s
This eliminates most newLLVM family of functions.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 02:07:13 +09:00
cat eeffefd22b internal/rosa/llvm: migrate compiler-rt helper
Test / Create distribution (push) Successful in 1m16s
Test / Sandbox (push) Successful in 3m23s
Test / Hakurei (push) Successful in 4m29s
Test / ShareFS (push) Successful in 4m34s
Test / Sandbox (race detector) (push) Successful in 5m51s
Test / Hakurei (race detector) (push) Successful in 6m58s
Test / Flake checks (push) Successful in 1m26s
This also removes unused dependencies.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 01:12:56 +09:00
cat ac825640ab internal/rosa/llvm: migrate musl
Test / Create distribution (push) Successful in 2m48s
Test / Sandbox (push) Successful in 6m4s
Test / Hakurei (push) Successful in 7m14s
Test / ShareFS (push) Successful in 7m23s
Test / Sandbox (race detector) (push) Successful in 9m34s
Test / Hakurei (race detector) (push) Successful in 4m2s
Test / Flake checks (push) Successful in 1m33s
This removes the pointless special treatment given to musl.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:35:42 +09:00
cat a7f7ce1795 internal/rosa/llvm: migrate compiler-rt
Test / Create distribution (push) Successful in 1m24s
Test / Sandbox (push) Successful in 3m33s
Test / Hakurei (push) Successful in 6m17s
Test / ShareFS (push) Successful in 6m14s
Test / Hakurei (race detector) (push) Successful in 8m34s
Test / Sandbox (race detector) (push) Successful in 5m39s
Test / Flake checks (push) Successful in 1m40s
The newLLVM family of functions predate the package system. This change migrates compiler-rt without changing any resulting artifacts.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:19:33 +09:00
cat 38c639e35c internal/rosa/llvm: remove project/runtime helper
Test / Create distribution (push) Successful in 2m48s
Test / Sandbox (push) Successful in 7m21s
Test / ShareFS (push) Successful in 8m34s
Test / Sandbox (race detector) (push) Successful in 9m6s
Test / Hakurei (race detector) (push) Successful in 10m14s
Test / Hakurei (push) Successful in 3m30s
Test / Flake checks (push) Successful in 2m54s
More remnants from early days, these are not reusable at all but that was not known at the time.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-14 00:03:23 +09:00
cat b2cb13e94c internal/rosa/llvm: centralise patches
Test / Create distribution (push) Successful in 1m18s
Test / Sandbox (push) Successful in 3m19s
Test / Hakurei (push) Successful in 4m19s
Test / ShareFS (push) Successful in 4m30s
Test / Sandbox (race detector) (push) Successful in 5m48s
Test / Hakurei (race detector) (push) Successful in 8m20s
Test / Flake checks (push) Successful in 4m32s
This enables easier reuse of the patchset.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 23:52:44 +09:00
cat 46f98d12d6 internal/rosa/llvm: remove conditional flags in helper
Test / Create distribution (push) Successful in 1m15s
Test / Sandbox (push) Successful in 3m13s
Test / Hakurei (push) Successful in 4m17s
Test / ShareFS (push) Successful in 4m27s
Test / Sandbox (race detector) (push) Successful in 5m41s
Test / Hakurei (race detector) (push) Successful in 6m50s
Test / Flake checks (push) Successful in 1m27s
The llvm helper is a remnant from very early days, and ended up not being very useful, but was never removed. This change begins its removal, without changing the resulting artifacts for now.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 23:38:11 +09:00
cat 503c7f953c internal/rosa/x: libpciaccess artifact
Test / Create distribution (push) Successful in 1m17s
Test / Sandbox (push) Successful in 3m19s
Test / Hakurei (push) Successful in 7m51s
Test / ShareFS (push) Successful in 8m40s
Test / Sandbox (race detector) (push) Successful in 4m32s
Test / Hakurei (race detector) (push) Successful in 4m14s
Test / Flake checks (push) Successful in 1m34s
Required by userspace gpu drivers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 19:04:38 +09:00
cat 15c9f6545d internal/rosa/perl: populate anitya identifiers
Test / Create distribution (push) Successful in 1m23s
Test / Sandbox (push) Successful in 3m27s
Test / Hakurei (push) Successful in 4m29s
Test / ShareFS (push) Successful in 4m37s
Test / Sandbox (race detector) (push) Successful in 5m59s
Test / Hakurei (race detector) (push) Successful in 7m3s
Test / Flake checks (push) Successful in 1m28s
These are also tracked by Anitya.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 18:44:43 +09:00
cat 83b0e32c55 internal/rosa: helpers for common url formats
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m16s
Test / ShareFS (push) Successful in 4m39s
Test / Sandbox (race detector) (push) Successful in 5m49s
Test / Hakurei (race detector) (push) Successful in 6m59s
Test / Hakurei (push) Successful in 3m13s
Test / Flake checks (push) Successful in 1m37s
This cleans up call site of NewPackage.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 18:02:57 +09:00
cat eeaf26e7a2 internal/rosa: wrapper around git helper
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m46s
Test / Hakurei (push) Successful in 3m43s
Test / ShareFS (push) Successful in 3m50s
Test / Sandbox (race detector) (push) Successful in 5m21s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m23s
This results in much cleaner call site for the majority of use cases.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 15:20:51 +09:00
cat b587caf2e8 internal/rosa: assume file source is xz-compressed
Test / Create distribution (push) Successful in 1m4s
Test / Sandbox (push) Successful in 2m51s
Test / Hakurei (push) Successful in 3m54s
Test / ShareFS (push) Successful in 3m55s
Test / Sandbox (race detector) (push) Successful in 5m15s
Test / Hakurei (race detector) (push) Successful in 6m20s
Test / Flake checks (push) Successful in 1m22s
XZ happens to be the only widely-used format that is awful to deal with, everything else is natively supported.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 15:07:30 +09:00
cat f1c2ca4928 internal/rosa/mesa: libdrm artifact
Test / Create distribution (push) Successful in 1m23s
Test / Sandbox (push) Successful in 3m21s
Test / Hakurei (push) Successful in 4m34s
Test / ShareFS (push) Successful in 4m41s
Test / Sandbox (race detector) (push) Successful in 6m0s
Test / Hakurei (race detector) (push) Successful in 7m3s
Test / Flake checks (push) Successful in 1m29s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:27:09 +09:00
cat 0ca301219f internal/rosa/python: pyyaml artifact
Test / Create distribution (push) Successful in 1m39s
Test / Sandbox (push) Successful in 4m9s
Test / Hakurei (push) Successful in 5m59s
Test / ShareFS (push) Successful in 6m2s
Test / Sandbox (race detector) (push) Successful in 7m0s
Test / Hakurei (race detector) (push) Successful in 8m19s
Test / Flake checks (push) Successful in 1m54s
Mesa unfortunately requires this horrible format.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:18:47 +09:00
cat e2199e1276 internal/rosa/python: mako artifact
Test / Create distribution (push) Successful in 1m42s
Test / Sandbox (push) Successful in 4m3s
Test / Hakurei (push) Successful in 5m56s
Test / ShareFS (push) Successful in 6m2s
Test / Sandbox (race detector) (push) Successful in 6m59s
Test / Hakurei (race detector) (push) Successful in 8m18s
Test / Flake checks (push) Successful in 1m53s
This unfortunately pulls in platform-specific package.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:11:38 +09:00
cat 86eacb3208 cmd/mbf: checksum command
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m16s
Test / Hakurei (push) Successful in 4m36s
Test / ShareFS (push) Successful in 4m43s
Test / Sandbox (race detector) (push) Successful in 5m55s
Test / Hakurei (race detector) (push) Successful in 7m11s
Test / Flake checks (push) Successful in 1m31s
This computes and encodes sha384 checksum of data streamed from standard input.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 03:09:21 +09:00
cat 8541bdd858 internal/rosa: wrap per-arch values
Test / Create distribution (push) Successful in 1m16s
Test / Sandbox (push) Successful in 3m12s
Test / Hakurei (push) Successful in 4m35s
Test / ShareFS (push) Successful in 4m38s
Test / Sandbox (race detector) (push) Successful in 5m52s
Test / Hakurei (race detector) (push) Successful in 7m9s
Test / Flake checks (push) Successful in 1m31s
This is cleaner syntax in some specific cases.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:59:55 +09:00
cat 46be0b0dc8 internal/rosa/nss: buildcatrust 0.4.0 to 0.5.1
Test / Create distribution (push) Successful in 1m51s
Test / Sandbox (push) Successful in 7m29s
Test / ShareFS (push) Successful in 9m35s
Test / Sandbox (race detector) (push) Successful in 9m52s
Test / Hakurei (race detector) (push) Successful in 4m13s
Test / Hakurei (push) Successful in 2m53s
Test / Flake checks (push) Successful in 1m31s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:18:21 +09:00
cat cbe37e87e7 internal/rosa/python: pytest 9.0.2 to 9.0.3
Test / Create distribution (push) Successful in 1m27s
Test / ShareFS (push) Successful in 7m24s
Test / Sandbox (race detector) (push) Successful in 8m9s
Test / Hakurei (race detector) (push) Successful in 11m13s
Test / Sandbox (push) Successful in 2m57s
Test / Hakurei (push) Successful in 4m20s
Test / Flake checks (push) Successful in 1m36s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:18:02 +09:00
cat 66d741fb07 internal/rosa/python: pygments 2.19.2 to 2.20.0
Test / Create distribution (push) Successful in 2m13s
Test / ShareFS (push) Successful in 9m37s
Test / Sandbox (race detector) (push) Successful in 9m41s
Test / Hakurei (race detector) (push) Successful in 11m21s
Test / Sandbox (push) Successful in 2m40s
Test / Hakurei (push) Successful in 4m2s
Test / Flake checks (push) Successful in 1m37s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:13:04 +09:00
cat 0d449011f6 internal/rosa/python: use predictable URLs
Test / Create distribution (push) Successful in 1m18s
Test / Sandbox (push) Successful in 3m25s
Test / Hakurei (push) Successful in 5m54s
Test / ShareFS (push) Successful in 5m53s
Test / Sandbox (race detector) (push) Successful in 7m30s
Test / Hakurei (race detector) (push) Successful in 9m6s
Test / Flake checks (push) Successful in 2m53s
This is much cleaner and more maintainable than specifying URL prefix manually. This change also populates Anitya project identifiers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 02:08:22 +09:00
cat 46428ed85d internal/rosa/python: url pip wheel helper
Test / Create distribution (push) Successful in 1m36s
Test / Sandbox (push) Successful in 3m34s
Test / Hakurei (push) Successful in 4m54s
Test / ShareFS (push) Successful in 4m55s
Test / Sandbox (race detector) (push) Successful in 6m9s
Test / Hakurei (race detector) (push) Successful in 4m44s
Test / Flake checks (push) Successful in 3m0s
This enables a cleaner higher-level helper.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-13 01:59:28 +09:00
cat 081d6b463c internal/rosa/llvm: libclc artifact
Test / Create distribution (push) Successful in 1m6s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 3m51s
Test / ShareFS (push) Successful in 3m54s
Test / Sandbox (race detector) (push) Successful in 5m19s
Test / Hakurei (race detector) (push) Successful in 6m21s
Test / Flake checks (push) Successful in 1m24s
This is built independently of llvm build system to avoid having to build llvm again.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:57:04 +09:00
cat 11b3171180 internal/rosa/glslang: glslang artifact
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m56s
Test / ShareFS (push) Successful in 3m50s
Test / Hakurei (push) Successful in 4m3s
Test / Sandbox (race detector) (push) Successful in 5m12s
Test / Hakurei (race detector) (push) Successful in 6m20s
Test / Flake checks (push) Successful in 1m21s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:34:17 +09:00
cat adbb84c3dd internal/rosa/glslang: spirv-tools artifact
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m47s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m20s
Test / Hakurei (race detector) (push) Successful in 6m24s
Test / Flake checks (push) Successful in 1m31s
Required by glslang.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:27:49 +09:00
cat 1084e31d95 internal/rosa/glslang: spirv-headers artifact
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m42s
Test / Hakurei (push) Successful in 3m45s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (race detector) (push) Successful in 6m25s
Test / Flake checks (push) Successful in 1m21s
Required by spirv-tools.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 22:16:29 +09:00
cat 27a1b8fe0a internal/rosa/mesa: libglvnd artifact
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m51s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 5m14s
Test / Hakurei (race detector) (push) Successful in 6m18s
Test / Flake checks (push) Successful in 1m22s
Required by mesa.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 21:27:30 +09:00
cat b2141a41d7 internal/rosa/dbus: xdg-dbus-proxy artifact
Test / Create distribution (push) Successful in 1m13s
Test / Sandbox (push) Successful in 3m0s
Test / Hakurei (push) Successful in 4m8s
Test / ShareFS (push) Successful in 4m19s
Test / Sandbox (race detector) (push) Successful in 5m31s
Test / Hakurei (race detector) (push) Successful in 6m35s
Test / Flake checks (push) Successful in 1m22s
This is currently a hakurei runtime dependency, but will eventually be removed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 19:41:49 +09:00
cat c0dff5bc87 internal/rosa/gnu: gcc set with-multilib-list as needed
Test / Create distribution (push) Successful in 1m22s
Test / Sandbox (push) Successful in 3m19s
Test / Hakurei (push) Successful in 6m57s
Test / ShareFS (push) Successful in 6m54s
Test / Sandbox (race detector) (push) Successful in 7m38s
Test / Hakurei (race detector) (push) Successful in 8m50s
Test / Flake checks (push) Successful in 2m8s
This breaks riscv64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 18:03:45 +09:00
cat 04513c0510 internal/rosa/gnu: gmp explicit CC
Test / Create distribution (push) Successful in 1m25s
Test / Sandbox (push) Successful in 3m20s
Test / Hakurei (push) Successful in 4m26s
Test / ShareFS (push) Successful in 4m23s
Test / Sandbox (race detector) (push) Successful in 5m50s
Test / Hakurei (race detector) (push) Successful in 6m51s
Test / Flake checks (push) Successful in 1m54s
The configure script is hard coded to use gcc without fallback on riscv64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-12 17:25:15 +09:00
cat 28ebf973d6 nix: add sharefs supplementary group
Test / Sandbox (push) Successful in 1m1s
Test / Hakurei (push) Successful in 1m5s
Test / Sandbox (race detector) (push) Successful in 1m2s
Test / Hakurei (race detector) (push) Successful in 1m10s
Test / Create distribution (push) Successful in 1m19s
Test / ShareFS (push) Successful in 3m6s
Test / Flake checks (push) Successful in 1m33s
This works around vfs inode file attribute race.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 23:28:18 +09:00
cat 41aeb404ec internal/rosa/hakurei: 0.3.7 to 0.4.0
Test / Create distribution (push) Successful in 1m16s
Test / Sandbox (push) Successful in 3m9s
Test / Hakurei (push) Successful in 4m27s
Test / ShareFS (push) Successful in 4m32s
Test / Sandbox (race detector) (push) Successful in 5m37s
Test / Hakurei (race detector) (push) Successful in 6m49s
Test / Flake checks (push) Successful in 1m28s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 10:53:29 +09:00
cat 0b1009786f release: 0.4.0
Test / Create distribution (push) Successful in 37s
Release / Create release (push) Successful in 45s
Test / ShareFS (push) Successful in 43s
Test / Sandbox (push) Successful in 48s
Test / Hakurei (push) Successful in 52s
Test / Sandbox (race detector) (push) Successful in 48s
Test / Hakurei (race detector) (push) Successful in 55s
Test / Flake checks (push) Successful in 1m28s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-11 10:49:43 +09:00
cat b390640376 internal/landlock: relocate from package container
Test / Create distribution (push) Successful in 35s
Test / ShareFS (push) Successful in 39s
Test / Sandbox (push) Successful in 45s
Test / Hakurei (push) Successful in 50s
Test / Sandbox (race detector) (push) Successful in 45s
Test / Hakurei (race detector) (push) Successful in 50s
Test / Flake checks (push) Successful in 1m21s
This is not possible to use directly, so remove it from the public API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 23:56:45 +09:00
cat ad2c9f36cd container: unexport PR_SET_NO_NEW_PRIVS wrapper
Test / Create distribution (push) Successful in 1m1s
Test / Sandbox (push) Successful in 2m46s
Test / Hakurei (push) Successful in 3m52s
Test / ShareFS (push) Successful in 3m47s
Test / Sandbox (race detector) (push) Successful in 5m6s
Test / Hakurei (race detector) (push) Successful in 6m16s
Test / Flake checks (push) Successful in 1m23s
This is subtle to use correctly. It also does not make sense as part of the container API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 23:45:51 +09:00
cat 67db3fbb8d check: use encoding interfaces
Test / Create distribution (push) Successful in 1m13s
Test / Sandbox (push) Successful in 3m5s
Test / Hakurei (push) Successful in 4m15s
Test / ShareFS (push) Successful in 4m16s
Test / Sandbox (race detector) (push) Successful in 5m34s
Test / Hakurei (race detector) (push) Successful in 6m41s
Test / Flake checks (push) Successful in 1m25s
This turned out not to require specific treatment, so the shared interfaces are cleaner.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 22:11:53 +09:00
cat 560cb626a1 hst: remove enablement json adapter
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m46s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m11s
Test / Hakurei (race detector) (push) Successful in 6m20s
Test / Flake checks (push) Successful in 1m23s
The go116 behaviour of built-in new function makes this cleaner.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 20:47:30 +09:00
cat c33a6a5b7e hst: optionally reject insecure options
Test / Create distribution (push) Successful in 1m31s
Test / Sandbox (push) Successful in 3m35s
Test / Hakurei (push) Successful in 4m43s
Test / ShareFS (push) Successful in 4m49s
Test / Sandbox (race detector) (push) Successful in 5m59s
Test / Hakurei (race detector) (push) Successful in 7m13s
Test / Flake checks (push) Successful in 1m30s
This prevents inadvertent use of insecure compatibility features.

Closes #30.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 19:34:02 +09:00
cat 952082bd9b internal/rosa/python: 3.14.3 to 3.14.4
Test / Create distribution (push) Successful in 1m31s
Test / Sandbox (push) Successful in 5m0s
Test / ShareFS (push) Successful in 13m54s
Test / Hakurei (race detector) (push) Successful in 21m43s
Test / Sandbox (race detector) (push) Successful in 8m37s
Test / Hakurei (push) Successful in 6m30s
Test / Flake checks (push) Successful in 1m29s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:38:22 +09:00
cat 24a9b24823 internal/rosa/openssl: 3.6.1 to 3.6.2
Test / Sandbox (push) Successful in 5m7s
Test / Create distribution (push) Successful in 2m36s
Test / Sandbox (race detector) (push) Successful in 4m48s
Test / ShareFS (push) Successful in 7m20s
Test / Hakurei (race detector) (push) Successful in 9m30s
Test / Hakurei (push) Successful in 5m34s
Test / Flake checks (push) Successful in 1m32s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:38:02 +09:00
cat c2e61e7987 internal/rosa/libcap: 2.77 to 2.78
Test / Create distribution (push) Successful in 7m4s
Test / Sandbox (push) Successful in 11m33s
Test / ShareFS (push) Successful in 15m46s
Test / Sandbox (race detector) (push) Successful in 4m12s
Test / Hakurei (push) Successful in 1m5s
Test / Hakurei (race detector) (push) Successful in 1m4s
Test / Flake checks (push) Successful in 5m25s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:37:04 +09:00
cat 86787b3bc5 internal/rosa/tamago: 1.26.1 to 1.26.2
Test / Create distribution (push) Successful in 7m2s
Test / Sandbox (push) Successful in 12m41s
Test / Sandbox (race detector) (push) Successful in 15m28s
Test / ShareFS (push) Successful in 16m13s
Test / Hakurei (race detector) (push) Successful in 1m6s
Test / Hakurei (push) Successful in 5m57s
Test / Flake checks (push) Successful in 1m35s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:31:57 +09:00
cat cdfcfe6ce0 internal/rosa/go: 1.26.1 to 1.26.2
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 7m33s
Test / ShareFS (push) Successful in 7m32s
Test / Hakurei (race detector) (push) Successful in 11m11s
Test / Hakurei (push) Successful in 9m13s
Test / Flake checks (push) Successful in 5m39s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:18:27 +09:00
cat 68a2f0c240 internal/rosa/llvm: remove unused field
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 4m18s
Test / Sandbox (race detector) (push) Successful in 6m53s
Test / ShareFS (push) Successful in 7m0s
Test / Hakurei (race detector) (push) Successful in 10m52s
Test / Hakurei (push) Successful in 6m57s
Test / Flake checks (push) Successful in 1m34s
This change also renames confusingly named flags field and corrects its doc comment.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 02:13:26 +09:00
cat 7319c7adf9 internal/rosa/llvm: use latest version on arm64
Test / Create distribution (push) Successful in 2m33s
Test / Sandbox (race detector) (push) Successful in 10m0s
Test / ShareFS (push) Successful in 11m16s
Test / Sandbox (push) Successful in 2m19s
Test / Hakurei (push) Successful in 6m19s
Test / Hakurei (race detector) (push) Successful in 7m9s
Test / Flake checks (push) Successful in 1m28s
This also removes arch-specific patches because they were not useful.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 01:07:25 +09:00
cat e9c890cbb2 internal/rosa/llvm: enable cross compilation
Test / Create distribution (push) Successful in 2m16s
Test / ShareFS (push) Successful in 14m13s
Test / Sandbox (push) Successful in 4m24s
Test / Sandbox (race detector) (push) Successful in 5m48s
Test / Hakurei (push) Successful in 10m2s
Test / Hakurei (race detector) (push) Successful in 7m42s
Test / Flake checks (push) Successful in 1m44s
This now passes the test suite.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 00:59:14 +09:00
cat 6f924336fc internal/rosa/llvm: increase stack size
Test / Create distribution (push) Successful in 1m56s
Test / Sandbox (push) Successful in 9m57s
Test / Sandbox (race detector) (push) Successful in 12m34s
Test / ShareFS (push) Successful in 2m0s
Test / Hakurei (race detector) (push) Successful in 11m33s
Test / Hakurei (push) Successful in 6m3s
Test / Flake checks (push) Successful in 1m40s
Some aarch64 regression tests fail intermittently on the default size.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-10 00:56:51 +09:00
cat bd88f10524 internal/rosa/llvm: 22.1.2 to 22.1.3
Test / Create distribution (push) Successful in 1m7s
Test / ShareFS (push) Successful in 1m24s
Test / Sandbox (push) Successful in 4m46s
Test / Sandbox (race detector) (push) Successful in 6m30s
Test / Hakurei (race detector) (push) Successful in 14m5s
Test / Hakurei (push) Successful in 9m59s
Test / Flake checks (push) Successful in 1m59s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-09 17:36:23 +09:00
cat e34e3b917e internal/kobject: process uevent message
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m21s
Test / Hakurei (push) Successful in 4m37s
Test / ShareFS (push) Successful in 4m36s
Test / Sandbox (race detector) (push) Successful in 5m54s
Test / Hakurei (race detector) (push) Successful in 7m7s
Test / Flake checks (push) Successful in 1m29s
This deals with environment variables generally present in every message.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 18:00:04 +09:00
cat b0ba165107 cmd/sharefs: group-accessible permission bits
Test / Create distribution (push) Successful in 1m16s
Test / Sandbox (push) Successful in 3m17s
Test / Hakurei (push) Successful in 4m21s
Test / ShareFS (push) Successful in 4m30s
Test / Sandbox (race detector) (push) Successful in 5m43s
Test / Hakurei (race detector) (push) Successful in 6m48s
Test / Flake checks (push) Successful in 1m28s
This works around the race in vfs via supplementary group.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 16:14:47 +09:00
cat 351d6c5a35 cmd/sharefs: reproduce vfs inode file attribute race
Test / ShareFS (push) Successful in 46s
Test / Sandbox (race detector) (push) Successful in 53s
Test / Sandbox (push) Successful in 55s
Test / Hakurei (push) Successful in 1m1s
Test / Hakurei (race detector) (push) Successful in 1m0s
Test / Create distribution (push) Successful in 1m11s
Test / Flake checks (push) Successful in 1m28s
This happens in the vfs permissions check only and stale data appears to never reach userspace.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-08 15:51:36 +09:00
cat f23f73701c cmd/mbf: optional host abstract
Test / Create distribution (push) Successful in 1m30s
Test / Sandbox (push) Successful in 3m25s
Test / Hakurei (push) Successful in 4m28s
Test / ShareFS (push) Successful in 4m29s
Test / Sandbox (race detector) (push) Successful in 5m47s
Test / Hakurei (race detector) (push) Successful in 6m54s
Test / Flake checks (push) Successful in 1m23s
This works around kernels with Landlock LSM disabled. Does not affect cure outcome.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 18:15:49 +09:00
cat 876917229a internal/rosa/go: enable riscv64 bootstrap path
Test / Create distribution (push) Successful in 44s
Test / Sandbox (push) Successful in 1m7s
Test / ShareFS (push) Successful in 3m7s
Test / Hakurei (push) Successful in 3m21s
Test / Sandbox (race detector) (push) Successful in 5m34s
Test / Hakurei (race detector) (push) Successful in 6m43s
Test / Flake checks (push) Successful in 1m24s
This is quite expensive, but no other option, unfortunately.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 18:11:42 +09:00
cat 0558032c2d container: do not set static deadline
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m25s
Test / Hakurei (push) Successful in 5m0s
Test / ShareFS (push) Successful in 4m53s
Test / Sandbox (race detector) (push) Successful in 6m1s
Test / Hakurei (race detector) (push) Successful in 3m37s
Test / Flake checks (push) Successful in 1m27s
This usually ends up in the buffer, or completes well before the deadline, however this can still timeout on a very slow system.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 17:00:20 +09:00
cat c61cdc505f internal/params: relocate from package container
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m20s
Test / Hakurei (push) Successful in 4m26s
Test / ShareFS (push) Successful in 4m28s
Test / Sandbox (race detector) (push) Successful in 5m42s
Test / Hakurei (race detector) (push) Successful in 6m57s
Test / Flake checks (push) Successful in 1m27s
This does not make sense as part of the public API, so make it internal.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 16:37:44 +09:00
cat 062edb3487 container: remove setup pipe helper
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m10s
Test / Hakurei (push) Successful in 4m24s
Test / ShareFS (push) Successful in 4m34s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 6m53s
Test / Flake checks (push) Successful in 1m30s
The API forces use of finalizer to close the read end of the setup pipe, which is no longer considered acceptable. Exporting this as part of package container also imposes unnecessary maintenance burden.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 16:05:33 +09:00
cat e4355279a1 all: optionally forbid degrading in tests
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 3m27s
Test / Hakurei (push) Successful in 4m41s
Test / ShareFS (push) Successful in 4m47s
Test / Sandbox (race detector) (push) Successful in 5m57s
Test / Hakurei (race detector) (push) Successful in 7m3s
Test / Flake checks (push) Successful in 1m29s
This enables transparently degradable tests to be forced on in environments known to support them.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 15:22:52 +09:00
cat 289fdebead container: transparently degrade landlock in tests
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m15s
Test / Hakurei (push) Successful in 4m31s
Test / ShareFS (push) Successful in 4m37s
Test / Sandbox (race detector) (push) Successful in 5m53s
Test / Hakurei (race detector) (push) Successful in 6m56s
Test / Flake checks (push) Successful in 1m28s
Explicitly requiring landlock in tests will be supported in a future change.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 15:03:48 +09:00
cat 9c9e190db9 ldd: remove timeout
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m16s
Test / Hakurei (push) Successful in 4m19s
Test / ShareFS (push) Successful in 4m30s
Test / Sandbox (race detector) (push) Successful in 5m49s
Test / Hakurei (race detector) (push) Successful in 6m53s
Test / Flake checks (push) Successful in 1m28s
The program generally never blocks, and it is more flexible to leave it up to the caller to set a timeout.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:49:20 +09:00
cat d7d42c69a1 internal/pkg: transparently degrade landlock in tests
Test / Create distribution (push) Successful in 1m23s
Test / Sandbox (push) Successful in 3m26s
Test / Hakurei (push) Successful in 4m52s
Test / ShareFS (push) Successful in 4m57s
Test / Sandbox (race detector) (push) Successful in 5m56s
Test / Hakurei (race detector) (push) Successful in 7m8s
Test / Flake checks (push) Successful in 1m29s
This does not test package container, so should transparently cope with Landlock LSM being unavailable.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:44:34 +09:00
cat c758e762bd container: skip landlock on hostnet
Test / Create distribution (push) Successful in 1m34s
Test / Sandbox (push) Successful in 3m46s
Test / ShareFS (push) Successful in 5m0s
Test / Sandbox (race detector) (push) Successful in 6m10s
Test / Hakurei (race detector) (push) Successful in 7m29s
Test / Hakurei (push) Successful in 3m21s
Test / Flake checks (push) Successful in 1m43s
This overlaps with net namespace, so can be skipped without degrading security.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 14:36:44 +09:00
cat 10f8b1c221 internal/pkg: optional landlock LSM
Test / Create distribution (push) Successful in 1m15s
Test / Sandbox (push) Successful in 3m8s
Test / Hakurei (push) Successful in 4m17s
Test / ShareFS (push) Successful in 4m22s
Test / Sandbox (race detector) (push) Successful in 5m43s
Test / Hakurei (race detector) (push) Successful in 6m51s
Test / Flake checks (push) Successful in 1m29s
The alpine linux riscv64 kernel does not enable Landlock LSM, and kernel compilation is not yet feasible.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-07 12:44:07 +09:00
cat 6907700d67 cmd/dist: set hsu tar header mode bits
Test / Create distribution (push) Successful in 1m15s
Test / Sandbox (push) Successful in 3m1s
Test / Hakurei (push) Successful in 4m5s
Test / ShareFS (push) Successful in 4m14s
Test / Sandbox (race detector) (push) Successful in 5m27s
Test / Hakurei (race detector) (push) Successful in 6m30s
Test / Flake checks (push) Successful in 1m20s
This has no effect, but is nice to have.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 23:37:38 +09:00
cat 0243f3ffbd internal/rosa/stage0: add riscv64 tarball
Test / Create distribution (push) Successful in 39s
Test / ShareFS (push) Successful in 3m44s
Test / Sandbox (race detector) (push) Successful in 49s
Test / Sandbox (push) Successful in 49s
Test / Hakurei (push) Successful in 54s
Test / Hakurei (race detector) (push) Successful in 1m48s
Test / Flake checks (push) Successful in 1m29s
This had not yet passed all test suites because emulator is prohibitively slow.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 13:57:43 +09:00
cat cd0beeaf8e internal/uevent: optionally pass UUID during coldboot
Test / Create distribution (push) Successful in 1m3s
Test / Sandbox (push) Successful in 2m42s
Test / Hakurei (push) Successful in 3m49s
Test / ShareFS (push) Successful in 3m47s
Test / Sandbox (race detector) (push) Successful in 5m12s
Test / Hakurei (race detector) (push) Successful in 6m20s
Test / Flake checks (push) Successful in 1m20s
This enables rejection of non-coldboot synthetic events.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-06 12:42:47 +09:00
cat a69273ab2a cmd/dist: replace dist/release.sh
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m12s
Test / Hakurei (push) Successful in 4m26s
Test / ShareFS (push) Successful in 4m31s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 7m1s
Test / Flake checks (push) Successful in 1m29s
This is much more robust than a shell script.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 23:58:08 +09:00
cat 4cd0f57e48 dist: remove redundant cleanup
Test / Create distribution (push) Successful in 1m10s
Test / Sandbox (push) Successful in 2m57s
Test / Hakurei (push) Successful in 4m20s
Test / ShareFS (push) Successful in 4m17s
Test / Sandbox (race detector) (push) Successful in 5m47s
Test / Hakurei (race detector) (push) Successful in 6m47s
Test / Flake checks (push) Successful in 1m27s
This breaks on shells that do not evaluate pathnames.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 16:16:37 +09:00
cat 33a0e6c01b hst: conditionally skip root remount
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 56s
Test / Hakurei (push) Successful in 4m25s
Test / ShareFS (push) Successful in 2m19s
Test / Sandbox (race detector) (push) Successful in 3m22s
Test / Hakurei (race detector) (push) Successful in 6m55s
Test / Flake checks (push) Successful in 1m31s
This enables the writable root overlay use case.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 14:04:21 +09:00
cat d58f5c7590 dist: destroy workdir on exist
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m22s
Test / Hakurei (push) Successful in 4m27s
Test / ShareFS (push) Successful in 4m32s
Test / Sandbox (race detector) (push) Successful in 5m47s
Test / Hakurei (race detector) (push) Successful in 6m55s
Test / Flake checks (push) Successful in 1m26s
This no longer relies on the hermetic build system to clean up.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:32:16 +09:00
cat 1da992e342 dist: prefix from environment
Test / Create distribution (push) Successful in 1m5s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 3m48s
Test / ShareFS (push) Successful in 3m49s
Test / Sandbox (race detector) (push) Successful in 5m11s
Test / Hakurei (race detector) (push) Successful in 6m22s
Test / Flake checks (push) Successful in 1m27s
These are baked in, so make them configurable for the build.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:11:44 +09:00
cat 9641805ec2 container/init: ignore finished process
Test / Create distribution (push) Successful in 1m25s
Test / Sandbox (push) Successful in 4m19s
Test / Hakurei (push) Successful in 5m36s
Test / ShareFS (push) Successful in 5m38s
Test / Sandbox (race detector) (push) Successful in 6m25s
Test / Hakurei (race detector) (push) Successful in 7m38s
Test / Flake checks (push) Successful in 1m35s
This is not considered an error, if the process finishes while the signal is being delivered.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:38:33 +09:00
cat 0738f4889a internal/rosa/gnu: fetch mpc source via git
Test / Create distribution (push) Successful in 1m34s
Test / Sandbox (push) Successful in 3m38s
Test / Hakurei (push) Successful in 4m55s
Test / ShareFS (push) Successful in 4m57s
Test / Sandbox (race detector) (push) Successful in 6m13s
Test / Hakurei (race detector) (push) Successful in 7m19s
Test / Flake checks (push) Successful in 2m12s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:35:15 +09:00
cat 7de3cfe221 internal/rosa/netfilter: fetch iptables source via git
Test / Create distribution (push) Successful in 1m43s
Test / Sandbox (push) Successful in 3m54s
Test / ShareFS (push) Successful in 5m41s
Test / Hakurei (push) Successful in 5m46s
Test / Sandbox (race detector) (push) Successful in 6m46s
Test / Hakurei (race detector) (push) Successful in 8m3s
Test / Flake checks (push) Successful in 1m32s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:19:40 +09:00
cat 8b0648dd5d internal/rosa/netfilter: fetch libnftnl source via git
Test / Create distribution (push) Successful in 1m43s
Test / Sandbox (push) Successful in 3m49s
Test / Hakurei (push) Successful in 5m38s
Test / ShareFS (push) Successful in 5m36s
Test / Sandbox (race detector) (push) Successful in 6m44s
Test / Hakurei (race detector) (push) Successful in 8m0s
Test / Flake checks (push) Successful in 1m32s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:14:45 +09:00
cat 4667fac76c internal/rosa/libbsd: fetch source via git
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m16s
Test / Hakurei (push) Successful in 4m28s
Test / ShareFS (push) Successful in 4m38s
Test / Sandbox (race detector) (push) Successful in 5m46s
Test / Hakurei (race detector) (push) Successful in 7m7s
Test / Flake checks (push) Successful in 2m20s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:05:56 +09:00
cat 52e5443b0e internal/rosa/libbsd: fetch libmd source via git
Test / Create distribution (push) Successful in 1m28s
Test / Sandbox (push) Successful in 3m26s
Test / Hakurei (push) Successful in 4m46s
Test / ShareFS (push) Successful in 4m50s
Test / Sandbox (race detector) (push) Successful in 6m4s
Test / Hakurei (race detector) (push) Successful in 7m10s
Test / Flake checks (push) Successful in 1m50s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:02:56 +09:00
cat 130e470b60 internal/rosa/libxslt: fetch source via git
Test / Create distribution (push) Successful in 49s
Test / Sandbox (push) Successful in 4m14s
Test / Hakurei (push) Successful in 7m3s
Test / ShareFS (push) Successful in 7m16s
Test / Hakurei (race detector) (push) Successful in 9m54s
Test / Sandbox (race detector) (push) Successful in 2m35s
Test / Flake checks (push) Successful in 1m30s
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:05:10 +09:00
cat ba5ee8e3ee internal/rosa/libxml2: fetch source via git
Test / Create distribution (push) Successful in 1m41s
Test / Sandbox (push) Successful in 4m29s
Test / Hakurei (push) Successful in 0s
Test / ShareFS (push) Successful in 7m20s
Test / Sandbox (race detector) (push) Successful in 8m9s
Test / Hakurei (race detector) (push) Successful in 10m5s
Test / Flake checks (push) Successful in 1m33s
Eliminates the xz dependency. This also switches to meson to avoid pulling in autotools.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:58:29 +09:00
cat d1cef30877 internal/rosa/gtk: fetch glib source via git
Test / Create distribution (push) Successful in 1m53s
Test / Sandbox (push) Successful in 4m28s
Test / ShareFS (push) Successful in 5m30s
Test / Hakurei (push) Successful in 7m13s
Test / Sandbox (race detector) (push) Successful in 6m48s
Test / Hakurei (race detector) (push) Successful in 4m17s
Test / Flake checks (push) Successful in 1m36s
This eliminates xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:47:10 +09:00
cat 0188a3f0c7 internal/rosa/gnu: gnutls disable arm64 hardware acceleration
Test / Create distribution (push) Successful in 1m36s
Test / ShareFS (push) Successful in 7m39s
Test / Sandbox (race detector) (push) Successful in 8m11s
Test / Hakurei (race detector) (push) Successful in 9m55s
Test / Sandbox (push) Successful in 1m43s
Test / Hakurei (push) Successful in 2m57s
Test / Flake checks (push) Successful in 1m29s
Hardware on arm64 is quite messy, this miscompiles.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:37:18 +09:00
cat 04fe3b24ce internal/rosa/gnu: gnutls configure trust store
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m18s
Test / Hakurei (push) Successful in 4m31s
Test / ShareFS (push) Successful in 4m45s
Test / Sandbox (race detector) (push) Successful in 5m49s
Test / Hakurei (race detector) (push) Successful in 6m55s
Test / Flake checks (push) Successful in 1m53s
The test suite is somehow happy on amd64 but fails on arm64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:33:51 +09:00
cat 93ad551054 internal/rosa/git: shallow clone
Test / Create distribution (push) Successful in 1m24s
Test / Sandbox (push) Successful in 3m19s
Test / Hakurei (push) Successful in 4m32s
Test / ShareFS (push) Successful in 4m34s
Test / Sandbox (race detector) (push) Successful in 5m58s
Test / Hakurei (race detector) (push) Successful in 7m2s
Test / Flake checks (push) Successful in 1m29s
The .git directory is destroyed anyway, so no point fetching more than the bare minimum.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 20:14:40 +09:00
cat 3d54d1f176 internal/rosa: drop caches
Test / Create distribution (push) Successful in 1m21s
Test / Sandbox (push) Successful in 3m14s
Test / Hakurei (push) Successful in 4m27s
Test / ShareFS (push) Successful in 4m31s
Test / Sandbox (race detector) (push) Successful in 5m55s
Test / Hakurei (race detector) (push) Successful in 6m55s
Test / Flake checks (push) Successful in 1m30s
This enables accurate benchmarking of the toolchain abstraction.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:58:23 +09:00
cat 9feac7738f internal/pkg: optionally suppress via assumed checksum
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m24s
Test / Hakurei (push) Successful in 4m31s
Test / ShareFS (push) Successful in 4m41s
Test / Sandbox (race detector) (push) Successful in 5m53s
Test / Hakurei (race detector) (push) Successful in 6m58s
Test / Flake checks (push) Successful in 1m28s
This is quite error-prone and causes cache inconsistency similar to the store inconsistency seen on nix when a similar condition happens. Keep this behind a flag in case it is ever beneficial.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:33:59 +09:00
cat 591a60bac9 internal/pkg: per-cache SCHED_IDLE
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m11s
Test / Hakurei (push) Successful in 4m22s
Test / ShareFS (push) Successful in 4m27s
Test / Sandbox (race detector) (push) Successful in 5m49s
Test / Hakurei (race detector) (push) Successful in 6m48s
Test / Flake checks (push) Successful in 1m27s
This is cleaner than setting it globally, and is impossible to race.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:49:34 +09:00
cat 5093a06026 internal/pkg: cache flags
Test / Create distribution (push) Successful in 1m19s
Test / Sandbox (push) Successful in 3m11s
Test / Hakurei (push) Successful in 4m25s
Test / ShareFS (push) Successful in 4m30s
Test / Sandbox (race detector) (push) Successful in 5m48s
Test / Hakurei (race detector) (push) Successful in 6m56s
Test / Flake checks (push) Successful in 1m29s
This is cleaner for extending the API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:16:55 +09:00
cat 50c1d7f880 internal/rosa/kernel: 6.12.78 to 6.12.80
Test / ShareFS (push) Successful in 47s
Test / Sandbox (push) Successful in 54s
Test / Hakurei (push) Successful in 59s
Test / Hakurei (race detector) (push) Successful in 57s
Test / Sandbox (race detector) (push) Successful in 52s
Test / Create distribution (push) Successful in 1m10s
Test / Flake checks (push) Successful in 1m27s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:14:16 +09:00
cat 9e63633fbc container: remove test timeouts
Test / Create distribution (push) Successful in 3m2s
Test / Sandbox (push) Successful in 5m25s
Test / Hakurei (push) Successful in 7m45s
Test / ShareFS (push) Successful in 7m54s
Test / Sandbox (race detector) (push) Successful in 8m47s
Test / Hakurei (race detector) (push) Successful in 10m38s
Test / Flake checks (push) Successful in 1m49s
These timeouts are no longer useful, and causes spurious test failures under load.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:51:37 +09:00
cat 61f981a34a internal/rosa/perl: 5.42.1 to 5.42.2
Test / Create distribution (push) Successful in 1m25s
Test / Sandbox (push) Successful in 2m36s
Test / Hakurei (push) Successful in 4m11s
Test / ShareFS (push) Successful in 3m51s
Test / Sandbox (race detector) (push) Successful in 6m1s
Test / Hakurei (race detector) (push) Successful in 4m24s
Test / Flake checks (push) Successful in 4m19s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:25 +09:00
cat d717c41bbe internal/rosa/cmake: 4.3.0 to 4.3.1
Test / Sandbox (push) Successful in 2m37s
Test / Create distribution (push) Successful in 1m14s
Test / Hakurei (push) Successful in 3m50s
Test / ShareFS (push) Successful in 3m19s
Test / Sandbox (race detector) (push) Successful in 5m35s
Test / Hakurei (race detector) (push) Successful in 4m14s
Test / Flake checks (push) Successful in 3m46s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:07 +09:00
cat b896eec9b7 internal/rosa/gnu: parallel 20260222 to 20260322
Test / Create distribution (push) Successful in 39s
Test / Sandbox (push) Successful in 2m6s
Test / ShareFS (push) Successful in 3m43s
Test / Sandbox (race detector) (push) Successful in 6m32s
Test / Hakurei (race detector) (push) Successful in 9m45s
Test / Hakurei (push) Successful in 2m52s
Test / Flake checks (push) Successful in 2m32s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:48 +09:00
cat 8ab99e5e40 internal/rosa/util-linux: 2.41.3 to 2.42
Test / Sandbox (push) Successful in 3m25s
Test / Create distribution (push) Successful in 1m14s
Test / ShareFS (push) Successful in 4m35s
Test / Sandbox (race detector) (push) Successful in 6m50s
Test / Hakurei (race detector) (push) Successful in 4m12s
Test / Hakurei (push) Successful in 3m47s
Test / Flake checks (push) Successful in 3m13s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:25 +09:00
cat 2b6160ef7d internal/rosa/wayland: wayland-protocols 1.47 to 1.48
Test / Create distribution (push) Successful in 1m29s
Test / Sandbox (push) Successful in 5m36s
Test / Hakurei (push) Successful in 7m50s
Test / ShareFS (push) Successful in 5m41s
Test / Sandbox (race detector) (push) Successful in 8m49s
Test / Hakurei (race detector) (push) Successful in 3m41s
Test / Flake checks (push) Successful in 3m51s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:04 +09:00
cat 4dcac7f133 internal/rosa/xz: 5.8.2 to 5.8.3
Test / Sandbox (push) Successful in 5m33s
Test / ShareFS (push) Successful in 7m47s
Test / Sandbox (race detector) (push) Successful in 8m53s
Test / Hakurei (race detector) (push) Successful in 10m40s
Test / Hakurei (push) Successful in 2m45s
Test / Create distribution (push) Successful in 1m51s
Test / Flake checks (push) Successful in 3m51s
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:43:37 +09:00
cat 966fd4df9e internal/rosa: connman artifact
Test / Create distribution (push) Successful in 1m20s
Test / Sandbox (push) Successful in 3m12s
Test / Hakurei (push) Successful in 4m28s
Test / ShareFS (push) Successful in 4m29s
Test / Sandbox (race detector) (push) Successful in 5m42s
Test / Hakurei (race detector) (push) Successful in 6m51s
Test / Flake checks (push) Successful in 1m26s
Will be gradually replaced with a native implementation.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:17:10 +09:00
cat a2cf59b989 internal/rosa/ncurses: also build dynamic library
Test / Create distribution (push) Successful in 1m16s
Test / Sandbox (push) Successful in 3m3s
Test / Hakurei (push) Successful in 4m22s
Test / ShareFS (push) Successful in 4m26s
Test / Sandbox (race detector) (push) Successful in 5m40s
Test / Hakurei (race detector) (push) Successful in 6m45s
Test / Flake checks (push) Successful in 1m47s
GNU readline breaks without this.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:13:04 +09:00
207 changed files with 19399 additions and 2763 deletions
+5 -27
View File
@@ -1,37 +1,15 @@
# Binaries for programs and plugins # produced by tools and text editors
*.exe *.qcow2
*.exe~
*.dll
*.so
*.dylib
*.pkg
/hakurei
# Test binary, built with `go test -c`
*.test *.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out *.out
# Dependency directories (remove the comment below to include it)
# vendor/
# Go workspace file
go.work
go.work.sum
# env file
.env
.idea .idea
.vscode .vscode
# go generate # go generate
/cmd/hakurei/LICENSE /cmd/hakurei/LICENSE
/cmd/mbf/internal/pkgserver/ui/static
/internal/pkg/testdata/testtool /internal/pkg/testdata/testtool
/internal/rosa/hakurei_current.tar.gz /internal/rosa/hakurei_current.tar.gz
# release # cmd/dist default destination
/dist/hakurei-* /dist
# interactive nixos vm
nixos.qcow2
Executable
+10
View File
@@ -0,0 +1,10 @@
#!/bin/sh -e
TOOLCHAIN_VERSION="$(go version)"
cd "$(dirname -- "$0")/"
echo "Building cmd/dist using ${TOOLCHAIN_VERSION}."
FLAGS=''
if test -n "$VERBOSE"; then
FLAGS="$FLAGS -v"
fi
go run $FLAGS --tags=dist ./cmd/dist
+22 -23
View File
@@ -2,7 +2,7 @@
package check package check
import ( import (
"encoding/json" "encoding"
"errors" "errors"
"fmt" "fmt"
"path/filepath" "path/filepath"
@@ -30,6 +30,16 @@ func (e AbsoluteError) Is(target error) bool {
// Absolute holds a pathname checked to be absolute. // Absolute holds a pathname checked to be absolute.
type Absolute struct{ pathname unique.Handle[string] } type Absolute struct{ pathname unique.Handle[string] }
var (
_ encoding.TextAppender = new(Absolute)
_ encoding.TextMarshaler = new(Absolute)
_ encoding.TextUnmarshaler = new(Absolute)
_ encoding.BinaryAppender = new(Absolute)
_ encoding.BinaryMarshaler = new(Absolute)
_ encoding.BinaryUnmarshaler = new(Absolute)
)
// ok returns whether [Absolute] is not the zero value. // ok returns whether [Absolute] is not the zero value.
func (a *Absolute) ok() bool { return a != nil && *a != (Absolute{}) } func (a *Absolute) ok() bool { return a != nil && *a != (Absolute{}) }
@@ -84,13 +94,16 @@ func (a *Absolute) Append(elem ...string) *Absolute {
// Dir calls [filepath.Dir] with [Absolute] as its argument. // Dir calls [filepath.Dir] with [Absolute] as its argument.
func (a *Absolute) Dir() *Absolute { return unsafeAbs(filepath.Dir(a.String())) } func (a *Absolute) Dir() *Absolute { return unsafeAbs(filepath.Dir(a.String())) }
// GobEncode returns the checked pathname. // AppendText appends the checked pathname.
func (a *Absolute) GobEncode() ([]byte, error) { func (a *Absolute) AppendText(data []byte) ([]byte, error) {
return []byte(a.String()), nil return append(data, a.String()...), nil
} }
// GobDecode stores data if it represents an absolute pathname. // MarshalText returns the checked pathname.
func (a *Absolute) GobDecode(data []byte) error { func (a *Absolute) MarshalText() ([]byte, error) { return a.AppendText(nil) }
// UnmarshalText stores data if it represents an absolute pathname.
func (a *Absolute) UnmarshalText(data []byte) error {
pathname := string(data) pathname := string(data)
if !filepath.IsAbs(pathname) { if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname) return AbsoluteError(pathname)
@@ -99,23 +112,9 @@ func (a *Absolute) GobDecode(data []byte) error {
return nil return nil
} }
// MarshalJSON returns a JSON representation of the checked pathname. func (a *Absolute) AppendBinary(data []byte) ([]byte, error) { return a.AppendText(data) }
func (a *Absolute) MarshalJSON() ([]byte, error) { func (a *Absolute) MarshalBinary() ([]byte, error) { return a.MarshalText() }
return json.Marshal(a.String()) func (a *Absolute) UnmarshalBinary(data []byte) error { return a.UnmarshalText(data) }
}
// UnmarshalJSON stores data if it represents an absolute pathname.
func (a *Absolute) UnmarshalJSON(data []byte) error {
var pathname string
if err := json.Unmarshal(data, &pathname); err != nil {
return err
}
if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname)
}
a.pathname = unique.Make(pathname)
return nil
}
// SortAbs calls [slices.SortFunc] for a slice of [Absolute]. // SortAbs calls [slices.SortFunc] for a slice of [Absolute].
func SortAbs(x []*Absolute) { func SortAbs(x []*Absolute) {
+6 -15
View File
@@ -170,20 +170,20 @@ func TestCodecAbsolute(t *testing.T) {
{"good", MustAbs("/etc"), {"good", MustAbs("/etc"),
nil, nil,
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\b\xff\x80\x00\x04/etc", "\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\b\xff\x80\x00\x04/etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x04/etc\x01\xfc\xc0\xed\x00\x00\x00", ",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x04/etc\x01\xfc\xc0\xed\x00\x00\x00",
`"/etc"`, `{"val":"/etc","magic":3236757504}`}, `"/etc"`, `{"val":"/etc","magic":3236757504}`},
{"not absolute", nil, {"not absolute", nil,
AbsoluteError("etc"), AbsoluteError("etc"),
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\a\xff\x80\x00\x03etc", "\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\a\xff\x80\x00\x03etc",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x03etc\x01\xfb\x01\x81\xda\x00\x00\x00", ",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x0f\xff\x84\x01\x03etc\x01\xfb\x01\x81\xda\x00\x00\x00",
`"etc"`, `{"val":"etc","magic":3236757504}`}, `"etc"`, `{"val":"etc","magic":3236757504}`},
{"zero", nil, {"zero", nil,
new(AbsoluteError), new(AbsoluteError),
"\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\x04\xff\x80\x00\x00", "\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\x04\xff\x80\x00\x00",
",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x05\x01\x02\xff\x82\x00\x00\x00\f\xff\x84\x01\x00\x01\xfb\x01\x81\xda\x00\x00\x00", ",\xff\x83\x03\x01\x01\x06sCheck\x01\xff\x84\x00\x01\x02\x01\bPathname\x01\xff\x80\x00\x01\x05Magic\x01\x06\x00\x00\x00\t\x7f\x06\x01\x02\xff\x82\x00\x00\x00\f\xff\x84\x01\x00\x01\xfb\x01\x81\xda\x00\x00\x00",
`""`, `{"val":"","magic":3236757504}`}, `""`, `{"val":"","magic":3236757504}`},
} }
@@ -347,15 +347,6 @@ func TestCodecAbsolute(t *testing.T) {
}) })
}) })
} }
t.Run("json passthrough", func(t *testing.T) {
t.Parallel()
wantErr := "invalid character ':' looking for beginning of value"
if err := new(Absolute).UnmarshalJSON([]byte(":3")); err == nil || err.Error() != wantErr {
t.Errorf("UnmarshalJSON: error = %v, want %s", err, wantErr)
}
})
} }
func TestAbsoluteWrap(t *testing.T) { func TestAbsoluteWrap(t *testing.T) {
+8
View File
@@ -4,15 +4,23 @@ import "strings"
const ( const (
// SpecialOverlayEscape is the escape string for overlay mount options. // SpecialOverlayEscape is the escape string for overlay mount options.
//
// Deprecated: This is no longer used and will be removed in 0.5.
SpecialOverlayEscape = `\` SpecialOverlayEscape = `\`
// SpecialOverlayOption is the separator string between overlay mount options. // SpecialOverlayOption is the separator string between overlay mount options.
//
// Deprecated: This is no longer used and will be removed in 0.5.
SpecialOverlayOption = "," SpecialOverlayOption = ","
// SpecialOverlayPath is the separator string between overlay paths. // SpecialOverlayPath is the separator string between overlay paths.
//
// Deprecated: This is no longer used and will be removed in 0.5.
SpecialOverlayPath = ":" SpecialOverlayPath = ":"
) )
// EscapeOverlayDataSegment escapes a string for formatting into the data // EscapeOverlayDataSegment escapes a string for formatting into the data
// argument of an overlay mount system call. // argument of an overlay mount system call.
//
// Deprecated: This is no longer used and will be removed in 0.5.
func EscapeOverlayDataSegment(s string) string { func EscapeOverlayDataSegment(s string) string {
if s == "" { if s == "" {
return "" return ""
View File
+246
View File
@@ -0,0 +1,246 @@
//go:build dist
package main
import (
"archive/tar"
"compress/gzip"
"context"
"crypto/sha512"
_ "embed"
"encoding/hex"
"fmt"
"io"
"io/fs"
"log"
"os"
"os/exec"
"os/signal"
"path/filepath"
"runtime"
)
// getenv looks up an environment variable, and returns fallback if it is unset.
func getenv(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok {
return v
}
return fallback
}
// mustRun runs a command with the current process's environment and panics
// on error or non-zero exit code.
func mustRun(ctx context.Context, name string, arg ...string) {
cmd := exec.CommandContext(ctx, name, arg...)
cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
if err := cmd.Run(); err != nil {
panic(err)
}
}
//go:embed comp/_hakurei
var comp []byte
func main() {
log.SetFlags(0)
log.SetPrefix("")
verbose := os.Getenv("VERBOSE") != ""
version := getenv("HAKUREI_VERSION", "untagged")
prefix := getenv("PREFIX", "/usr")
destdir := getenv("DESTDIR", "dist")
if verbose {
log.Println()
}
if err := os.MkdirAll(destdir, 0755); err != nil {
log.Fatal(err)
}
s, err := os.MkdirTemp(destdir, ".dist.*")
if err != nil {
log.Fatal(err)
}
defer func() {
var code int
if err = os.RemoveAll(s); err != nil {
code = 1
log.Println(err)
}
if r := recover(); r != nil {
code = 1
log.Println(r)
}
os.Exit(code)
}()
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()
verboseFlag := "-v"
if !verbose {
verboseFlag = "-buildvcs=false"
}
log.Printf("Building hakurei for %s/%s.", runtime.GOOS, runtime.GOARCH)
mustRun(ctx, "go", "generate", "./...")
mustRun(
ctx, "go", "build",
"-trimpath",
verboseFlag, "-o", s,
"-ldflags=-s -w "+
"-buildid= -linkmode external -extldflags=-static "+
"-X hakurei.app/internal/info.buildVersion="+version+" "+
"-X hakurei.app/internal/info.hakureiPath="+prefix+"/bin/hakurei "+
"-X hakurei.app/internal/info.hsuPath="+prefix+"/bin/hsu "+
"-X main.hakureiPath="+prefix+"/bin/hakurei",
"./...",
)
log.Println()
log.Println("##### Testing Hakurei.")
mustRun(
ctx, "go", "test",
"-ldflags=-buildid= -linkmode external -extldflags=-static",
"./...",
)
log.Println()
log.Println("##### Creating distribution.")
const suffix = ".tar.gz"
distName := "hakurei-" + version + "-" + runtime.GOARCH
var f *os.File
if f, err = os.OpenFile(
filepath.Join(s, distName+suffix),
os.O_CREATE|os.O_EXCL|os.O_WRONLY,
0644,
); err != nil {
panic(err)
}
defer func() {
if f == nil {
return
}
if err = f.Close(); err != nil {
log.Println(err)
}
}()
h := sha512.New()
gw, _ := gzip.NewWriterLevel(io.MultiWriter(f, h), gzip.BestCompression)
tw := tar.NewWriter(gw)
mustWriteHeader := func(name string, size int64, mode os.FileMode) {
header := tar.Header{
Name: filepath.Join(distName, name),
Size: size,
Mode: int64(mode),
Uname: "root",
Gname: "root",
}
if mode&os.ModeDir != 0 {
header.Typeflag = tar.TypeDir
fmt.Printf("%s %s\n", mode, name)
} else {
header.Typeflag = tar.TypeReg
fmt.Printf("%s %s (%d bytes)\n", mode, name, size)
}
if err = tw.WriteHeader(&header); err != nil {
panic(err)
}
}
mustWriteFile := func(name string, data []byte, mode os.FileMode) {
mustWriteHeader(name, int64(len(data)), mode)
if mode&os.ModeDir != 0 {
return
}
if _, err = tw.Write(data); err != nil {
panic(err)
}
}
mustWriteFromPath := func(dst, src string, mode os.FileMode) {
var r *os.File
if r, err = os.Open(src); err != nil {
panic(err)
}
var fi os.FileInfo
if fi, err = r.Stat(); err != nil {
_ = r.Close()
panic(err)
}
if mode == 0 {
mode = fi.Mode()
}
mustWriteHeader(dst, fi.Size(), mode)
if _, err = io.Copy(tw, r); err != nil {
_ = r.Close()
panic(err)
} else if err = r.Close(); err != nil {
panic(err)
}
}
mustWriteFile(".", nil, fs.ModeDir|0755)
mustWriteFile("comp/", nil, os.ModeDir|0755)
mustWriteFile("comp/_hakurei", comp, 0644)
mustWriteFile("install.sh", []byte(`#!/bin/sh -e
cd "$(dirname -- "$0")" || exit 1
install -vDm0755 "bin/hakurei" "${DESTDIR}`+prefix+`/bin/hakurei"
install -vDm0755 "bin/sharefs" "${DESTDIR}`+prefix+`/bin/sharefs"
install -vDm4511 "bin/hsu" "${DESTDIR}`+prefix+`/bin/hsu"
if [ ! -f "${DESTDIR}/etc/hsurc" ]; then
install -vDm0400 "hsurc.default" "${DESTDIR}/etc/hsurc"
fi
install -vDm0644 "comp/_hakurei" "${DESTDIR}`+prefix+`/share/zsh/site-functions/_hakurei"
`), 0755)
mustWriteFromPath("README.md", "README.md", 0)
mustWriteFile("hsurc.default", []byte("1000 0"), 0400)
mustWriteFromPath("bin/hsu", filepath.Join(s, "hsu"), 04511)
for _, name := range []string{
"hakurei",
"sharefs",
} {
mustWriteFromPath(
filepath.Join("bin", name),
filepath.Join(s, name),
0,
)
}
if err = tw.Close(); err != nil {
panic(err)
} else if err = gw.Close(); err != nil {
panic(err)
} else if err = f.Close(); err != nil {
panic(err)
}
f = nil
if err = os.WriteFile(
filepath.Join(destdir, distName+suffix+".sha512"),
append(hex.AppendEncode(nil, h.Sum(nil)), " "+distName+suffix+"\n"...),
0644,
); err != nil {
panic(err)
}
if err = os.Rename(
filepath.Join(s, distName+suffix),
filepath.Join(destdir, distName+suffix),
); err != nil {
panic(err)
}
}
+13 -6
View File
@@ -38,8 +38,9 @@ var errSuccess = errors.New("success")
func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErrs, out io.Writer) command.Command { func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErrs, out io.Writer) command.Command {
var ( var (
flagVerbose bool flagVerbose bool
flagJSON bool flagInsecure bool
flagJSON bool
) )
c := command.New(out, log.Printf, "hakurei", func([]string) error { c := command.New(out, log.Printf, "hakurei", func([]string) error {
msg.SwapVerbose(flagVerbose) msg.SwapVerbose(flagVerbose)
@@ -57,6 +58,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
return nil return nil
}). }).
Flag(&flagVerbose, "v", command.BoolFlag(false), "Increase log verbosity"). Flag(&flagVerbose, "v", command.BoolFlag(false), "Increase log verbosity").
Flag(&flagInsecure, "insecure", command.BoolFlag(false), "Allow use of insecure compatibility options").
Flag(&flagJSON, "json", command.BoolFlag(false), "Serialise output in JSON when applicable") Flag(&flagJSON, "json", command.BoolFlag(false), "Serialise output in JSON when applicable")
c.Command("shim", command.UsageInternal, func([]string) error { outcome.Shim(msg); return errSuccess }) c.Command("shim", command.UsageInternal, func([]string) error { outcome.Shim(msg); return errSuccess })
@@ -75,7 +77,12 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
config.Container.Args = append(config.Container.Args, args[1:]...) config.Container.Args = append(config.Container.Args, args[1:]...)
} }
outcome.Main(ctx, msg, config, flagIdentifierFile) var flags int
if flagInsecure {
flags |= hst.VAllowInsecure
}
outcome.Main(ctx, msg, config, flags, flagIdentifierFile)
panic("unreachable") panic("unreachable")
}). }).
Flag(&flagIdentifierFile, "identifier-fd", command.IntFlag(-1), Flag(&flagIdentifierFile, "identifier-fd", command.IntFlag(-1),
@@ -145,7 +152,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
} }
} }
var et hst.Enablement var et hst.Enablements
if flagWayland { if flagWayland {
et |= hst.EWayland et |= hst.EWayland
} }
@@ -163,7 +170,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
ID: flagID, ID: flagID,
Identity: flagIdentity, Identity: flagIdentity,
Groups: flagGroups, Groups: flagGroups,
Enablements: hst.NewEnablements(et), Enablements: &et,
Container: &hst.ContainerConfig{ Container: &hst.ContainerConfig{
Filesystem: []hst.FilesystemConfigJSON{ Filesystem: []hst.FilesystemConfigJSON{
@@ -282,7 +289,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
} }
} }
outcome.Main(ctx, msg, &config, -1) outcome.Main(ctx, msg, &config, 0, -1)
panic("unreachable") panic("unreachable")
}). }).
Flag(&flagDBusConfigSession, "dbus-config", command.StringFlag("builtin"), Flag(&flagDBusConfigSession, "dbus-config", command.StringFlag("builtin"),
+1 -1
View File
@@ -20,7 +20,7 @@ func TestHelp(t *testing.T) {
}{ }{
{ {
"main", []string{}, ` "main", []string{}, `
Usage: hakurei [-h | --help] [-v] [--json] COMMAND [OPTIONS] Usage: hakurei [-h | --help] [-v] [--insecure] [--json] COMMAND [OPTIONS]
Commands: Commands:
run Load and start container from configuration file run Load and start container from configuration file
+1 -1
View File
@@ -56,7 +56,7 @@ func printShowInstance(
t := newPrinter(output) t := newPrinter(output)
defer t.MustFlush() defer t.MustFlush()
if err := config.Validate(); err != nil { if err := config.Validate(hst.VAllowInsecure); err != nil {
valid = false valid = false
if m, ok := message.GetMessage(err); ok { if m, ok := message.GetMessage(err); ok {
mustPrint(output, "Error: "+m+"!\n\n") mustPrint(output, "Error: "+m+"!\n\n")
+1 -1
View File
@@ -32,7 +32,7 @@ var (
PID: 0xbeef, PID: 0xbeef,
ShimPID: 0xcafe, ShimPID: 0xcafe,
Config: &hst.Config{ Config: &hst.Config{
Enablements: hst.NewEnablements(hst.EWayland | hst.EPipeWire), Enablements: new(hst.EWayland | hst.EPipeWire),
Identity: 1, Identity: 1,
Container: &hst.ContainerConfig{ Container: &hst.ContainerConfig{
Shell: check.MustAbs("/bin/sh"), Shell: check.MustAbs("/bin/sh"),
+91
View File
@@ -0,0 +1,91 @@
package main
import (
"context"
"os"
"path/filepath"
"testing"
"hakurei.app/check"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
// cache refers to an instance of [pkg.Cache] that might be open.
type cache struct {
ctx context.Context
msg message.Msg
// Should generally not be used directly.
c *pkg.Cache
cures, jobs int
hostAbstract, idle bool
base string
}
// open opens the underlying [pkg.Cache].
func (cache *cache) open() (err error) {
if cache.c != nil {
return os.ErrInvalid
}
var base *check.Absolute
if cache.base, err = filepath.Abs(cache.base); err != nil {
return
} else if base, err = check.NewAbs(cache.base); err != nil {
return
}
var flags int
if cache.idle {
flags |= pkg.CSchedIdle
}
if cache.hostAbstract {
flags |= pkg.CHostAbstract
}
done := make(chan struct{})
defer close(done)
go func() {
select {
case <-cache.ctx.Done():
if testing.Testing() {
return
}
os.Exit(2)
case <-done:
return
}
}()
cache.msg.Verbosef("opening cache at %s", base)
cache.c, err = pkg.Open(
cache.ctx,
cache.msg,
flags,
cache.cures,
cache.jobs,
base,
)
return
}
// Close closes the underlying [pkg.Cache] if it is open.
func (cache *cache) Close() {
if cache.c != nil {
cache.c.Close()
}
}
// Do calls f on the underlying cache and returns its error value.
func (cache *cache) Do(f func(cache *pkg.Cache) error) error {
if cache.c == nil {
if err := cache.open(); err != nil {
return err
}
}
return f(cache.c)
}
+37
View File
@@ -0,0 +1,37 @@
package main
import (
"log"
"os"
"testing"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
func TestCache(t *testing.T) {
t.Parallel()
cm := cache{
ctx: t.Context(),
msg: message.New(log.New(os.Stderr, "check: ", 0)),
base: t.TempDir(),
hostAbstract: true, idle: true,
}
defer cm.Close()
cm.Close()
if err := cm.open(); err != nil {
t.Fatalf("open: error = %v", err)
}
if err := cm.open(); err != os.ErrInvalid {
t.Errorf("(duplicate) open: error = %v", err)
}
if err := cm.Do(func(cache *pkg.Cache) error {
return cache.Scrub(0)
}); err != nil {
t.Errorf("Scrub: error = %v", err)
}
}
+354
View File
@@ -0,0 +1,354 @@
package main
import (
"context"
"encoding/binary"
"errors"
"io"
"log"
"math"
"net"
"os"
"sync"
"syscall"
"testing"
"time"
"unique"
"hakurei.app/check"
"hakurei.app/internal/pkg"
)
// daemonTimeout is the maximum amount of time cureFromIR will wait on I/O.
const daemonTimeout = 30 * time.Second
// daemonDeadline returns the deadline corresponding to daemonTimeout, or the
// zero value when running in a test.
func daemonDeadline() time.Time {
if testing.Testing() {
return time.Time{}
}
return time.Now().Add(daemonTimeout)
}
const (
// remoteNoReply notifies that the client will not receive a cure reply.
remoteNoReply = 1 << iota
)
// cureFromIR services an IR curing request.
func cureFromIR(
cache *pkg.Cache,
conn net.Conn,
flags uint64,
) (pkg.Artifact, error) {
a, decodeErr := cache.NewDecoder(conn).Decode()
if decodeErr != nil {
_, err := conn.Write([]byte("\x00" + decodeErr.Error()))
return nil, errors.Join(decodeErr, err, conn.Close())
}
pathname, _, cureErr := cache.Cure(a)
if flags&remoteNoReply != 0 {
return a, errors.Join(cureErr, conn.Close())
}
if err := conn.SetWriteDeadline(daemonDeadline()); err != nil {
return a, errors.Join(cureErr, err, conn.Close())
}
if cureErr != nil {
_, err := conn.Write([]byte("\x00" + cureErr.Error()))
return a, errors.Join(cureErr, err, conn.Close())
}
_, err := conn.Write([]byte(pathname.String()))
if testing.Testing() && errors.Is(err, io.ErrClosedPipe) {
return a, nil
}
return a, errors.Join(err, conn.Close())
}
const (
// specialCancel is a message consisting of a single identifier referring
// to a curing artifact to be cancelled.
specialCancel = iota
// specialAbort requests for all pending cures to be aborted. It has no
// message body.
specialAbort
// remoteSpecial denotes a special message with custom layout.
remoteSpecial = math.MaxUint64
)
// writeSpecialHeader writes the header of a remoteSpecial message.
func writeSpecialHeader(conn net.Conn, kind uint64) error {
var sh [16]byte
binary.LittleEndian.PutUint64(sh[:], remoteSpecial)
binary.LittleEndian.PutUint64(sh[8:], kind)
if n, err := conn.Write(sh[:]); err != nil {
return err
} else if n != len(sh) {
return io.ErrShortWrite
}
return nil
}
// cancelIdent reads an identifier from conn and cancels the corresponding cure.
func cancelIdent(
cache *pkg.Cache,
conn net.Conn,
) (*pkg.ID, bool, error) {
var ident pkg.ID
if _, err := io.ReadFull(conn, ident[:]); err != nil {
return nil, false, errors.Join(err, conn.Close())
}
ok := cache.Cancel(unique.Make(ident))
return &ident, ok, conn.Close()
}
// serve services connections from a [net.UnixListener].
func serve(
ctx context.Context,
log *log.Logger,
cm *cache,
ul *net.UnixListener,
) error {
ul.SetUnlinkOnClose(true)
if cm.c == nil {
if err := cm.open(); err != nil {
return errors.Join(err, ul.Close())
}
}
var wg sync.WaitGroup
defer wg.Wait()
wg.Go(func() {
for {
if ctx.Err() != nil {
break
}
conn, err := ul.AcceptUnix()
if err != nil {
if !errors.Is(err, os.ErrDeadlineExceeded) {
log.Println(err)
}
continue
}
wg.Go(func() {
done := make(chan struct{})
defer close(done)
go func() {
select {
case <-ctx.Done():
_ = conn.SetDeadline(time.Now())
case <-done:
return
}
}()
if _err := conn.SetReadDeadline(daemonDeadline()); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
var word [8]byte
if _, _err := io.ReadFull(conn, word[:]); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
flags := binary.LittleEndian.Uint64(word[:])
if flags == remoteSpecial {
if _, _err := io.ReadFull(conn, word[:]); _err != nil {
log.Println(_err)
if _err = conn.Close(); _err != nil {
log.Println(_err)
}
return
}
switch special := binary.LittleEndian.Uint64(word[:]); special {
default:
log.Printf("invalid special %d", special)
case specialCancel:
if id, ok, _err := cancelIdent(cm.c, conn); _err != nil {
log.Println(_err)
} else if !ok {
log.Println(
"attempting to cancel invalid artifact",
pkg.Encode(*id),
)
} else {
log.Println(
"cancelled artifact",
pkg.Encode(*id),
)
}
case specialAbort:
log.Println("aborting all pending cures")
cm.c.Abort()
if _err := conn.Close(); _err != nil {
log.Println(_err)
}
}
return
}
if a, _err := cureFromIR(cm.c, conn, flags); _err != nil {
log.Println(_err)
} else {
log.Printf(
"fulfilled artifact %s",
pkg.Encode(cm.c.Ident(a).Value()),
)
}
})
}
})
<-ctx.Done()
if err := ul.SetDeadline(time.Now()); err != nil {
return errors.Join(err, ul.Close())
}
wg.Wait()
return ul.Close()
}
// dial wraps [net.DialUnix] with a context.
func dial(ctx context.Context, addr *net.UnixAddr) (
done chan<- struct{},
conn *net.UnixConn,
err error,
) {
conn, err = net.DialUnix("unix", nil, addr)
if err != nil {
return
}
d := make(chan struct{})
done = d
go func() {
select {
case <-ctx.Done():
_ = conn.SetDeadline(time.Now())
case <-d:
return
}
}()
return
}
// cureRemote cures a [pkg.Artifact] on a daemon.
func cureRemote(
ctx context.Context,
addr *net.UnixAddr,
a pkg.Artifact,
flags uint64,
) (*check.Absolute, error) {
if flags == remoteSpecial {
return nil, syscall.EINVAL
}
done, conn, err := dial(ctx, addr)
if err != nil {
return nil, err
}
defer close(done)
if n, flagErr := conn.Write(binary.LittleEndian.AppendUint64(nil, flags)); flagErr != nil {
return nil, errors.Join(flagErr, conn.Close())
} else if n != 8 {
return nil, errors.Join(io.ErrShortWrite, conn.Close())
}
if err = pkg.NewIR().EncodeAll(conn, a); err != nil {
return nil, errors.Join(err, conn.Close())
} else if err = conn.CloseWrite(); err != nil {
return nil, errors.Join(err, conn.Close())
}
if flags&remoteNoReply != 0 {
return nil, conn.Close()
}
payload, recvErr := io.ReadAll(conn)
if err = errors.Join(recvErr, conn.Close()); err != nil {
if errors.Is(err, os.ErrDeadlineExceeded) {
if cancelErr := ctx.Err(); cancelErr != nil {
err = cancelErr
}
}
return nil, err
}
if len(payload) > 0 && payload[0] == 0 {
return nil, errors.New(string(payload[1:]))
}
var p *check.Absolute
p, err = check.NewAbs(string(payload))
return p, err
}
// cancelRemote cancels a [pkg.Artifact] curing on a daemon.
func cancelRemote(
ctx context.Context,
addr *net.UnixAddr,
a pkg.Artifact,
wait bool,
) error {
done, conn, err := dial(ctx, addr)
if err != nil {
return err
}
defer close(done)
if err = writeSpecialHeader(conn, specialCancel); err != nil {
return errors.Join(err, conn.Close())
}
var n int
id := pkg.NewIR().Ident(a).Value()
if n, err = conn.Write(id[:]); err != nil {
return errors.Join(err, conn.Close())
} else if n != len(id) {
return errors.Join(io.ErrShortWrite, conn.Close())
}
if wait {
if _, err = conn.Read(make([]byte, 1)); err == io.EOF {
err = nil
}
}
return errors.Join(err, conn.Close())
}
// abortRemote aborts all [pkg.Artifact] curing on a daemon.
func abortRemote(
ctx context.Context,
addr *net.UnixAddr,
wait bool,
) error {
done, conn, err := dial(ctx, addr)
if err != nil {
return err
}
defer close(done)
err = writeSpecialHeader(conn, specialAbort)
if wait && err == nil {
if _, err = conn.Read(make([]byte, 1)); err == io.EOF {
err = nil
}
}
return errors.Join(err, conn.Close())
}
+146
View File
@@ -0,0 +1,146 @@
package main
import (
"bytes"
"context"
"errors"
"io"
"log"
"net"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"hakurei.app/check"
"hakurei.app/internal/pkg"
"hakurei.app/message"
)
func TestNoReply(t *testing.T) {
t.Parallel()
if !daemonDeadline().IsZero() {
t.Fatal("daemonDeadline did not return the zero value")
}
c, err := pkg.Open(
t.Context(),
message.New(log.New(os.Stderr, "cir: ", 0)),
0, 0, 0,
check.MustAbs(t.TempDir()),
)
if err != nil {
t.Fatalf("Open: error = %v", err)
}
defer c.Close()
client, server := net.Pipe()
done := make(chan struct{})
go func() {
defer close(done)
go func() {
<-t.Context().Done()
if _err := client.SetDeadline(time.Now()); _err != nil && !errors.Is(_err, io.ErrClosedPipe) {
panic(_err)
}
}()
if _err := c.EncodeAll(
client,
pkg.NewFile("check", []byte{0}),
); _err != nil {
panic(_err)
} else if _err = client.Close(); _err != nil {
panic(_err)
}
}()
a, cureErr := cureFromIR(c, server, remoteNoReply)
if cureErr != nil {
t.Fatalf("cureFromIR: error = %v", cureErr)
}
<-done
wantIdent := pkg.MustDecode("fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG")
if gotIdent := c.Ident(a).Value(); gotIdent != wantIdent {
t.Errorf(
"cureFromIR: %s, want %s",
pkg.Encode(gotIdent), pkg.Encode(wantIdent),
)
}
}
func TestDaemon(t *testing.T) {
t.Parallel()
var buf bytes.Buffer
logger := log.New(&buf, "daemon: ", 0)
addr := net.UnixAddr{
Name: filepath.Join(t.TempDir(), "daemon"),
Net: "unix",
}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
cm := cache{
ctx: ctx,
msg: message.New(logger),
base: t.TempDir(),
}
defer cm.Close()
ul, err := net.ListenUnix("unix", &addr)
if err != nil {
t.Fatalf("ListenUnix: error = %v", err)
}
done := make(chan struct{})
go func() {
defer close(done)
if _err := serve(ctx, logger, &cm, ul); _err != nil {
panic(_err)
}
}()
if err = cancelRemote(ctx, &addr, pkg.NewFile("nonexistent", nil), true); err != nil {
t.Fatalf("cancelRemote: error = %v", err)
}
if err = abortRemote(ctx, &addr, true); err != nil {
t.Fatalf("abortRemote: error = %v", err)
}
// keep this last for synchronisation
var p *check.Absolute
p, err = cureRemote(ctx, &addr, pkg.NewFile("check", []byte{0}), 0)
if err != nil {
t.Fatalf("cureRemote: error = %v", err)
}
cancel()
<-done
const want = "fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG"
if got := filepath.Base(p.String()); got != want {
t.Errorf("cureRemote: %s, want %s", got, want)
}
wantLog := []string{
"",
"daemon: aborting all pending cures",
"daemon: attempting to cancel invalid artifact kQm9fmnCmXST1-MMmxzcau2oKZCXXrlZydo4PkeV5hO_2PKfeC8t98hrbV_ZZx_j",
"daemon: fulfilled artifact fiZf-ZY_Yq6qxJNrHbMiIPYCsGkUiKCRsZrcSELXTqZWtCnESlHmzV5ThhWWGGYG",
}
gotLog := strings.Split(buf.String(), "\n")
slices.Sort(gotLog)
if !slices.Equal(gotLog, wantLog) {
t.Errorf(
"serve: logged\n%s\nwant\n%s",
strings.Join(gotLog, "\n"), strings.Join(wantLog, "\n"),
)
}
}
+114
View File
@@ -0,0 +1,114 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"strings"
"hakurei.app/internal/pkg"
"hakurei.app/internal/rosa"
)
// commandInfo implements the info subcommand.
func commandInfo(
cm *cache,
args []string,
w io.Writer,
writeStatus bool,
r *rosa.Report,
) (err error) {
if len(args) == 0 {
return errors.New("info requires at least 1 argument")
}
// recovered by HandleAccess
mustPrintln := func(a ...any) {
if _, _err := fmt.Fprintln(w, a...); _err != nil {
panic(_err)
}
}
mustPrint := func(a ...any) {
if _, _err := fmt.Fprint(w, a...); _err != nil {
panic(_err)
}
}
for i, name := range args {
if p, ok := rosa.ResolveName(name); !ok {
return fmt.Errorf("unknown artifact %q", name)
} else {
var suffix string
if version := rosa.Std.Version(p); version != rosa.Unversioned {
suffix += "-" + version
}
mustPrintln("name : " + name + suffix)
meta := rosa.GetMetadata(p)
mustPrintln("description : " + meta.Description)
if meta.Website != "" {
mustPrintln("website : " +
strings.TrimSuffix(meta.Website, "/"))
}
if len(meta.Dependencies) > 0 {
mustPrint("depends on :")
for _, d := range meta.Dependencies {
s := rosa.GetMetadata(d).Name
if version := rosa.Std.Version(d); version != rosa.Unversioned {
s += "-" + version
}
mustPrint(" " + s)
}
mustPrintln()
}
const statusPrefix = "status : "
if writeStatus {
if r == nil {
var f io.ReadSeekCloser
err = cm.Do(func(cache *pkg.Cache) (err error) {
f, err = cache.OpenStatus(rosa.Std.Load(p))
return
})
if err != nil {
if errors.Is(err, os.ErrNotExist) {
mustPrintln(
statusPrefix + "not yet cured",
)
} else {
return
}
} else {
mustPrint(statusPrefix)
_, err = io.Copy(w, f)
if err = errors.Join(err, f.Close()); err != nil {
return
}
}
} else if err = cm.Do(func(cache *pkg.Cache) (err error) {
status, n := r.ArtifactOf(cache.Ident(rosa.Std.Load(p)))
if status == nil {
mustPrintln(
statusPrefix + "not in report",
)
} else {
mustPrintln("size :", n)
mustPrint(statusPrefix)
if _, err = w.Write(status); err != nil {
return
}
}
return
}); err != nil {
return
}
}
if i != len(args)-1 {
mustPrintln()
}
}
}
return nil
}
+181
View File
@@ -0,0 +1,181 @@
package main
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"reflect"
"strings"
"syscall"
"testing"
"unsafe"
"hakurei.app/internal/pkg"
"hakurei.app/internal/rosa"
"hakurei.app/message"
)
func TestInfo(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
args []string
status map[string]string
report string
want string
wantErr any
}{
{"qemu", []string{"qemu"}, nil, "", `
name : qemu-` + rosa.Std.Version(rosa.QEMU) + `
description : a generic and open source machine emulator and virtualizer
website : https://www.qemu.org
depends on : glib-` + rosa.Std.Version(rosa.GLib) + ` zstd-` + rosa.Std.Version(rosa.Zstd) + `
`, nil},
{"multi", []string{"hakurei", "hakurei-dist"}, nil, "", `
name : hakurei-` + rosa.Std.Version(rosa.Hakurei) + `
description : low-level userspace tooling for Rosa OS
website : https://hakurei.app
name : hakurei-dist-` + rosa.Std.Version(rosa.HakureiDist) + `
description : low-level userspace tooling for Rosa OS (distribution tarball)
website : https://hakurei.app
`, nil},
{"nonexistent", []string{"zlib", "\x00"}, nil, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
`, fmt.Errorf("unknown artifact %q", "\x00")},
{"status cache", []string{"zlib", "zstd"}, map[string]string{
"zstd": "internal/pkg (amd64) on satori\n",
"hakurei": "internal/pkg (amd64) on satori\n\n",
}, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
status : not yet cured
name : zstd-` + rosa.Std.Version(rosa.Zstd) + `
description : a fast compression algorithm
website : https://facebook.github.io/zstd
status : internal/pkg (amd64) on satori
`, nil},
{"status cache perm", []string{"zlib"}, map[string]string{
"zlib": "\x00",
}, "", `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
`, func(cm *cache) error {
return &os.PathError{
Op: "open",
Path: filepath.Join(cm.base, "status", pkg.Encode(cm.c.Ident(rosa.Std.Load(rosa.Zlib)).Value())),
Err: syscall.EACCES,
}
}},
{"status report", []string{"zlib"}, nil, strings.Repeat("\x00", len(pkg.Checksum{})+8), `
name : zlib-` + rosa.Std.Version(rosa.Zlib) + `
description : lossless data-compression library
website : https://zlib.net
status : not in report
`, nil},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var (
cm *cache
buf strings.Builder
r *rosa.Report
)
if tc.status != nil || tc.report != "" {
cm = &cache{
ctx: context.Background(),
msg: message.New(log.New(os.Stderr, "info: ", 0)),
base: t.TempDir(),
}
defer cm.Close()
}
if tc.report != "" {
pathname := filepath.Join(t.TempDir(), "report")
err := os.WriteFile(
pathname,
unsafe.Slice(unsafe.StringData(tc.report), len(tc.report)),
0400,
)
if err != nil {
t.Fatal(err)
}
r, err = rosa.OpenReport(pathname)
if err != nil {
t.Fatal(err)
}
defer func() {
if err = r.Close(); err != nil {
t.Fatal(err)
}
}()
}
if tc.status != nil {
for name, status := range tc.status {
p, ok := rosa.ResolveName(name)
if !ok {
t.Fatalf("invalid name %q", name)
}
perm := os.FileMode(0400)
if status == "\x00" {
perm = 0
}
if err := cm.Do(func(cache *pkg.Cache) error {
return os.WriteFile(filepath.Join(
cm.base,
"status",
pkg.Encode(cache.Ident(rosa.Std.Load(p)).Value()),
), unsafe.Slice(unsafe.StringData(status), len(status)), perm)
}); err != nil {
t.Fatalf("Do: error = %v", err)
}
}
}
var wantErr error
switch c := tc.wantErr.(type) {
case error:
wantErr = c
case func(cm *cache) error:
wantErr = c(cm)
default:
if tc.wantErr != nil {
t.Fatalf("invalid wantErr %#v", tc.wantErr)
}
}
if err := commandInfo(
cm,
tc.args,
&buf,
cm != nil,
r,
); !reflect.DeepEqual(err, wantErr) {
t.Fatalf("commandInfo: error = %v, want %v", err, wantErr)
}
if got := buf.String(); got != strings.TrimPrefix(tc.want, "\n") {
t.Errorf("commandInfo:\n%s\nwant\n%s", got, tc.want)
}
})
}
}
+202
View File
@@ -0,0 +1,202 @@
// Package pkgserver implements the package metadata service backend.
package pkgserver
import (
"context"
"encoding/json"
"log"
"net/http"
"net/url"
"path"
"strconv"
"sync"
"time"
"hakurei.app/internal/info"
"hakurei.app/internal/rosa"
)
// for lazy initialisation of serveInfo
var (
infoPayload struct {
// Current package count.
Count int `json:"count"`
// Hakurei version, set at link time.
HakureiVersion string `json:"hakurei_version"`
}
infoPayloadOnce sync.Once
)
// handleInfo writes constant system information.
func handleInfo(w http.ResponseWriter, _ *http.Request) {
infoPayloadOnce.Do(func() {
infoPayload.Count = int(rosa.PresetUnexportedStart)
infoPayload.HakureiVersion = info.Version()
})
// TODO(mae): cache entire response if no additional fields are planned
writeAPIPayload(w, infoPayload)
}
// newStatusHandler returns a [http.HandlerFunc] that offers status files for
// viewing or download, if available.
func (index *packageIndex) newStatusHandler(disposition bool) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
m, ok := index.names[path.Base(r.URL.Path)]
if !ok || !m.HasReport {
http.NotFound(w, r)
return
}
contentType := "text/plain; charset=utf-8"
if disposition {
contentType = "application/octet-stream"
// quoting like this is unsound, but okay, because metadata is hardcoded
contentDisposition := `attachment; filename="`
contentDisposition += m.Name + "-"
if m.Version != "" {
contentDisposition += m.Version + "-"
}
contentDisposition += m.ids + `.log"`
w.Header().Set("Content-Disposition", contentDisposition)
}
w.Header().Set("Content-Type", contentType)
w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate")
if err := func() (err error) {
defer index.handleAccess(&err)()
_, err = w.Write(m.status)
return
}(); err != nil {
log.Println(err)
http.Error(
w, "cannot deliver status, contact maintainers",
http.StatusInternalServerError,
)
}
}
}
// handleGet writes a slice of metadata with specified order.
func (index *packageIndex) handleGet(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
limit, err := strconv.Atoi(q.Get("limit"))
if err != nil || limit > 100 || limit < 1 {
http.Error(
w, "limit must be an integer between 1 and 100",
http.StatusBadRequest,
)
return
}
i, err := strconv.Atoi(q.Get("index"))
if err != nil || i >= len(index.sorts[0]) || i < 0 {
http.Error(
w, "index must be an integer between 0 and "+
strconv.Itoa(int(rosa.PresetUnexportedStart-1)),
http.StatusBadRequest,
)
return
}
sort, err := strconv.Atoi(q.Get("sort"))
if err != nil || sort >= len(index.sorts) || sort < 0 {
http.Error(
w, "sort must be an integer between 0 and "+
strconv.Itoa(sortOrderEnd),
http.StatusBadRequest,
)
return
}
values := index.sorts[sort][i:min(i+limit, len(index.sorts[sort]))]
writeAPIPayload(w, &struct {
Values []*metadata `json:"values"`
}{values})
}
func (index *packageIndex) handleSearch(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
limit, err := strconv.Atoi(q.Get("limit"))
if err != nil || limit > 100 || limit < 1 {
http.Error(
w, "limit must be an integer between 1 and 100",
http.StatusBadRequest,
)
return
}
i, err := strconv.Atoi(q.Get("index"))
if err != nil || i >= len(index.sorts[0]) || i < 0 {
http.Error(
w, "index must be an integer between 0 and "+
strconv.Itoa(int(rosa.PresetUnexportedStart-1)),
http.StatusBadRequest,
)
return
}
search, err := url.QueryUnescape(q.Get("search"))
if len(search) > 100 || err != nil {
http.Error(
w, "search must be a string between 0 and 100 characters long",
http.StatusBadRequest,
)
return
}
desc := q.Get("desc") == "true"
n, res, err := index.performSearchQuery(limit, i, search, desc)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
writeAPIPayload(w, &struct {
Count int `json:"count"`
Values []searchResult `json:"values"`
}{n, res})
}
// apiVersion is the name of the current API revision, as part of the pattern.
const apiVersion = "v1"
// registerAPI registers API handler functions.
func (index *packageIndex) registerAPI(mux *http.ServeMux) {
mux.HandleFunc("GET /api/"+apiVersion+"/info", handleInfo)
mux.HandleFunc("GET /api/"+apiVersion+"/get", index.handleGet)
mux.HandleFunc("GET /api/"+apiVersion+"/search", index.handleSearch)
mux.HandleFunc("GET /api/"+apiVersion+"/status/", index.newStatusHandler(false))
mux.HandleFunc("GET /status/", index.newStatusHandler(true))
}
// Register arranges for mux to service API requests.
func Register(ctx context.Context, mux *http.ServeMux, report *rosa.Report) error {
var index packageIndex
index.search = make(searchCache)
if err := index.populate(report); err != nil {
return err
}
ticker := time.NewTicker(1 * time.Minute)
go func() {
for {
select {
case <-ctx.Done():
ticker.Stop()
return
case <-ticker.C:
index.search.clean()
}
}
}()
index.registerAPI(mux)
return nil
}
// writeAPIPayload sets headers common to API responses and encodes payload as
// JSON for the response body.
func writeAPIPayload(w http.ResponseWriter, payload any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate")
w.Header().Set("Pragma", "no-cache")
w.Header().Set("Expires", "0")
if err := json.NewEncoder(w).Encode(payload); err != nil {
log.Println(err)
http.Error(
w, "cannot encode payload, contact maintainers",
http.StatusInternalServerError,
)
}
}
+181
View File
@@ -0,0 +1,181 @@
package pkgserver
import (
"net/http"
"net/http/httptest"
"slices"
"strconv"
"testing"
"hakurei.app/internal/info"
"hakurei.app/internal/rosa"
)
// prefix is prepended to every API path.
const prefix = "/api/" + apiVersion + "/"
func TestAPIInfo(t *testing.T) {
t.Parallel()
w := httptest.NewRecorder()
handleInfo(w, httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
prefix+"info",
nil,
))
resp := w.Result()
checkStatus(t, resp, http.StatusOK)
checkAPIHeader(t, w.Header())
checkPayload(t, resp, struct {
Count int `json:"count"`
HakureiVersion string `json:"hakurei_version"`
}{int(rosa.PresetUnexportedStart), info.Version()})
}
func TestAPIGet(t *testing.T) {
t.Parallel()
const target = prefix + "get"
index := newIndex(t)
newRequest := func(suffix string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
index.handleGet(w, httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
target+suffix,
nil,
))
return w
}
checkValidate := func(t *testing.T, suffix string, vmin, vmax int, wantErr string) {
t.Run("invalid", func(t *testing.T) {
t.Parallel()
w := newRequest("?" + suffix + "=invalid")
resp := w.Result()
checkError(t, resp, wantErr, http.StatusBadRequest)
})
t.Run("min", func(t *testing.T) {
t.Parallel()
w := newRequest("?" + suffix + "=" + strconv.Itoa(vmin-1))
resp := w.Result()
checkError(t, resp, wantErr, http.StatusBadRequest)
w = newRequest("?" + suffix + "=" + strconv.Itoa(vmin))
resp = w.Result()
checkStatus(t, resp, http.StatusOK)
})
t.Run("max", func(t *testing.T) {
t.Parallel()
w := newRequest("?" + suffix + "=" + strconv.Itoa(vmax+1))
resp := w.Result()
checkError(t, resp, wantErr, http.StatusBadRequest)
w = newRequest("?" + suffix + "=" + strconv.Itoa(vmax))
resp = w.Result()
checkStatus(t, resp, http.StatusOK)
})
}
t.Run("limit", func(t *testing.T) {
t.Parallel()
checkValidate(
t, "index=0&sort=0&limit", 1, 100,
"limit must be an integer between 1 and 100",
)
})
t.Run("index", func(t *testing.T) {
t.Parallel()
checkValidate(
t, "limit=1&sort=0&index", 0, int(rosa.PresetUnexportedStart-1),
"index must be an integer between 0 and "+strconv.Itoa(int(rosa.PresetUnexportedStart-1)),
)
})
t.Run("sort", func(t *testing.T) {
t.Parallel()
checkValidate(
t, "index=0&limit=1&sort", 0, int(sortOrderEnd),
"sort must be an integer between 0 and "+strconv.Itoa(int(sortOrderEnd)),
)
})
checkWithSuffix := func(name, suffix string, want []*metadata) {
t.Run(name, func(t *testing.T) {
t.Parallel()
w := newRequest(suffix)
resp := w.Result()
checkStatus(t, resp, http.StatusOK)
checkAPIHeader(t, w.Header())
checkPayloadFunc(t, resp, func(got *struct {
Values []*metadata `json:"values"`
}) bool {
return slices.EqualFunc(got.Values, want, func(a, b *metadata) bool {
return (a.Version == b.Version ||
a.Version == rosa.Unversioned ||
b.Version == rosa.Unversioned) &&
a.HasReport == b.HasReport &&
a.Name == b.Name &&
a.Description == b.Description &&
a.Website == b.Website
})
})
})
}
checkWithSuffix("declarationAscending", "?limit=2&index=1&sort=0", []*metadata{
{
Metadata: rosa.GetMetadata(1),
Version: rosa.Std.Version(1),
},
{
Metadata: rosa.GetMetadata(2),
Version: rosa.Std.Version(2),
},
})
checkWithSuffix("declarationAscending offset", "?limit=3&index=5&sort=0", []*metadata{
{
Metadata: rosa.GetMetadata(5),
Version: rosa.Std.Version(5),
},
{
Metadata: rosa.GetMetadata(6),
Version: rosa.Std.Version(6),
},
{
Metadata: rosa.GetMetadata(7),
Version: rosa.Std.Version(7),
},
})
checkWithSuffix("declarationDescending", "?limit=3&index=0&sort=1", []*metadata{
{
Metadata: rosa.GetMetadata(rosa.PresetUnexportedStart - 1),
Version: rosa.Std.Version(rosa.PresetUnexportedStart - 1),
},
{
Metadata: rosa.GetMetadata(rosa.PresetUnexportedStart - 2),
Version: rosa.Std.Version(rosa.PresetUnexportedStart - 2),
},
{
Metadata: rosa.GetMetadata(rosa.PresetUnexportedStart - 3),
Version: rosa.Std.Version(rosa.PresetUnexportedStart - 3),
},
})
checkWithSuffix("declarationDescending offset", "?limit=1&index=37&sort=1", []*metadata{
{
Metadata: rosa.GetMetadata(rosa.PresetUnexportedStart - 38),
Version: rosa.Std.Version(rosa.PresetUnexportedStart - 38),
},
})
}
+106
View File
@@ -0,0 +1,106 @@
package pkgserver
import (
"cmp"
"errors"
"slices"
"strings"
"hakurei.app/internal/pkg"
"hakurei.app/internal/rosa"
)
const (
declarationAscending = iota
declarationDescending
nameAscending
nameDescending
sizeAscending
sizeDescending
sortOrderEnd = iota - 1
)
// packageIndex refers to metadata by name and various sort orders.
type packageIndex struct {
sorts [sortOrderEnd + 1][rosa.PresetUnexportedStart]*metadata
names map[string]*metadata
search searchCache
// Taken from [rosa.Report] if available.
handleAccess func(*error) func()
}
// metadata holds [rosa.Metadata] extended with additional information.
type metadata struct {
p rosa.PArtifact
*rosa.Metadata
// Populated via [rosa.Toolchain.Version], [rosa.Unversioned] is equivalent
// to the zero value. Otherwise, the zero value is invalid.
Version string `json:"version,omitempty"`
// Output data size, available if present in report.
Size int64 `json:"size,omitempty"`
// Whether the underlying [pkg.Artifact] is present in the report.
HasReport bool `json:"report"`
// Ident string encoded ahead of time.
ids string
// Backed by [rosa.Report], access must be prepared by HandleAccess.
status []byte
}
// populate deterministically populates packageIndex, optionally with a report.
func (index *packageIndex) populate(report *rosa.Report) (err error) {
if report != nil {
defer report.HandleAccess(&err)()
index.handleAccess = report.HandleAccess
}
var work [rosa.PresetUnexportedStart]*metadata
index.names = make(map[string]*metadata)
ir := pkg.NewIR()
for p := range rosa.PresetUnexportedStart {
m := metadata{
p: p,
Metadata: rosa.GetMetadata(p),
Version: rosa.Std.Version(p),
}
if m.Version == "" {
return errors.New("invalid version from " + m.Name)
}
if m.Version == rosa.Unversioned {
m.Version = ""
}
if report != nil {
id := ir.Ident(rosa.Std.Load(p))
m.ids = pkg.Encode(id.Value())
m.status, m.Size = report.ArtifactOf(id)
m.HasReport = m.Size >= 0
}
work[p] = &m
index.names[m.Name] = &m
}
index.sorts[declarationAscending] = work
index.sorts[declarationDescending] = work
slices.Reverse(index.sorts[declarationDescending][:])
index.sorts[nameAscending] = work
slices.SortFunc(index.sorts[nameAscending][:], func(a, b *metadata) int {
return strings.Compare(a.Name, b.Name)
})
index.sorts[nameDescending] = index.sorts[nameAscending]
slices.Reverse(index.sorts[nameDescending][:])
index.sorts[sizeAscending] = work
slices.SortFunc(index.sorts[sizeAscending][:], func(a, b *metadata) int {
return cmp.Compare(a.Size, b.Size)
})
index.sorts[sizeDescending] = index.sorts[sizeAscending]
slices.Reverse(index.sorts[sizeDescending][:])
return
}
+96
View File
@@ -0,0 +1,96 @@
package pkgserver
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"reflect"
"testing"
)
// newIndex returns the address of a newly populated packageIndex.
func newIndex(t *testing.T) *packageIndex {
t.Helper()
var index packageIndex
if err := index.populate(nil); err != nil {
t.Fatalf("populate: error = %v", err)
}
return &index
}
// checkStatus checks response status code.
func checkStatus(t *testing.T, resp *http.Response, want int) {
t.Helper()
if resp.StatusCode != want {
t.Errorf(
"StatusCode: %s, want %s",
http.StatusText(resp.StatusCode),
http.StatusText(want),
)
}
}
// checkHeader checks the value of a header entry.
func checkHeader(t *testing.T, h http.Header, key, want string) {
t.Helper()
if got := h.Get(key); got != want {
t.Errorf("%s: %q, want %q", key, got, want)
}
}
// checkAPIHeader checks common entries set for API endpoints.
func checkAPIHeader(t *testing.T, h http.Header) {
t.Helper()
checkHeader(t, h, "Content-Type", "application/json; charset=utf-8")
checkHeader(t, h, "Cache-Control", "no-cache, no-store, must-revalidate")
checkHeader(t, h, "Pragma", "no-cache")
checkHeader(t, h, "Expires", "0")
}
// checkPayloadFunc checks the JSON response of an API endpoint by passing it to f.
func checkPayloadFunc[T any](
t *testing.T,
resp *http.Response,
f func(got *T) bool,
) {
t.Helper()
var got T
r := io.Reader(resp.Body)
if testing.Verbose() {
var buf bytes.Buffer
r = io.TeeReader(r, &buf)
defer func() { t.Helper(); t.Log(buf.String()) }()
}
if err := json.NewDecoder(r).Decode(&got); err != nil {
t.Fatalf("Decode: error = %v", err)
}
if !f(&got) {
t.Errorf("Body: %#v", got)
}
}
// checkPayload checks the JSON response of an API endpoint.
func checkPayload[T any](t *testing.T, resp *http.Response, want T) {
t.Helper()
checkPayloadFunc(t, resp, func(got *T) bool {
return reflect.DeepEqual(got, &want)
})
}
func checkError(t *testing.T, resp *http.Response, error string, code int) {
t.Helper()
checkStatus(t, resp, code)
if got, _ := io.ReadAll(resp.Body); string(got) != fmt.Sprintln(error) {
t.Errorf("Body: %q, want %q", string(got), error)
}
}
+81
View File
@@ -0,0 +1,81 @@
package pkgserver
import (
"cmp"
"maps"
"regexp"
"slices"
"time"
)
type searchCache map[string]searchCacheEntry
type searchResult struct {
NameIndices [][]int `json:"name_matches"`
DescIndices [][]int `json:"desc_matches,omitempty"`
Score float64 `json:"score"`
*metadata
}
type searchCacheEntry struct {
query string
results []searchResult
expiry time.Time
}
func (index *packageIndex) performSearchQuery(limit int, i int, search string, desc bool) (int, []searchResult, error) {
query := search
if desc {
query += ";withDesc"
}
entry, ok := index.search[query]
if ok && len(entry.results) > 0 {
return len(entry.results), entry.results[min(i, len(entry.results)-1):min(i+limit, len(entry.results))], nil
}
regex, err := regexp.Compile(search)
if err != nil {
return 0, make([]searchResult, 0), err
}
res := make([]searchResult, 0)
for p := range maps.Values(index.names) {
nameIndices := regex.FindAllIndex([]byte(p.Name), -1)
var descIndices [][]int = nil
if desc {
descIndices = regex.FindAllIndex([]byte(p.Description), -1)
}
if nameIndices == nil && descIndices == nil {
continue
}
score := float64(indexsum(nameIndices)) / (float64(len(nameIndices)) + 1)
if desc {
score += float64(indexsum(descIndices)) / (float64(len(descIndices)) + 1) / 10.0
}
res = append(res, searchResult{
NameIndices: nameIndices,
DescIndices: descIndices,
Score: score,
metadata: p,
})
}
slices.SortFunc(res[:], func(a, b searchResult) int { return -cmp.Compare(a.Score, b.Score) })
expiry := time.Now().Add(1 * time.Minute)
entry = searchCacheEntry{
query: search,
results: res,
expiry: expiry,
}
index.search[query] = entry
return len(res), res[i:min(i+limit, len(entry.results))], nil
}
func (s *searchCache) clean() {
maps.DeleteFunc(*s, func(_ string, v searchCacheEntry) bool {
return v.expiry.Before(time.Now())
})
}
func indexsum(in [][]int) int {
sum := 0
for i := 0; i < len(in); i++ {
sum += in[i][1] - in[i][0]
}
return sum
}
+57
View File
@@ -0,0 +1,57 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="style.css">
<title>Hakurei PkgServer</title>
<script src="index.js"></script>
</head>
<body>
<h1>Hakurei PkgServer</h1>
<div class="top-controls" id="top-controls-regular">
<p>Showing entries <span id="entry-counter"></span>.</p>
<span id="search-bar">
<label for="search">Search: </label>
<input type="text" name="search" id="search"/>
<button onclick="doSearch()">Find</button>
<label for="include-desc">Include descriptions: </label>
<input type="checkbox" name="include-desc" id="include-desc" checked/>
</span>
<div><label for="count">Entries per page: </label><select name="count" id="count">
<option value="10">10</option>
<option value="20">20</option>
<option value="30">30</option>
<option value="50">50</option>
</select></div>
<div><label for="sort">Sort by: </label><select name="sort" id="sort">
<option value="0">Definition (ascending)</option>
<option value="1">Definition (descending)</option>
<option value="2">Name (ascending)</option>
<option value="3">Name (descending)</option>
<option value="4">Size (ascending)</option>
<option value="5">Size (descending)</option>
</select></div>
</div>
<div class="top-controls" id="search-top-controls" hidden>
<p>Showing search results <span id="search-entry-counter"></span> for query "<span id="search-query"></span>".</p>
<button onclick="exitSearch()">Back</button>
<div><label for="search-count">Entries per page: </label><select name="search-count" id="search-count">
<option value="10">10</option>
<option value="20">20</option>
<option value="30">30</option>
<option value="50">50</option>
</select></div>
<p>Sorted by best match</p>
</div>
<div class="page-controls"><a href="javascript:prevPage()">&laquo; Previous</a> <input type="text" class="page-number" value="1"/> <a href="javascript:nextPage()">Next &raquo;</a></div>
<table id="pkg-list">
<tr><td>Loading...</td></tr>
</table>
<div class="page-controls"><a href="javascript:prevPage()">&laquo; Previous</a> <input type="text" class="page-number" value="1"/> <a href="javascript:nextPage()">Next &raquo;</a></div>
<footer>
<p>&copy;<a href="https://hakurei.app/">Hakurei</a> (<span id="hakurei-version">unknown</span>). Licensed under the MIT license.</p>
</footer>
<script>main();</script>
</body>
</html>
+331
View File
@@ -0,0 +1,331 @@
interface PackageIndexEntry {
name: string
size?: number
description?: string
website?: string
version?: string
report?: boolean
}
function entryToHTML(entry: PackageIndexEntry | SearchResult): HTMLTableRowElement {
let v = entry.version != null ? `<span>${escapeHtml(entry.version)}</span>` : ""
let s = entry.size != null && entry.size > 0 ? `<p>Size: ${toByteSizeString(entry.size)} (${entry.size})</p>` : ""
let n: string
let d: string
if ('name_matches' in entry) {
n = `<h2>${nameMatches(entry as SearchResult)} ${v}</h2>`
} else {
n = `<h2>${escapeHtml(entry.name)} ${v}</h2>`
}
if ('desc_matches' in entry && STATE.getIncludeDescriptions()) {
d = descMatches(entry as SearchResult)
} else {
d = (entry as PackageIndexEntry).description != null ? `<p>${escapeHtml((entry as PackageIndexEntry).description)}</p>` : ""
}
let w = entry.website != null ? `<a href="${encodeURI(entry.website)}">Website</a>` : ""
let r = entry.report ? `Log (<a href=\"${encodeURI('/api/v1/status/' + entry.name)}\">View</a> | <a href=\"${encodeURI('/status/' + entry.name)}\">Download</a>)` : ""
let row = <HTMLTableRowElement>(document.createElement('tr'))
row.innerHTML = `<td>
${n}
${d}
${s}
${w}
${r}
</td>`
return row
}
function nameMatches(sr: SearchResult): string {
return markMatches(sr.name, sr.name_matches)
}
function descMatches(sr: SearchResult): string {
return markMatches(sr.description!, sr.desc_matches)
}
function markMatches(str: string, indices: [number, number][]): string {
if (indices == null) {
return str
}
let out: string = ""
let j = 0
for (let i = 0; i < str.length; i++) {
if (j < indices.length) {
if (i === indices[j][0]) {
out += `<mark>${escapeHtmlChar(str[i])}`
continue
}
if (i === indices[j][1]) {
out += `</mark>${escapeHtmlChar(str[i])}`
j++
continue
}
}
out += escapeHtmlChar(str[i])
}
if (indices[j] !== undefined) {
out += "</mark>"
}
return out
}
function toByteSizeString(bytes: number): string {
if (bytes == null) return `unspecified`
if (bytes < 1024) return `${bytes}B`
if (bytes < Math.pow(1024, 2)) return `${(bytes / 1024).toFixed(2)}kiB`
if (bytes < Math.pow(1024, 3)) return `${(bytes / Math.pow(1024, 2)).toFixed(2)}MiB`
if (bytes < Math.pow(1024, 4)) return `${(bytes / Math.pow(1024, 3)).toFixed(2)}GiB`
if (bytes < Math.pow(1024, 5)) return `${(bytes / Math.pow(1024, 4)).toFixed(2)}TiB`
return "not only is it big, it's large"
}
const API_VERSION = 1
const ENDPOINT = `/api/v${API_VERSION}`
interface InfoPayload {
count?: number
hakurei_version?: string
}
async function infoRequest(): Promise<InfoPayload> {
const res = await fetch(`${ENDPOINT}/info`)
const payload = await res.json()
return payload as InfoPayload
}
interface GetPayload {
values?: PackageIndexEntry[]
}
enum SortOrders {
DeclarationAscending,
DeclarationDescending,
NameAscending,
NameDescending
}
async function getRequest(limit: number, index: number, sort: SortOrders): Promise<GetPayload> {
const res = await fetch(`${ENDPOINT}/get?limit=${limit}&index=${index}&sort=${sort.valueOf()}`)
const payload = await res.json()
return payload as GetPayload
}
interface SearchResult extends PackageIndexEntry {
name_matches: [number, number][]
desc_matches: [number, number][]
score: number
}
interface SearchPayload {
count?: number
values?: SearchResult[]
}
async function searchRequest(limit: number, index: number, search: string, desc: boolean): Promise<SearchPayload> {
const res = await fetch(`${ENDPOINT}/search?limit=${limit}&index=${index}&search=${encodeURIComponent(search)}&desc=${desc}`)
if (!res.ok) {
exitSearch()
alert("invalid search query!")
return Promise.reject(res.statusText)
}
const payload = await res.json()
return payload as SearchPayload
}
class State {
entriesPerPage: number = 10
entryIndex: number = 0
maxTotal: number = 0
maxEntries: number = 0
sort: SortOrders = SortOrders.DeclarationAscending
search: boolean = false
getEntriesPerPage(): number {
return this.entriesPerPage
}
setEntriesPerPage(entriesPerPage: number) {
this.entriesPerPage = entriesPerPage
this.setEntryIndex(Math.floor(this.getEntryIndex() / entriesPerPage) * entriesPerPage)
}
getEntryIndex(): number {
return this.entryIndex
}
setEntryIndex(entryIndex: number) {
this.entryIndex = entryIndex
this.updatePage()
this.updateRange()
this.updateListings()
}
getMaxTotal(): number {
return this.maxTotal
}
setMaxTotal(max: number) {
this.maxTotal = max
}
getSortOrder(): SortOrders {
return this.sort
}
setSortOrder(sortOrder: SortOrders) {
this.sort = sortOrder
this.setEntryIndex(0)
}
updatePage() {
let page = Math.ceil(((this.getEntryIndex() + this.getEntriesPerPage()) - 1) / this.getEntriesPerPage())
for (let e of document.getElementsByClassName("page-number")) {
(e as HTMLInputElement).value = String(page)
}
}
updateRange() {
let max = Math.min(this.getEntryIndex() + this.getEntriesPerPage(), this.getMaxTotal())
document.getElementById("entry-counter")!.textContent = `${this.getEntryIndex() + 1}-${max} of ${this.getMaxTotal()}`
if (this.search) {
document.getElementById("search-entry-counter")!.textContent = `${this.getEntryIndex() + 1}-${max} of ${this.maxTotal}/${this.maxEntries}`
document.getElementById("search-query")!.innerHTML = `<code>${escapeHtml(this.getSearchQuery())}</code>`
}
}
getSearchQuery(): string {
let queryString = document.getElementById("search")!;
return (queryString as HTMLInputElement).value
}
getIncludeDescriptions(): boolean {
let includeDesc = document.getElementById("include-desc")!;
return (includeDesc as HTMLInputElement).checked
}
updateListings() {
if (this.search) {
searchRequest(this.getEntriesPerPage(), this.getEntryIndex(), this.getSearchQuery(), this.getIncludeDescriptions())
.then(res => {
let table = document.getElementById("pkg-list")!
table.innerHTML = ''
for (let row of res.values!) {
table.appendChild(entryToHTML(row))
}
STATE.maxTotal = res.count!
STATE.updateRange()
if(res.count! < 1) {
exitSearch()
alert("no results found!")
}
})
} else {
getRequest(this.getEntriesPerPage(), this.getEntryIndex(), this.getSortOrder())
.then(res => {
let table = document.getElementById("pkg-list")!
table.innerHTML = ''
for (let row of res.values!) {
table.appendChild(entryToHTML(row))
}
})
}
}
}
let STATE: State
function lastPageIndex(): number {
return Math.floor(STATE.getMaxTotal() / STATE.getEntriesPerPage()) * STATE.getEntriesPerPage()
}
function setPage(page: number) {
STATE.setEntryIndex(Math.max(0, Math.min(STATE.getEntriesPerPage() * (page - 1), lastPageIndex())))
}
function escapeHtml(str?: string): string {
let out: string = ''
if (str == undefined) return ""
for (let i = 0; i < str.length; i++) {
out += escapeHtmlChar(str[i])
}
return out
}
function escapeHtmlChar(char: string): string {
if (char.length != 1) return char
switch (char[0]) {
case '&':
return "&amp;"
case '<':
return "&lt;"
case '>':
return "&gt;"
case '"':
return "&quot;"
case "'":
return "&apos;"
default:
return char
}
}
function firstPage() {
STATE.setEntryIndex(0)
}
function prevPage() {
let index = STATE.getEntryIndex()
STATE.setEntryIndex(Math.max(0, index - STATE.getEntriesPerPage()))
}
function lastPage() {
STATE.setEntryIndex(lastPageIndex())
}
function nextPage() {
let index = STATE.getEntryIndex()
STATE.setEntryIndex(Math.min(lastPageIndex(), index + STATE.getEntriesPerPage()))
}
function doSearch() {
document.getElementById("top-controls-regular")!.toggleAttribute("hidden");
document.getElementById("search-top-controls")!.toggleAttribute("hidden");
STATE.search = true;
STATE.setEntryIndex(0);
}
function exitSearch() {
document.getElementById("top-controls-regular")!.toggleAttribute("hidden");
document.getElementById("search-top-controls")!.toggleAttribute("hidden");
STATE.search = false;
STATE.setMaxTotal(STATE.maxEntries)
STATE.setEntryIndex(0)
}
function main() {
STATE = new State()
infoRequest()
.then(res => {
STATE.maxEntries = res.count!
STATE.setMaxTotal(STATE.maxEntries)
document.getElementById("hakurei-version")!.textContent = res.hakurei_version!
STATE.updateRange()
STATE.updateListings()
})
for (let e of document.getElementsByClassName("page-number")) {
e.addEventListener("change", (_) => {
setPage(parseInt((e as HTMLInputElement).value))
})
}
document.getElementById("count")?.addEventListener("change", (event) => {
STATE.setEntriesPerPage(parseInt((event.target as HTMLSelectElement).value))
})
document.getElementById("sort")?.addEventListener("change", (event) => {
STATE.setSortOrder(parseInt((event.target as HTMLSelectElement).value))
})
document.getElementById("search")?.addEventListener("keyup", (event) => {
if (event.key === 'Enter') doSearch()
})
}
+21
View File
@@ -0,0 +1,21 @@
.page-number {
width: 2em;
text-align: center;
}
.page-number {
width: 2em;
text-align: center;
}
@media (prefers-color-scheme: dark) {
html {
background-color: #2c2c2c;
color: ghostwhite;
}
}
@media (prefers-color-scheme: light) {
html {
background-color: #d3d3d3;
color: black;
}
}
@@ -0,0 +1,8 @@
{
"compilerOptions": {
"target": "ES2024",
"strict": true,
"alwaysStrict": true,
"outDir": "static"
}
}
+9
View File
@@ -0,0 +1,9 @@
// Package ui holds the static web UI.
package ui
import "net/http"
// Register arranges for mux to serve the embedded frontend.
func Register(mux *http.ServeMux) {
mux.Handle("GET /", http.FileServer(http.FS(static)))
}
+21
View File
@@ -0,0 +1,21 @@
//go:build frontend
package ui
import (
"embed"
"io/fs"
)
//go:generate tsc
//go:generate cp index.html style.css static
//go:embed static
var _static embed.FS
var static = func() fs.FS {
if f, err := fs.Sub(_static, "static"); err != nil {
panic(err)
} else {
return f
}
}()
+7
View File
@@ -0,0 +1,7 @@
//go:build !frontend
package ui
import "testing/fstest"
var static fstest.MapFS
+316 -229
View File
@@ -14,16 +14,18 @@ package main
import ( import (
"context" "context"
"crypto/sha512"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log" "log"
"net"
"net/http"
"os" "os"
"os/signal" "os/signal"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strconv" "strconv"
"strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"syscall" "syscall"
@@ -40,6 +42,9 @@ import (
"hakurei.app/internal/pkg" "hakurei.app/internal/pkg"
"hakurei.app/internal/rosa" "hakurei.app/internal/rosa"
"hakurei.app/message" "hakurei.app/message"
"hakurei.app/cmd/mbf/internal/pkgserver"
"hakurei.app/cmd/mbf/internal/pkgserver/ui"
) )
func main() { func main() {
@@ -53,77 +58,106 @@ func main() {
log.Fatal("this program must not run as root") log.Fatal("this program must not run as root")
} }
var cache *pkg.Cache
ctx, stop := signal.NotifyContext(context.Background(), ctx, stop := signal.NotifyContext(context.Background(),
syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP) syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
defer stop() defer stop()
defer func() {
if cache != nil {
cache.Close()
}
if r := recover(); r != nil { var cm cache
fmt.Println(r) defer func() { cm.Close() }()
log.Fatal("consider scrubbing the on-disk cache")
}
}()
var ( var (
flagQuiet bool flagQuiet bool
flagCures int flagCheck bool
flagBase string flagLTO bool
flagTShift int
flagIdle bool addr net.UnixAddr
) )
c := command.New(os.Stderr, log.Printf, "mbf", func([]string) (err error) { c := command.New(os.Stderr, log.Printf, "mbf", func([]string) error {
msg.SwapVerbose(!flagQuiet) msg.SwapVerbose(!flagQuiet)
cm.ctx, cm.msg = ctx, msg
flagBase = os.ExpandEnv(flagBase) cm.base = os.ExpandEnv(cm.base)
if flagBase == "" { if cm.base == "" {
flagBase = "cache" cm.base = "cache"
} }
var base *check.Absolute addr.Net = "unix"
if flagBase, err = filepath.Abs(flagBase); err != nil { addr.Name = os.ExpandEnv(addr.Name)
return if addr.Name == "" {
} else if base, err = check.NewAbs(flagBase); err != nil { addr.Name = filepath.Join(cm.base, "daemon")
return
}
if cache, err = pkg.Open(ctx, msg, flagCures, base); err == nil {
if flagTShift < 0 {
cache.SetThreshold(0)
} else if flagTShift > 31 {
cache.SetThreshold(1 << 31)
} else {
cache.SetThreshold(1 << flagTShift)
}
} }
if flagIdle { var flags int
pkg.SetSchedIdle = true if !flagCheck {
flags |= rosa.OptSkipCheck
} }
if !flagLTO {
flags |= rosa.OptLLVMNoLTO
}
rosa.DropCaches(flags)
return return nil
}).Flag( }).Flag(
&flagQuiet, &flagQuiet,
"q", command.BoolFlag(false), "q", command.BoolFlag(false),
"Do not print cure messages", "Do not print cure messages",
).Flag( ).Flag(
&flagCures, &flagLTO,
"lto", command.BoolFlag(false),
"Enable LTO in stage2 and stage3 LLVM toolchains",
).Flag(
&flagCheck,
"check", command.BoolFlag(true),
"Run test suites",
).Flag(
&cm.cures,
"cures", command.IntFlag(0), "cures", command.IntFlag(0),
"Maximum number of dependencies to cure at any given time", "Maximum number of dependencies to cure at any given time",
).Flag( ).Flag(
&flagBase, &cm.jobs,
"jobs", command.IntFlag(0),
"Preferred number of jobs to run, when applicable",
).Flag(
&cm.base,
"d", command.StringFlag("$MBF_CACHE_DIR"), "d", command.StringFlag("$MBF_CACHE_DIR"),
"Directory to store cured artifacts", "Directory to store cured artifacts",
).Flag( ).Flag(
&flagTShift, &cm.idle,
"tshift", command.IntFlag(-1),
"Dependency graph size exponent, to the power of 2",
).Flag(
&flagIdle,
"sched-idle", command.BoolFlag(false), "sched-idle", command.BoolFlag(false),
"Set SCHED_IDLE scheduling policy", "Set SCHED_IDLE scheduling policy",
).Flag(
&cm.hostAbstract,
"host-abstract", command.BoolFlag(
os.Getenv("MBF_HOST_ABSTRACT") != "",
),
"Do not restrict networked cure containers from connecting to host "+
"abstract UNIX sockets",
).Flag(
&addr.Name,
"socket", command.StringFlag("$MBF_DAEMON_SOCKET"),
"Pathname of socket to bind to",
)
c.NewCommand(
"checksum", "Compute checksum of data read from standard input",
func([]string) error {
done := make(chan struct{})
defer close(done)
go func() {
select {
case <-ctx.Done():
os.Exit(1)
case <-done:
return
}
}()
h := sha512.New384()
if _, err := io.Copy(h, os.Stdin); err != nil {
return err
}
log.Println(pkg.Encode(pkg.Checksum(h.Sum(nil))))
return nil
},
) )
{ {
@@ -137,7 +171,9 @@ func main() {
if flagShifts < 0 || flagShifts > 31 { if flagShifts < 0 || flagShifts > 31 {
flagShifts = 12 flagShifts = 12
} }
return cache.Scrub(runtime.NumCPU() << flagShifts) return cm.Do(func(cache *pkg.Cache) error {
return cache.Scrub(runtime.NumCPU() << flagShifts)
})
}, },
).Flag( ).Flag(
&flagShifts, &flagShifts,
@@ -148,6 +184,7 @@ func main() {
{ {
var ( var (
flagBind string
flagStatus bool flagStatus bool
flagReport string flagReport string
) )
@@ -155,9 +192,7 @@ func main() {
"info", "info",
"Display out-of-band metadata of an artifact", "Display out-of-band metadata of an artifact",
func(args []string) (err error) { func(args []string) (err error) {
if len(args) == 0 { const shutdownTimeout = 15 * time.Second
return errors.New("info requires at least 1 argument")
}
var r *rosa.Report var r *rosa.Report
if flagReport != "" { if flagReport != "" {
@@ -172,88 +207,46 @@ func main() {
defer r.HandleAccess(&err)() defer r.HandleAccess(&err)()
} }
for i, name := range args { if flagBind == "" {
if p, ok := rosa.ResolveName(name); !ok { return commandInfo(&cm, args, os.Stdout, flagStatus, r)
return fmt.Errorf("unknown artifact %q", name)
} else {
var suffix string
if version := rosa.Std.Version(p); version != rosa.Unversioned {
suffix += "-" + version
}
fmt.Println("name : " + name + suffix)
meta := rosa.GetMetadata(p)
fmt.Println("description : " + meta.Description)
if meta.Website != "" {
fmt.Println("website : " +
strings.TrimSuffix(meta.Website, "/"))
}
if len(meta.Dependencies) > 0 {
fmt.Print("depends on :")
for _, d := range meta.Dependencies {
s := rosa.GetMetadata(d).Name
if version := rosa.Std.Version(d); version != rosa.Unversioned {
s += "-" + version
}
fmt.Print(" " + s)
}
fmt.Println()
}
const statusPrefix = "status : "
if flagStatus {
if r == nil {
var f io.ReadSeekCloser
f, err = cache.OpenStatus(rosa.Std.Load(p))
if err != nil {
if errors.Is(err, os.ErrNotExist) {
fmt.Println(
statusPrefix + "not yet cured",
)
} else {
return
}
} else {
fmt.Print(statusPrefix)
_, err = io.Copy(os.Stdout, f)
if err = errors.Join(err, f.Close()); err != nil {
return
}
}
} else {
status, n := r.ArtifactOf(cache.Ident(rosa.Std.Load(p)))
if status == nil {
fmt.Println(
statusPrefix + "not in report",
)
} else {
fmt.Println("size :", n)
fmt.Print(statusPrefix)
if _, err = os.Stdout.Write(status); err != nil {
return
}
}
}
}
if i != len(args)-1 {
fmt.Println()
}
}
} }
return nil
var mux http.ServeMux
ui.Register(&mux)
if err = pkgserver.Register(ctx, &mux, r); err != nil {
return
}
server := http.Server{Addr: flagBind, Handler: &mux}
go func() {
<-ctx.Done()
cc, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
defer cancel()
if _err := server.Shutdown(cc); _err != nil {
log.Fatal(_err)
}
}()
msg.Verbosef("listening on %q", flagBind)
err = server.ListenAndServe()
if errors.Is(err, http.ErrServerClosed) {
err = nil
}
return
}, },
). ).Flag(
Flag( &flagBind,
&flagStatus, "bind", command.StringFlag(""),
"status", command.BoolFlag(false), "TCP address for the server to listen on",
"Display cure status if available", ).Flag(
). &flagStatus,
Flag( "status", command.BoolFlag(false),
&flagReport, "Display cure status if available",
"report", command.StringFlag(""), ).Flag(
"Load cure status from this report file instead of cache", &flagReport,
) "report", command.StringFlag(""),
"Load cure status from this report file instead of cache",
)
} }
c.NewCommand( c.NewCommand(
@@ -287,7 +280,9 @@ func main() {
if ext.Isatty(int(w.Fd())) { if ext.Isatty(int(w.Fd())) {
return errors.New("output appears to be a terminal") return errors.New("output appears to be a terminal")
} }
return rosa.WriteReport(msg, w, cache) return cm.Do(func(cache *pkg.Cache) error {
return rosa.WriteReport(msg, w, cache)
})
}, },
) )
@@ -350,14 +345,26 @@ func main() {
" package(s) are out of date")) " package(s) are out of date"))
} }
return errors.Join(errs...) return errors.Join(errs...)
}). }).Flag(
Flag( &flagJobs,
&flagJobs, "j", command.IntFlag(32),
"j", command.IntFlag(32), "Maximum number of simultaneous connections",
"Maximum number of simultaneous connections", )
)
} }
c.NewCommand(
"daemon",
"Service artifact IR with Rosa OS extensions",
func(args []string) error {
ul, err := net.ListenUnix("unix", &addr)
if err != nil {
return err
}
log.Printf("listening on pathname socket at %s", addr.Name)
return serve(ctx, log.Default(), &cm, ul)
},
)
{ {
var ( var (
flagGentoo string flagGentoo string
@@ -382,25 +389,37 @@ func main() {
rosa.SetGentooStage3(flagGentoo, checksum) rosa.SetGentooStage3(flagGentoo, checksum)
} }
_, _, _, stage1 := (t - 2).NewLLVM()
_, _, _, stage2 := (t - 1).NewLLVM()
_, _, _, stage3 := t.NewLLVM()
var ( var (
pathname *check.Absolute pathname *check.Absolute
checksum [2]unique.Handle[pkg.Checksum] checksum [2]unique.Handle[pkg.Checksum]
) )
if pathname, _, err = cache.Cure(stage1); err != nil { if err = cm.Do(func(cache *pkg.Cache) (err error) {
return err pathname, _, err = cache.Cure(
(t - 2).Load(rosa.LLVM),
)
return
}); err != nil {
return
} }
log.Println("stage1:", pathname) log.Println("stage1:", pathname)
if pathname, checksum[0], err = cache.Cure(stage2); err != nil { if err = cm.Do(func(cache *pkg.Cache) (err error) {
return err pathname, checksum[0], err = cache.Cure(
(t - 1).Load(rosa.LLVM),
)
return
}); err != nil {
return
} }
log.Println("stage2:", pathname) log.Println("stage2:", pathname)
if pathname, checksum[1], err = cache.Cure(stage3); err != nil { if err = cm.Do(func(cache *pkg.Cache) (err error) {
return err pathname, checksum[1], err = cache.Cure(
t.Load(rosa.LLVM),
)
return
}); err != nil {
return
} }
log.Println("stage3:", pathname) log.Println("stage3:", pathname)
@@ -417,39 +436,44 @@ func main() {
} }
if flagStage0 { if flagStage0 {
if pathname, _, err = cache.Cure( if err = cm.Do(func(cache *pkg.Cache) (err error) {
t.Load(rosa.Stage0), pathname, _, err = cache.Cure(
); err != nil { t.Load(rosa.Stage0),
return err )
return
}); err != nil {
return
} }
log.Println(pathname) log.Println(pathname)
} }
return return
}, },
). ).Flag(
Flag( &flagGentoo,
&flagGentoo, "gentoo", command.StringFlag(""),
"gentoo", command.StringFlag(""), "Bootstrap from a Gentoo stage3 tarball",
"Bootstrap from a Gentoo stage3 tarball", ).Flag(
). &flagChecksum,
Flag( "checksum", command.StringFlag(""),
&flagChecksum, "Checksum of Gentoo stage3 tarball",
"checksum", command.StringFlag(""), ).Flag(
"Checksum of Gentoo stage3 tarball", &flagStage0,
). "stage0", command.BoolFlag(false),
Flag( "Create bootstrap stage0 tarball",
&flagStage0, )
"stage0", command.BoolFlag(false),
"Create bootstrap stage0 tarball",
)
} }
{ {
var ( var (
flagDump string flagDump string
flagEnter bool flagEnter bool
flagExport string flagExport string
flagRemote bool
flagNoReply bool
flagBoot bool
flagStd bool
) )
c.NewCommand( c.NewCommand(
"cure", "cure",
@@ -463,9 +487,20 @@ func main() {
return fmt.Errorf("unknown artifact %q", args[0]) return fmt.Errorf("unknown artifact %q", args[0])
} }
t := rosa.Std
if flagBoot {
t -= 2
} else if flagStd {
t -= 1
}
switch { switch {
default: default:
pathname, _, err := cache.Cure(rosa.Std.Load(p)) var pathname *check.Absolute
err := cm.Do(func(cache *pkg.Cache) (err error) {
pathname, _, err = cache.Cure(t.Load(p))
return
})
if err != nil { if err != nil {
return err return err
} }
@@ -505,7 +540,7 @@ func main() {
return err return err
} }
if err = cache.EncodeAll(f, rosa.Std.Load(p)); err != nil { if err = pkg.NewIR().EncodeAll(f, rosa.Std.Load(p)); err != nil {
_ = f.Close() _ = f.Close()
return err return err
} }
@@ -513,33 +548,76 @@ func main() {
return f.Close() return f.Close()
case flagEnter: case flagEnter:
return cache.EnterExec( return cm.Do(func(cache *pkg.Cache) error {
ctx, return cache.EnterExec(
rosa.Std.Load(p), ctx,
true, os.Stdin, os.Stdout, os.Stderr, t.Load(p),
rosa.AbsSystem.Append("bin", "mksh"), true, os.Stdin, os.Stdout, os.Stderr,
"sh", rosa.AbsSystem.Append("bin", "mksh"),
) "sh",
)
})
case flagRemote:
var flags uint64
if flagNoReply {
flags |= remoteNoReply
}
a := t.Load(p)
pathname, err := cureRemote(ctx, &addr, a, flags)
if !flagNoReply && err == nil {
log.Println(pathname)
}
if errors.Is(err, context.Canceled) {
cc, cancel := context.WithDeadline(context.Background(), daemonDeadline())
defer cancel()
if _err := cancelRemote(cc, &addr, a, false); _err != nil {
log.Println(err)
}
}
return err
} }
}, },
). ).Flag(
Flag( &flagDump,
&flagDump, "dump", command.StringFlag(""),
"dump", command.StringFlag(""), "Write IR to specified pathname and terminate",
"Write IR to specified pathname and terminate", ).Flag(
). &flagExport,
Flag( "export", command.StringFlag(""),
&flagExport, "Export cured artifact to specified pathname",
"export", command.StringFlag(""), ).Flag(
"Export cured artifact to specified pathname", &flagEnter,
). "enter", command.BoolFlag(false),
Flag( "Enter cure container with an interactive shell",
&flagEnter, ).Flag(
"enter", command.BoolFlag(false), &flagRemote,
"Enter cure container with an interactive shell", "daemon", command.BoolFlag(false),
) "Cure artifact on the daemon",
).Flag(
&flagNoReply,
"no-reply", command.BoolFlag(false),
"Do not receive a reply from the daemon",
).Flag(
&flagBoot,
"boot", command.BoolFlag(false),
"Build on the stage0 toolchain",
).Flag(
&flagStd,
"std", command.BoolFlag(false),
"Build on the intermediate toolchain",
)
} }
c.NewCommand(
"abort",
"Abort all pending cures on the daemon",
func([]string) error { return abortRemote(ctx, &addr, false) },
)
{ {
var ( var (
flagNet bool flagNet bool
@@ -551,7 +629,7 @@ func main() {
"shell", "shell",
"Interactive shell in the specified Rosa OS environment", "Interactive shell in the specified Rosa OS environment",
func(args []string) error { func(args []string) error {
presets := make([]rosa.PArtifact, len(args)) presets := make([]rosa.PArtifact, len(args)+3)
for i, arg := range args { for i, arg := range args {
p, ok := rosa.ResolveName(arg) p, ok := rosa.ResolveName(arg)
if !ok { if !ok {
@@ -559,21 +637,24 @@ func main() {
} }
presets[i] = p presets[i] = p
} }
base := rosa.LLVM
if !flagWithToolchain {
base = rosa.Musl
}
presets = append(presets,
base,
rosa.Mksh,
rosa.Toybox,
)
root := make(pkg.Collect, 0, 6+len(args)) root := make(pkg.Collect, 0, 6+len(args))
root = rosa.Std.AppendPresets(root, presets...) root = rosa.Std.AppendPresets(root, presets...)
if flagWithToolchain { if err := cm.Do(func(cache *pkg.Cache) error {
musl, compilerRT, runtimes, clang := (rosa.Std - 1).NewLLVM() _, _, err := cache.Cure(&root)
root = append(root, musl, compilerRT, runtimes, clang) return err
} else { }); err == nil {
root = append(root, rosa.Std.Load(rosa.Musl))
}
root = append(root,
rosa.Std.Load(rosa.Mksh),
rosa.Std.Load(rosa.Toybox),
)
if _, _, err := cache.Cure(&root); err == nil {
return errors.New("unreachable") return errors.New("unreachable")
} else if !pkg.IsCollected(err) { } else if !pkg.IsCollected(err) {
return err return err
@@ -585,11 +666,22 @@ func main() {
} }
cured := make(map[pkg.Artifact]cureRes) cured := make(map[pkg.Artifact]cureRes)
for _, a := range root { for _, a := range root {
pathname, checksum, err := cache.Cure(a) if err := cm.Do(func(cache *pkg.Cache) error {
if err != nil { pathname, checksum, err := cache.Cure(a)
if err == nil {
cured[a] = cureRes{pathname, checksum}
}
return err
}); err != nil {
return err
}
}
// explicitly open for direct error-free use from this point
if cm.c == nil {
if err := cm.open(); err != nil {
return err return err
} }
cured[a] = cureRes{pathname, checksum}
} }
layers := pkg.PromoteLayers(root, func(a pkg.Artifact) ( layers := pkg.PromoteLayers(root, func(a pkg.Artifact) (
@@ -599,7 +691,7 @@ func main() {
res := cured[a] res := cured[a]
return res.pathname, res.checksum return res.pathname, res.checksum
}, func(i int, d pkg.Artifact) { }, func(i int, d pkg.Artifact) {
r := pkg.Encode(cache.Ident(d).Value()) r := pkg.Encode(cm.c.Ident(d).Value())
if s, ok := d.(fmt.Stringer); ok { if s, ok := d.(fmt.Stringer); ok {
if name := s.String(); name != "" { if name := s.String(); name != "" {
r += "-" + name r += "-" + name
@@ -663,22 +755,19 @@ func main() {
} }
return z.Wait() return z.Wait()
}, },
). ).Flag(
Flag( &flagNet,
&flagNet, "net", command.BoolFlag(false),
"net", command.BoolFlag(false), "Share host net namespace",
"Share host net namespace", ).Flag(
). &flagSession,
Flag( "session", command.BoolFlag(true),
&flagSession, "Retain session",
"session", command.BoolFlag(true), ).Flag(
"Retain session", &flagWithToolchain,
). "with-toolchain", command.BoolFlag(false),
Flag( "Include the stage2 LLVM toolchain",
&flagWithToolchain, )
"with-toolchain", command.BoolFlag(false),
"Include the stage2 LLVM toolchain",
)
} }
@@ -689,9 +778,7 @@ func main() {
) )
c.MustParse(os.Args[1:], func(err error) { c.MustParse(os.Args[1:], func(err error) {
if cache != nil { cm.Close()
cache.Close()
}
if w, ok := err.(interface{ Unwrap() []error }); !ok { if w, ok := err.(interface{ Unwrap() []error }); !ok {
log.Fatal(err) log.Fatal(err)
} else { } else {
+47
View File
@@ -0,0 +1,47 @@
package main
import (
"net"
"os"
"testing"
"hakurei.app/internal/rosa"
)
func TestMain(m *testing.M) {
rosa.DropCaches(rosa.OptLLVMNoLTO)
os.Exit(m.Run())
}
func TestCureAll(t *testing.T) {
t.Parallel()
const env = "ROSA_TEST_DAEMON"
if !testing.Verbose() {
t.Skip("verbose flag not set")
}
pathname, ok := os.LookupEnv(env)
if !ok {
t.Skip(env + " not set")
}
addr := net.UnixAddr{Net: "unix", Name: pathname}
t.Cleanup(func() {
if t.Failed() {
if err := abortRemote(t.Context(), &addr, false); err != nil {
t.Fatal(err)
}
}
})
for i := range rosa.PresetEnd {
p := rosa.PArtifact(i)
t.Run(rosa.GetMetadata(p).Name, func(t *testing.T) {
_, err := cureRemote(t.Context(), &addr, rosa.Std.Load(p), 0)
if err != nil {
t.Error(err)
}
})
}
}
+2 -2
View File
@@ -7,8 +7,8 @@
#endif #endif
#define SHAREFS_MEDIA_RW_ID (1 << 10) - 1 /* owning gid presented to userspace */ #define SHAREFS_MEDIA_RW_ID (1 << 10) - 1 /* owning gid presented to userspace */
#define SHAREFS_PERM_DIR 0700 /* permission bits for directories presented to userspace */ #define SHAREFS_PERM_DIR 0770 /* permission bits for directories presented to userspace */
#define SHAREFS_PERM_REG 0600 /* permission bits for regular files presented to userspace */ #define SHAREFS_PERM_REG 0660 /* permission bits for regular files presented to userspace */
#define SHAREFS_FORBIDDEN_FLAGS O_DIRECT /* these open flags are cleared unconditionally */ #define SHAREFS_FORBIDDEN_FLAGS O_DIRECT /* these open flags are cleared unconditionally */
/* sharefs_private is populated by sharefs_init and contains process-wide context */ /* sharefs_private is populated by sharefs_init and contains process-wide context */
+10 -7
View File
@@ -19,7 +19,6 @@ import (
"encoding/gob" "encoding/gob"
"errors" "errors"
"fmt" "fmt"
"io"
"log" "log"
"os" "os"
"os/exec" "os/exec"
@@ -470,13 +469,14 @@ func _main(s ...string) (exitCode int) {
os.NewFile(uintptr(C.fuse_session_fd(se)), "fuse"), os.NewFile(uintptr(C.fuse_session_fd(se)), "fuse"),
)) ))
var setupWriter io.WriteCloser var setupPipe [2]*os.File
if fd, w, err := container.Setup(&z.ExtraFiles); err != nil { if r, w, err := os.Pipe(); err != nil {
log.Println(err) log.Println(err)
return 5 return 5
} else { } else {
z.Args = append(z.Args, "-osetup="+strconv.Itoa(fd)) z.Args = append(z.Args, "-osetup="+strconv.Itoa(3+len(z.ExtraFiles)))
setupWriter = w z.ExtraFiles = append(z.ExtraFiles, r)
setupPipe[0], setupPipe[1] = r, w
} }
if err := z.Start(); err != nil { if err := z.Start(); err != nil {
@@ -487,6 +487,9 @@ func _main(s ...string) (exitCode int) {
} }
return 5 return 5
} }
if err := setupPipe[0].Close(); err != nil {
log.Println(err)
}
if err := z.Serve(); err != nil { if err := z.Serve(); err != nil {
if m, ok := message.GetMessage(err); ok { if m, ok := message.GetMessage(err); ok {
log.Println(m) log.Println(m)
@@ -496,10 +499,10 @@ func _main(s ...string) (exitCode int) {
return 5 return 5
} }
if err := gob.NewEncoder(setupWriter).Encode(&setup); err != nil { if err := gob.NewEncoder(setupPipe[1]).Encode(&setup); err != nil {
log.Println(err) log.Println(err)
return 5 return 5
} else if err = setupWriter.Close(); err != nil { } else if err = setupPipe[1].Close(); err != nil {
log.Println(err) log.Println(err)
} }
+122
View File
@@ -0,0 +1,122 @@
//go:build raceattr
// The raceattr program reproduces vfs inode file attribute race.
//
// Even though libfuse high-level API presents the address of a struct stat
// alongside struct fuse_context, file attributes are actually inherent to the
// inode, instead of the specific call from userspace. The kernel implementation
// in fs/fuse/xattr.c appears to make stale data in the inode (set by a previous
// call) impossible or very unlikely to reach userspace via the stat family of
// syscalls. However, when using default_permissions to have the VFS check
// permissions, this race still happens, despite the resulting struct stat being
// correct when overriding the check via capabilities otherwise.
//
// This program reproduces the failure, but because of its continuous nature, it
// is provided independent of the vm integration test suite.
package main
import (
"context"
"flag"
"log"
"os"
"os/signal"
"runtime"
"sync"
"sync/atomic"
"syscall"
)
func newStatAs(
ctx context.Context, cancel context.CancelFunc,
n *atomic.Uint64, ok *atomic.Bool,
uid uint32, pathname string,
continuous bool,
) func() {
return func() {
runtime.LockOSThread()
defer cancel()
if _, _, errno := syscall.Syscall(
syscall.SYS_SETUID, uintptr(uid),
0, 0,
); errno != 0 {
cancel()
log.Printf("cannot set uid to %d: %s", uid, errno)
}
var stat syscall.Stat_t
for {
if ctx.Err() != nil {
return
}
if err := syscall.Lstat(pathname, &stat); err != nil {
// SHAREFS_PERM_DIR not world executable, or
// SHAREFS_PERM_REG not world readable
if !continuous {
cancel()
}
ok.Store(true)
log.Printf("uid %d: %v", uid, err)
} else if stat.Uid != uid {
// appears to be unreachable
if !continuous {
cancel()
}
ok.Store(true)
log.Printf("got uid %d instead of %d", stat.Uid, uid)
}
n.Add(1)
}
}
}
func main() {
log.SetFlags(0)
log.SetPrefix("raceattr: ")
p := flag.String("target", "/sdcard/raceattr", "pathname of test file")
u0 := flag.Int("uid0", 1<<10-1, "first uid")
u1 := flag.Int("uid1", 1<<10-2, "second uid")
count := flag.Int("count", 1, "threads per uid")
continuous := flag.Bool("continuous", false, "keep running even after reproduce")
flag.Parse()
if os.Geteuid() != 0 {
log.Fatal("this program must run as root")
}
ctx, cancel := signal.NotifyContext(
context.Background(),
syscall.SIGINT,
syscall.SIGTERM,
syscall.SIGHUP,
)
if err := os.WriteFile(*p, nil, 0); err != nil {
log.Fatal(err)
}
var (
wg sync.WaitGroup
n atomic.Uint64
ok atomic.Bool
)
if *count < 1 {
*count = 1
}
for range *count {
wg.Go(newStatAs(ctx, cancel, &n, &ok, uint32(*u0), *p, *continuous))
if *u1 >= 0 {
wg.Go(newStatAs(ctx, cancel, &n, &ok, uint32(*u1), *p, *continuous))
}
}
wg.Wait()
if !*continuous && ok.Load() {
log.Printf("reproduced after %d calls", n.Load())
}
}
+46 -26
View File
@@ -21,6 +21,7 @@ import (
"hakurei.app/container/std" "hakurei.app/container/std"
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/internal/landlock"
"hakurei.app/message" "hakurei.app/message"
) )
@@ -28,9 +29,6 @@ const (
// CancelSignal is the signal expected by container init on context cancel. // CancelSignal is the signal expected by container init on context cancel.
// A custom [Container.Cancel] function must eventually deliver this signal. // A custom [Container.Cancel] function must eventually deliver this signal.
CancelSignal = SIGUSR2 CancelSignal = SIGUSR2
// Timeout for writing initParams to Container.setup.
initSetupTimeout = 5 * time.Second
) )
type ( type (
@@ -53,7 +51,7 @@ type (
ExtraFiles []*os.File ExtraFiles []*os.File
// Write end of a pipe connected to the init to deliver [Params]. // Write end of a pipe connected to the init to deliver [Params].
setup *os.File setup [2]*os.File
// Cancels the context passed to the underlying cmd. // Cancels the context passed to the underlying cmd.
cancel context.CancelFunc cancel context.CancelFunc
// Closed after Wait returns. Keeps the spawning thread alive. // Closed after Wait returns. Keeps the spawning thread alive.
@@ -287,14 +285,16 @@ func (p *Container) Start() error {
} }
// place setup pipe before user supplied extra files, this is later restored by init // place setup pipe before user supplied extra files, this is later restored by init
if fd, f, err := Setup(&p.cmd.ExtraFiles); err != nil { if r, w, err := os.Pipe(); err != nil {
return &StartError{ return &StartError{
Fatal: true, Fatal: true,
Step: "set up params stream", Step: "set up params stream",
Err: err, Err: err,
} }
} else { } else {
p.setup = f fd := 3 + len(p.cmd.ExtraFiles)
p.cmd.ExtraFiles = append(p.cmd.ExtraFiles, r)
p.setup[0], p.setup[1] = r, w
p.cmd.Env = []string{setupEnv + "=" + strconv.Itoa(fd)} p.cmd.Env = []string{setupEnv + "=" + strconv.Itoa(fd)}
} }
p.cmd.ExtraFiles = append(p.cmd.ExtraFiles, p.ExtraFiles...) p.cmd.ExtraFiles = append(p.cmd.ExtraFiles, p.ExtraFiles...)
@@ -308,7 +308,7 @@ func (p *Container) Start() error {
done <- func() error { done <- func() error {
// PR_SET_NO_NEW_PRIVS: thread-directed but acts on all processes // PR_SET_NO_NEW_PRIVS: thread-directed but acts on all processes
// created from the calling thread // created from the calling thread
if err := SetNoNewPrivs(); err != nil { if err := setNoNewPrivs(); err != nil {
return &StartError{ return &StartError{
Fatal: true, Fatal: true,
Step: "prctl(PR_SET_NO_NEW_PRIVS)", Step: "prctl(PR_SET_NO_NEW_PRIVS)",
@@ -318,15 +318,17 @@ func (p *Container) Start() error {
// landlock: depends on per-thread state but acts on a process group // landlock: depends on per-thread state but acts on a process group
{ {
rulesetAttr := &RulesetAttr{Scoped: LANDLOCK_SCOPE_SIGNAL} rulesetAttr := &landlock.RulesetAttr{
Scoped: landlock.LANDLOCK_SCOPE_SIGNAL,
}
if !p.HostAbstract { if !p.HostAbstract {
rulesetAttr.Scoped |= LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET rulesetAttr.Scoped |= landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET
} }
if abi, err := LandlockGetABI(); err != nil { if abi, err := landlock.GetABI(); err != nil {
if p.HostAbstract { if p.HostAbstract || !p.HostNet {
// landlock can be skipped here as it restricts access // landlock can be skipped here as it restricts access
// to resources already covered by namespaces (pid) // to resources already covered by namespaces (pid, net)
goto landlockOut goto landlockOut
} }
return &StartError{Step: "get landlock ABI", Err: err} return &StartError{Step: "get landlock ABI", Err: err}
@@ -352,7 +354,7 @@ func (p *Container) Start() error {
} }
} else { } else {
p.msg.Verbosef("enforcing landlock ruleset %s", rulesetAttr) p.msg.Verbosef("enforcing landlock ruleset %s", rulesetAttr)
if err = LandlockRestrictSelf(rulesetFd, 0); err != nil { if err = landlock.RestrictSelf(rulesetFd, 0); err != nil {
_ = Close(rulesetFd) _ = Close(rulesetFd)
return &StartError{ return &StartError{
Fatal: true, Fatal: true,
@@ -428,24 +430,33 @@ func (p *Container) Start() error {
// Serve serves [Container.Params] to the container init. // Serve serves [Container.Params] to the container init.
// //
// Serve must only be called once. // Serve must only be called once.
func (p *Container) Serve() error { func (p *Container) Serve() (err error) {
if p.setup == nil { if p.setup[0] == nil || p.setup[1] == nil {
panic("invalid serve") panic("invalid serve")
} }
setup := p.setup done := make(chan struct{})
p.setup = nil defer func() {
if err := setup.SetDeadline(time.Now().Add(initSetupTimeout)); err != nil { if closeErr := p.setup[1].Close(); err == nil {
err = closeErr
}
if err != nil {
p.cancel()
}
close(done)
p.setup[0], p.setup[1] = nil, nil
}()
if err = p.setup[0].Close(); err != nil {
return &StartError{ return &StartError{
Fatal: true, Fatal: true,
Step: "set init pipe deadline", Step: "close read end of init pipe",
Err: err, Err: err,
Passthrough: true, Passthrough: true,
} }
} }
if p.Path == nil { if p.Path == nil {
p.cancel()
return &StartError{ return &StartError{
Step: "invalid executable pathname", Step: "invalid executable pathname",
Err: EINVAL, Err: EINVAL,
@@ -461,18 +472,27 @@ func (p *Container) Serve() error {
p.SeccompRules = make([]std.NativeRule, 0) p.SeccompRules = make([]std.NativeRule, 0)
} }
err := gob.NewEncoder(setup).Encode(&initParams{ t := time.Now().UTC()
go func(f *os.File) {
select {
case <-p.ctx.Done():
if cancelErr := f.SetWriteDeadline(t); cancelErr != nil {
p.msg.Verbose(err)
}
case <-done:
p.msg.Verbose("setup payload took", time.Since(t))
return
}
}(p.setup[1])
return gob.NewEncoder(p.setup[1]).Encode(&initParams{
p.Params, p.Params,
Getuid(), Getuid(),
Getgid(), Getgid(),
len(p.ExtraFiles), len(p.ExtraFiles),
p.msg.IsVerbose(), p.msg.IsVerbose(),
}) })
_ = setup.Close()
if err != nil {
p.cancel()
}
return err
} }
// Wait blocks until the container init process to exit and releases any // Wait blocks until the container init process to exit and releases any
+33 -20
View File
@@ -16,7 +16,7 @@ import (
"strings" "strings"
"syscall" "syscall"
"testing" "testing"
"time" _ "unsafe" // for go:linkname
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/command" "hakurei.app/command"
@@ -26,6 +26,9 @@ import (
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/hst" "hakurei.app/hst"
"hakurei.app/internal/info"
"hakurei.app/internal/landlock"
"hakurei.app/internal/params"
"hakurei.app/ldd" "hakurei.app/ldd"
"hakurei.app/message" "hakurei.app/message"
"hakurei.app/vfs" "hakurei.app/vfs"
@@ -84,9 +87,9 @@ func TestStartError(t *testing.T) {
{"params env", &container.StartError{ {"params env", &container.StartError{
Fatal: true, Fatal: true,
Step: "set up params stream", Step: "set up params stream",
Err: container.ErrReceiveEnv, Err: params.ErrReceiveEnv,
}, "set up params stream: environment variable not set", }, "set up params stream: environment variable not set",
container.ErrReceiveEnv, syscall.EBADF, params.ErrReceiveEnv, syscall.EBADF,
"cannot set up params stream: environment variable not set"}, "cannot set up params stream: environment variable not set"},
{"params", &container.StartError{ {"params", &container.StartError{
@@ -231,6 +234,9 @@ func earlyMnt(mnt ...*vfs.MountInfoEntry) func(*testing.T, context.Context) []*v
return func(*testing.T, context.Context) []*vfs.MountInfoEntry { return mnt } return func(*testing.T, context.Context) []*vfs.MountInfoEntry { return mnt }
} }
//go:linkname toHost hakurei.app/container.toHost
func toHost(name string) string
var containerTestCases = []struct { var containerTestCases = []struct {
name string name string
filter bool filter bool
@@ -330,13 +336,15 @@ var containerTestCases = []struct {
func(t *testing.T, ctx context.Context) []*vfs.MountInfoEntry { func(t *testing.T, ctx context.Context) []*vfs.MountInfoEntry {
return []*vfs.MountInfoEntry{ return []*vfs.MountInfoEntry{
ent("/", hst.PrivateTmp, "rw", "overlay", "overlay", ent("/", hst.PrivateTmp, "rw", "overlay", "overlay",
"rw,lowerdir="+ "rw"+
container.InternalToHostOvlEscape(ctx.Value(testVal("lower0")).(*check.Absolute).String())+":"+ ",lowerdir+="+
container.InternalToHostOvlEscape(ctx.Value(testVal("lower1")).(*check.Absolute).String())+ toHost(ctx.Value(testVal("lower0")).(*check.Absolute).String())+
",lowerdir+="+
toHost(ctx.Value(testVal("lower1")).(*check.Absolute).String())+
",upperdir="+ ",upperdir="+
container.InternalToHostOvlEscape(ctx.Value(testVal("upper")).(*check.Absolute).String())+ toHost(ctx.Value(testVal("upper")).(*check.Absolute).String())+
",workdir="+ ",workdir="+
container.InternalToHostOvlEscape(ctx.Value(testVal("work")).(*check.Absolute).String())+ toHost(ctx.Value(testVal("work")).(*check.Absolute).String())+
",redirect_dir=nofollow,uuid=on,userxattr"), ",redirect_dir=nofollow,uuid=on,userxattr"),
} }
}, },
@@ -386,9 +394,11 @@ var containerTestCases = []struct {
func(t *testing.T, ctx context.Context) []*vfs.MountInfoEntry { func(t *testing.T, ctx context.Context) []*vfs.MountInfoEntry {
return []*vfs.MountInfoEntry{ return []*vfs.MountInfoEntry{
ent("/", hst.PrivateTmp, "rw", "overlay", "overlay", ent("/", hst.PrivateTmp, "rw", "overlay", "overlay",
"ro,lowerdir="+ "ro"+
container.InternalToHostOvlEscape(ctx.Value(testVal("lower0")).(*check.Absolute).String())+":"+ ",lowerdir+="+
container.InternalToHostOvlEscape(ctx.Value(testVal("lower1")).(*check.Absolute).String())+ toHost(ctx.Value(testVal("lower0")).(*check.Absolute).String())+
",lowerdir+="+
toHost(ctx.Value(testVal("lower1")).(*check.Absolute).String())+
",redirect_dir=nofollow,userxattr"), ",redirect_dir=nofollow,userxattr"),
} }
}, },
@@ -436,11 +446,8 @@ func TestContainer(t *testing.T) {
wantOps, wantOpsCtx := tc.ops(t) wantOps, wantOpsCtx := tc.ops(t)
wantMnt := tc.mnt(t, wantOpsCtx) wantMnt := tc.mnt(t, wantOpsCtx)
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout)
defer cancel()
var libPaths []*check.Absolute var libPaths []*check.Absolute
c := helperNewContainerLibPaths(ctx, &libPaths, "container", strconv.Itoa(i)) c := helperNewContainerLibPaths(t.Context(), &libPaths, "container", strconv.Itoa(i))
c.Uid = tc.uid c.Uid = tc.uid
c.Gid = tc.gid c.Gid = tc.gid
c.Hostname = hostnameFromTestCase(tc.name) c.Hostname = hostnameFromTestCase(tc.name)
@@ -450,7 +457,6 @@ func TestContainer(t *testing.T) {
} else { } else {
c.Stdout, c.Stderr = os.Stdout, os.Stderr c.Stdout, c.Stderr = os.Stdout, os.Stderr
} }
c.WaitDelay = helperDefaultTimeout
*c.Ops = append(*c.Ops, *wantOps...) *c.Ops = append(*c.Ops, *wantOps...)
c.SeccompRules = tc.rules c.SeccompRules = tc.rules
c.SeccompFlags = tc.flags | seccomp.AllowMultiarch c.SeccompFlags = tc.flags | seccomp.AllowMultiarch
@@ -458,6 +464,15 @@ func TestContainer(t *testing.T) {
c.SeccompDisable = !tc.filter c.SeccompDisable = !tc.filter
c.RetainSession = tc.session c.RetainSession = tc.session
c.HostNet = tc.net c.HostNet = tc.net
if info.CanDegrade {
if _, err := landlock.GetABI(); err != nil {
if !errors.Is(err, syscall.ENOSYS) {
t.Fatalf("LandlockGetABI: error = %v", err)
}
c.HostAbstract = true
t.Log("Landlock LSM is unavailable, enabling HostAbstract")
}
}
c. c.
Readonly(check.MustAbs(pathReadonly), 0755). Readonly(check.MustAbs(pathReadonly), 0755).
@@ -553,11 +568,10 @@ func testContainerCancel(
) func(t *testing.T) { ) func(t *testing.T) {
return func(t *testing.T) { return func(t *testing.T) {
t.Parallel() t.Parallel()
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout) ctx, cancel := context.WithCancel(t.Context())
c := helperNewContainer(ctx, "block") c := helperNewContainer(ctx, "block")
c.Stdout, c.Stderr = os.Stdout, os.Stderr c.Stdout, c.Stderr = os.Stdout, os.Stderr
c.WaitDelay = helperDefaultTimeout
if containerExtra != nil { if containerExtra != nil {
containerExtra(c) containerExtra(c)
} }
@@ -738,8 +752,7 @@ func init() {
const ( const (
envDoCheck = "HAKUREI_TEST_DO_CHECK" envDoCheck = "HAKUREI_TEST_DO_CHECK"
helperDefaultTimeout = 5 * time.Second helperInnerPath = "/usr/bin/helper"
helperInnerPath = "/usr/bin/helper"
) )
var ( var (
+10 -4
View File
@@ -16,6 +16,7 @@ import (
"hakurei.app/container/std" "hakurei.app/container/std"
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/internal/netlink" "hakurei.app/internal/netlink"
"hakurei.app/internal/params"
"hakurei.app/message" "hakurei.app/message"
) )
@@ -56,7 +57,7 @@ type syscallDispatcher interface {
// isatty provides [Isatty]. // isatty provides [Isatty].
isatty(fd int) bool isatty(fd int) bool
// receive provides [Receive]. // receive provides [Receive].
receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) receive(key string, e any, fdp *int) (closeFunc func() error, err error)
// bindMount provides procPaths.bindMount. // bindMount provides procPaths.bindMount.
bindMount(msg message.Msg, source, target string, flags uintptr) error bindMount(msg message.Msg, source, target string, flags uintptr) error
@@ -64,6 +65,8 @@ type syscallDispatcher interface {
remount(msg message.Msg, target string, flags uintptr) error remount(msg message.Msg, target string, flags uintptr) error
// mountTmpfs provides mountTmpfs. // mountTmpfs provides mountTmpfs.
mountTmpfs(fsname, target string, flags uintptr, size int, perm os.FileMode) error mountTmpfs(fsname, target string, flags uintptr, size int, perm os.FileMode) error
// mountOverlay provides mountOverlay.
mountOverlay(target string, options [][2]string) error
// ensureFile provides ensureFile. // ensureFile provides ensureFile.
ensureFile(name string, perm, pperm os.FileMode) error ensureFile(name string, perm, pperm os.FileMode) error
// mustLoopback provides mustLoopback. // mustLoopback provides mustLoopback.
@@ -147,7 +150,7 @@ func (direct) lockOSThread() { runtime.LockOSThread() }
func (direct) setPtracer(pid uintptr) error { return ext.SetPtracer(pid) } func (direct) setPtracer(pid uintptr) error { return ext.SetPtracer(pid) }
func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) } func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) }
func (direct) setNoNewPrivs() error { return SetNoNewPrivs() } func (direct) setNoNewPrivs() error { return setNoNewPrivs() }
func (direct) lastcap(msg message.Msg) uintptr { return LastCap(msg) } func (direct) lastcap(msg message.Msg) uintptr { return LastCap(msg) }
func (direct) capset(hdrp *capHeader, datap *[2]capData) error { return capset(hdrp, datap) } func (direct) capset(hdrp *capHeader, datap *[2]capData) error { return capset(hdrp, datap) }
@@ -155,8 +158,8 @@ func (direct) capBoundingSetDrop(cap uintptr) error { return capBound
func (direct) capAmbientClearAll() error { return capAmbientClearAll() } func (direct) capAmbientClearAll() error { return capAmbientClearAll() }
func (direct) capAmbientRaise(cap uintptr) error { return capAmbientRaise(cap) } func (direct) capAmbientRaise(cap uintptr) error { return capAmbientRaise(cap) }
func (direct) isatty(fd int) bool { return ext.Isatty(fd) } func (direct) isatty(fd int) bool { return ext.Isatty(fd) }
func (direct) receive(key string, e any, fdp *uintptr) (func() error, error) { func (direct) receive(key string, e any, fdp *int) (func() error, error) {
return Receive(key, e, fdp) return params.Receive(key, e, fdp)
} }
func (direct) bindMount(msg message.Msg, source, target string, flags uintptr) error { func (direct) bindMount(msg message.Msg, source, target string, flags uintptr) error {
@@ -168,6 +171,9 @@ func (direct) remount(msg message.Msg, target string, flags uintptr) error {
func (k direct) mountTmpfs(fsname, target string, flags uintptr, size int, perm os.FileMode) error { func (k direct) mountTmpfs(fsname, target string, flags uintptr, size int, perm os.FileMode) error {
return mountTmpfs(k, fsname, target, flags, size, perm) return mountTmpfs(k, fsname, target, flags, size, perm)
} }
func (k direct) mountOverlay(target string, options [][2]string) error {
return mountOverlay(target, options)
}
func (direct) ensureFile(name string, perm, pperm os.FileMode) error { func (direct) ensureFile(name string, perm, pperm os.FileMode) error {
return ensureFile(name, perm, pperm) return ensureFile(name, perm, pperm)
} }
+18 -3
View File
@@ -390,7 +390,7 @@ func (k *kstub) isatty(fd int) bool {
return expect.Ret.(bool) return expect.Ret.(bool)
} }
func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) { func (k *kstub) receive(key string, e any, fdp *int) (closeFunc func() error, err error) {
k.Helper() k.Helper()
expect := k.Expects("receive") expect := k.Expects("receive")
@@ -408,10 +408,17 @@ func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error
} }
return nil return nil
} }
// avoid changing test cases
var fdpComp *uintptr
if fdp != nil {
fdpComp = new(uintptr(*fdp))
}
err = expect.Error( err = expect.Error(
stub.CheckArg(k.Stub, "key", key, 0), stub.CheckArg(k.Stub, "key", key, 0),
stub.CheckArgReflect(k.Stub, "e", e, 1), stub.CheckArgReflect(k.Stub, "e", e, 1),
stub.CheckArgReflect(k.Stub, "fdp", fdp, 2)) stub.CheckArgReflect(k.Stub, "fdp", fdpComp, 2))
// 3 is unused so stores params // 3 is unused so stores params
if expect.Args[3] != nil { if expect.Args[3] != nil {
@@ -426,7 +433,7 @@ func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error
if expect.Args[4] != nil { if expect.Args[4] != nil {
if v, ok := expect.Args[4].(uintptr); ok && v >= 3 { if v, ok := expect.Args[4].(uintptr); ok && v >= 3 {
if fdp != nil { if fdp != nil {
*fdp = v *fdp = int(v)
} }
} }
} }
@@ -461,6 +468,14 @@ func (k *kstub) mountTmpfs(fsname, target string, flags uintptr, size int, perm
stub.CheckArg(k.Stub, "perm", perm, 4)) stub.CheckArg(k.Stub, "perm", perm, 4))
} }
func (k *kstub) mountOverlay(target string, options [][2]string) error {
k.Helper()
return k.Expects("mountOverlay").Error(
stub.CheckArg(k.Stub, "target", target, 0),
stub.CheckArgReflect(k.Stub, "options", options, 1),
)
}
func (k *kstub) ensureFile(name string, perm, pperm os.FileMode) error { func (k *kstub) ensureFile(name string, perm, pperm os.FileMode) error {
k.Helper() k.Helper()
return k.Expects("ensureFile").Error( return k.Expects("ensureFile").Error(
+4
View File
@@ -118,6 +118,10 @@ func errnoFallback(op, path string, err error) (syscall.Errno, *os.PathError) {
// mount wraps syscall.Mount for error handling. // mount wraps syscall.Mount for error handling.
func mount(source, target, fstype string, flags uintptr, data string) error { func mount(source, target, fstype string, flags uintptr, data string) error {
if max(len(source), len(target), len(data))+1 > os.Getpagesize() {
return &MountError{source, target, fstype, flags, data, syscall.ENOMEM}
}
err := syscall.Mount(source, target, fstype, flags, data) err := syscall.Mount(source, target, fstype, flags, data)
if err == nil { if err == nil {
return nil return nil
+36 -37
View File
@@ -19,6 +19,7 @@ import (
"hakurei.app/container/seccomp" "hakurei.app/container/seccomp"
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/internal/params"
"hakurei.app/message" "hakurei.app/message"
) )
@@ -147,35 +148,33 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
} }
var ( var (
params initParams param initParams
closeSetup func() error closeSetup func() error
setupFd uintptr setupFd int
offsetSetup int
) )
if f, err := k.receive(setupEnv, &params, &setupFd); err != nil { if f, err := k.receive(setupEnv, &param, &setupFd); err != nil {
if errors.Is(err, EBADF) { if errors.Is(err, EBADF) {
k.fatal(msg, "invalid setup descriptor") k.fatal(msg, "invalid setup descriptor")
} }
if errors.Is(err, ErrReceiveEnv) { if errors.Is(err, params.ErrReceiveEnv) {
k.fatal(msg, setupEnv+" not set") k.fatal(msg, setupEnv+" not set")
} }
k.fatalf(msg, "cannot decode init setup payload: %v", err) k.fatalf(msg, "cannot decode init setup payload: %v", err)
} else { } else {
if params.Ops == nil { if param.Ops == nil {
k.fatal(msg, "invalid setup parameters") k.fatal(msg, "invalid setup parameters")
} }
if params.ParentPerm == 0 { if param.ParentPerm == 0 {
params.ParentPerm = 0755 param.ParentPerm = 0755
} }
msg.SwapVerbose(params.Verbose) msg.SwapVerbose(param.Verbose)
msg.Verbose("received setup parameters") msg.Verbose("received setup parameters")
closeSetup = f closeSetup = f
offsetSetup = int(setupFd + 1)
} }
if !params.HostNet { if !param.HostNet {
ctx, cancel := signal.NotifyContext(context.Background(), CancelSignal, ctx, cancel := signal.NotifyContext(context.Background(), CancelSignal,
os.Interrupt, SIGTERM, SIGQUIT) os.Interrupt, SIGTERM, SIGQUIT)
defer cancel() // for panics defer cancel() // for panics
@@ -188,7 +187,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot set SUID_DUMP_USER: %v", err) k.fatalf(msg, "cannot set SUID_DUMP_USER: %v", err)
} }
if err := k.writeFile(fhs.Proc+"self/uid_map", if err := k.writeFile(fhs.Proc+"self/uid_map",
append([]byte{}, strconv.Itoa(params.Uid)+" "+strconv.Itoa(params.HostUid)+" 1\n"...), append([]byte{}, strconv.Itoa(param.Uid)+" "+strconv.Itoa(param.HostUid)+" 1\n"...),
0); err != nil { 0); err != nil {
k.fatalf(msg, "%v", err) k.fatalf(msg, "%v", err)
} }
@@ -198,7 +197,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "%v", err) k.fatalf(msg, "%v", err)
} }
if err := k.writeFile(fhs.Proc+"self/gid_map", if err := k.writeFile(fhs.Proc+"self/gid_map",
append([]byte{}, strconv.Itoa(params.Gid)+" "+strconv.Itoa(params.HostGid)+" 1\n"...), append([]byte{}, strconv.Itoa(param.Gid)+" "+strconv.Itoa(param.HostGid)+" 1\n"...),
0); err != nil { 0); err != nil {
k.fatalf(msg, "%v", err) k.fatalf(msg, "%v", err)
} }
@@ -207,8 +206,8 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
} }
oldmask := k.umask(0) oldmask := k.umask(0)
if params.Hostname != "" { if param.Hostname != "" {
if err := k.sethostname([]byte(params.Hostname)); err != nil { if err := k.sethostname([]byte(param.Hostname)); err != nil {
k.fatalf(msg, "cannot set hostname: %v", err) k.fatalf(msg, "cannot set hostname: %v", err)
} }
} }
@@ -221,7 +220,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
} }
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
state := &setupState{process: make(map[int]WaitStatus), Params: &params.Params, Msg: msg, Context: ctx} state := &setupState{process: make(map[int]WaitStatus), Params: &param.Params, Msg: msg, Context: ctx}
defer cancel() defer cancel()
/* early is called right before pivot_root into intermediate root; /* early is called right before pivot_root into intermediate root;
@@ -229,7 +228,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
difficult to obtain via library functions after pivot_root, and difficult to obtain via library functions after pivot_root, and
implementations are expected to avoid changing the state of the mount implementations are expected to avoid changing the state of the mount
namespace */ namespace */
for i, op := range *params.Ops { for i, op := range *param.Ops {
if op == nil || !op.Valid() { if op == nil || !op.Valid() {
k.fatalf(msg, "invalid op at index %d", i) k.fatalf(msg, "invalid op at index %d", i)
} }
@@ -272,7 +271,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
step sets up the container filesystem, and implementations are expected to step sets up the container filesystem, and implementations are expected to
keep the host root and sysroot mount points intact but otherwise can do keep the host root and sysroot mount points intact but otherwise can do
whatever they need to. Calling chdir is allowed but discouraged. */ whatever they need to. Calling chdir is allowed but discouraged. */
for i, op := range *params.Ops { for i, op := range *param.Ops {
// ops already checked during early setup // ops already checked during early setup
if prefix, ok := op.prefix(); ok { if prefix, ok := op.prefix(); ok {
msg.Verbosef("%s %s", prefix, op) msg.Verbosef("%s %s", prefix, op)
@@ -328,7 +327,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot clear the ambient capability set: %v", err) k.fatalf(msg, "cannot clear the ambient capability set: %v", err)
} }
for i := uintptr(0); i <= lastcap; i++ { for i := uintptr(0); i <= lastcap; i++ {
if params.Privileged && i == CAP_SYS_ADMIN { if param.Privileged && i == CAP_SYS_ADMIN {
continue continue
} }
if err := k.capBoundingSetDrop(i); err != nil { if err := k.capBoundingSetDrop(i); err != nil {
@@ -337,7 +336,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
} }
var keep [2]uint32 var keep [2]uint32
if params.Privileged { if param.Privileged {
keep[capToIndex(CAP_SYS_ADMIN)] |= capToMask(CAP_SYS_ADMIN) keep[capToIndex(CAP_SYS_ADMIN)] |= capToMask(CAP_SYS_ADMIN)
if err := k.capAmbientRaise(CAP_SYS_ADMIN); err != nil { if err := k.capAmbientRaise(CAP_SYS_ADMIN); err != nil {
@@ -351,13 +350,13 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot capset: %v", err) k.fatalf(msg, "cannot capset: %v", err)
} }
if !params.SeccompDisable { if !param.SeccompDisable {
rules := params.SeccompRules rules := param.SeccompRules
if len(rules) == 0 { // non-empty rules slice always overrides presets if len(rules) == 0 { // non-empty rules slice always overrides presets
msg.Verbosef("resolving presets %#x", params.SeccompPresets) msg.Verbosef("resolving presets %#x", param.SeccompPresets)
rules = seccomp.Preset(params.SeccompPresets, params.SeccompFlags) rules = seccomp.Preset(param.SeccompPresets, param.SeccompFlags)
} }
if err := k.seccompLoad(rules, params.SeccompFlags); err != nil { if err := k.seccompLoad(rules, param.SeccompFlags); err != nil {
// this also indirectly asserts PR_SET_NO_NEW_PRIVS // this also indirectly asserts PR_SET_NO_NEW_PRIVS
k.fatalf(msg, "cannot load syscall filter: %v", err) k.fatalf(msg, "cannot load syscall filter: %v", err)
} }
@@ -366,10 +365,10 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
msg.Verbose("syscall filter not configured") msg.Verbose("syscall filter not configured")
} }
extraFiles := make([]*os.File, params.Count) extraFiles := make([]*os.File, param.Count)
for i := range extraFiles { for i := range extraFiles {
// setup fd is placed before all extra files // setup fd is placed before all extra files
extraFiles[i] = k.newFile(uintptr(offsetSetup+i), "extra file "+strconv.Itoa(i)) extraFiles[i] = k.newFile(uintptr(setupFd+1+i), "extra file "+strconv.Itoa(i))
} }
k.umask(oldmask) k.umask(oldmask)
@@ -447,7 +446,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
// called right before startup of initial process, all state changes to the // called right before startup of initial process, all state changes to the
// current process is prohibited during late // current process is prohibited during late
for i, op := range *params.Ops { for i, op := range *param.Ops {
// ops already checked during early setup // ops already checked during early setup
if err := op.late(state, k); err != nil { if err := op.late(state, k); err != nil {
if m, ok := messageFromError(err); ok { if m, ok := messageFromError(err); ok {
@@ -468,14 +467,14 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
k.fatalf(msg, "cannot close setup pipe: %v", err) k.fatalf(msg, "cannot close setup pipe: %v", err)
} }
cmd := exec.Command(params.Path.String()) cmd := exec.Command(param.Path.String())
cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
cmd.Args = params.Args cmd.Args = param.Args
cmd.Env = params.Env cmd.Env = param.Env
cmd.ExtraFiles = extraFiles cmd.ExtraFiles = extraFiles
cmd.Dir = params.Dir.String() cmd.Dir = param.Dir.String()
msg.Verbosef("starting initial process %s", params.Path) msg.Verbosef("starting initial process %s", param.Path)
if err := k.start(cmd); err != nil { if err := k.start(cmd); err != nil {
k.fatalf(msg, "%v", err) k.fatalf(msg, "%v", err)
} }
@@ -493,9 +492,9 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
for { for {
select { select {
case s := <-sig: case s := <-sig:
if s == CancelSignal && params.ForwardCancel && cmd.Process != nil { if s == CancelSignal && param.ForwardCancel && cmd.Process != nil {
msg.Verbose("forwarding context cancellation") msg.Verbose("forwarding context cancellation")
if err := k.signal(cmd, os.Interrupt); err != nil { if err := k.signal(cmd, os.Interrupt); err != nil && !errors.Is(err, os.ErrProcessDone) {
k.printf(msg, "cannot forward cancellation: %v", err) k.printf(msg, "cannot forward cancellation: %v", err)
} }
continue continue
@@ -525,7 +524,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
cancel() cancel()
// start timeout early // start timeout early
go func() { time.Sleep(params.AdoptWaitDelay); close(timeout) }() go func() { time.Sleep(param.AdoptWaitDelay); close(timeout) }()
// close initial process files; this also keeps them alive // close initial process files; this also keeps them alive
for _, f := range extraFiles { for _, f := range extraFiles {
+2 -1
View File
@@ -10,6 +10,7 @@ import (
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/container/seccomp" "hakurei.app/container/seccomp"
"hakurei.app/container/std" "hakurei.app/container/std"
"hakurei.app/internal/params"
"hakurei.app/internal/stub" "hakurei.app/internal/stub"
) )
@@ -40,7 +41,7 @@ func TestInitEntrypoint(t *testing.T) {
call("lockOSThread", stub.ExpectArgs{}, nil, nil), call("lockOSThread", stub.ExpectArgs{}, nil, nil),
call("getpid", stub.ExpectArgs{}, 1, nil), call("getpid", stub.ExpectArgs{}, 1, nil),
call("setPtracer", stub.ExpectArgs{uintptr(0)}, nil, nil), call("setPtracer", stub.ExpectArgs{uintptr(0)}, nil, nil),
call("receive", stub.ExpectArgs{"HAKUREI_SETUP", new(initParams), new(uintptr)}, nil, ErrReceiveEnv), call("receive", stub.ExpectArgs{"HAKUREI_SETUP", new(initParams), new(uintptr)}, nil, params.ErrReceiveEnv),
call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SETUP not set"}}, nil, nil), call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SETUP not set"}}, nil, nil),
}, },
}, nil}, }, nil},
+40 -12
View File
@@ -4,9 +4,9 @@ import (
"encoding/gob" "encoding/gob"
"fmt" "fmt"
"slices" "slices"
"strings"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/ext"
"hakurei.app/fhs" "hakurei.app/fhs"
) )
@@ -150,7 +150,7 @@ func (o *MountOverlayOp) early(_ *setupState, k syscallDispatcher) error {
if v, err := k.evalSymlinks(o.Upper.String()); err != nil { if v, err := k.evalSymlinks(o.Upper.String()); err != nil {
return err return err
} else { } else {
o.upper = check.EscapeOverlayDataSegment(toHost(v)) o.upper = toHost(v)
} }
} }
@@ -158,7 +158,7 @@ func (o *MountOverlayOp) early(_ *setupState, k syscallDispatcher) error {
if v, err := k.evalSymlinks(o.Work.String()); err != nil { if v, err := k.evalSymlinks(o.Work.String()); err != nil {
return err return err
} else { } else {
o.work = check.EscapeOverlayDataSegment(toHost(v)) o.work = toHost(v)
} }
} }
} }
@@ -168,12 +168,39 @@ func (o *MountOverlayOp) early(_ *setupState, k syscallDispatcher) error {
if v, err := k.evalSymlinks(a.String()); err != nil { if v, err := k.evalSymlinks(a.String()); err != nil {
return err return err
} else { } else {
o.lower[i] = check.EscapeOverlayDataSegment(toHost(v)) o.lower[i] = toHost(v)
} }
} }
return nil return nil
} }
// mountOverlay sets up an overlay mount via [ext.FS].
func mountOverlay(target string, options [][2]string) error {
fs, err := ext.OpenFS(SourceOverlay, 0)
if err != nil {
return err
}
if err = fs.SetString("source", SourceOverlay); err != nil {
_ = fs.Close()
return err
}
for _, option := range options {
if err = fs.SetString(option[0], option[1]); err != nil {
_ = fs.Close()
return err
}
}
if err = fs.SetFlag(OptionOverlayUserxattr); err != nil {
_ = fs.Close()
return err
}
if err = fs.Mount(target, 0); err != nil {
_ = fs.Close()
return err
}
return fs.Close()
}
func (o *MountOverlayOp) apply(state *setupState, k syscallDispatcher) error { func (o *MountOverlayOp) apply(state *setupState, k syscallDispatcher) error {
target := o.Target.String() target := o.Target.String()
if !o.noPrefix { if !o.noPrefix {
@@ -194,7 +221,7 @@ func (o *MountOverlayOp) apply(state *setupState, k syscallDispatcher) error {
} }
} }
options := make([]string, 0, 4) options := make([][2]string, 0, 2+len(o.lower))
if o.upper == zeroString && o.work == zeroString { // readonly if o.upper == zeroString && o.work == zeroString { // readonly
if len(o.Lower) < 2 { if len(o.Lower) < 2 {
@@ -205,15 +232,16 @@ func (o *MountOverlayOp) apply(state *setupState, k syscallDispatcher) error {
if len(o.Lower) == 0 { if len(o.Lower) == 0 {
return &OverlayArgumentError{OverlayEmptyLower, zeroString} return &OverlayArgumentError{OverlayEmptyLower, zeroString}
} }
options = append(options, options = append(options, [][2]string{
OptionOverlayUpperdir+"="+o.upper, {OptionOverlayUpperdir, o.upper},
OptionOverlayWorkdir+"="+o.work) {OptionOverlayWorkdir, o.work},
}...)
}
for _, lower := range o.lower {
options = append(options, [2]string{OptionOverlayLowerdir + "+", lower})
} }
options = append(options,
OptionOverlayLowerdir+"="+strings.Join(o.lower, check.SpecialOverlayPath),
OptionOverlayUserxattr)
return k.mount(SourceOverlay, target, FstypeOverlay, 0, strings.Join(options, check.SpecialOverlayOption)) return k.mountOverlay(target, options)
} }
func (o *MountOverlayOp) late(*setupState, syscallDispatcher) error { return nil } func (o *MountOverlayOp) late(*setupState, syscallDispatcher) error { return nil }
+33 -33
View File
@@ -97,13 +97,12 @@ func TestMountOverlayOp(t *testing.T) {
call("mkdirAll", stub.ExpectArgs{"/sysroot", os.FileMode(0705)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/sysroot", os.FileMode(0705)}, nil, nil),
call("mkdirTemp", stub.ExpectArgs{"/", "overlay.upper.*"}, "overlay.upper.32768", nil), call("mkdirTemp", stub.ExpectArgs{"/", "overlay.upper.*"}, "overlay.upper.32768", nil),
call("mkdirTemp", stub.ExpectArgs{"/", "overlay.work.*"}, "overlay.work.32768", nil), call("mkdirTemp", stub.ExpectArgs{"/", "overlay.work.*"}, "overlay.work.32768", nil),
call("mount", stub.ExpectArgs{"overlay", "/sysroot", "overlay", uintptr(0), "" + call("mountOverlay", stub.ExpectArgs{"/sysroot", [][2]string{
"upperdir=overlay.upper.32768," + {"upperdir", "overlay.upper.32768"},
"workdir=overlay.work.32768," + {"workdir", "overlay.work.32768"},
"lowerdir=" + {"lowerdir+", `/host/var/lib/planterette/base/debian:f92c9052`},
`/host/var/lib/planterette/base/debian\:f92c9052:` + {"lowerdir+", `/host/var/lib/planterette/app/org.chromium.Chromium@debian:f92c9052`},
`/host/var/lib/planterette/app/org.chromium.Chromium@debian\:f92c9052,` + }}, nil, nil),
"userxattr"}, nil, nil),
}, nil}, }, nil},
{"short lower ro", &Params{ParentPerm: 0755}, &MountOverlayOp{ {"short lower ro", &Params{ParentPerm: 0755}, &MountOverlayOp{
@@ -129,11 +128,10 @@ func TestMountOverlayOp(t *testing.T) {
call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store0"}, "/mnt-root/nix/.ro-store0", nil), call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store0"}, "/mnt-root/nix/.ro-store0", nil),
}, nil, []stub.Call{ }, nil, []stub.Call{
call("mkdirAll", stub.ExpectArgs{"/nix/store", os.FileMode(0755)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/nix/store", os.FileMode(0755)}, nil, nil),
call("mount", stub.ExpectArgs{"overlay", "/nix/store", "overlay", uintptr(0), "" + call("mountOverlay", stub.ExpectArgs{"/nix/store", [][2]string{
"lowerdir=" + {"lowerdir+", "/host/mnt-root/nix/.ro-store"},
"/host/mnt-root/nix/.ro-store:" + {"lowerdir+", "/host/mnt-root/nix/.ro-store0"},
"/host/mnt-root/nix/.ro-store0," + }}, nil, nil),
"userxattr"}, nil, nil),
}, nil}, }, nil},
{"success ro", &Params{ParentPerm: 0755}, &MountOverlayOp{ {"success ro", &Params{ParentPerm: 0755}, &MountOverlayOp{
@@ -147,11 +145,10 @@ func TestMountOverlayOp(t *testing.T) {
call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store0"}, "/mnt-root/nix/.ro-store0", nil), call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store0"}, "/mnt-root/nix/.ro-store0", nil),
}, nil, []stub.Call{ }, nil, []stub.Call{
call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0755)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0755)}, nil, nil),
call("mount", stub.ExpectArgs{"overlay", "/sysroot/nix/store", "overlay", uintptr(0), "" + call("mountOverlay", stub.ExpectArgs{"/sysroot/nix/store", [][2]string{
"lowerdir=" + {"lowerdir+", "/host/mnt-root/nix/.ro-store"},
"/host/mnt-root/nix/.ro-store:" + {"lowerdir+", "/host/mnt-root/nix/.ro-store0"},
"/host/mnt-root/nix/.ro-store0," + }}, nil, nil),
"userxattr"}, nil, nil),
}, nil}, }, nil},
{"nil lower", &Params{ParentPerm: 0700}, &MountOverlayOp{ {"nil lower", &Params{ParentPerm: 0700}, &MountOverlayOp{
@@ -219,7 +216,11 @@ func TestMountOverlayOp(t *testing.T) {
call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store"}, "/mnt-root/nix/ro-store", nil), call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store"}, "/mnt-root/nix/ro-store", nil),
}, nil, []stub.Call{ }, nil, []stub.Call{
call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil),
call("mount", stub.ExpectArgs{"overlay", "/sysroot/nix/store", "overlay", uintptr(0), "upperdir=/host/mnt-root/nix/.rw-store/.upper,workdir=/host/mnt-root/nix/.rw-store/.work,lowerdir=/host/mnt-root/nix/ro-store,userxattr"}, nil, stub.UniqueError(0)), call("mountOverlay", stub.ExpectArgs{"/sysroot/nix/store", [][2]string{
{"upperdir", "/host/mnt-root/nix/.rw-store/.upper"},
{"workdir", "/host/mnt-root/nix/.rw-store/.work"},
{"lowerdir+", "/host/mnt-root/nix/ro-store"},
}}, nil, stub.UniqueError(0)),
}, stub.UniqueError(0)}, }, stub.UniqueError(0)},
{"success single layer", &Params{ParentPerm: 0700}, &MountOverlayOp{ {"success single layer", &Params{ParentPerm: 0700}, &MountOverlayOp{
@@ -233,11 +234,11 @@ func TestMountOverlayOp(t *testing.T) {
call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store"}, "/mnt-root/nix/ro-store", nil), call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store"}, "/mnt-root/nix/ro-store", nil),
}, nil, []stub.Call{ }, nil, []stub.Call{
call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil),
call("mount", stub.ExpectArgs{"overlay", "/sysroot/nix/store", "overlay", uintptr(0), "" + call("mountOverlay", stub.ExpectArgs{"/sysroot/nix/store", [][2]string{
"upperdir=/host/mnt-root/nix/.rw-store/.upper," + {"upperdir", "/host/mnt-root/nix/.rw-store/.upper"},
"workdir=/host/mnt-root/nix/.rw-store/.work," + {"workdir", "/host/mnt-root/nix/.rw-store/.work"},
"lowerdir=/host/mnt-root/nix/ro-store," + {"lowerdir+", "/host/mnt-root/nix/ro-store"},
"userxattr"}, nil, nil), }}, nil, nil),
}, nil}, }, nil},
{"success", &Params{ParentPerm: 0700}, &MountOverlayOp{ {"success", &Params{ParentPerm: 0700}, &MountOverlayOp{
@@ -261,16 +262,15 @@ func TestMountOverlayOp(t *testing.T) {
call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store3"}, "/mnt-root/nix/ro-store3", nil), call("evalSymlinks", stub.ExpectArgs{"/mnt-root/nix/.ro-store3"}, "/mnt-root/nix/ro-store3", nil),
}, nil, []stub.Call{ }, nil, []stub.Call{
call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil), call("mkdirAll", stub.ExpectArgs{"/sysroot/nix/store", os.FileMode(0700)}, nil, nil),
call("mount", stub.ExpectArgs{"overlay", "/sysroot/nix/store", "overlay", uintptr(0), "" + call("mountOverlay", stub.ExpectArgs{"/sysroot/nix/store", [][2]string{
"upperdir=/host/mnt-root/nix/.rw-store/.upper," + {"upperdir", "/host/mnt-root/nix/.rw-store/.upper"},
"workdir=/host/mnt-root/nix/.rw-store/.work," + {"workdir", "/host/mnt-root/nix/.rw-store/.work"},
"lowerdir=" + {"lowerdir+", "/host/mnt-root/nix/ro-store"},
"/host/mnt-root/nix/ro-store:" + {"lowerdir+", "/host/mnt-root/nix/ro-store0"},
"/host/mnt-root/nix/ro-store0:" + {"lowerdir+", "/host/mnt-root/nix/ro-store1"},
"/host/mnt-root/nix/ro-store1:" + {"lowerdir+", "/host/mnt-root/nix/ro-store2"},
"/host/mnt-root/nix/ro-store2:" + {"lowerdir+", "/host/mnt-root/nix/ro-store3"},
"/host/mnt-root/nix/ro-store3," + }}, nil, nil),
"userxattr"}, nil, nil),
}, nil}, }, nil},
}) })
-65
View File
@@ -1,65 +0,0 @@
package container_test
import (
"testing"
"unsafe"
"hakurei.app/container"
)
func TestLandlockString(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
rulesetAttr *container.RulesetAttr
want string
}{
{"nil", nil, "NULL"},
{"zero", new(container.RulesetAttr), "0"},
{"some", &container.RulesetAttr{Scoped: container.LANDLOCK_SCOPE_SIGNAL}, "scoped: signal"},
{"set", &container.RulesetAttr{
HandledAccessFS: container.LANDLOCK_ACCESS_FS_MAKE_SYM | container.LANDLOCK_ACCESS_FS_IOCTL_DEV | container.LANDLOCK_ACCESS_FS_WRITE_FILE,
HandledAccessNet: container.LANDLOCK_ACCESS_NET_BIND_TCP,
Scoped: container.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET | container.LANDLOCK_SCOPE_SIGNAL,
}, "fs: write_file make_sym fs_ioctl_dev, net: bind_tcp, scoped: abstract_unix_socket signal"},
{"all", &container.RulesetAttr{
HandledAccessFS: container.LANDLOCK_ACCESS_FS_EXECUTE |
container.LANDLOCK_ACCESS_FS_WRITE_FILE |
container.LANDLOCK_ACCESS_FS_READ_FILE |
container.LANDLOCK_ACCESS_FS_READ_DIR |
container.LANDLOCK_ACCESS_FS_REMOVE_DIR |
container.LANDLOCK_ACCESS_FS_REMOVE_FILE |
container.LANDLOCK_ACCESS_FS_MAKE_CHAR |
container.LANDLOCK_ACCESS_FS_MAKE_DIR |
container.LANDLOCK_ACCESS_FS_MAKE_REG |
container.LANDLOCK_ACCESS_FS_MAKE_SOCK |
container.LANDLOCK_ACCESS_FS_MAKE_FIFO |
container.LANDLOCK_ACCESS_FS_MAKE_BLOCK |
container.LANDLOCK_ACCESS_FS_MAKE_SYM |
container.LANDLOCK_ACCESS_FS_REFER |
container.LANDLOCK_ACCESS_FS_TRUNCATE |
container.LANDLOCK_ACCESS_FS_IOCTL_DEV,
HandledAccessNet: container.LANDLOCK_ACCESS_NET_BIND_TCP |
container.LANDLOCK_ACCESS_NET_CONNECT_TCP,
Scoped: container.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET |
container.LANDLOCK_SCOPE_SIGNAL,
}, "fs: execute write_file read_file read_dir remove_dir remove_file make_char make_dir make_reg make_sock make_fifo make_block make_sym fs_refer fs_truncate fs_ioctl_dev, net: bind_tcp connect_tcp, scoped: abstract_unix_socket signal"},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if got := tc.rulesetAttr.String(); got != tc.want {
t.Errorf("String: %s, want %s", got, tc.want)
}
})
}
}
func TestLandlockAttrSize(t *testing.T) {
t.Parallel()
want := 24
if got := unsafe.Sizeof(container.RulesetAttr{}); got != uintptr(want) {
t.Errorf("Sizeof: %d, want %d", got, want)
}
}
-47
View File
@@ -1,47 +0,0 @@
package container
import (
"encoding/gob"
"errors"
"os"
"strconv"
"syscall"
)
// Setup appends the read end of a pipe for setup params transmission and returns its fd.
func Setup(extraFiles *[]*os.File) (int, *os.File, error) {
if r, w, err := os.Pipe(); err != nil {
return -1, nil, err
} else {
fd := 3 + len(*extraFiles)
*extraFiles = append(*extraFiles, r)
return fd, w, nil
}
}
var (
ErrReceiveEnv = errors.New("environment variable not set")
)
// Receive retrieves setup fd from the environment and receives params.
func Receive(key string, e any, fdp *uintptr) (func() error, error) {
var setup *os.File
if s, ok := os.LookupEnv(key); !ok {
return nil, ErrReceiveEnv
} else {
if fd, err := strconv.Atoi(s); err != nil {
return nil, optionalErrorUnwrap(err)
} else {
setup = os.NewFile(uintptr(fd), "setup")
if setup == nil {
return nil, syscall.EDOM
}
if fdp != nil {
*fdp = setup.Fd()
}
}
}
return setup.Close, gob.NewDecoder(setup).Decode(e)
}
-4
View File
@@ -10,7 +10,6 @@ import (
"testing" "testing"
"unsafe" "unsafe"
"hakurei.app/check"
"hakurei.app/vfs" "hakurei.app/vfs"
) )
@@ -50,9 +49,6 @@ func TestToHost(t *testing.T) {
} }
} }
// InternalToHostOvlEscape exports toHost passed to [check.EscapeOverlayDataSegment].
func InternalToHostOvlEscape(s string) string { return check.EscapeOverlayDataSegment(toHost(s)) }
func TestCreateFile(t *testing.T) { func TestCreateFile(t *testing.T) {
t.Run("nonexistent", func(t *testing.T) { t.Run("nonexistent", func(t *testing.T) {
t.Run("mkdir", func(t *testing.T) { t.Run("mkdir", func(t *testing.T) {
+2 -2
View File
@@ -7,8 +7,8 @@ import (
"hakurei.app/ext" "hakurei.app/ext"
) )
// SetNoNewPrivs sets the calling thread's no_new_privs attribute. // setNoNewPrivs sets the calling thread's no_new_privs attribute.
func SetNoNewPrivs() error { func setNoNewPrivs() error {
return ext.Prctl(PR_SET_NO_NEW_PRIVS, 1, 0) return ext.Prctl(PR_SET_NO_NEW_PRIVS, 1, 0)
} }
-1
View File
@@ -1 +0,0 @@
1000 0
-12
View File
@@ -1,12 +0,0 @@
#!/bin/sh
cd "$(dirname -- "$0")" || exit 1
install -vDm0755 "bin/hakurei" "${DESTDIR}/usr/bin/hakurei"
install -vDm0755 "bin/sharefs" "${DESTDIR}/usr/bin/sharefs"
install -vDm4511 "bin/hsu" "${DESTDIR}/usr/bin/hsu"
if [ ! -f "${DESTDIR}/etc/hsurc" ]; then
install -vDm0400 "hsurc.default" "${DESTDIR}/etc/hsurc"
fi
install -vDm0644 "comp/_hakurei" "${DESTDIR}/usr/share/zsh/site-functions/_hakurei"
-31
View File
@@ -1,31 +0,0 @@
#!/bin/sh -e
cd "$(dirname -- "$0")/.."
VERSION="${HAKUREI_VERSION:-untagged}"
pname="hakurei-${VERSION}-$(go env GOARCH)"
out="${DESTDIR:-dist}/${pname}"
echo '# Preparing distribution files.'
mkdir -p "${out}"
cp -v "README.md" "dist/hsurc.default" "dist/install.sh" "${out}"
cp -rv "dist/comp" "${out}"
echo
echo '# Building hakurei.'
go generate ./...
go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w
-buildid= -linkmode external -extldflags=-static
-X hakurei.app/internal/info.buildVersion=${VERSION}
-X hakurei.app/internal/info.hakureiPath=/usr/bin/hakurei
-X hakurei.app/internal/info.hsuPath=/usr/bin/hsu
-X main.hakureiPath=/usr/bin/hakurei" ./...
echo
echo '# Testing hakurei.'
go test -ldflags='-buildid= -linkmode external -extldflags=-static' ./...
echo
echo '# Creating distribution.'
rm -f "${out}.tar.gz" && tar -C "${out}/.." -vczf "${out}.tar.gz" "${pname}"
rm -rf "${out}"
(cd "${out}/.." && sha512sum "${pname}.tar.gz" > "${pname}.tar.gz.sha512")
echo
+267
View File
@@ -0,0 +1,267 @@
package ext
import (
"os"
"runtime"
"syscall"
"unsafe"
)
// include/uapi/linux/mount.h
/*
* move_mount() flags.
*/
const (
MOVE_MOUNT_F_SYMLINKS = 1 << iota /* Follow symlinks on from path */
MOVE_MOUNT_F_AUTOMOUNTS /* Follow automounts on from path */
MOVE_MOUNT_F_EMPTY_PATH /* Empty from path permitted */
_
MOVE_MOUNT_T_SYMLINKS /* Follow symlinks on to path */
MOVE_MOUNT_T_AUTOMOUNTS /* Follow automounts on to path */
MOVE_MOUNT_T_EMPTY_PATH /* Empty to path permitted */
_
MOVE_MOUNT_SET_GROUP /* Set sharing group instead */
MOVE_MOUNT_BENEATH /* Mount beneath top mount */
)
/*
* fsopen() flags.
*/
const (
FSOPEN_CLOEXEC = 1 << iota
)
/*
* fspick() flags.
*/
const (
FSPICK_CLOEXEC = 1 << iota
FSPICK_SYMLINK_NOFOLLOW
FSPICK_NO_AUTOMOUNT
FSPICK_EMPTY_PATH
)
/*
* The type of fsconfig() call made.
*/
const (
FSCONFIG_SET_FLAG = iota /* Set parameter, supplying no value */
FSCONFIG_SET_STRING /* Set parameter, supplying a string value */
FSCONFIG_SET_BINARY /* Set parameter, supplying a binary blob value */
FSCONFIG_SET_PATH /* Set parameter, supplying an object by path */
FSCONFIG_SET_PATH_EMPTY /* Set parameter, supplying an object by (empty) path */
FSCONFIG_SET_FD /* Set parameter, supplying an object by fd */
FSCONFIG_CMD_CREATE /* Create new or reuse existing superblock */
FSCONFIG_CMD_RECONFIGURE /* Invoke superblock reconfiguration */
FSCONFIG_CMD_CREATE_EXCL /* Create new superblock, fail if reusing existing superblock */
)
/*
* fsmount() flags.
*/
const (
FSMOUNT_CLOEXEC = 1 << iota
)
/*
* Mount attributes.
*/
const (
MOUNT_ATTR_RDONLY = 0x00000001 /* Mount read-only */
MOUNT_ATTR_NOSUID = 0x00000002 /* Ignore suid and sgid bits */
MOUNT_ATTR_NODEV = 0x00000004 /* Disallow access to device special files */
MOUNT_ATTR_NOEXEC = 0x00000008 /* Disallow program execution */
MOUNT_ATTR__ATIME = 0x00000070 /* Setting on how atime should be updated */
MOUNT_ATTR_RELATIME = 0x00000000 /* - Update atime relative to mtime/ctime. */
MOUNT_ATTR_NOATIME = 0x00000010 /* - Do not update access times. */
MOUNT_ATTR_STRICTATIME = 0x00000020 /* - Always perform atime updates */
MOUNT_ATTR_NODIRATIME = 0x00000080 /* Do not update directory access times */
MOUNT_ATTR_IDMAP = 0x00100000 /* Idmap mount to @userns_fd in struct mount_attr. */
MOUNT_ATTR_NOSYMFOLLOW = 0x00200000 /* Do not follow symlinks */
)
// FS provides low-level wrappers around the suite of file-descriptor-based
// mount facilities in Linux.
type FS struct {
fd uintptr
c runtime.Cleanup
}
// newFS allocates a new [FS] for the specified fd.
func newFS(fd uintptr) *FS {
fs := FS{fd: fd}
fs.c = runtime.AddCleanup(&fs, func(fd uintptr) {
_ = syscall.Close(int(fd))
}, fd)
return &fs
}
// Close closes the underlying filesystem context.
func (fs *FS) Close() error {
if fs == nil {
return syscall.EINVAL
}
err := syscall.Close(int(fs.fd))
fs.c.Stop()
return err
}
// OpenFS creates a new filesystem context.
func OpenFS(fsname string, flags int) (fs *FS, err error) {
var s *byte
s, err = syscall.BytePtrFromString(fsname)
if err != nil {
return
}
fd, _, errno := syscall.Syscall(
SYS_FSOPEN,
uintptr(unsafe.Pointer(s)),
uintptr(flags|FSOPEN_CLOEXEC),
0,
)
if errno != 0 {
err = os.NewSyscallError("fsopen", errno)
} else {
fs = newFS(fd)
}
return
}
// PickFS selects filesystem for reconfiguration.
func PickFS(dirfd int, pathname string, flags int) (fs *FS, err error) {
var s *byte
s, err = syscall.BytePtrFromString(pathname)
if err != nil {
return
}
fd, _, errno := syscall.Syscall(
SYS_FSPICK,
uintptr(dirfd),
uintptr(unsafe.Pointer(s)),
uintptr(flags|FSPICK_CLOEXEC),
)
if errno != 0 {
err = os.NewSyscallError("fspick", errno)
} else {
fs = newFS(fd)
}
return
}
// config configures new or existing filesystem context.
func (fs *FS) config(cmd uint, key *byte, value unsafe.Pointer, aux int) (err error) {
_, _, errno := syscall.Syscall6(
SYS_FSCONFIG,
fs.fd,
uintptr(cmd),
uintptr(unsafe.Pointer(key)),
uintptr(value),
uintptr(aux),
0,
)
if errno != 0 {
err = os.NewSyscallError("fsconfig", errno)
}
return
}
// SetFlag sets the flag parameter named by key. ([FSCONFIG_SET_FLAG])
func (fs *FS) SetFlag(key string) (err error) {
var s *byte
s, err = syscall.BytePtrFromString(key)
if err != nil {
return
}
return fs.config(FSCONFIG_SET_FLAG, s, nil, 0)
}
// SetString sets the string parameter named by key to the value specified by
// value. ([FSCONFIG_SET_STRING])
func (fs *FS) SetString(key, value string) (err error) {
var s0 *byte
s0, err = syscall.BytePtrFromString(key)
if err != nil {
return
}
var s1 *byte
s1, err = syscall.BytePtrFromString(value)
if err != nil {
return
}
return fs.config(FSCONFIG_SET_STRING, s0, unsafe.Pointer(s1), 0)
}
// mount instantiates mount object from filesystem context.
func (fs *FS) mount(flags, attrFlags int) (fsfd int, err error) {
r, _, errno := syscall.Syscall(
SYS_FSMOUNT,
fs.fd,
uintptr(flags|FSMOUNT_CLOEXEC),
uintptr(attrFlags),
)
fsfd = int(r)
if errno != 0 {
err = os.NewSyscallError("fsmount", errno)
}
return
}
// MoveMount moves or attaches mount object to filesystem.
func MoveMount(
fromDirfd int,
fromPathname string,
toDirfd int,
toPathname string,
flags int,
) (err error) {
var s0 *byte
s0, err = syscall.BytePtrFromString(fromPathname)
if err != nil {
return
}
var s1 *byte
s1, err = syscall.BytePtrFromString(toPathname)
if err != nil {
return
}
_, _, errno := syscall.Syscall6(
SYS_MOVE_MOUNT,
uintptr(fromDirfd),
uintptr(unsafe.Pointer(s0)),
uintptr(toDirfd),
uintptr(unsafe.Pointer(s1)),
uintptr(flags),
0,
)
if errno != 0 {
err = os.NewSyscallError("move_mount", errno)
}
return
}
// Mount attaches the underlying filesystem context to the specified pathname.
func (fs *FS) Mount(pathname string, attrFlags int) error {
if err := fs.config(FSCONFIG_CMD_CREATE_EXCL, nil, nil, 0); err != nil {
return err
}
fd, err := fs.mount(0, attrFlags)
if err != nil {
return err
}
err = MoveMount(
fd, "",
-1, pathname,
MOVE_MOUNT_F_EMPTY_PATH,
)
closeErr := syscall.Close(fd)
if err == nil {
err = closeErr
}
return err
}
+2 -2
View File
@@ -137,11 +137,10 @@
CC="musl-clang -O3 -Werror -Qunused-arguments" \ CC="musl-clang -O3 -Werror -Qunused-arguments" \
GOCACHE="$(mktemp -d)" \ GOCACHE="$(mktemp -d)" \
HAKUREI_TEST_SKIP_ACL=1 \
PATH="${pkgs.pkgsStatic.musl.bin}/bin:$PATH" \ PATH="${pkgs.pkgsStatic.musl.bin}/bin:$PATH" \
DESTDIR="$out" \ DESTDIR="$out" \
HAKUREI_VERSION="v${hakurei.version}" \ HAKUREI_VERSION="v${hakurei.version}" \
./dist/release.sh ./all.sh
''; '';
} }
); );
@@ -196,6 +195,7 @@
./test/interactive/vm.nix ./test/interactive/vm.nix
./test/interactive/hakurei.nix ./test/interactive/hakurei.nix
./test/interactive/trace.nix ./test/interactive/trace.nix
./test/interactive/raceattr.nix
self.nixosModules.hakurei self.nixosModules.hakurei
home-manager.nixosModules.home-manager home-manager.nixosModules.home-manager
+36 -11
View File
@@ -140,21 +140,29 @@ var (
ErrInsecure = errors.New("configuration is insecure") ErrInsecure = errors.New("configuration is insecure")
) )
const (
// VAllowInsecure allows use of compatibility options considered insecure
// under any configuration, to work around ecosystem-wide flaws.
VAllowInsecure = 1 << iota
)
// Validate checks [Config] and returns [AppError] if an invalid value is encountered. // Validate checks [Config] and returns [AppError] if an invalid value is encountered.
func (config *Config) Validate() error { func (config *Config) Validate(flags int) error {
const step = "validate configuration"
if config == nil { if config == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull, return &AppError{Step: step, Err: ErrConfigNull,
Msg: "invalid configuration"} Msg: "invalid configuration"}
} }
// this is checked again in hsu // this is checked again in hsu
if config.Identity < IdentityStart || config.Identity > IdentityEnd { if config.Identity < IdentityStart || config.Identity > IdentityEnd {
return &AppError{Step: "validate configuration", Err: ErrIdentityBounds, return &AppError{Step: step, Err: ErrIdentityBounds,
Msg: "identity " + strconv.Itoa(config.Identity) + " out of range"} Msg: "identity " + strconv.Itoa(config.Identity) + " out of range"}
} }
if config.SchedPolicy < 0 || config.SchedPolicy > ext.SCHED_LAST { if config.SchedPolicy < 0 || config.SchedPolicy > ext.SCHED_LAST {
return &AppError{Step: "validate configuration", Err: ErrSchedPolicyBounds, return &AppError{Step: step, Err: ErrSchedPolicyBounds,
Msg: "scheduling policy " + Msg: "scheduling policy " +
strconv.Itoa(int(config.SchedPolicy)) + strconv.Itoa(int(config.SchedPolicy)) +
" out of range"} " out of range"}
@@ -168,34 +176,51 @@ func (config *Config) Validate() error {
} }
if config.Container == nil { if config.Container == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull, return &AppError{Step: step, Err: ErrConfigNull,
Msg: "configuration missing container state"} Msg: "configuration missing container state"}
} }
if config.Container.Home == nil { if config.Container.Home == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull, return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to home directory"} Msg: "container configuration missing path to home directory"}
} }
if config.Container.Shell == nil { if config.Container.Shell == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull, return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to shell"} Msg: "container configuration missing path to shell"}
} }
if config.Container.Path == nil { if config.Container.Path == nil {
return &AppError{Step: "validate configuration", Err: ErrConfigNull, return &AppError{Step: step, Err: ErrConfigNull,
Msg: "container configuration missing path to initial program"} Msg: "container configuration missing path to initial program"}
} }
for key := range config.Container.Env { for key := range config.Container.Env {
if strings.IndexByte(key, '=') != -1 || strings.IndexByte(key, 0) != -1 { if strings.IndexByte(key, '=') != -1 || strings.IndexByte(key, 0) != -1 {
return &AppError{Step: "validate configuration", Err: ErrEnviron, return &AppError{Step: step, Err: ErrEnviron,
Msg: "invalid environment variable " + strconv.Quote(key)} Msg: "invalid environment variable " + strconv.Quote(key)}
} }
} }
if et := config.Enablements.Unwrap(); !config.DirectPulse && et&EPulse != 0 { et := config.Enablements.Unwrap()
return &AppError{Step: "validate configuration", Err: ErrInsecure, if !config.DirectPulse && et&EPulse != 0 {
return &AppError{Step: step, Err: ErrInsecure,
Msg: "enablement PulseAudio is insecure and no longer supported"} Msg: "enablement PulseAudio is insecure and no longer supported"}
} }
if flags&VAllowInsecure == 0 {
switch {
case et&EWayland != 0 && config.DirectWayland:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_wayland is insecure and no longer supported"}
case et&EPipeWire != 0 && config.DirectPipeWire:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_pipewire is insecure and no longer supported"}
case et&EPulse != 0 && config.DirectPulse:
return &AppError{Step: step, Err: ErrInsecure,
Msg: "direct_pulse is insecure and no longer supported"}
}
}
return nil return nil
} }
+61 -17
View File
@@ -14,65 +14,109 @@ func TestConfigValidate(t *testing.T) {
testCases := []struct { testCases := []struct {
name string name string
config *hst.Config config *hst.Config
flags int
wantErr error wantErr error
}{ }{
{"nil", nil, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull, {"nil", nil, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "invalid configuration"}}, Msg: "invalid configuration"}},
{"identity lower", &hst.Config{Identity: -1}, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
{"identity lower", &hst.Config{Identity: -1}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
Msg: "identity -1 out of range"}}, Msg: "identity -1 out of range"}},
{"identity upper", &hst.Config{Identity: 10000}, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds, {"identity upper", &hst.Config{Identity: 10000}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrIdentityBounds,
Msg: "identity 10000 out of range"}}, Msg: "identity 10000 out of range"}},
{"sched lower", &hst.Config{SchedPolicy: -1}, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
{"sched lower", &hst.Config{SchedPolicy: -1}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
Msg: "scheduling policy -1 out of range"}}, Msg: "scheduling policy -1 out of range"}},
{"sched upper", &hst.Config{SchedPolicy: 0xcafe}, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds, {"sched upper", &hst.Config{SchedPolicy: 0xcafe}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrSchedPolicyBounds,
Msg: "scheduling policy 51966 out of range"}}, Msg: "scheduling policy 51966 out of range"}},
{"dbus session", &hst.Config{SessionBus: &hst.BusConfig{See: []string{""}}},
{"dbus session", &hst.Config{SessionBus: &hst.BusConfig{See: []string{""}}}, 0,
&hst.BadInterfaceError{Interface: "", Segment: "session"}}, &hst.BadInterfaceError{Interface: "", Segment: "session"}},
{"dbus system", &hst.Config{SystemBus: &hst.BusConfig{See: []string{""}}}, {"dbus system", &hst.Config{SystemBus: &hst.BusConfig{See: []string{""}}}, 0,
&hst.BadInterfaceError{Interface: "", Segment: "system"}}, &hst.BadInterfaceError{Interface: "", Segment: "system"}},
{"container", &hst.Config{}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
{"container", &hst.Config{}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "configuration missing container state"}}, Msg: "configuration missing container state"}},
{"home", &hst.Config{Container: &hst.ContainerConfig{}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull, {"home", &hst.Config{Container: &hst.ContainerConfig{}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to home directory"}}, Msg: "container configuration missing path to home directory"}},
{"shell", &hst.Config{Container: &hst.ContainerConfig{ {"shell", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull, }}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to shell"}}, Msg: "container configuration missing path to shell"}},
{"path", &hst.Config{Container: &hst.ContainerConfig{ {"path", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
Shell: fhs.AbsTmp, Shell: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull, }}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrConfigNull,
Msg: "container configuration missing path to initial program"}}, Msg: "container configuration missing path to initial program"}},
{"env equals", &hst.Config{Container: &hst.ContainerConfig{ {"env equals", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
Shell: fhs.AbsTmp, Shell: fhs.AbsTmp,
Path: fhs.AbsTmp, Path: fhs.AbsTmp,
Env: map[string]string{"TERM=": ""}, Env: map[string]string{"TERM=": ""},
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron, }}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
Msg: `invalid environment variable "TERM="`}}, Msg: `invalid environment variable "TERM="`}},
{"env NUL", &hst.Config{Container: &hst.ContainerConfig{ {"env NUL", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
Shell: fhs.AbsTmp, Shell: fhs.AbsTmp,
Path: fhs.AbsTmp, Path: fhs.AbsTmp,
Env: map[string]string{"TERM\x00": ""}, Env: map[string]string{"TERM\x00": ""},
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron, }}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrEnviron,
Msg: `invalid environment variable "TERM\x00"`}}, Msg: `invalid environment variable "TERM\x00"`}},
{"insecure pulse", &hst.Config{Enablements: hst.NewEnablements(hst.EPulse), Container: &hst.ContainerConfig{
{"insecure pulse", &hst.Config{Enablements: new(hst.EPulse), Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
Shell: fhs.AbsTmp, Shell: fhs.AbsTmp,
Path: fhs.AbsTmp, Path: fhs.AbsTmp,
}}, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure, }}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "enablement PulseAudio is insecure and no longer supported"}}, Msg: "enablement PulseAudio is insecure and no longer supported"}},
{"direct wayland", &hst.Config{Enablements: new(hst.EWayland), DirectWayland: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_wayland is insecure and no longer supported"}},
{"direct wayland allow", &hst.Config{Enablements: new(hst.EWayland), DirectWayland: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"direct pipewire", &hst.Config{Enablements: new(hst.EPipeWire), DirectPipeWire: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_pipewire is insecure and no longer supported"}},
{"direct pipewire allow", &hst.Config{Enablements: new(hst.EPipeWire), DirectPipeWire: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"direct pulse", &hst.Config{Enablements: new(hst.EPulse), DirectPulse: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, 0, &hst.AppError{Step: "validate configuration", Err: hst.ErrInsecure,
Msg: "direct_pulse is insecure and no longer supported"}},
{"direct pulse allow", &hst.Config{Enablements: new(hst.EPulse), DirectPulse: true, Container: &hst.ContainerConfig{
Home: fhs.AbsTmp,
Shell: fhs.AbsTmp,
Path: fhs.AbsTmp,
}}, hst.VAllowInsecure, nil},
{"valid", &hst.Config{Container: &hst.ContainerConfig{ {"valid", &hst.Config{Container: &hst.ContainerConfig{
Home: fhs.AbsTmp, Home: fhs.AbsTmp,
Shell: fhs.AbsTmp, Shell: fhs.AbsTmp,
Path: fhs.AbsTmp, Path: fhs.AbsTmp,
}}, nil}, }}, 0, nil},
} }
for _, tc := range testCases { for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
if err := tc.config.Validate(); !reflect.DeepEqual(err, tc.wantErr) { if err := tc.config.Validate(tc.flags); !reflect.DeepEqual(err, tc.wantErr) {
t.Errorf("Validate: error = %#v, want %#v", err, tc.wantErr) t.Errorf("Validate: error = %#v, want %#v", err, tc.wantErr)
} }
}) })
+21 -31
View File
@@ -7,12 +7,12 @@ import (
"syscall" "syscall"
) )
// Enablement represents an optional host service to export to the target user. // Enablements denotes optional host service to export to the target user.
type Enablement byte type Enablements byte
const ( const (
// EWayland exposes a Wayland pathname socket via security-context-v1. // EWayland exposes a Wayland pathname socket via security-context-v1.
EWayland Enablement = 1 << iota EWayland Enablements = 1 << iota
// EX11 adds the target user via X11 ChangeHosts and exposes the X11 // EX11 adds the target user via X11 ChangeHosts and exposes the X11
// pathname socket. // pathname socket.
EX11 EX11
@@ -28,8 +28,8 @@ const (
EM EM
) )
// String returns a string representation of the flags set on [Enablement]. // String returns a string representation of the flags set on [Enablements].
func (e Enablement) String() string { func (e Enablements) String() string {
switch e { switch e {
case 0: case 0:
return "(no enablements)" return "(no enablements)"
@@ -47,7 +47,7 @@ func (e Enablement) String() string {
buf := new(strings.Builder) buf := new(strings.Builder)
buf.Grow(32) buf.Grow(32)
for i := Enablement(1); i < EM; i <<= 1 { for i := Enablements(1); i < EM; i <<= 1 {
if e&i != 0 { if e&i != 0 {
buf.WriteString(", " + i.String()) buf.WriteString(", " + i.String())
} }
@@ -60,12 +60,6 @@ func (e Enablement) String() string {
} }
} }
// NewEnablements returns the address of [Enablement] as [Enablements].
func NewEnablements(e Enablement) *Enablements { return (*Enablements)(&e) }
// Enablements is the [json] adapter for [Enablement].
type Enablements Enablement
// enablementsJSON is the [json] representation of [Enablements]. // enablementsJSON is the [json] representation of [Enablements].
type enablementsJSON = struct { type enablementsJSON = struct {
Wayland bool `json:"wayland,omitempty"` Wayland bool `json:"wayland,omitempty"`
@@ -75,24 +69,21 @@ type enablementsJSON = struct {
Pulse bool `json:"pulse,omitempty"` Pulse bool `json:"pulse,omitempty"`
} }
// Unwrap returns the underlying [Enablement]. // Unwrap returns the value pointed to by e.
func (e *Enablements) Unwrap() Enablement { func (e *Enablements) Unwrap() Enablements {
if e == nil { if e == nil {
return 0 return 0
} }
return Enablement(*e) return *e
} }
func (e *Enablements) MarshalJSON() ([]byte, error) { func (e Enablements) MarshalJSON() ([]byte, error) {
if e == nil {
return nil, syscall.EINVAL
}
return json.Marshal(&enablementsJSON{ return json.Marshal(&enablementsJSON{
Wayland: Enablement(*e)&EWayland != 0, Wayland: e&EWayland != 0,
X11: Enablement(*e)&EX11 != 0, X11: e&EX11 != 0,
DBus: Enablement(*e)&EDBus != 0, DBus: e&EDBus != 0,
PipeWire: Enablement(*e)&EPipeWire != 0, PipeWire: e&EPipeWire != 0,
Pulse: Enablement(*e)&EPulse != 0, Pulse: e&EPulse != 0,
}) })
} }
@@ -106,22 +97,21 @@ func (e *Enablements) UnmarshalJSON(data []byte) error {
return err return err
} }
var ve Enablement *e = 0
if v.Wayland { if v.Wayland {
ve |= EWayland *e |= EWayland
} }
if v.X11 { if v.X11 {
ve |= EX11 *e |= EX11
} }
if v.DBus { if v.DBus {
ve |= EDBus *e |= EDBus
} }
if v.PipeWire { if v.PipeWire {
ve |= EPipeWire *e |= EPipeWire
} }
if v.Pulse { if v.Pulse {
ve |= EPulse *e |= EPulse
} }
*e = Enablements(ve)
return nil return nil
} }
+9 -12
View File
@@ -13,7 +13,7 @@ func TestEnablementString(t *testing.T) {
t.Parallel() t.Parallel()
testCases := []struct { testCases := []struct {
flags hst.Enablement flags hst.Enablements
want string want string
}{ }{
{0, "(no enablements)"}, {0, "(no enablements)"},
@@ -59,13 +59,13 @@ func TestEnablements(t *testing.T) {
sData string sData string
}{ }{
{"nil", nil, "null", `{"value":null,"magic":3236757504}`}, {"nil", nil, "null", `{"value":null,"magic":3236757504}`},
{"zero", hst.NewEnablements(0), `{}`, `{"value":{},"magic":3236757504}`}, {"zero", new(hst.Enablements(0)), `{}`, `{"value":{},"magic":3236757504}`},
{"wayland", hst.NewEnablements(hst.EWayland), `{"wayland":true}`, `{"value":{"wayland":true},"magic":3236757504}`}, {"wayland", new(hst.EWayland), `{"wayland":true}`, `{"value":{"wayland":true},"magic":3236757504}`},
{"x11", hst.NewEnablements(hst.EX11), `{"x11":true}`, `{"value":{"x11":true},"magic":3236757504}`}, {"x11", new(hst.EX11), `{"x11":true}`, `{"value":{"x11":true},"magic":3236757504}`},
{"dbus", hst.NewEnablements(hst.EDBus), `{"dbus":true}`, `{"value":{"dbus":true},"magic":3236757504}`}, {"dbus", new(hst.EDBus), `{"dbus":true}`, `{"value":{"dbus":true},"magic":3236757504}`},
{"pipewire", hst.NewEnablements(hst.EPipeWire), `{"pipewire":true}`, `{"value":{"pipewire":true},"magic":3236757504}`}, {"pipewire", new(hst.EPipeWire), `{"pipewire":true}`, `{"value":{"pipewire":true},"magic":3236757504}`},
{"pulse", hst.NewEnablements(hst.EPulse), `{"pulse":true}`, `{"value":{"pulse":true},"magic":3236757504}`}, {"pulse", new(hst.EPulse), `{"pulse":true}`, `{"value":{"pulse":true},"magic":3236757504}`},
{"all", hst.NewEnablements(hst.EM - 1), `{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true}`, `{"value":{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true},"magic":3236757504}`}, {"all", new(hst.EM - 1), `{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true}`, `{"value":{"wayland":true,"x11":true,"dbus":true,"pipewire":true,"pulse":true},"magic":3236757504}`},
} }
for _, tc := range testCases { for _, tc := range testCases {
@@ -137,7 +137,7 @@ func TestEnablements(t *testing.T) {
}) })
t.Run("val", func(t *testing.T) { t.Run("val", func(t *testing.T) {
if got := hst.NewEnablements(hst.EWayland | hst.EPulse).Unwrap(); got != hst.EWayland|hst.EPulse { if got := new(hst.EWayland | hst.EPulse).Unwrap(); got != hst.EWayland|hst.EPulse {
t.Errorf("Unwrap: %v", got) t.Errorf("Unwrap: %v", got)
} }
}) })
@@ -146,9 +146,6 @@ func TestEnablements(t *testing.T) {
t.Run("passthrough", func(t *testing.T) { t.Run("passthrough", func(t *testing.T) {
t.Parallel() t.Parallel()
if _, err := (*hst.Enablements)(nil).MarshalJSON(); !errors.Is(err, syscall.EINVAL) {
t.Errorf("MarshalJSON: error = %v", err)
}
if err := (*hst.Enablements)(nil).UnmarshalJSON(nil); !errors.Is(err, syscall.EINVAL) { if err := (*hst.Enablements)(nil).UnmarshalJSON(nil); !errors.Is(err, syscall.EINVAL) {
t.Errorf("UnmarshalJSON: error = %v", err) t.Errorf("UnmarshalJSON: error = %v", err)
} }
+3 -1
View File
@@ -56,8 +56,10 @@ type Ops interface {
// ApplyState holds the address of [Ops] and any relevant application state. // ApplyState holds the address of [Ops] and any relevant application state.
type ApplyState struct { type ApplyState struct {
// AutoEtcPrefix is the prefix for [FSBind] in autoetc [FSBind.Special] condition. // Prefix for [FSBind] in autoetc [FSBind.Special] condition.
AutoEtcPrefix string AutoEtcPrefix string
// Whether to skip remounting root.
NoRemountRoot bool
Ops Ops
} }
+6 -2
View File
@@ -5,6 +5,7 @@ import (
"strings" "strings"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/fhs"
) )
func init() { gob.Register(new(FSOverlay)) } func init() { gob.Register(new(FSOverlay)) }
@@ -69,9 +70,12 @@ func (o *FSOverlay) Apply(z *ApplyState) {
return return
} }
if o.Upper != nil && o.Work != nil { // rw if o.Upper != nil && o.Work != nil {
z.Overlay(o.Target, o.Upper, o.Work, o.Lower...) z.Overlay(o.Target, o.Upper, o.Work, o.Lower...)
} else { // ro if o.Target.Is(fhs.AbsRoot) {
z.NoRemountRoot = true
}
} else {
z.OverlayReadonly(o.Target, o.Lower...) z.OverlayReadonly(o.Target, o.Lower...)
} }
} }
+13
View File
@@ -49,5 +49,18 @@ func TestFSOverlay(t *testing.T) {
Lower: ms("/tmp/.src0", "/tmp/.src1"), Lower: ms("/tmp/.src0", "/tmp/.src1"),
}}, m("/mnt/src"), ms("/tmp/.src0", "/tmp/.src1"), }}, m("/mnt/src"), ms("/tmp/.src0", "/tmp/.src1"),
"*/mnt/src:/tmp/.src0:/tmp/.src1"}, "*/mnt/src:/tmp/.src0:/tmp/.src1"},
{"no remount root", &hst.FSOverlay{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}, true, container.Ops{&container.MountOverlayOp{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}}, m("/"), ms("/tmp/upper", "/tmp/work", "/tmp/.src0", "/tmp/.src1"),
"w*/:/tmp/upper:/tmp/work:/tmp/.src0:/tmp/.src1"},
}) })
} }
+1 -1
View File
@@ -72,7 +72,7 @@ func Template() *Config {
return &Config{ return &Config{
ID: "org.chromium.Chromium", ID: "org.chromium.Chromium",
Enablements: NewEnablements(EWayland | EDBus | EPipeWire), Enablements: new(EWayland | EDBus | EPipeWire),
SessionBus: &BusConfig{ SessionBus: &BusConfig{
See: nil, See: nil,
+10 -2
View File
@@ -11,9 +11,11 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
"syscall"
"testing" "testing"
"hakurei.app/internal/acl" "hakurei.app/internal/acl"
"hakurei.app/internal/info"
) )
const testFileName = "acl.test" const testFileName = "acl.test"
@@ -24,8 +26,14 @@ var (
) )
func TestUpdate(t *testing.T) { func TestUpdate(t *testing.T) {
if os.Getenv("HAKUREI_TEST_SKIP_ACL") == "1" { if info.CanDegrade {
t.Skip("acl test skipped") name := filepath.Join(t.TempDir(), "check-degrade")
if err := os.WriteFile(name, nil, 0); err != nil {
t.Fatal(err)
}
if err := acl.Update(name, os.Geteuid()); errors.Is(err, syscall.ENOTSUP) {
t.Skip(err)
}
} }
testFilePath := filepath.Join(t.TempDir(), testFileName) testFilePath := filepath.Join(t.TempDir(), testFileName)
+7
View File
@@ -0,0 +1,7 @@
//go:build !noskip
package info
// CanDegrade is whether tests are allowed to transparently degrade or skip due
// to required system features being denied or unavailable.
const CanDegrade = true
+5
View File
@@ -0,0 +1,5 @@
//go:build noskip
package info
const CanDegrade = false
+90
View File
@@ -0,0 +1,90 @@
package kobject
import (
"errors"
"strconv"
"strings"
"unsafe"
"hakurei.app/internal/uevent"
)
// Event is a [uevent.Message] with known environment variables processed.
type Event struct {
// alloc_uevent_skb: action_string
Action uevent.KobjectAction `json:"action"`
// alloc_uevent_skb: devpath
DevPath string `json:"devpath"`
// Uninterpreted environment variable pairs. An entry missing a separator
// gains the value "\x00".
Env map[string]string `json:"env"`
// SEQNUM value set by the kernel.
Sequence uint64 `json:"seqnum"`
// SYNTH_UUID value set on trigger, nil denotes a non-synthetic event.
Synth *uevent.UUID `json:"synth_uuid,omitempty"`
// SUBSYSTEM value set by the kernel.
Subsystem string `json:"subsystem"`
}
// Populate populates e with the contents of a [uevent.Message].
//
// The ACTION and DEVPATH environment variables are ignored and assumed to be
// consistent with the header.
func (e *Event) Populate(reportErr func(error), m *uevent.Message) {
if reportErr == nil {
reportErr = func(error) {}
}
*e = Event{
Action: m.Action,
DevPath: m.DevPath,
Env: make(map[string]string),
}
for _, s := range m.Env {
k, v, ok := strings.Cut(s, "=")
if !ok {
if _, ok = e.Env[s]; !ok {
e.Env[s] = "\x00"
}
continue
}
switch k {
case "ACTION", "DEVPATH":
continue
case "SEQNUM":
seq, err := strconv.ParseUint(v, 10, 64)
if err != nil {
if _e := errors.Unwrap(err); _e != nil {
err = _e
}
reportErr(err)
e.Env[k] = v
continue
}
e.Sequence = seq
case "SYNTH_UUID":
var uuid uevent.UUID
err := uuid.UnmarshalText(unsafe.Slice(unsafe.StringData(v), len(v)))
if err != nil {
reportErr(err)
e.Env[k] = v
continue
}
e.Synth = &uuid
case "SUBSYSTEM":
e.Subsystem = v
default:
e.Env[k] = v
}
}
}
+92
View File
@@ -0,0 +1,92 @@
package kobject_test
import (
"reflect"
"strconv"
"testing"
"hakurei.app/internal/kobject"
"hakurei.app/internal/uevent"
)
func TestEvent(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
msg uevent.Message
want kobject.Event
errs []error
}{
{"sample coldboot qemu", uevent.Message{
Action: uevent.KOBJ_ADD,
DevPath: "/devices/LNXSYSTM:00/LNXPWRBN:00",
Env: []string{
"ACTION=add",
"DEVPATH=/devices/LNXSYSTM:00/LNXPWRBN:00",
"SUBSYSTEM=acpi",
"SYNTH_UUID=fe4d7c9d-b8c6-4a70-9ef1-3d8a58d18eed",
"MODALIAS=acpi:LNXPWRBN:",
"SEQNUM=777",
}}, kobject.Event{
Action: uevent.KOBJ_ADD,
DevPath: "/devices/LNXSYSTM:00/LNXPWRBN:00",
Env: map[string]string{
"MODALIAS": "acpi:LNXPWRBN:",
},
Sequence: 777,
Synth: &uevent.UUID{
0xfe, 0x4d, 0x7c, 0x9d,
0xb8, 0xc6,
0x4a, 0x70,
0x9e, 0xf1,
0x3d, 0x8a, 0x58, 0xd1, 0x8e, 0xed,
},
Subsystem: "acpi",
}, []error{}},
{"nil reportErr", uevent.Message{Env: []string{
"SEQNUM=\x00",
}}, kobject.Event{Env: map[string]string{
"SEQNUM": "\x00",
}}, nil},
{"bad SEQNUM SYNTH_UUID", uevent.Message{Env: []string{
"SEQNUM=\x00",
"SYNTH_UUID=\x00",
"SUBSYSTEM=\x00",
}}, kobject.Event{Subsystem: "\x00", Env: map[string]string{
"SEQNUM": "\x00",
"SYNTH_UUID": "\x00",
}}, []error{strconv.ErrSyntax, uevent.UUIDSizeError(1)}},
{"bad sep", uevent.Message{Env: []string{
"SYNTH_UUID",
}}, kobject.Event{Env: map[string]string{
"SYNTH_UUID": "\x00",
}}, []error{}},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
var f func(error)
gotErrs := make([]error, 0)
if tc.errs != nil {
f = func(err error) {
gotErrs = append(gotErrs, err)
}
}
var got kobject.Event
got.Populate(f, &tc.msg)
if !reflect.DeepEqual(&got, &tc.want) {
t.Errorf("Populate: %#v, want %#v", got, tc.want)
}
if tc.errs != nil && !reflect.DeepEqual(gotErrs, tc.errs) {
t.Errorf("Populate: errs = %v, want %v", gotErrs, tc.errs)
}
})
}
}
@@ -1,4 +1,4 @@
package container package landlock
import ( import (
"strings" "strings"
@@ -14,11 +14,11 @@ const (
LANDLOCK_CREATE_RULESET_VERSION = 1 << iota LANDLOCK_CREATE_RULESET_VERSION = 1 << iota
) )
// LandlockAccessFS is bitmask of handled filesystem actions. // AccessFS is bitmask of handled filesystem actions.
type LandlockAccessFS uint64 type AccessFS uint64
const ( const (
LANDLOCK_ACCESS_FS_EXECUTE LandlockAccessFS = 1 << iota LANDLOCK_ACCESS_FS_EXECUTE AccessFS = 1 << iota
LANDLOCK_ACCESS_FS_WRITE_FILE LANDLOCK_ACCESS_FS_WRITE_FILE
LANDLOCK_ACCESS_FS_READ_FILE LANDLOCK_ACCESS_FS_READ_FILE
LANDLOCK_ACCESS_FS_READ_DIR LANDLOCK_ACCESS_FS_READ_DIR
@@ -38,8 +38,8 @@ const (
_LANDLOCK_ACCESS_FS_DELIM _LANDLOCK_ACCESS_FS_DELIM
) )
// String returns a space-separated string of [LandlockAccessFS] flags. // String returns a space-separated string of [AccessFS] flags.
func (f LandlockAccessFS) String() string { func (f AccessFS) String() string {
switch f { switch f {
case LANDLOCK_ACCESS_FS_EXECUTE: case LANDLOCK_ACCESS_FS_EXECUTE:
return "execute" return "execute"
@@ -90,8 +90,8 @@ func (f LandlockAccessFS) String() string {
return "fs_ioctl_dev" return "fs_ioctl_dev"
default: default:
var c []LandlockAccessFS var c []AccessFS
for i := LandlockAccessFS(1); i < _LANDLOCK_ACCESS_FS_DELIM; i <<= 1 { for i := AccessFS(1); i < _LANDLOCK_ACCESS_FS_DELIM; i <<= 1 {
if f&i != 0 { if f&i != 0 {
c = append(c, i) c = append(c, i)
} }
@@ -107,18 +107,18 @@ func (f LandlockAccessFS) String() string {
} }
} }
// LandlockAccessNet is bitmask of handled network actions. // AccessNet is bitmask of handled network actions.
type LandlockAccessNet uint64 type AccessNet uint64
const ( const (
LANDLOCK_ACCESS_NET_BIND_TCP LandlockAccessNet = 1 << iota LANDLOCK_ACCESS_NET_BIND_TCP AccessNet = 1 << iota
LANDLOCK_ACCESS_NET_CONNECT_TCP LANDLOCK_ACCESS_NET_CONNECT_TCP
_LANDLOCK_ACCESS_NET_DELIM _LANDLOCK_ACCESS_NET_DELIM
) )
// String returns a space-separated string of [LandlockAccessNet] flags. // String returns a space-separated string of [AccessNet] flags.
func (f LandlockAccessNet) String() string { func (f AccessNet) String() string {
switch f { switch f {
case LANDLOCK_ACCESS_NET_BIND_TCP: case LANDLOCK_ACCESS_NET_BIND_TCP:
return "bind_tcp" return "bind_tcp"
@@ -127,8 +127,8 @@ func (f LandlockAccessNet) String() string {
return "connect_tcp" return "connect_tcp"
default: default:
var c []LandlockAccessNet var c []AccessNet
for i := LandlockAccessNet(1); i < _LANDLOCK_ACCESS_NET_DELIM; i <<= 1 { for i := AccessNet(1); i < _LANDLOCK_ACCESS_NET_DELIM; i <<= 1 {
if f&i != 0 { if f&i != 0 {
c = append(c, i) c = append(c, i)
} }
@@ -144,18 +144,18 @@ func (f LandlockAccessNet) String() string {
} }
} }
// LandlockScope is bitmask of scopes restricting a Landlock domain from accessing outside resources. // Scope is bitmask of scopes restricting a Landlock domain from accessing outside resources.
type LandlockScope uint64 type Scope uint64
const ( const (
LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET LandlockScope = 1 << iota LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET Scope = 1 << iota
LANDLOCK_SCOPE_SIGNAL LANDLOCK_SCOPE_SIGNAL
_LANDLOCK_SCOPE_DELIM _LANDLOCK_SCOPE_DELIM
) )
// String returns a space-separated string of [LandlockScope] flags. // String returns a space-separated string of [Scope] flags.
func (f LandlockScope) String() string { func (f Scope) String() string {
switch f { switch f {
case LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET: case LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET:
return "abstract_unix_socket" return "abstract_unix_socket"
@@ -164,8 +164,8 @@ func (f LandlockScope) String() string {
return "signal" return "signal"
default: default:
var c []LandlockScope var c []Scope
for i := LandlockScope(1); i < _LANDLOCK_SCOPE_DELIM; i <<= 1 { for i := Scope(1); i < _LANDLOCK_SCOPE_DELIM; i <<= 1 {
if f&i != 0 { if f&i != 0 {
c = append(c, i) c = append(c, i)
} }
@@ -184,12 +184,12 @@ func (f LandlockScope) String() string {
// RulesetAttr is equivalent to struct landlock_ruleset_attr. // RulesetAttr is equivalent to struct landlock_ruleset_attr.
type RulesetAttr struct { type RulesetAttr struct {
// Bitmask of handled filesystem actions. // Bitmask of handled filesystem actions.
HandledAccessFS LandlockAccessFS HandledAccessFS AccessFS
// Bitmask of handled network actions. // Bitmask of handled network actions.
HandledAccessNet LandlockAccessNet HandledAccessNet AccessNet
// Bitmask of scopes restricting a Landlock domain from accessing outside // Bitmask of scopes restricting a Landlock domain from accessing outside
// resources (e.g. IPCs). // resources (e.g. IPCs).
Scoped LandlockScope Scoped Scope
} }
// String returns a user-facing description of [RulesetAttr]. // String returns a user-facing description of [RulesetAttr].
@@ -239,13 +239,13 @@ func (rulesetAttr *RulesetAttr) Create(flags uintptr) (fd int, err error) {
return fd, nil return fd, nil
} }
// LandlockGetABI returns the ABI version supported by the kernel. // GetABI returns the ABI version supported by the kernel.
func LandlockGetABI() (int, error) { func GetABI() (int, error) {
return (*RulesetAttr)(nil).Create(LANDLOCK_CREATE_RULESET_VERSION) return (*RulesetAttr)(nil).Create(LANDLOCK_CREATE_RULESET_VERSION)
} }
// LandlockRestrictSelf applies a loaded ruleset to the calling thread. // RestrictSelf applies a loaded ruleset to the calling thread.
func LandlockRestrictSelf(rulesetFd int, flags uintptr) error { func RestrictSelf(rulesetFd int, flags uintptr) error {
r, _, errno := syscall.Syscall( r, _, errno := syscall.Syscall(
ext.SYS_LANDLOCK_RESTRICT_SELF, ext.SYS_LANDLOCK_RESTRICT_SELF,
uintptr(rulesetFd), uintptr(rulesetFd),
+65
View File
@@ -0,0 +1,65 @@
package landlock_test
import (
"testing"
"unsafe"
"hakurei.app/internal/landlock"
)
func TestLandlockString(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
rulesetAttr *landlock.RulesetAttr
want string
}{
{"nil", nil, "NULL"},
{"zero", new(landlock.RulesetAttr), "0"},
{"some", &landlock.RulesetAttr{Scoped: landlock.LANDLOCK_SCOPE_SIGNAL}, "scoped: signal"},
{"set", &landlock.RulesetAttr{
HandledAccessFS: landlock.LANDLOCK_ACCESS_FS_MAKE_SYM | landlock.LANDLOCK_ACCESS_FS_IOCTL_DEV | landlock.LANDLOCK_ACCESS_FS_WRITE_FILE,
HandledAccessNet: landlock.LANDLOCK_ACCESS_NET_BIND_TCP,
Scoped: landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET | landlock.LANDLOCK_SCOPE_SIGNAL,
}, "fs: write_file make_sym fs_ioctl_dev, net: bind_tcp, scoped: abstract_unix_socket signal"},
{"all", &landlock.RulesetAttr{
HandledAccessFS: landlock.LANDLOCK_ACCESS_FS_EXECUTE |
landlock.LANDLOCK_ACCESS_FS_WRITE_FILE |
landlock.LANDLOCK_ACCESS_FS_READ_FILE |
landlock.LANDLOCK_ACCESS_FS_READ_DIR |
landlock.LANDLOCK_ACCESS_FS_REMOVE_DIR |
landlock.LANDLOCK_ACCESS_FS_REMOVE_FILE |
landlock.LANDLOCK_ACCESS_FS_MAKE_CHAR |
landlock.LANDLOCK_ACCESS_FS_MAKE_DIR |
landlock.LANDLOCK_ACCESS_FS_MAKE_REG |
landlock.LANDLOCK_ACCESS_FS_MAKE_SOCK |
landlock.LANDLOCK_ACCESS_FS_MAKE_FIFO |
landlock.LANDLOCK_ACCESS_FS_MAKE_BLOCK |
landlock.LANDLOCK_ACCESS_FS_MAKE_SYM |
landlock.LANDLOCK_ACCESS_FS_REFER |
landlock.LANDLOCK_ACCESS_FS_TRUNCATE |
landlock.LANDLOCK_ACCESS_FS_IOCTL_DEV,
HandledAccessNet: landlock.LANDLOCK_ACCESS_NET_BIND_TCP |
landlock.LANDLOCK_ACCESS_NET_CONNECT_TCP,
Scoped: landlock.LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET |
landlock.LANDLOCK_SCOPE_SIGNAL,
}, "fs: execute write_file read_file read_dir remove_dir remove_file make_char make_dir make_reg make_sock make_fifo make_block make_sym fs_refer fs_truncate fs_ioctl_dev, net: bind_tcp connect_tcp, scoped: abstract_unix_socket signal"},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if got := tc.rulesetAttr.String(); got != tc.want {
t.Errorf("String: %s, want %s", got, tc.want)
}
})
}
}
func TestLandlockAttrSize(t *testing.T) {
t.Parallel()
want := 24
if got := unsafe.Sizeof(landlock.RulesetAttr{}); got != uintptr(want) {
t.Errorf("Sizeof: %d, want %d", got, want)
}
}
+4 -3
View File
@@ -17,6 +17,7 @@ import (
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/internal/dbus" "hakurei.app/internal/dbus"
"hakurei.app/internal/info" "hakurei.app/internal/info"
"hakurei.app/internal/params"
"hakurei.app/message" "hakurei.app/message"
) )
@@ -84,7 +85,7 @@ type syscallDispatcher interface {
// setDumpable provides [container.SetDumpable]. // setDumpable provides [container.SetDumpable].
setDumpable(dumpable uintptr) error setDumpable(dumpable uintptr) error
// receive provides [container.Receive]. // receive provides [container.Receive].
receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) receive(key string, e any, fdp *int) (closeFunc func() error, err error)
// containerStart provides the Start method of [container.Container]. // containerStart provides the Start method of [container.Container].
containerStart(z *container.Container) error containerStart(z *container.Container) error
@@ -154,8 +155,8 @@ func (direct) prctl(op, arg2, arg3 uintptr) error { return ext.Prctl(op, arg2, a
func (direct) overflowUid(msg message.Msg) int { return container.OverflowUid(msg) } func (direct) overflowUid(msg message.Msg) int { return container.OverflowUid(msg) }
func (direct) overflowGid(msg message.Msg) int { return container.OverflowGid(msg) } func (direct) overflowGid(msg message.Msg) int { return container.OverflowGid(msg) }
func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) } func (direct) setDumpable(dumpable uintptr) error { return ext.SetDumpable(dumpable) }
func (direct) receive(key string, e any, fdp *uintptr) (func() error, error) { func (direct) receive(key string, e any, fdp *int) (func() error, error) {
return container.Receive(key, e, fdp) return params.Receive(key, e, fdp)
} }
func (direct) containerStart(z *container.Container) error { return z.Start() } func (direct) containerStart(z *container.Container) error { return z.Start() }
+34 -34
View File
@@ -401,12 +401,12 @@ func (k *kstub) setDumpable(dumpable uintptr) error {
stub.CheckArg(k.Stub, "dumpable", dumpable, 0)) stub.CheckArg(k.Stub, "dumpable", dumpable, 0))
} }
func (k *kstub) receive(key string, e any, fdp *uintptr) (closeFunc func() error, err error) { func (k *kstub) receive(key string, e any, fdp *int) (closeFunc func() error, err error) {
k.Helper() k.Helper()
expect := k.Expects("receive") expect := k.Expects("receive")
reflect.ValueOf(e).Elem().Set(reflect.ValueOf(expect.Args[1])) reflect.ValueOf(e).Elem().Set(reflect.ValueOf(expect.Args[1]))
if expect.Args[2] != nil { if expect.Args[2] != nil {
*fdp = expect.Args[2].(uintptr) *fdp = int(expect.Args[2].(uintptr))
} }
return func() error { return k.Expects("closeReceive").Err }, expect.Error( return func() error { return k.Expects("closeReceive").Err }, expect.Error(
stub.CheckArg(k.Stub, "key", key, 0)) stub.CheckArg(k.Stub, "key", key, 0))
@@ -690,38 +690,38 @@ func (panicMsgContext) Value(any) any { panic("unreachable") }
// This type is meant to be embedded in partial syscallDispatcher implementations. // This type is meant to be embedded in partial syscallDispatcher implementations.
type panicDispatcher struct{} type panicDispatcher struct{}
func (panicDispatcher) new(func(k syscallDispatcher, msg message.Msg)) { panic("unreachable") } func (panicDispatcher) new(func(k syscallDispatcher, msg message.Msg)) { panic("unreachable") }
func (panicDispatcher) getppid() int { panic("unreachable") } func (panicDispatcher) getppid() int { panic("unreachable") }
func (panicDispatcher) getpid() int { panic("unreachable") } func (panicDispatcher) getpid() int { panic("unreachable") }
func (panicDispatcher) getuid() int { panic("unreachable") } func (panicDispatcher) getuid() int { panic("unreachable") }
func (panicDispatcher) getgid() int { panic("unreachable") } func (panicDispatcher) getgid() int { panic("unreachable") }
func (panicDispatcher) lookupEnv(string) (string, bool) { panic("unreachable") } func (panicDispatcher) lookupEnv(string) (string, bool) { panic("unreachable") }
func (panicDispatcher) pipe() (*os.File, *os.File, error) { panic("unreachable") } func (panicDispatcher) pipe() (*os.File, *os.File, error) { panic("unreachable") }
func (panicDispatcher) stat(string) (os.FileInfo, error) { panic("unreachable") } func (panicDispatcher) stat(string) (os.FileInfo, error) { panic("unreachable") }
func (panicDispatcher) open(string) (osFile, error) { panic("unreachable") } func (panicDispatcher) open(string) (osFile, error) { panic("unreachable") }
func (panicDispatcher) readdir(string) ([]os.DirEntry, error) { panic("unreachable") } func (panicDispatcher) readdir(string) ([]os.DirEntry, error) { panic("unreachable") }
func (panicDispatcher) tempdir() string { panic("unreachable") } func (panicDispatcher) tempdir() string { panic("unreachable") }
func (panicDispatcher) mkdir(string, os.FileMode) error { panic("unreachable") } func (panicDispatcher) mkdir(string, os.FileMode) error { panic("unreachable") }
func (panicDispatcher) removeAll(string) error { panic("unreachable") } func (panicDispatcher) removeAll(string) error { panic("unreachable") }
func (panicDispatcher) exit(int) { panic("unreachable") } func (panicDispatcher) exit(int) { panic("unreachable") }
func (panicDispatcher) evalSymlinks(string) (string, error) { panic("unreachable") } func (panicDispatcher) evalSymlinks(string) (string, error) { panic("unreachable") }
func (panicDispatcher) prctl(uintptr, uintptr, uintptr) error { panic("unreachable") } func (panicDispatcher) prctl(uintptr, uintptr, uintptr) error { panic("unreachable") }
func (panicDispatcher) lookupGroupId(string) (string, error) { panic("unreachable") } func (panicDispatcher) lookupGroupId(string) (string, error) { panic("unreachable") }
func (panicDispatcher) lookPath(string) (string, error) { panic("unreachable") } func (panicDispatcher) lookPath(string) (string, error) { panic("unreachable") }
func (panicDispatcher) cmdOutput(*exec.Cmd) ([]byte, error) { panic("unreachable") } func (panicDispatcher) cmdOutput(*exec.Cmd) ([]byte, error) { panic("unreachable") }
func (panicDispatcher) overflowUid(message.Msg) int { panic("unreachable") } func (panicDispatcher) overflowUid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) overflowGid(message.Msg) int { panic("unreachable") } func (panicDispatcher) overflowGid(message.Msg) int { panic("unreachable") }
func (panicDispatcher) setDumpable(uintptr) error { panic("unreachable") } func (panicDispatcher) setDumpable(uintptr) error { panic("unreachable") }
func (panicDispatcher) receive(string, any, *uintptr) (func() error, error) { panic("unreachable") } func (panicDispatcher) receive(string, any, *int) (func() error, error) { panic("unreachable") }
func (panicDispatcher) containerStart(*container.Container) error { panic("unreachable") } func (panicDispatcher) containerStart(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerServe(*container.Container) error { panic("unreachable") } func (panicDispatcher) containerServe(*container.Container) error { panic("unreachable") }
func (panicDispatcher) containerWait(*container.Container) error { panic("unreachable") } func (panicDispatcher) containerWait(*container.Container) error { panic("unreachable") }
func (panicDispatcher) mustHsuPath() *check.Absolute { panic("unreachable") } func (panicDispatcher) mustHsuPath() *check.Absolute { panic("unreachable") }
func (panicDispatcher) dbusAddress() (string, string) { panic("unreachable") } func (panicDispatcher) dbusAddress() (string, string) { panic("unreachable") }
func (panicDispatcher) setupContSignal(int) (io.ReadCloser, func(), error) { panic("unreachable") } func (panicDispatcher) setupContSignal(int) (io.ReadCloser, func(), error) { panic("unreachable") }
func (panicDispatcher) getMsg() message.Msg { panic("unreachable") } func (panicDispatcher) getMsg() message.Msg { panic("unreachable") }
func (panicDispatcher) fatal(...any) { panic("unreachable") } func (panicDispatcher) fatal(...any) { panic("unreachable") }
func (panicDispatcher) fatalf(string, ...any) { panic("unreachable") } func (panicDispatcher) fatalf(string, ...any) { panic("unreachable") }
func (panicDispatcher) notifyContext(context.Context, ...os.Signal) (context.Context, context.CancelFunc) { func (panicDispatcher) notifyContext(context.Context, ...os.Signal) (context.Context, context.CancelFunc) {
panic("unreachable") panic("unreachable")
+9 -2
View File
@@ -32,7 +32,14 @@ type outcome struct {
syscallDispatcher syscallDispatcher
} }
func (k *outcome) finalise(ctx context.Context, msg message.Msg, id *hst.ID, config *hst.Config) error { // finalise prepares an outcome for main.
func (k *outcome) finalise(
ctx context.Context,
msg message.Msg,
id *hst.ID,
config *hst.Config,
flags int,
) error {
if ctx == nil || id == nil { if ctx == nil || id == nil {
// unreachable // unreachable
panic("invalid call to finalise") panic("invalid call to finalise")
@@ -43,7 +50,7 @@ func (k *outcome) finalise(ctx context.Context, msg message.Msg, id *hst.ID, con
} }
k.ctx = ctx k.ctx = ctx
if err := config.Validate(); err != nil { if err := config.Validate(flags); err != nil {
return err return err
} }
+1 -1
View File
@@ -194,7 +194,7 @@ type outcomeStateSys struct {
// Copied from [hst.Config]. Safe for read by outcomeOp.toSystem. // Copied from [hst.Config]. Safe for read by outcomeOp.toSystem.
appId string appId string
// Copied from [hst.Config]. Safe for read by outcomeOp.toSystem. // Copied from [hst.Config]. Safe for read by outcomeOp.toSystem.
et hst.Enablement et hst.Enablements
// Copied from [hst.Config]. Safe for read by spWaylandOp.toSystem only. // Copied from [hst.Config]. Safe for read by spWaylandOp.toSystem only.
directWayland bool directWayland bool
+15 -11
View File
@@ -13,7 +13,6 @@ import (
"time" "time"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/container"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/hst" "hakurei.app/hst"
"hakurei.app/internal/info" "hakurei.app/internal/info"
@@ -298,12 +297,12 @@ func (k *outcome) main(msg message.Msg, identifierFd int) {
// accumulate enablements of remaining instances // accumulate enablements of remaining instances
var ( var (
// alive enablement bits // alive enablement bits
rt hst.Enablement rt hst.Enablements
// alive instance count // alive instance count
n int n int
) )
for eh := range entries { for eh := range entries {
var et hst.Enablement var et hst.Enablements
if et, err = eh.Load(nil); err != nil { if et, err = eh.Load(nil); err != nil {
perror(err, "read state header of instance "+eh.ID.String()) perror(err, "read state header of instance "+eh.ID.String())
} else { } else {
@@ -372,17 +371,18 @@ func (k *outcome) start(ctx context.Context, msg message.Msg,
// shim runs in the same session as monitor; see shim.go for behaviour // shim runs in the same session as monitor; see shim.go for behaviour
cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGCONT) } cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGCONT) }
var shimPipe *os.File var shimPipe [2]*os.File
if fd, w, err := container.Setup(&cmd.ExtraFiles); err != nil { if r, w, err := os.Pipe(); err != nil {
return cmd, nil, &hst.AppError{Step: "create shim setup pipe", Err: err} return cmd, nil, &hst.AppError{Step: "create shim setup pipe", Err: err}
} else { } else {
shimPipe = w
cmd.Env = []string{ cmd.Env = []string{
// passed through to shim by hsu // passed through to shim by hsu
shimEnv + "=" + strconv.Itoa(fd), shimEnv + "=" + strconv.Itoa(3+len(cmd.ExtraFiles)),
// interpreted by hsu // interpreted by hsu
"HAKUREI_IDENTITY=" + k.state.identity.String(), "HAKUREI_IDENTITY=" + k.state.identity.String(),
} }
cmd.ExtraFiles = append(cmd.ExtraFiles, r)
shimPipe[0], shimPipe[1] = r, w
} }
if len(k.supp) > 0 { if len(k.supp) > 0 {
@@ -393,12 +393,16 @@ func (k *outcome) start(ctx context.Context, msg message.Msg,
msg.Verbosef("setuid helper at %s", hsuPath) msg.Verbosef("setuid helper at %s", hsuPath)
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
_, _ = shimPipe[0].Close(), shimPipe[1].Close()
msg.Resume() msg.Resume()
return cmd, shimPipe, &hst.AppError{Step: "start setuid wrapper", Err: err} return cmd, nil, &hst.AppError{Step: "start setuid wrapper", Err: err}
}
if err := shimPipe[0].Close(); err != nil {
msg.Verbose(err)
} }
*startTime = time.Now().UTC() *startTime = time.Now().UTC()
return cmd, shimPipe, nil return cmd, shimPipe[1], nil
} }
// serveShim serves outcomeState through the shim setup pipe. // serveShim serves outcomeState through the shim setup pipe.
@@ -411,11 +415,11 @@ func serveShim(msg message.Msg, shimPipe *os.File, state *outcomeState) error {
msg.Verbose(err.Error()) msg.Verbose(err.Error())
} }
if err := gob.NewEncoder(shimPipe).Encode(state); err != nil { if err := gob.NewEncoder(shimPipe).Encode(state); err != nil {
_ = shimPipe.Close()
msg.Resume() msg.Resume()
return &hst.AppError{Step: "transmit shim config", Err: err} return &hst.AppError{Step: "transmit shim config", Err: err}
} }
_ = shimPipe.Close() return shimPipe.Close()
return nil
} }
// printMessageError prints the error message according to [message.GetMessage], // printMessageError prints the error message according to [message.GetMessage],
+8 -2
View File
@@ -18,7 +18,13 @@ import (
func IsPollDescriptor(fd uintptr) bool func IsPollDescriptor(fd uintptr) bool
// Main runs an app according to [hst.Config] and terminates. Main does not return. // Main runs an app according to [hst.Config] and terminates. Main does not return.
func Main(ctx context.Context, msg message.Msg, config *hst.Config, fd int) { func Main(
ctx context.Context,
msg message.Msg,
config *hst.Config,
flags int,
fd int,
) {
// avoids runtime internals or standard streams // avoids runtime internals or standard streams
if fd >= 0 { if fd >= 0 {
if IsPollDescriptor(uintptr(fd)) || fd < 3 { if IsPollDescriptor(uintptr(fd)) || fd < 3 {
@@ -34,7 +40,7 @@ func Main(ctx context.Context, msg message.Msg, config *hst.Config, fd int) {
k := outcome{syscallDispatcher: direct{msg}} k := outcome{syscallDispatcher: direct{msg}}
finaliseTime := time.Now() finaliseTime := time.Now()
if err := k.finalise(ctx, msg, &id, config); err != nil { if err := k.finalise(ctx, msg, &id, config, flags); err != nil {
printMessageError(msg.GetLogger().Fatalln, "cannot seal app:", err) printMessageError(msg.GetLogger().Fatalln, "cannot seal app:", err)
panic("unreachable") panic("unreachable")
} }
+2 -2
View File
@@ -288,7 +288,7 @@ func TestOutcomeRun(t *testing.T) {
}, },
Filter: true, Filter: true,
}, },
Enablements: hst.NewEnablements(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse), Enablements: new(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Container: &hst.ContainerConfig{ Container: &hst.ContainerConfig{
Filesystem: []hst.FilesystemConfigJSON{ Filesystem: []hst.FilesystemConfigJSON{
@@ -427,7 +427,7 @@ func TestOutcomeRun(t *testing.T) {
DirectPipeWire: true, DirectPipeWire: true,
ID: "org.chromium.Chromium", ID: "org.chromium.Chromium",
Enablements: hst.NewEnablements(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse), Enablements: new(hst.EWayland | hst.EDBus | hst.EPipeWire | hst.EPulse),
Container: &hst.ContainerConfig{ Container: &hst.ContainerConfig{
Env: nil, Env: nil,
Filesystem: []hst.FilesystemConfigJSON{ Filesystem: []hst.FilesystemConfigJSON{
+2 -1
View File
@@ -20,6 +20,7 @@ import (
"hakurei.app/ext" "hakurei.app/ext"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/hst" "hakurei.app/hst"
"hakurei.app/internal/params"
"hakurei.app/internal/pipewire" "hakurei.app/internal/pipewire"
"hakurei.app/message" "hakurei.app/message"
) )
@@ -197,7 +198,7 @@ func shimEntrypoint(k syscallDispatcher) {
if errors.Is(err, syscall.EBADF) { if errors.Is(err, syscall.EBADF) {
k.fatal("invalid config descriptor") k.fatal("invalid config descriptor")
} }
if errors.Is(err, container.ErrReceiveEnv) { if errors.Is(err, params.ErrReceiveEnv) {
k.fatal(shimEnv + " not set") k.fatal(shimEnv + " not set")
} }
+2 -1
View File
@@ -16,6 +16,7 @@ import (
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/hst" "hakurei.app/hst"
"hakurei.app/internal/env" "hakurei.app/internal/env"
"hakurei.app/internal/params"
"hakurei.app/internal/stub" "hakurei.app/internal/stub"
) )
@@ -172,7 +173,7 @@ func TestShimEntrypoint(t *testing.T) {
call("setDumpable", stub.ExpectArgs{uintptr(ext.SUID_DUMP_DISABLE)}, nil, nil), call("setDumpable", stub.ExpectArgs{uintptr(ext.SUID_DUMP_DISABLE)}, nil, nil),
call("getppid", stub.ExpectArgs{}, 0xbad, nil), call("getppid", stub.ExpectArgs{}, 0xbad, nil),
call("setupContSignal", stub.ExpectArgs{0xbad}, 0, nil), call("setupContSignal", stub.ExpectArgs{0xbad}, 0, nil),
call("receive", stub.ExpectArgs{"HAKUREI_SHIM", outcomeState{}, nil}, nil, container.ErrReceiveEnv), call("receive", stub.ExpectArgs{"HAKUREI_SHIM", outcomeState{}, nil}, nil, params.ErrReceiveEnv),
call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SHIM not set"}}, nil, nil), call("fatal", stub.ExpectArgs{[]any{"HAKUREI_SHIM not set"}}, nil, nil),
// deferred // deferred
+3 -1
View File
@@ -290,7 +290,9 @@ func (s *spFilesystemOp) toContainer(state *outcomeStateParams) error {
if state.Container.Flags&hst.FDevice == 0 { if state.Container.Flags&hst.FDevice == 0 {
state.params.Remount(fhs.AbsDev, syscall.MS_RDONLY) state.params.Remount(fhs.AbsDev, syscall.MS_RDONLY)
} }
state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY) if !state.as.NoRemountRoot {
state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY)
}
state.params.Env = make([]string, 0, len(state.env)) state.params.Env = make([]string, 0, len(state.env))
for key, value := range state.env { for key, value := range state.env {
+1 -1
View File
@@ -21,7 +21,7 @@ func TestSpPulseOp(t *testing.T) {
newConfig := func() *hst.Config { newConfig := func() *hst.Config {
config := hst.Template() config := hst.Template()
config.DirectPulse = true config.DirectPulse = true
config.Enablements = hst.NewEnablements(hst.EPulse) config.Enablements = new(hst.EPulse)
return config return config
} }
+42
View File
@@ -0,0 +1,42 @@
// Package params provides helpers for receiving setup payload from parent.
package params
import (
"encoding/gob"
"errors"
"os"
"strconv"
"syscall"
)
// ErrReceiveEnv is returned by [Receive] if setup fd is not present in environment.
var ErrReceiveEnv = errors.New("environment variable not set")
// Receive retrieves setup fd from the environment and receives params.
//
// The file descriptor written to the value pointed to by fdp must not be passed
// to any system calls. It is made available for ordering file descriptor only.
func Receive(key string, v any, fdp *int) (func() error, error) {
var setup *os.File
if s, ok := os.LookupEnv(key); !ok {
return nil, ErrReceiveEnv
} else {
if fd, err := strconv.Atoi(s); err != nil {
if _err := errors.Unwrap(err); _err != nil {
err = _err
}
return nil, err
} else {
setup = os.NewFile(uintptr(fd), "setup")
if setup == nil {
return nil, syscall.EDOM
}
if fdp != nil {
*fdp = fd
}
}
}
return setup.Close, gob.NewDecoder(setup).Decode(v)
}
@@ -1,4 +1,4 @@
package container_test package params_test
import ( import (
"encoding/gob" "encoding/gob"
@@ -9,7 +9,7 @@ import (
"syscall" "syscall"
"testing" "testing"
"hakurei.app/container" "hakurei.app/internal/params"
) )
func TestSetupReceive(t *testing.T) { func TestSetupReceive(t *testing.T) {
@@ -30,8 +30,8 @@ func TestSetupReceive(t *testing.T) {
}) })
} }
if _, err := container.Receive(key, nil, nil); !errors.Is(err, container.ErrReceiveEnv) { if _, err := params.Receive(key, nil, nil); !errors.Is(err, params.ErrReceiveEnv) {
t.Errorf("Receive: error = %v, want %v", err, container.ErrReceiveEnv) t.Errorf("Receive: error = %v, want %v", err, params.ErrReceiveEnv)
} }
}) })
@@ -39,7 +39,7 @@ func TestSetupReceive(t *testing.T) {
const key = "TEST_ENV_FORMAT" const key = "TEST_ENV_FORMAT"
t.Setenv(key, "") t.Setenv(key, "")
if _, err := container.Receive(key, nil, nil); !errors.Is(err, strconv.ErrSyntax) { if _, err := params.Receive(key, nil, nil); !errors.Is(err, strconv.ErrSyntax) {
t.Errorf("Receive: error = %v, want %v", err, strconv.ErrSyntax) t.Errorf("Receive: error = %v, want %v", err, strconv.ErrSyntax)
} }
}) })
@@ -48,7 +48,7 @@ func TestSetupReceive(t *testing.T) {
const key = "TEST_ENV_RANGE" const key = "TEST_ENV_RANGE"
t.Setenv(key, "-1") t.Setenv(key, "-1")
if _, err := container.Receive(key, nil, nil); !errors.Is(err, syscall.EDOM) { if _, err := params.Receive(key, nil, nil); !errors.Is(err, syscall.EDOM) {
t.Errorf("Receive: error = %v, want %v", err, syscall.EDOM) t.Errorf("Receive: error = %v, want %v", err, syscall.EDOM)
} }
}) })
@@ -60,16 +60,22 @@ func TestSetupReceive(t *testing.T) {
encoderDone := make(chan error, 1) encoderDone := make(chan error, 1)
extraFiles := make([]*os.File, 0, 1) extraFiles := make([]*os.File, 0, 1)
deadline, _ := t.Deadline() if r, w, err := os.Pipe(); err != nil {
if fd, f, err := container.Setup(&extraFiles); err != nil {
t.Fatalf("Setup: error = %v", err) t.Fatalf("Setup: error = %v", err)
} else if fd != 3 {
t.Fatalf("Setup: fd = %d, want 3", fd)
} else { } else {
if err = f.SetDeadline(deadline); err != nil { t.Cleanup(func() {
t.Fatal(err.Error()) if err = errors.Join(r.Close(), w.Close()); err != nil {
t.Fatal(err)
}
})
extraFiles = append(extraFiles, r)
if deadline, ok := t.Deadline(); ok {
if err = w.SetDeadline(deadline); err != nil {
t.Fatal(err)
}
} }
go func() { encoderDone <- gob.NewEncoder(f).Encode(payload) }() go func() { encoderDone <- gob.NewEncoder(w).Encode(payload) }()
} }
if len(extraFiles) != 1 { if len(extraFiles) != 1 {
@@ -87,13 +93,13 @@ func TestSetupReceive(t *testing.T) {
var ( var (
gotPayload []uint64 gotPayload []uint64
fdp *uintptr fdp *int
) )
if !useNilFdp { if !useNilFdp {
fdp = new(uintptr) fdp = new(int)
} }
var closeFile func() error var closeFile func() error
if f, err := container.Receive(key, &gotPayload, fdp); err != nil { if f, err := params.Receive(key, &gotPayload, fdp); err != nil {
t.Fatalf("Receive: error = %v", err) t.Fatalf("Receive: error = %v", err)
} else { } else {
closeFile = f closeFile = f
@@ -103,7 +109,7 @@ func TestSetupReceive(t *testing.T) {
} }
} }
if !useNilFdp { if !useNilFdp {
if int(*fdp) != dupFd { if *fdp != dupFd {
t.Errorf("Fd: %d, want %d", *fdp, dupFd) t.Errorf("Fd: %d, want %d", *fdp, dupFd)
} }
} }
+26
View File
@@ -184,6 +184,32 @@ func TestFlatten(t *testing.T) {
{Mode: fs.ModeDir | 0700, Path: "work"}, {Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d"), nil}, }, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d"), nil},
{"sample no assume checksum", fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},
"checksum": {Mode: fs.ModeDir | 0700},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M": {Mode: fs.ModeDir | 0500},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check": {Mode: 0400, Data: []byte{}},
"identifier": {Mode: fs.ModeDir | 0700},
"identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"work": {Mode: fs.ModeDir | 0700},
}, []pkg.FlatEntry{
{Mode: fs.ModeDir | 0700, Path: "."},
{Mode: fs.ModeDir | 0700, Path: "checksum"},
{Mode: fs.ModeDir | 0500, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M"},
{Mode: 0400, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check", Data: []byte{}},
{Mode: fs.ModeDir | 0700, Path: "identifier"},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w"), nil},
{"sample tar step unpack", fstest.MapFS{ {"sample tar step unpack", fstest.MapFS{
".": {Mode: fs.ModeDir | 0500}, ".": {Mode: fs.ModeDir | 0500},
+13 -7
View File
@@ -27,6 +27,11 @@ import (
// AbsWork is the container pathname [TContext.GetWorkDir] is mounted on. // AbsWork is the container pathname [TContext.GetWorkDir] is mounted on.
var AbsWork = fhs.AbsRoot.Append("work/") var AbsWork = fhs.AbsRoot.Append("work/")
// EnvJobs is the name of the environment variable holding a decimal
// representation of the preferred job count. Its value must not affect cure
// outcome.
const EnvJobs = "CURE_JOBS"
// ExecPath is a slice of [Artifact] and the [check.Absolute] pathname to make // ExecPath is a slice of [Artifact] and the [check.Absolute] pathname to make
// it available at under in the container. // it available at under in the container.
type ExecPath struct { type ExecPath struct {
@@ -40,9 +45,6 @@ type ExecPath struct {
W bool W bool
} }
// SetSchedIdle is whether to set [ext.SCHED_IDLE] scheduling priority.
var SetSchedIdle bool
// GetArtifactFunc is the function signature of [FContext.GetArtifact]. // GetArtifactFunc is the function signature of [FContext.GetArtifact].
type GetArtifactFunc func(Artifact) (*check.Absolute, unique.Handle[Checksum]) type GetArtifactFunc func(Artifact) (*check.Absolute, unique.Handle[Checksum])
@@ -400,6 +402,7 @@ const SeccompPresets = std.PresetStrict &
func (a *execArtifact) makeContainer( func (a *execArtifact) makeContainer(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
flags, jobs int,
hostNet bool, hostNet bool,
temp, work *check.Absolute, temp, work *check.Absolute,
getArtifact GetArtifactFunc, getArtifact GetArtifactFunc,
@@ -426,15 +429,16 @@ func (a *execArtifact) makeContainer(
z.SeccompFlags |= seccomp.AllowMultiarch z.SeccompFlags |= seccomp.AllowMultiarch
z.ParentPerm = 0700 z.ParentPerm = 0700
z.HostNet = hostNet z.HostNet = hostNet
z.HostAbstract = flags&CHostAbstract != 0
z.Hostname = "cure" z.Hostname = "cure"
z.SetScheduler = SetSchedIdle z.SetScheduler = flags&CSchedIdle != 0
z.SchedPolicy = ext.SCHED_IDLE z.SchedPolicy = ext.SCHED_IDLE
if z.HostNet { if z.HostNet {
z.Hostname = "cure-net" z.Hostname = "cure-net"
} }
z.Uid, z.Gid = (1<<10)-1, (1<<10)-1 z.Uid, z.Gid = (1<<10)-1, (1<<10)-1
z.Dir, z.Path, z.Args = a.dir, a.path, a.args
z.Dir, z.Env, z.Path, z.Args = a.dir, a.env, a.path, a.args z.Env = slices.Concat(a.env, []string{EnvJobs + "=" + strconv.Itoa(jobs)})
z.Grow(len(a.paths) + 4) z.Grow(len(a.paths) + 4)
for i, b := range a.paths { for i, b := range a.paths {
@@ -563,6 +567,8 @@ func (c *Cache) EnterExec(
var z *container.Container var z *container.Container
z, err = e.makeContainer( z, err = e.makeContainer(
ctx, c.msg, ctx, c.msg,
c.flags,
c.jobs,
hostNet, hostNet,
temp, work, temp, work,
func(a Artifact) (*check.Absolute, unique.Handle[Checksum]) { func(a Artifact) (*check.Absolute, unique.Handle[Checksum]) {
@@ -602,7 +608,7 @@ func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
msg := f.GetMessage() msg := f.GetMessage()
var z *container.Container var z *container.Container
if z, err = a.makeContainer( if z, err = a.makeContainer(
ctx, msg, hostNet, ctx, msg, f.cache.flags, f.GetJobs(), hostNet,
f.GetTempDir(), f.GetWorkDir(), f.GetTempDir(), f.GetWorkDir(),
f.GetArtifact, f.GetArtifact,
f.cache.Ident, f.cache.Ident,
+6 -12
View File
@@ -33,8 +33,7 @@ func TestExec(t *testing.T) {
) )
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"offline", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"offline", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -111,8 +110,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("Q5DluWQCAeohLoiGRImurwFp3vdz9IfQCoj7Fuhh73s4KQPRHpEQEnHTdNHmB8Fx")}, }, pkg.MustDecode("Q5DluWQCAeohLoiGRImurwFp3vdz9IfQCoj7Fuhh73s4KQPRHpEQEnHTdNHmB8Fx")},
{"net", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"net", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
wantChecksum := pkg.MustDecode( wantChecksum := pkg.MustDecode(
@@ -146,8 +144,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("bPYvvqxpfV7xcC1EptqyKNK1klLJgYHMDkzBcoOyK6j_Aj5hb0mXNPwTwPSK5F6Z")}, }, pkg.MustDecode("bPYvvqxpfV7xcC1EptqyKNK1klLJgYHMDkzBcoOyK6j_Aj5hb0mXNPwTwPSK5F6Z")},
{"overlay root", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay root", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -172,8 +169,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("PO2DSSCa4yoSgEYRcCSZfQfwow1yRigL3Ry-hI0RDI4aGuFBha-EfXeSJnG_5_Rl")}, }, pkg.MustDecode("PO2DSSCa4yoSgEYRcCSZfQfwow1yRigL3Ry-hI0RDI4aGuFBha-EfXeSJnG_5_Rl")},
{"overlay work", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay work", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -203,8 +199,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("iaRt6l_Wm2n-h5UsDewZxQkCmjZjyL8r7wv32QT2kyV55-Lx09Dq4gfg9BiwPnKs")}, }, pkg.MustDecode("iaRt6l_Wm2n-h5UsDewZxQkCmjZjyL8r7wv32QT2kyV55-Lx09Dq4gfg9BiwPnKs")},
{"multiple layers", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"multiple layers", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -256,8 +251,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("O2YzyR7IUGU5J2CADy0hUZ3A5NkP_Vwzs4UadEdn2oMZZVWRtH0xZGJ3HXiimTnZ")}, }, pkg.MustDecode("O2YzyR7IUGU5J2CADy0hUZ3A5NkP_Vwzs4UadEdn2oMZZVWRtH0xZGJ3HXiimTnZ")},
{"overlay layer promotion", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay layer promotion", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
+1 -3
View File
@@ -11,9 +11,7 @@ func TestFile(t *testing.T) {
t.Parallel() t.Parallel()
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"file", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
{"short", pkg.NewFile("null", []byte{0}), base.Append( {"short", pkg.NewFile("null", []byte{0}), base.Append(
"identifier", "identifier",
+52 -23
View File
@@ -3,7 +3,6 @@ package pkg
import ( import (
"bufio" "bufio"
"bytes" "bytes"
"context"
"crypto/sha512" "crypto/sha512"
"encoding/binary" "encoding/binary"
"errors" "errors"
@@ -11,6 +10,7 @@ import (
"io" "io"
"slices" "slices"
"strconv" "strconv"
"sync"
"syscall" "syscall"
"unique" "unique"
"unsafe" "unsafe"
@@ -39,22 +39,45 @@ func panicToError(errP *error) {
} }
} }
// irCache implements [IRCache].
type irCache struct {
// Artifact to [unique.Handle] of identifier cache.
artifact sync.Map
// Identifier free list, must not be accessed directly.
identPool sync.Pool
}
// zeroIRCache returns the initialised value of irCache.
func zeroIRCache() irCache {
return irCache{
identPool: sync.Pool{New: func() any { return new(extIdent) }},
}
}
// IRCache provides memory management and caching primitives for IR and
// identifier operations against [Artifact] implementations.
//
// The zero value is not safe for use.
type IRCache struct{ irCache }
// NewIR returns the address of a new [IRCache].
func NewIR() *IRCache {
return &IRCache{zeroIRCache()}
}
// IContext is passed to [Artifact.Params] and provides methods for writing // IContext is passed to [Artifact.Params] and provides methods for writing
// values to the IR writer. It does not expose the underlying [io.Writer]. // values to the IR writer. It does not expose the underlying [io.Writer].
// //
// IContext is valid until [Artifact.Params] returns. // IContext is valid until [Artifact.Params] returns.
type IContext struct { type IContext struct {
// Address of underlying [Cache], should be zeroed or made unusable after // Address of underlying irCache, should be zeroed or made unusable after
// [Artifact.Params] returns and must not be exposed directly. // [Artifact.Params] returns and must not be exposed directly.
cache *Cache ic *irCache
// Written to by various methods, should be zeroed after [Artifact.Params] // Written to by various methods, should be zeroed after [Artifact.Params]
// returns and must not be exposed directly. // returns and must not be exposed directly.
w io.Writer w io.Writer
} }
// Unwrap returns the underlying [context.Context].
func (i *IContext) Unwrap() context.Context { return i.cache.ctx }
// irZero is a zero IR word. // irZero is a zero IR word.
var irZero [wordSize]byte var irZero [wordSize]byte
@@ -136,11 +159,11 @@ func (i *IContext) mustWrite(p []byte) {
// WriteIdent is not defined for an [Artifact] not part of the slice returned by // WriteIdent is not defined for an [Artifact] not part of the slice returned by
// [Artifact.Dependencies]. // [Artifact.Dependencies].
func (i *IContext) WriteIdent(a Artifact) { func (i *IContext) WriteIdent(a Artifact) {
buf := i.cache.getIdentBuf() buf := i.ic.getIdentBuf()
defer i.cache.putIdentBuf(buf) defer i.ic.putIdentBuf(buf)
IRKindIdent.encodeHeader(0).put(buf[:]) IRKindIdent.encodeHeader(0).put(buf[:])
*(*ID)(buf[wordSize:]) = i.cache.Ident(a).Value() *(*ID)(buf[wordSize:]) = i.ic.Ident(a).Value()
i.mustWrite(buf[:]) i.mustWrite(buf[:])
} }
@@ -183,19 +206,19 @@ func (i *IContext) WriteString(s string) {
// Encode writes a deterministic, efficient representation of a to w and returns // Encode writes a deterministic, efficient representation of a to w and returns
// the first non-nil error encountered while writing to w. // the first non-nil error encountered while writing to w.
func (c *Cache) Encode(w io.Writer, a Artifact) (err error) { func (ic *irCache) Encode(w io.Writer, a Artifact) (err error) {
deps := a.Dependencies() deps := a.Dependencies()
idents := make([]*extIdent, len(deps)) idents := make([]*extIdent, len(deps))
for i, d := range deps { for i, d := range deps {
dbuf, did := c.unsafeIdent(d, true) dbuf, did := ic.unsafeIdent(d, true)
if dbuf == nil { if dbuf == nil {
dbuf = c.getIdentBuf() dbuf = ic.getIdentBuf()
binary.LittleEndian.PutUint64(dbuf[:], uint64(d.Kind())) binary.LittleEndian.PutUint64(dbuf[:], uint64(d.Kind()))
*(*ID)(dbuf[wordSize:]) = did.Value() *(*ID)(dbuf[wordSize:]) = did.Value()
} else { } else {
c.storeIdent(d, dbuf) ic.storeIdent(d, dbuf)
} }
defer c.putIdentBuf(dbuf) defer ic.putIdentBuf(dbuf)
idents[i] = dbuf idents[i] = dbuf
} }
slices.SortFunc(idents, func(a, b *extIdent) int { slices.SortFunc(idents, func(a, b *extIdent) int {
@@ -221,10 +244,10 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
} }
func() { func() {
i := IContext{c, w} i := IContext{ic, w}
defer panicToError(&err) defer panicToError(&err)
defer func() { i.cache, i.w = nil, nil }() defer func() { i.ic, i.w = nil, nil }()
a.Params(&i) a.Params(&i)
}() }()
@@ -233,7 +256,7 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
} }
var f IREndFlag var f IREndFlag
kcBuf := c.getIdentBuf() kcBuf := ic.getIdentBuf()
sz := wordSize sz := wordSize
if kc, ok := a.(KnownChecksum); ok { if kc, ok := a.(KnownChecksum); ok {
f |= IREndKnownChecksum f |= IREndKnownChecksum
@@ -243,13 +266,13 @@ func (c *Cache) Encode(w io.Writer, a Artifact) (err error) {
IRKindEnd.encodeHeader(uint32(f)).put(kcBuf[:]) IRKindEnd.encodeHeader(uint32(f)).put(kcBuf[:])
_, err = w.Write(kcBuf[:sz]) _, err = w.Write(kcBuf[:sz])
c.putIdentBuf(kcBuf) ic.putIdentBuf(kcBuf)
return return
} }
// encodeAll implements EncodeAll by recursively encoding dependencies and // encodeAll implements EncodeAll by recursively encoding dependencies and
// performs deduplication by value via the encoded map. // performs deduplication by value via the encoded map.
func (c *Cache) encodeAll( func (ic *irCache) encodeAll(
w io.Writer, w io.Writer,
a Artifact, a Artifact,
encoded map[Artifact]struct{}, encoded map[Artifact]struct{},
@@ -259,13 +282,13 @@ func (c *Cache) encodeAll(
} }
for _, d := range a.Dependencies() { for _, d := range a.Dependencies() {
if err = c.encodeAll(w, d, encoded); err != nil { if err = ic.encodeAll(w, d, encoded); err != nil {
return return
} }
} }
encoded[a] = struct{}{} encoded[a] = struct{}{}
return c.Encode(w, a) return ic.Encode(w, a)
} }
// EncodeAll writes a self-describing IR stream of a to w and returns the first // EncodeAll writes a self-describing IR stream of a to w and returns the first
@@ -283,8 +306,8 @@ func (c *Cache) encodeAll(
// the ident cache, nor does it contribute identifiers it computes back to the // the ident cache, nor does it contribute identifiers it computes back to the
// ident cache. Because of this, multiple invocations of EncodeAll will have // ident cache. Because of this, multiple invocations of EncodeAll will have
// similar cost and does not amortise when combined with a call to Cure. // similar cost and does not amortise when combined with a call to Cure.
func (c *Cache) EncodeAll(w io.Writer, a Artifact) error { func (ic *irCache) EncodeAll(w io.Writer, a Artifact) error {
return c.encodeAll(w, a, make(map[Artifact]struct{})) return ic.encodeAll(w, a, make(map[Artifact]struct{}))
} }
// ErrRemainingIR is returned for a [IRReadFunc] that failed to call // ErrRemainingIR is returned for a [IRReadFunc] that failed to call
@@ -409,6 +432,12 @@ func (e InvalidKindError) Error() string {
// register is not safe for concurrent use. register must not be called after // register is not safe for concurrent use. register must not be called after
// the first instance of [Cache] has been opened. // the first instance of [Cache] has been opened.
func register(k Kind, f IRReadFunc) { func register(k Kind, f IRReadFunc) {
openMu.Lock()
defer openMu.Unlock()
if opened {
panic("attempting to register after open")
}
if _, ok := irArtifact[k]; ok { if _, ok := irArtifact[k]; ok {
panic("attempting to register " + strconv.Itoa(int(k)) + " twice") panic("attempting to register " + strconv.Itoa(int(k)) + " twice")
} }
+1 -1
View File
@@ -85,7 +85,7 @@ func TestIRRoundtrip(t *testing.T) {
testCasesCache := make([]cacheTestCase, len(testCases)) testCasesCache := make([]cacheTestCase, len(testCases))
for i, tc := range testCases { for i, tc := range testCases {
want := tc.a want := tc.a
testCasesCache[i] = cacheTestCase{tc.name, nil, testCasesCache[i] = cacheTestCase{tc.name, 0, nil,
func(t *testing.T, base *check.Absolute, c *pkg.Cache) { func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
r, w := io.Pipe() r, w := io.Pipe()
+10 -22
View File
@@ -8,7 +8,6 @@ import (
"testing" "testing"
"testing/fstest" "testing/fstest"
"unique" "unique"
"unsafe"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/internal/pkg" "hakurei.app/internal/pkg"
@@ -32,21 +31,15 @@ func TestHTTPGet(t *testing.T) {
})) }))
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"direct", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"direct", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext r := newRContext(t, c)
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
f := pkg.NewHTTPGet( f := pkg.NewHTTPGet(
&client, &client,
"file:///testdata", "file:///testdata",
testdataChecksum.Value(), testdataChecksum.Value(),
) )
var got []byte var got []byte
if rc, err := f.Cure(&r); err != nil { if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil { } else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err) t.Fatalf("ReadAll: error = %v", err)
@@ -65,7 +58,7 @@ func TestHTTPGet(t *testing.T) {
wantErrMismatch := &pkg.ChecksumMismatchError{ wantErrMismatch := &pkg.ChecksumMismatchError{
Got: testdataChecksum.Value(), Got: testdataChecksum.Value(),
} }
if rc, err := f.Cure(&r); err != nil { if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil { } else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err) t.Fatalf("ReadAll: error = %v", err)
@@ -76,7 +69,7 @@ func TestHTTPGet(t *testing.T) {
} }
// check fallback validation // check fallback validation
if rc, err := f.Cure(&r); err != nil { if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
} else if err = rc.Close(); !reflect.DeepEqual(err, wantErrMismatch) { } else if err = rc.Close(); !reflect.DeepEqual(err, wantErrMismatch) {
t.Fatalf("Close: error = %#v, want %#v", err, wantErrMismatch) t.Fatalf("Close: error = %#v, want %#v", err, wantErrMismatch)
@@ -89,18 +82,13 @@ func TestHTTPGet(t *testing.T) {
pkg.Checksum{}, pkg.Checksum{},
) )
wantErrNotFound := pkg.ResponseStatusError(http.StatusNotFound) wantErrNotFound := pkg.ResponseStatusError(http.StatusNotFound)
if _, err := f.Cure(&r); !reflect.DeepEqual(err, wantErrNotFound) { if _, err := f.Cure(r); !reflect.DeepEqual(err, wantErrNotFound) {
t.Fatalf("Cure: error = %#v, want %#v", err, wantErrNotFound) t.Fatalf("Cure: error = %#v, want %#v", err, wantErrNotFound)
} }
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")}, }, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"cure", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"cure", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext r := newRContext(t, c)
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
f := pkg.NewHTTPGet( f := pkg.NewHTTPGet(
&client, &client,
@@ -120,7 +108,7 @@ func TestHTTPGet(t *testing.T) {
} }
var got []byte var got []byte
if rc, err := f.Cure(&r); err != nil { if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil { } else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err) t.Fatalf("ReadAll: error = %v", err)
@@ -136,7 +124,7 @@ func TestHTTPGet(t *testing.T) {
"file:///testdata", "file:///testdata",
testdataChecksum.Value(), testdataChecksum.Value(),
) )
if rc, err := f.Cure(&r); err != nil { if rc, err := f.Cure(r); err != nil {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
} else if got, err = io.ReadAll(rc); err != nil { } else if got, err = io.ReadAll(rc); err != nil {
t.Fatalf("ReadAll: error = %v", err) t.Fatalf("ReadAll: error = %v", err)
+388 -136
View File
@@ -13,12 +13,12 @@ import (
"hash" "hash"
"io" "io"
"io/fs" "io/fs"
"iter"
"maps" "maps"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strconv"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -71,8 +71,70 @@ func MustDecode(s string) (checksum Checksum) {
return return
} }
var (
// extension is a string uniquely identifying a set of custom [Artifact]
// implementations registered by calling [Register].
extension string
// openMu synchronises access to global state for initialisation.
openMu sync.Mutex
// opened is false if [Open] was never called.
opened bool
)
// Extension returns a string uniquely identifying the currently registered set
// of custom [Artifact], or the zero value if none was registered.
func Extension() string { return extension }
// ValidExtension returns whether s is valid for use in a call to SetExtension.
func ValidExtension(s string) bool {
if l := len(s); l == 0 || l > 128 {
return false
}
for _, v := range s {
if v < 'a' || v > 'z' {
return false
}
}
return true
}
// ErrInvalidExtension is returned for a variant identification string for which
// [ValidExtension] returns false.
var ErrInvalidExtension = errors.New("invalid extension variant identification string")
// SetExtension sets the extension variant identification string. SetExtension
// must be called before [Open] if custom [Artifact] implementations had been
// recorded by calling [Register].
//
// The variant identification string must be between 1 and 128 bytes long and
// consists of only bytes between 'a' and 'z'.
//
// SetExtension is not safe for concurrent use. SetExtension is called at most
// once and must not be called after the first instance of Cache has been opened.
func SetExtension(s string) {
openMu.Lock()
defer openMu.Unlock()
if opened {
panic("attempting to set extension after open")
}
if extension != "" {
panic("attempting to set extension twice")
}
if !ValidExtension(s) {
panic(ErrInvalidExtension)
}
extension = s
statusHeader = makeStatusHeader(s)
}
// common holds elements and receives methods shared between different contexts. // common holds elements and receives methods shared between different contexts.
type common struct { type common struct {
// Context specific to this [Artifact]. The toplevel context in [Cache] must
// not be exposed directly.
ctx context.Context
// Address of underlying [Cache], should be zeroed or made unusable after // Address of underlying [Cache], should be zeroed or made unusable after
// Cure returns and must not be exposed directly. // Cure returns and must not be exposed directly.
cache *Cache cache *Cache
@@ -99,19 +161,27 @@ type TContext struct {
common common
} }
// statusHeader is the header written to all status files in dirStatus. // makeStatusHeader creates the header written to every status file. This should
var statusHeader = func() string { // not be called directly, its result is stored in statusHeader and will not
// change after the first [Cache] is opened.
func makeStatusHeader(extension string) string {
s := programName s := programName
if v := info.Version(); v != info.FallbackVersion { if v := info.Version(); v != info.FallbackVersion {
s += " " + v s += " " + v
} }
if extension != "" {
s += " with " + extension + " extensions"
}
s += " (" + runtime.GOARCH + ")" s += " (" + runtime.GOARCH + ")"
if name, err := os.Hostname(); err == nil { if name, err := os.Hostname(); err == nil {
s += " on " + name s += " on " + name
} }
s += "\n\n" s += "\n\n"
return s return s
}() }
// statusHeader is the header written to all status files in dirStatus.
var statusHeader = makeStatusHeader("")
// prepareStatus initialises the status file once. // prepareStatus initialises the status file once.
func (t *TContext) prepareStatus() error { func (t *TContext) prepareStatus() error {
@@ -184,11 +254,15 @@ func (t *TContext) destroy(errP *error) {
} }
// Unwrap returns the underlying [context.Context]. // Unwrap returns the underlying [context.Context].
func (c *common) Unwrap() context.Context { return c.cache.ctx } func (c *common) Unwrap() context.Context { return c.ctx }
// GetMessage returns [message.Msg] held by the underlying [Cache]. // GetMessage returns [message.Msg] held by the underlying [Cache].
func (c *common) GetMessage() message.Msg { return c.cache.msg } func (c *common) GetMessage() message.Msg { return c.cache.msg }
// GetJobs returns the preferred number of jobs to run, when applicable. Its
// value must not affect cure outcome.
func (c *common) GetJobs() int { return c.cache.jobs }
// GetWorkDir returns a pathname to a directory which [Artifact] is expected to // GetWorkDir returns a pathname to a directory which [Artifact] is expected to
// write its output to. This is not the final resting place of the [Artifact] // write its output to. This is not the final resting place of the [Artifact]
// and this pathname should not be directly referred to in the final contents. // and this pathname should not be directly referred to in the final contents.
@@ -208,11 +282,11 @@ func (t *TContext) GetTempDir() *check.Absolute { return t.temp }
// [ChecksumMismatchError], or the underlying implementation may block on Close. // [ChecksumMismatchError], or the underlying implementation may block on Close.
func (c *common) Open(a Artifact) (r io.ReadCloser, err error) { func (c *common) Open(a Artifact) (r io.ReadCloser, err error) {
if f, ok := a.(FileArtifact); ok { if f, ok := a.(FileArtifact); ok {
return c.cache.openFile(f) return c.cache.openFile(c.ctx, f)
} }
var pathname *check.Absolute var pathname *check.Absolute
if pathname, _, err = c.cache.Cure(a); err != nil { if pathname, _, err = c.cache.cure(a, true); err != nil {
return return
} }
@@ -331,23 +405,6 @@ type FloodArtifact interface {
Artifact Artifact
} }
// Flood returns an iterator over the dependency tree of an [Artifact].
func Flood(a Artifact) iter.Seq[Artifact] {
return func(yield func(Artifact) bool) {
for _, d := range a.Dependencies() {
if !yield(d) {
return
}
for d0 := range Flood(d) {
if !yield(d0) {
return
}
}
}
}
}
// TrivialArtifact refers to an [Artifact] that cures without requiring that // TrivialArtifact refers to an [Artifact] that cures without requiring that
// any other [Artifact] is cured before it. Its dependency tree is ignored after // any other [Artifact] is cured before it. Its dependency tree is ignored after
// computing its identifier. // computing its identifier.
@@ -390,6 +447,9 @@ type KnownChecksum interface {
} }
// FileArtifact refers to an [Artifact] backed by a single file. // FileArtifact refers to an [Artifact] backed by a single file.
//
// FileArtifact does not support fine-grained cancellation. Its context is
// inherited from the first [TrivialArtifact] or [FloodArtifact] that opens it.
type FileArtifact interface { type FileArtifact interface {
// Cure returns [io.ReadCloser] of the full contents of [FileArtifact]. If // Cure returns [io.ReadCloser] of the full contents of [FileArtifact]. If
// [FileArtifact] implements [KnownChecksum], Cure is responsible for // [FileArtifact] implements [KnownChecksum], Cure is responsible for
@@ -434,6 +494,9 @@ const (
// KindFile is the kind of [Artifact] returned by [NewFile]. // KindFile is the kind of [Artifact] returned by [NewFile].
KindFile KindFile
// _kindEnd is the total number of kinds and does not denote a kind.
_kindEnd
// KindCustomOffset is the first [Kind] value reserved for implementations // KindCustomOffset is the first [Kind] value reserved for implementations
// not from this package. // not from this package.
KindCustomOffset = 1 << 31 KindCustomOffset = 1 << 31
@@ -448,6 +511,9 @@ const (
// fileLock is the file name appended to Cache.base for guaranteeing // fileLock is the file name appended to Cache.base for guaranteeing
// exclusive access to the cache directory. // exclusive access to the cache directory.
fileLock = "lock" fileLock = "lock"
// fileVariant is the file name appended to Cache.base holding the variant
// identification string set by a prior call to [SetExtension].
fileVariant = "variant"
// dirIdentifier is the directory name appended to Cache.base for storing // dirIdentifier is the directory name appended to Cache.base for storing
// artifacts named after their [ID]. // artifacts named after their [ID].
@@ -506,6 +572,77 @@ type pendingArtifactDep struct {
*sync.WaitGroup *sync.WaitGroup
} }
const (
// CValidateKnown arranges for [KnownChecksum] outcomes to be validated to
// match its intended checksum.
//
// A correct implementation of [KnownChecksum] does not successfully cure
// with output not matching its intended checksum. When an implementation
// fails to perform this validation correctly, the on-disk format enters
// an inconsistent state (correctable by [Cache.Scrub]).
//
// This flag causes [Cache.Cure] to always compute the checksum, and reject
// a cure if it does not match the intended checksum.
//
// This behaviour significantly reduces performance and is not recommended
// outside of testing a custom [Artifact] implementation.
CValidateKnown = 1 << iota
// CSchedIdle arranges for the [ext.SCHED_IDLE] scheduling priority to be
// set for [KindExec] and [KindExecNet] containers.
CSchedIdle
// CAssumeChecksum enables the use of [KnownChecksum] for duplicate function
// call suppression via the on-disk cache.
//
// This may cause incorrect cure outcome if an impossible checksum is
// specified that matches an output already present in the on-disk cache.
// This may be avoided by purposefully specifying a statistically
// unattainable checksum, like the zero value.
//
// While this optimisation might seem appealing, it is almost never
// applicable in real world use. Almost every time this path was taken, it
// was caused by an incorrect checksum accidentally left behind while
// bumping a package. Only enable this if you are really sure you need it.
CAssumeChecksum
// CHostAbstract disables restriction of sandboxed processes from connecting
// to an abstract UNIX socket created by a host process.
//
// This is considered less secure in some systems, but does not introduce
// impurity due to [KindExecNet] being [KnownChecksum]. This flag exists
// to support kernels without Landlock LSM enabled.
CHostAbstract
// CPromoteVariant allows [pkg.Open] to promote an unextended on-disk cache
// to the current extension variant. This is a one-way operation.
CPromoteVariant
)
// toplevel holds [context.WithCancel] over caller-supplied context, where all
// [Artifact] context are derived from.
type toplevel struct {
ctx context.Context
cancel context.CancelFunc
}
// newToplevel returns the address of a new toplevel via ctx.
func newToplevel(ctx context.Context) *toplevel {
var t toplevel
t.ctx, t.cancel = context.WithCancel(ctx)
return &t
}
// pendingCure provides synchronisation and cancellation for pending cures.
type pendingCure struct {
// Closed on cure completion.
done <-chan struct{}
// Error outcome, safe to access after done is closed.
err error
// Cancels the corresponding cure.
cancel context.CancelFunc
}
// Cache is a support layer that implementations of [Artifact] can use to store // Cache is a support layer that implementations of [Artifact] can use to store
// cured [Artifact] data in a content addressed fashion. // cured [Artifact] data in a content addressed fashion.
type Cache struct { type Cache struct {
@@ -513,11 +650,10 @@ type Cache struct {
// implementation and receives an equal amount of elements after. // implementation and receives an equal amount of elements after.
cures chan struct{} cures chan struct{}
// [context.WithCancel] over caller-supplied context, used by [Artifact] and // Parent context which toplevel was derived from.
// all dependency curing goroutines. parent context.Context
ctx context.Context // For deriving curing context, must not be accessed directly.
// Cancels ctx. toplevel atomic.Pointer[toplevel]
cancel context.CancelFunc
// For waiting on dependency curing goroutines. // For waiting on dependency curing goroutines.
wg sync.WaitGroup wg sync.WaitGroup
// Reports new cures and passed to [Artifact]. // Reports new cures and passed to [Artifact].
@@ -525,17 +661,13 @@ type Cache struct {
// Directory where all [Cache] related files are placed. // Directory where all [Cache] related files are placed.
base *check.Absolute base *check.Absolute
// Immutable cure options set by [Open].
flags int
// Immutable job count, when applicable.
jobs int
// Whether to validate [FileArtifact.Cure] for a [KnownChecksum] file. This // Must not be exposed directly.
// significantly reduces performance. irCache
strict bool
// Maximum size of a dependency graph.
threshold uintptr
// Artifact to [unique.Handle] of identifier cache.
artifact sync.Map
// Identifier free list, must not be accessed directly.
identPool sync.Pool
// Synchronises access to dirChecksum. // Synchronises access to dirChecksum.
checksumMu sync.RWMutex checksumMu sync.RWMutex
@@ -545,9 +677,11 @@ type Cache struct {
// Identifier to error pair for unrecoverably faulted [Artifact]. // Identifier to error pair for unrecoverably faulted [Artifact].
identErr map[unique.Handle[ID]]error identErr map[unique.Handle[ID]]error
// Pending identifiers, accessed through Cure for entries not in ident. // Pending identifiers, accessed through Cure for entries not in ident.
identPending map[unique.Handle[ID]]<-chan struct{} identPending map[unique.Handle[ID]]*pendingCure
// Synchronises access to ident and corresponding filesystem entries. // Synchronises access to ident and corresponding filesystem entries.
identMu sync.RWMutex identMu sync.RWMutex
// Synchronises entry into Abort and Cure.
abortMu sync.RWMutex
// Synchronises entry into exclusive artifacts for the cure method. // Synchronises entry into exclusive artifacts for the cure method.
exclMu sync.Mutex exclMu sync.Mutex
@@ -556,51 +690,37 @@ type Cache struct {
// Unlocks the on-filesystem cache. Must only be called from Close. // Unlocks the on-filesystem cache. Must only be called from Close.
unlock func() unlock func()
// Synchronises calls to Close. // Whether [Cache] is considered closed.
closeOnce sync.Once closed bool
// Synchronises calls to Abort and Close.
closeMu sync.Mutex
// Whether EnterExec has not yet returned. // Whether EnterExec has not yet returned.
inExec atomic.Bool inExec atomic.Bool
} }
// IsStrict returns whether the [Cache] strictly verifies checksums.
func (c *Cache) IsStrict() bool { return c.strict }
// SetStrict sets whether the [Cache] strictly verifies checksums, even when
// the implementation promises to validate them internally. This significantly
// reduces performance and is not recommended outside of testing.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetStrict(strict bool) { c.strict = strict }
// SetThreshold imposes a maximum size on the dependency graph, checked on every
// call to Cure. The zero value disables this check entirely.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetThreshold(threshold uintptr) { c.threshold = threshold }
// extIdent is a [Kind] concatenated with [ID]. // extIdent is a [Kind] concatenated with [ID].
type extIdent [wordSize + len(ID{})]byte type extIdent [wordSize + len(ID{})]byte
// getIdentBuf returns the address of an extIdent for Ident. // getIdentBuf returns the address of an extIdent for Ident.
func (c *Cache) getIdentBuf() *extIdent { return c.identPool.Get().(*extIdent) } func (ic *irCache) getIdentBuf() *extIdent { return ic.identPool.Get().(*extIdent) }
// putIdentBuf adds buf to identPool. // putIdentBuf adds buf to identPool.
func (c *Cache) putIdentBuf(buf *extIdent) { c.identPool.Put(buf) } func (ic *irCache) putIdentBuf(buf *extIdent) { ic.identPool.Put(buf) }
// storeIdent adds an [Artifact] to the artifact cache. // storeIdent adds an [Artifact] to the artifact cache.
func (c *Cache) storeIdent(a Artifact, buf *extIdent) unique.Handle[ID] { func (ic *irCache) storeIdent(a Artifact, buf *extIdent) unique.Handle[ID] {
idu := unique.Make(ID(buf[wordSize:])) idu := unique.Make(ID(buf[wordSize:]))
c.artifact.Store(a, idu) ic.artifact.Store(a, idu)
return idu return idu
} }
// Ident returns the identifier of an [Artifact]. // Ident returns the identifier of an [Artifact].
func (c *Cache) Ident(a Artifact) unique.Handle[ID] { func (ic *irCache) Ident(a Artifact) unique.Handle[ID] {
buf, idu := c.unsafeIdent(a, false) buf, idu := ic.unsafeIdent(a, false)
if buf != nil { if buf != nil {
idu = c.storeIdent(a, buf) idu = ic.storeIdent(a, buf)
c.putIdentBuf(buf) ic.putIdentBuf(buf)
} }
return idu return idu
} }
@@ -608,17 +728,17 @@ func (c *Cache) Ident(a Artifact) unique.Handle[ID] {
// unsafeIdent implements Ident but returns the underlying buffer for a newly // unsafeIdent implements Ident but returns the underlying buffer for a newly
// computed identifier. Callers must return this buffer to identPool. encodeKind // computed identifier. Callers must return this buffer to identPool. encodeKind
// is only a hint, kind may still be encoded in the buffer. // is only a hint, kind may still be encoded in the buffer.
func (c *Cache) unsafeIdent(a Artifact, encodeKind bool) ( func (ic *irCache) unsafeIdent(a Artifact, encodeKind bool) (
buf *extIdent, buf *extIdent,
idu unique.Handle[ID], idu unique.Handle[ID],
) { ) {
if id, ok := c.artifact.Load(a); ok { if id, ok := ic.artifact.Load(a); ok {
idu = id.(unique.Handle[ID]) idu = id.(unique.Handle[ID])
return return
} }
if ki, ok := a.(KnownIdent); ok { if ki, ok := a.(KnownIdent); ok {
buf = c.getIdentBuf() buf = ic.getIdentBuf()
if encodeKind { if encodeKind {
binary.LittleEndian.PutUint64(buf[:], uint64(a.Kind())) binary.LittleEndian.PutUint64(buf[:], uint64(a.Kind()))
} }
@@ -626,9 +746,9 @@ func (c *Cache) unsafeIdent(a Artifact, encodeKind bool) (
return return
} }
buf = c.getIdentBuf() buf = ic.getIdentBuf()
h := sha512.New384() h := sha512.New384()
if err := c.Encode(h, a); err != nil { if err := ic.Encode(h, a); err != nil {
// unreachable // unreachable
panic(err) panic(err)
} }
@@ -997,7 +1117,11 @@ func (c *Cache) Scrub(checks int) error {
// loadOrStoreIdent attempts to load a cached [Artifact] by its identifier or // loadOrStoreIdent attempts to load a cached [Artifact] by its identifier or
// wait for a pending [Artifact] to cure. If neither is possible, the current // wait for a pending [Artifact] to cure. If neither is possible, the current
// identifier is stored in identPending and a non-nil channel is returned. // identifier is stored in identPending and a non-nil channel is returned.
//
// Since identErr is treated as grow-only, loadOrStoreIdent must not be entered
// without holding a read lock on abortMu.
func (c *Cache) loadOrStoreIdent(id unique.Handle[ID]) ( func (c *Cache) loadOrStoreIdent(id unique.Handle[ID]) (
ctx context.Context,
done chan<- struct{}, done chan<- struct{},
checksum unique.Handle[Checksum], checksum unique.Handle[Checksum],
err error, err error,
@@ -1014,20 +1138,23 @@ func (c *Cache) loadOrStoreIdent(id unique.Handle[ID]) (
return return
} }
var notify <-chan struct{} var pending *pendingCure
if notify, ok = c.identPending[id]; ok { if pending, ok = c.identPending[id]; ok {
c.identMu.Unlock() c.identMu.Unlock()
<-notify <-pending.done
c.identMu.RLock() c.identMu.RLock()
if checksum, ok = c.ident[id]; !ok { if checksum, ok = c.ident[id]; !ok {
err = c.identErr[id] err = pending.err
} }
c.identMu.RUnlock() c.identMu.RUnlock()
return return
} }
d := make(chan struct{}) d := make(chan struct{})
c.identPending[id] = d pending = &pendingCure{done: d}
ctx, pending.cancel = context.WithCancel(c.toplevel.Load().ctx)
c.wg.Add(1)
c.identPending[id] = pending
c.identMu.Unlock() c.identMu.Unlock()
done = d done = d
return return
@@ -1043,22 +1170,63 @@ func (c *Cache) finaliseIdent(
) { ) {
c.identMu.Lock() c.identMu.Lock()
if err != nil { if err != nil {
c.identPending[id].err = err
c.identErr[id] = err c.identErr[id] = err
} else { } else {
c.ident[id] = checksum c.ident[id] = checksum
} }
delete(c.identPending, id) delete(c.identPending, id)
c.identMu.Unlock() c.identMu.Unlock()
c.wg.Done()
close(done) close(done)
} }
// Done returns a channel that is closed when the ongoing cure of an [Artifact]
// referred to by the specified identifier completes. Done may return nil if
// no ongoing cure of the specified identifier exists.
func (c *Cache) Done(id unique.Handle[ID]) <-chan struct{} {
c.identMu.RLock()
pending, ok := c.identPending[id]
c.identMu.RUnlock()
if !ok || pending == nil {
return nil
}
return pending.done
}
// Cancel cancels the ongoing cure of an [Artifact] referred to by the specified
// identifier. Cancel returns whether the [context.CancelFunc] has been killed.
// Cancel returns after the cure is complete.
func (c *Cache) Cancel(id unique.Handle[ID]) bool {
c.identMu.RLock()
pending, ok := c.identPending[id]
c.identMu.RUnlock()
if !ok || pending == nil || pending.cancel == nil {
return false
}
pending.cancel()
<-pending.done
c.abortMu.Lock()
c.identMu.Lock()
delete(c.identErr, id)
c.identMu.Unlock()
c.abortMu.Unlock()
return true
}
// openFile tries to load [FileArtifact] from [Cache], and if that fails, // openFile tries to load [FileArtifact] from [Cache], and if that fails,
// obtains it via [FileArtifact.Cure] instead. Notably, it does not cure // obtains it via [FileArtifact.Cure] instead. Notably, it does not cure
// [FileArtifact] to the filesystem. If err is nil, the caller is responsible // [FileArtifact] to the filesystem. If err is nil, the caller is responsible
// for closing the resulting [io.ReadCloser]. // for closing the resulting [io.ReadCloser].
func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) { //
if kc, ok := f.(KnownChecksum); ok { // The context must originate from loadOrStoreIdent to enable cancellation.
func (c *Cache) openFile(
ctx context.Context,
f FileArtifact,
) (r io.ReadCloser, err error) {
if kc, ok := f.(KnownChecksum); c.flags&CAssumeChecksum != 0 && ok {
c.checksumMu.RLock() c.checksumMu.RLock()
r, err = os.Open(c.base.Append( r, err = os.Open(c.base.Append(
dirChecksum, dirChecksum,
@@ -1088,7 +1256,7 @@ func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) {
} }
}() }()
} }
return f.Cure(&RContext{common{c}}) return f.Cure(&RContext{common{ctx, c}})
} }
return return
} }
@@ -1229,14 +1397,6 @@ func (e InvalidArtifactError) Error() string {
return "artifact " + Encode(e) + " cannot be cured" return "artifact " + Encode(e) + " cannot be cured"
} }
// DependencyError refers to an artifact with a dependency tree larger than the
// threshold specified by a previous call to [Cache.SetThreshold].
type DependencyError struct{ A Artifact }
func (e DependencyError) Error() string {
return "artifact has too many dependencies"
}
// Cure cures the [Artifact] and returns its pathname and [Checksum]. Direct // Cure cures the [Artifact] and returns its pathname and [Checksum]. Direct
// calls to Cure are not subject to the cures limit. // calls to Cure are not subject to the cures limit.
func (c *Cache) Cure(a Artifact) ( func (c *Cache) Cure(a Artifact) (
@@ -1244,24 +1404,11 @@ func (c *Cache) Cure(a Artifact) (
checksum unique.Handle[Checksum], checksum unique.Handle[Checksum],
err error, err error,
) { ) {
select { c.abortMu.RLock()
case <-c.ctx.Done(): defer c.abortMu.RUnlock()
err = c.ctx.Err()
if err = c.toplevel.Load().ctx.Err(); err != nil {
return return
default:
}
if c.threshold > 0 {
var n uintptr
for range Flood(a) {
if n == c.threshold {
err = DependencyError{a}
return
}
n++
}
c.msg.Verbosef("visited %d artifacts", n)
} }
return c.cure(a, true) return c.cure(a, true)
@@ -1347,15 +1494,16 @@ func (c *Cache) enterCure(a Artifact, curesExempt bool) error {
return nil return nil
} }
ctx := c.toplevel.Load().ctx
select { select {
case c.cures <- struct{}{}: case c.cures <- struct{}{}:
return nil return nil
case <-c.ctx.Done(): case <-ctx.Done():
if a.IsExclusive() { if a.IsExclusive() {
c.exclMu.Unlock() c.exclMu.Unlock()
} }
return c.ctx.Err() return ctx.Err()
} }
} }
@@ -1453,7 +1601,8 @@ func (r *RContext) NewMeasuredReader(
return r.cache.newMeasuredReader(rc, checksum) return r.cache.newMeasuredReader(rc, checksum)
} }
// cure implements Cure without checking the full dependency graph. // cure implements Cure without acquiring a read lock on abortMu. cure must not
// be entered during Abort.
func (c *Cache) cure(a Artifact, curesExempt bool) ( func (c *Cache) cure(a Artifact, curesExempt bool) (
pathname *check.Absolute, pathname *check.Absolute,
checksum unique.Handle[Checksum], checksum unique.Handle[Checksum],
@@ -1472,8 +1621,11 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
} }
}() }()
var done chan<- struct{} var (
done, checksum, err = c.loadOrStoreIdent(id) ctx context.Context
done chan<- struct{}
)
ctx, done, checksum, err = c.loadOrStoreIdent(id)
if done == nil { if done == nil {
return return
} else { } else {
@@ -1521,16 +1673,18 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
checksums, checksums,
) )
c.checksumMu.RLock() if c.flags&CAssumeChecksum != 0 {
checksumFi, err = os.Stat(checksumPathname.String()) c.checksumMu.RLock()
c.checksumMu.RUnlock() checksumFi, err = os.Stat(checksumPathname.String())
c.checksumMu.RUnlock()
if err != nil { if err != nil {
if !errors.Is(err, os.ErrNotExist) { if !errors.Is(err, os.ErrNotExist) {
return return
}
checksumFi, err = nil, nil
} }
checksumFi, err = nil, nil
} }
} }
@@ -1584,9 +1738,9 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
if err = c.enterCure(a, curesExempt); err != nil { if err = c.enterCure(a, curesExempt); err != nil {
return return
} }
r, err = f.Cure(&RContext{common{c}}) r, err = f.Cure(&RContext{common{ctx, c}})
if err == nil { if err == nil {
if checksumPathname == nil || c.IsStrict() { if checksumPathname == nil || c.flags&CValidateKnown != 0 {
h := sha512.New384() h := sha512.New384()
hbw := c.getWriter(h) hbw := c.getWriter(h)
_, err = io.Copy(w, io.TeeReader(r, hbw)) _, err = io.Copy(w, io.TeeReader(r, hbw))
@@ -1603,7 +1757,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
if checksumPathname == nil { if checksumPathname == nil {
checksum = unique.Make(Checksum(buf[:])) checksum = unique.Make(Checksum(buf[:]))
checksums = Encode(Checksum(buf[:])) checksums = Encode(Checksum(buf[:]))
} else if c.IsStrict() { } else if c.flags&CValidateKnown != 0 {
if got := Checksum(buf[:]); got != checksum.Value() { if got := Checksum(buf[:]); got != checksum.Value() {
err = &ChecksumMismatchError{ err = &ChecksumMismatchError{
Got: got, Got: got,
@@ -1664,7 +1818,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
c.base.Append(dirWork, ids), c.base.Append(dirWork, ids),
c.base.Append(dirTemp, ids), c.base.Append(dirTemp, ids),
ids, nil, nil, nil, ids, nil, nil, nil,
common{c}, common{ctx, c},
} }
switch ca := a.(type) { switch ca := a.(type) {
case TrivialArtifact: case TrivialArtifact:
@@ -1815,23 +1969,65 @@ func (c *Cache) OpenStatus(a Artifact) (r io.ReadSeekCloser, err error) {
return return
} }
// Abort cancels all pending cures and waits for them to clean up, but does not
// close the cache.
func (c *Cache) Abort() {
c.closeMu.Lock()
defer c.closeMu.Unlock()
if c.closed {
return
}
c.toplevel.Load().cancel()
c.abortMu.Lock()
defer c.abortMu.Unlock()
// holding abortMu, identPending stays empty
c.wg.Wait()
c.identMu.Lock()
c.toplevel.Store(newToplevel(c.parent))
clear(c.identErr)
c.identMu.Unlock()
}
// Close cancels all pending cures and waits for them to clean up. // Close cancels all pending cures and waits for them to clean up.
func (c *Cache) Close() { func (c *Cache) Close() {
c.closeOnce.Do(func() { c.closeMu.Lock()
c.cancel() defer c.closeMu.Unlock()
c.wg.Wait()
close(c.cures) if c.closed {
c.unlock() return
}) }
c.closed = true
c.toplevel.Load().cancel()
c.wg.Wait()
close(c.cures)
c.unlock()
} }
// UnsupportedVariantError describes an on-disk cache with an extension variant
// identification string that differs from the value returned by [Extension].
type UnsupportedVariantError string
func (e UnsupportedVariantError) Error() string {
return "unsupported variant " + strconv.Quote(string(e))
}
var (
// ErrWouldPromote is returned by [Open] if the [CPromoteVariant] bit is not
// set and the on-disk cache requires variant promotion.
ErrWouldPromote = errors.New("operation would promote unextended cache")
)
// Open returns the address of a newly opened instance of [Cache]. // Open returns the address of a newly opened instance of [Cache].
// //
// Concurrent cures of a [FloodArtifact] dependency graph is limited to the // Concurrent cures of a [FloodArtifact] dependency graph is limited to the
// caller-supplied value, however direct calls to [Cache.Cure] is not subject // caller-supplied value, however direct calls to [Cache.Cure] is not subject
// to this limitation. // to this limitation.
// //
// A cures value of 0 or lower is equivalent to the value returned by // A cures or jobs value of 0 or lower is equivalent to the value returned by
// [runtime.NumCPU]. // [runtime.NumCPU].
// //
// A successful call to Open guarantees exclusive access to the on-filesystem // A successful call to Open guarantees exclusive access to the on-filesystem
@@ -1841,10 +2037,10 @@ func (c *Cache) Close() {
func Open( func Open(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures, jobs int,
base *check.Absolute, base *check.Absolute,
) (*Cache, error) { ) (*Cache, error) {
return open(ctx, msg, cures, base, true) return open(ctx, msg, flags, cures, jobs, base, true)
} }
// open implements Open but allows omitting the [lockedfile] lock when called // open implements Open but allows omitting the [lockedfile] lock when called
@@ -1852,13 +2048,24 @@ func Open(
func open( func open(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures, jobs int,
base *check.Absolute, base *check.Absolute,
lock bool, lock bool,
) (*Cache, error) { ) (*Cache, error) {
openMu.Lock()
defer openMu.Unlock()
opened = true
if extension == "" && len(irArtifact) != int(_kindEnd) {
panic("attempting to open cache with incomplete variant setup")
}
if cures < 1 { if cures < 1 {
cures = runtime.NumCPU() cures = runtime.NumCPU()
} }
if jobs < 1 {
jobs = runtime.NumCPU()
}
for _, name := range []string{ for _, name := range []string{
dirIdentifier, dirIdentifier,
@@ -1866,28 +2073,34 @@ func open(
dirStatus, dirStatus,
dirWork, dirWork,
} { } {
if err := os.MkdirAll(base.Append(name).String(), 0700); err != nil && if err := os.MkdirAll(
!errors.Is(err, os.ErrExist) { base.Append(name).String(),
0700,
); err != nil && !errors.Is(err, os.ErrExist) {
return nil, err return nil, err
} }
} }
c := Cache{ c := Cache{
parent: ctx,
cures: make(chan struct{}, cures), cures: make(chan struct{}, cures),
flags: flags,
jobs: jobs,
msg: msg, msg: msg,
base: base, base: base,
identPool: sync.Pool{New: func() any { return new(extIdent) }}, irCache: zeroIRCache(),
ident: make(map[unique.Handle[ID]]unique.Handle[Checksum]), ident: make(map[unique.Handle[ID]]unique.Handle[Checksum]),
identErr: make(map[unique.Handle[ID]]error), identErr: make(map[unique.Handle[ID]]error),
identPending: make(map[unique.Handle[ID]]<-chan struct{}), identPending: make(map[unique.Handle[ID]]*pendingCure),
brPool: sync.Pool{New: func() any { return new(bufio.Reader) }}, brPool: sync.Pool{New: func() any { return new(bufio.Reader) }},
bwPool: sync.Pool{New: func() any { return new(bufio.Writer) }}, bwPool: sync.Pool{New: func() any { return new(bufio.Writer) }},
} }
c.ctx, c.cancel = context.WithCancel(ctx) c.toplevel.Store(newToplevel(ctx))
if lock || !testing.Testing() { if lock || !testing.Testing() {
if unlock, err := lockedfile.MutexAt( if unlock, err := lockedfile.MutexAt(
@@ -1901,6 +2114,45 @@ func open(
c.unlock = func() {} c.unlock = func() {}
} }
variantPath := base.Append(fileVariant).String()
if p, err := os.ReadFile(variantPath); err != nil {
if !errors.Is(err, os.ErrNotExist) {
c.unlock()
return nil, err
}
// nonexistence implies newly created cache, or a cache predating
// variant identification strings, in which case it is silently promoted
if err = os.WriteFile(
variantPath,
[]byte(extension),
0400,
); err != nil {
c.unlock()
return nil, err
}
} else if s := string(p); s == "" {
if extension != "" {
if flags&CPromoteVariant == 0 {
c.unlock()
return nil, ErrWouldPromote
}
if err = os.WriteFile(
variantPath,
[]byte(extension),
0400,
); err != nil {
c.unlock()
return nil, err
}
}
} else if !ValidExtension(s) {
c.unlock()
return nil, ErrInvalidExtension
} else if s != extension {
c.unlock()
return nil, UnsupportedVariantError(s)
}
return &c, nil return &c, nil
} }
+406 -26
View File
@@ -16,6 +16,7 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
"sync"
"syscall" "syscall"
"testing" "testing"
"unique" "unique"
@@ -24,6 +25,8 @@ import (
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/container" "hakurei.app/container"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/internal/info"
"hakurei.app/internal/landlock"
"hakurei.app/internal/pkg" "hakurei.app/internal/pkg"
"hakurei.app/internal/stub" "hakurei.app/internal/stub"
"hakurei.app/message" "hakurei.app/message"
@@ -33,11 +36,47 @@ import (
func unsafeOpen( func unsafeOpen(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures, jobs int,
base *check.Absolute, base *check.Absolute,
lock bool, lock bool,
) (*pkg.Cache, error) ) (*pkg.Cache, error)
var (
// extension is a string uniquely identifying a set of custom [Artifact]
// implementations registered by calling [Register].
//
//go:linkname extension hakurei.app/internal/pkg.extension
extension string
// opened is false if [Open] was never called.
//
//go:linkname opened hakurei.app/internal/pkg.opened
opened bool
// irArtifact refers to artifact IR interpretation functions and must not be
// written to directly.
//
//go:linkname irArtifact hakurei.app/internal/pkg.irArtifact
irArtifact map[pkg.Kind]pkg.IRReadFunc
)
// newRContext returns the address of a new [pkg.RContext] unsafely created for
// the specified [testing.TB].
func newRContext(tb testing.TB, c *pkg.Cache) *pkg.RContext {
var r pkg.RContext
rContextVal := reflect.ValueOf(&r).Elem().FieldByName("ctx")
reflect.NewAt(
rContextVal.Type(),
unsafe.Pointer(rContextVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(tb.Context()))
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt(
rCacheVal.Type(),
unsafe.Pointer(rCacheVal.UnsafeAddr()),
).Elem().Set(reflect.ValueOf(c))
return &r
}
func TestMain(m *testing.M) { container.TryArgv0(nil); os.Exit(m.Run()) } func TestMain(m *testing.M) { container.TryArgv0(nil); os.Exit(m.Run()) }
// overrideIdent overrides the ID method of [Artifact]. // overrideIdent overrides the ID method of [Artifact].
@@ -228,7 +267,7 @@ func TestIdent(t *testing.T) {
var cache *pkg.Cache var cache *pkg.Cache
if a, err := check.NewAbs(t.TempDir()); err != nil { if a, err := check.NewAbs(t.TempDir()); err != nil {
t.Fatal(err) t.Fatal(err)
} else if cache, err = pkg.Open(t.Context(), msg, 0, a); err != nil { } else if cache, err = pkg.Open(t.Context(), msg, 0, 0, 0, a); err != nil {
t.Fatal(err) t.Fatal(err)
} }
t.Cleanup(cache.Close) t.Cleanup(cache.Close)
@@ -252,6 +291,7 @@ func TestIdent(t *testing.T) {
// on test completion. // on test completion.
type cacheTestCase struct { type cacheTestCase struct {
name string name string
flags int
early func(t *testing.T, base *check.Absolute) early func(t *testing.T, base *check.Absolute)
f func(t *testing.T, base *check.Absolute, c *pkg.Cache) f func(t *testing.T, base *check.Absolute, c *pkg.Cache)
want pkg.Checksum want pkg.Checksum
@@ -288,8 +328,20 @@ func checkWithCache(t *testing.T, testCases []cacheTestCase) {
msg := message.New(log.New(os.Stderr, "cache: ", 0)) msg := message.New(log.New(os.Stderr, "cache: ", 0))
msg.SwapVerbose(testing.Verbose()) msg.SwapVerbose(testing.Verbose())
flags := tc.flags
if info.CanDegrade {
if _, err := landlock.GetABI(); err != nil {
if !errors.Is(err, syscall.ENOSYS) {
t.Fatalf("LandlockGetABI: error = %v", err)
}
flags |= pkg.CHostAbstract
t.Log("Landlock LSM is unavailable, setting CHostAbstract")
}
}
var scrubFunc func() error // scrub after hashing var scrubFunc func() error // scrub after hashing
if c, err := pkg.Open(t.Context(), msg, 1<<4, base); err != nil { if c, err := pkg.Open(t.Context(), msg, flags, 1<<4, 0, base); err != nil {
t.Fatalf("Open: error = %v", err) t.Fatalf("Open: error = %v", err)
} else { } else {
t.Cleanup(c.Close) t.Cleanup(c.Close)
@@ -309,9 +361,20 @@ func checkWithCache(t *testing.T, testCases []cacheTestCase) {
restoreTemp = true restoreTemp = true
} }
// destroy lock file to avoid changing cache checksums // destroy lock and variant file to avoid changing cache checksums
if err := os.Remove(base.Append("lock").String()); err != nil { for _, s := range []string{
t.Fatal(err) "lock",
"variant",
} {
pathname := base.Append(s)
if p, err := os.ReadFile(pathname.String()); err != nil {
t.Fatal(err)
} else if len(p) != 0 {
t.Fatalf("file %q: %q", s, string(p))
}
if err := os.Remove(pathname.String()); err != nil {
t.Fatal(err)
}
} }
// destroy non-deterministic status files // destroy non-deterministic status files
@@ -468,9 +531,7 @@ func TestCache(t *testing.T) {
}() }()
testCases := []cacheTestCase{ testCases := []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"file", pkg.CValidateKnown | pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
identifier := (pkg.ID)(bytes.Repeat([]byte{ identifier := (pkg.ID)(bytes.Repeat([]byte{
0x75, 0xe6, 0x9d, 0x6d, 0xe7, 0x9f, 0x75, 0xe6, 0x9d, 0x6d, 0xe7, 0x9f,
}, 8)) }, 8))
@@ -593,7 +654,7 @@ func TestCache(t *testing.T) {
if c0, err := unsafeOpen( if c0, err := unsafeOpen(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, base, false, 0, 0, 0, base, false,
); err != nil { ); err != nil {
t.Fatalf("open: error = %v", err) t.Fatalf("open: error = %v", err)
} else { } else {
@@ -627,7 +688,7 @@ func TestCache(t *testing.T) {
} }
}, pkg.MustDecode("St9rlE-mGZ5gXwiv_hzQ_B8bZP-UUvSNmf4nHUZzCMOumb6hKnheZSe0dmnuc4Q2")}, }, pkg.MustDecode("St9rlE-mGZ5gXwiv_hzQ_B8bZP-UUvSNmf4nHUZzCMOumb6hKnheZSe0dmnuc4Q2")},
{"directory", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"directory", pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
id := pkg.MustDecode( id := pkg.MustDecode(
"HnySzeLQvSBZuTUcvfmLEX_OmH4yJWWH788NxuLuv7kVn8_uPM6Ks4rqFWM2NZJY", "HnySzeLQvSBZuTUcvfmLEX_OmH4yJWWH788NxuLuv7kVn8_uPM6Ks4rqFWM2NZJY",
) )
@@ -804,9 +865,7 @@ func TestCache(t *testing.T) {
}) })
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d")}, }, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d")},
{"pending", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"pending", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
wantErr := stub.UniqueError(0xcafe) wantErr := stub.UniqueError(0xcafe)
n, ready := make(chan struct{}), make(chan struct{}) n, ready := make(chan struct{}), make(chan struct{})
go func() { go func() {
@@ -865,18 +924,117 @@ func TestCache(t *testing.T) {
t.Fatalf("Scrub: error = %#v, want %#v", err, wantErrScrub) t.Fatalf("Scrub: error = %#v, want %#v", err, wantErrScrub)
} }
identPendingVal := reflect.ValueOf(c).Elem().FieldByName("identPending") notify := c.Done(unique.Make(pkg.ID{0xff}))
identPending := reflect.NewAt(
identPendingVal.Type(),
unsafe.Pointer(identPendingVal.UnsafeAddr()),
).Elem().Interface().(map[unique.Handle[pkg.ID]]<-chan struct{})
notify := identPending[unique.Make(pkg.ID{0xff})]
go close(n) go close(n)
<-notify if notify != nil {
<-notify
}
for c.Done(unique.Make(pkg.ID{0xff})) != nil {
}
<-wCureDone <-wCureDone
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")}, }, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"scrub", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"cancel abort block", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var wg sync.WaitGroup
defer wg.Wait()
var started sync.WaitGroup
defer started.Wait()
blockCures := func(d byte, e stub.UniqueError, n int) {
started.Add(n)
for i := range n {
wg.Go(func() {
if _, _, err := c.Cure(overrideIdent{pkg.ID{d, byte(i)}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
started.Done()
<-t.Unwrap().Done()
return e + stub.UniqueError(i)
},
}}); !reflect.DeepEqual(err, e+stub.UniqueError(i)) {
panic(err)
}
})
}
started.Wait()
}
blockCures(0xfd, 0xbad, 16)
c.Abort()
wg.Wait()
blockCures(0xfd, 0xcafe, 16)
c.Abort()
wg.Wait()
blockCures(0xff, 0xbad, 1)
if !c.Cancel(unique.Make(pkg.ID{0xff})) {
t.Fatal("missed cancellation")
}
wg.Wait()
blockCures(0xff, 0xcafe, 1)
if !c.Cancel(unique.Make(pkg.ID{0xff})) {
t.Fatal("missed cancellation")
}
wg.Wait()
for c.Cancel(unique.Make(pkg.ID{0xff})) {
}
c.Close()
c.Abort()
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"no assume checksum", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
makeGarbage := func(work *check.Absolute, wantErr error) error {
if err := os.Mkdir(work.String(), 0700); err != nil {
return err
}
if err := os.WriteFile(work.Append(
"check",
).String(), nil, 0400); err != nil {
return err
}
return wantErr
}
wantChecksum := pkg.MustDecode("Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")
cureMany(t, c, []cureStep{
{"create", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 0}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 0}),
), wantChecksum, nil},
{"reject", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xfe, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), stub.UniqueError(0xbad))
},
}}}, nil, pkg.Checksum{}, stub.UniqueError(0xbad)},
{"match", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 1}),
), wantChecksum, nil},
})
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w")},
{"scrub", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
{"bad measured file", newStubFile( {"bad measured file", newStubFile(
pkg.KindHTTPGet, pkg.KindHTTPGet,
@@ -973,6 +1131,10 @@ func TestErrors(t *testing.T) {
Want: pkg.IRKindIdent, Want: pkg.IRKindIdent,
Ancillary: 0xcafe, Ancillary: 0xcafe,
}, "got invalid kind 48879 IR value (0xcafe) instead of ident"}, }, "got invalid kind 48879 IR value (0xcafe) instead of ident"},
{"UnsupportedVariantError", pkg.UnsupportedVariantError(
"rosa",
), `unsupported variant "rosa"`},
} }
for _, tc := range testCases { for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
@@ -1181,8 +1343,10 @@ func (a earlyFailureF) Cure(*pkg.FContext) error {
} }
func TestDependencyCureErrorEarly(t *testing.T) { func TestDependencyCureErrorEarly(t *testing.T) {
t.Parallel()
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"early", nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) { {"early", 0, nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) {
_, _, err := c.Cure(earlyFailureF(8)) _, _, err := c.Cure(earlyFailureF(8))
if !errors.Is(err, stub.UniqueError(0xcafe)) { if !errors.Is(err, stub.UniqueError(0xcafe)) {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
@@ -1191,7 +1355,7 @@ func TestDependencyCureErrorEarly(t *testing.T) {
}) })
} }
func TestNew(t *testing.T) { func TestOpen(t *testing.T) {
t.Parallel() t.Parallel()
t.Run("nonexistent", func(t *testing.T) { t.Run("nonexistent", func(t *testing.T) {
@@ -1205,7 +1369,7 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open( if _, err := pkg.Open(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, check.MustAbs(container.Nonexistent), 0, 0, 0, check.MustAbs(container.Nonexistent),
); !reflect.DeepEqual(err, wantErr) { ); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr) t.Errorf("Open: error = %#v, want %#v", err, wantErr)
} }
@@ -1233,9 +1397,225 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open( if _, err := pkg.Open(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, tempDir.Append("cache"), 0, 0, 0, tempDir.Append("cache"),
); !reflect.DeepEqual(err, wantErr) { ); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr) t.Errorf("Open: error = %#v, want %#v", err, wantErr)
} }
}) })
} }
func TestExtensionRegister(t *testing.T) {
extensionOld := extension
openedOld := opened
t.Cleanup(func() { extension = extensionOld; opened = openedOld })
extension = ""
opened = false
t.Run("set", func(t *testing.T) {
t.Cleanup(func() { extension = "" })
const want = "rosa"
pkg.SetExtension(want)
if got := pkg.Extension(); got != want {
t.Fatalf("Extension: %q, want %q", got, want)
}
})
t.Run("twice", func(t *testing.T) {
t.Cleanup(func() { extension = "" })
defer func() {
const wantPanic = "attempting to set extension twice"
if r := recover(); r != wantPanic {
t.Errorf("panic: %#v, want %q", r, wantPanic)
}
}()
pkg.SetExtension("rosa")
pkg.SetExtension("rosa")
})
t.Run("invalid", func(t *testing.T) {
defer func() {
var wantPanic = pkg.ErrInvalidExtension
if r := recover(); r != wantPanic {
t.Errorf("panic: %#v, want %#v", r, wantPanic)
}
}()
pkg.SetExtension(" ")
})
t.Run("opened", func(t *testing.T) {
t.Cleanup(func() { opened = false })
if _, err := pkg.Open(
t.Context(),
message.New(log.Default()),
0, 0, 0,
check.MustAbs(container.Nonexistent),
); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("Open: error = %v", err)
}
t.Run("variant", func(t *testing.T) {
defer func() {
const wantPanic = "attempting to set extension after open"
if r := recover(); r != wantPanic {
t.Errorf("panic: %#v, want %q", r, wantPanic)
}
}()
pkg.SetExtension("rosa")
})
t.Run("register", func(t *testing.T) {
defer func() {
const wantPanic = "attempting to register after open"
if r := recover(); r != wantPanic {
t.Errorf("panic: %#v, want %q", r, wantPanic)
}
}()
pkg.Register(pkg.KindCustomOffset, nil)
})
})
t.Run("incomplete", func(t *testing.T) {
t.Cleanup(func() { delete(irArtifact, pkg.KindCustomOffset) })
defer func() {
const wantPanic = "attempting to open cache with incomplete variant setup"
if r := recover(); r != wantPanic {
t.Errorf("panic: %#v, want %q", r, wantPanic)
}
}()
pkg.Register(pkg.KindCustomOffset, nil)
t.Cleanup(func() { opened = false })
_, _ = pkg.Open(nil, nil, 0, 0, 0, nil)
panic("unreachable")
})
t.Run("create", func(t *testing.T) {
t.Cleanup(func() { extension = "" })
const want = "rosa"
pkg.SetExtension(want)
base := check.MustAbs(t.TempDir())
t.Cleanup(func() { opened = false })
if c, err := pkg.Open(
t.Context(), nil,
0, 0, 0,
base,
); err != nil {
t.Fatal(err)
} else {
c.Close()
}
if got, err := os.ReadFile(base.Append("variant").String()); err != nil {
t.Fatal(err)
} else if string(got) != want {
t.Fatalf("variant: %q", string(got))
}
})
t.Run("access", func(t *testing.T) {
base := check.MustAbs(t.TempDir())
t.Cleanup(func() { opened = false })
if err := os.WriteFile(base.Append("variant").String(), nil, 0); err != nil {
t.Fatal(err)
}
wantErr := &os.PathError{
Op: "open",
Path: base.Append("variant").String(),
Err: syscall.EACCES,
}
if _, err := pkg.Open(
t.Context(), nil,
0, 0, 0,
base,
); !reflect.DeepEqual(err, wantErr) {
t.Fatalf("Open: error = %v, want %v", err, wantErr)
}
})
t.Run("promote", func(t *testing.T) {
t.Cleanup(func() { extension = "" })
const want = "rosa"
pkg.SetExtension(want)
base := check.MustAbs(t.TempDir())
t.Cleanup(func() { opened = false })
variantPath := base.Append("variant")
if err := os.WriteFile(variantPath.String(), nil, 0600); err != nil {
t.Fatal(err)
}
if _, err := pkg.Open(
t.Context(), nil,
0, 0, 0,
base,
); !reflect.DeepEqual(err, pkg.ErrWouldPromote) {
t.Fatalf("Open: error = %v", err)
}
if p, err := os.ReadFile(variantPath.String()); err != nil {
t.Fatal(err)
} else if len(p) != 0 {
t.Fatalf("variant: %q", string(p))
}
if c, err := pkg.Open(
t.Context(), nil,
pkg.CPromoteVariant, 0, 0,
base,
); err != nil {
t.Fatalf("Open: error = %v", err)
} else {
c.Close()
}
if p, err := os.ReadFile(variantPath.String()); err != nil {
t.Fatal(err)
} else if string(p) != want {
t.Fatalf("variant: %q, want %q", string(p), want)
}
})
t.Run("open invalid", func(t *testing.T) {
base := check.MustAbs(t.TempDir())
t.Cleanup(func() { opened = false })
variantPath := base.Append("variant")
if err := os.WriteFile(variantPath.String(), make([]byte, 129), 0400); err != nil {
t.Fatal(err)
}
if _, err := pkg.Open(
t.Context(), nil,
0, 0, 0,
base,
); !reflect.DeepEqual(err, pkg.ErrInvalidExtension) {
t.Fatalf("Open: error = %v", err)
}
})
t.Run("unsupported", func(t *testing.T) {
base := check.MustAbs(t.TempDir())
t.Cleanup(func() { opened = false })
variantPath := base.Append("variant")
if err := os.WriteFile(variantPath.String(), []byte("rosa"), 0400); err != nil {
t.Fatal(err)
}
if _, err := pkg.Open(
t.Context(), nil,
0, 0, 0,
base,
); !reflect.DeepEqual(err, pkg.UnsupportedVariantError("rosa")) {
t.Fatalf("Open: error = %v", err)
}
})
}
+1 -2
View File
@@ -43,8 +43,7 @@ var _ fmt.Stringer = new(tarArtifactNamed)
func (a *tarArtifactNamed) String() string { return a.name + "-unpack" } func (a *tarArtifactNamed) String() string { return a.name + "-unpack" }
// NewTar returns a new [Artifact] backed by the supplied [Artifact] and // NewTar returns a new [Artifact] backed by the supplied [Artifact] and
// compression method. The source [Artifact] must be compatible with // compression method. The source [Artifact] must be a [FileArtifact].
// [TContext.Open].
func NewTar(a Artifact, compression uint32) Artifact { func NewTar(a Artifact, compression uint32) Artifact {
ta := tarArtifact{a, compression} ta := tarArtifact{a, compression}
if s, ok := a.(fmt.Stringer); ok { if s, ok := a.(fmt.Stringer); ok {
+2 -2
View File
@@ -21,7 +21,7 @@ func TestTar(t *testing.T) {
t.Parallel() t.Parallel()
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"http", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"http", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{ checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700}, ".": {Mode: fs.ModeDir | 0700},
@@ -42,7 +42,7 @@ func TestTar(t *testing.T) {
)) ))
}, pkg.MustDecode("NQTlc466JmSVLIyWklm_u8_g95jEEb98PxJU-kjwxLpfdjwMWJq0G8ze9R4Vo1Vu")}, }, pkg.MustDecode("NQTlc466JmSVLIyWklm_u8_g95jEEb98PxJU-kjwxLpfdjwMWJq0G8ze9R4Vo1Vu")},
{"http expand", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"http expand", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{ checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700}, ".": {Mode: fs.ModeDir | 0700},
+17 -31
View File
@@ -9,10 +9,11 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"reflect" "reflect"
"runtime"
"slices" "slices"
"strconv"
"strings" "strings"
"hakurei.app/check"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/vfs" "hakurei.app/vfs"
) )
@@ -21,6 +22,10 @@ func main() {
log.SetFlags(0) log.SetFlags(0)
log.SetPrefix("testtool: ") log.SetPrefix("testtool: ")
environ := slices.DeleteFunc(slices.Clone(os.Environ()), func(s string) bool {
return s == "CURE_JOBS="+strconv.Itoa(runtime.NumCPU())
})
var hostNet, layers, promote bool var hostNet, layers, promote bool
if len(os.Args) == 2 && os.Args[0] == "testtool" { if len(os.Args) == 2 && os.Args[0] == "testtool" {
switch os.Args[1] { switch os.Args[1] {
@@ -48,15 +53,15 @@ func main() {
var overlayRoot bool var overlayRoot bool
wantEnv := []string{"HAKUREI_TEST=1"} wantEnv := []string{"HAKUREI_TEST=1"}
if len(os.Environ()) == 2 { if len(environ) == 2 {
overlayRoot = true overlayRoot = true
if !layers && !promote { if !layers && !promote {
log.SetPrefix("testtool(overlay root): ") log.SetPrefix("testtool(overlay root): ")
} }
wantEnv = []string{"HAKUREI_TEST=1", "HAKUREI_ROOT=1"} wantEnv = []string{"HAKUREI_TEST=1", "HAKUREI_ROOT=1"}
} }
if !slices.Equal(wantEnv, os.Environ()) { if !slices.Equal(wantEnv, environ) {
log.Fatalf("Environ: %q, want %q", os.Environ(), wantEnv) log.Fatalf("Environ: %q, want %q", environ, wantEnv)
} }
var overlayWork bool var overlayWork bool
@@ -153,43 +158,24 @@ func main() {
m.Source != "overlay" || m.FsType != "overlay" { m.Source != "overlay" || m.FsType != "overlay" {
log.Fatal("unexpected root mount entry") log.Fatal("unexpected root mount entry")
} }
var lowerdir string var lowerdir []string
for _, o := range strings.Split(m.FsOptstr, ",") { for _, o := range strings.Split(m.FsOptstr, ",") {
const lowerdirKey = "lowerdir=" const lowerdirKey = "lowerdir+="
if strings.HasPrefix(o, lowerdirKey) { if strings.HasPrefix(o, lowerdirKey) {
lowerdir = o[len(lowerdirKey):] lowerdir = append(lowerdir, o[len(lowerdirKey):])
} }
} }
if !layers { if !layers {
if filepath.Base(lowerdir) != checksumEmptyDir { if len(lowerdir) != 1 || filepath.Base(lowerdir[0]) != checksumEmptyDir {
log.Fatal("unexpected artifact checksum") log.Fatal("unexpected artifact checksum")
} }
} else { } else {
ident = "p1t_drXr34i-jZNuxDMLaMOdL6tZvQqhavNafGynGqxOZoXAUTSn7kqNh3Ovv3DT" ident = "p1t_drXr34i-jZNuxDMLaMOdL6tZvQqhavNafGynGqxOZoXAUTSn7kqNh3Ovv3DT"
lowerdirsEscaped := strings.Split(lowerdir, ":") if len(lowerdir) != 2 ||
lowerdirs := lowerdirsEscaped[:0] filepath.Base(lowerdir[0]) != "MGWmEfjut2QE2xPJwTsmUzpff4BN_FEnQ7T0j7gvUCCiugJQNwqt9m151fm9D1yU" ||
// ignore the option separator since it does not appear in ident filepath.Base(lowerdir[1]) != "nY_CUdiaUM1OL4cPr5TS92FCJ3rCRV7Hm5oVTzAvMXwC03_QnTRfQ5PPs7mOU9fK" {
for i, e := range lowerdirsEscaped { log.Fatalf("unexpected lowerdirs %s", strings.Join(lowerdir, ", "))
if len(e) > 0 &&
e[len(e)-1] == check.SpecialOverlayEscape[0] &&
(len(e) == 1 || e[len(e)-2] != check.SpecialOverlayEscape[0]) {
// ignore escaped pathname separator since it does not
// appear in ident
e = e[:len(e)-1]
if len(lowerdirsEscaped) != i {
lowerdirsEscaped[i+1] = e + lowerdirsEscaped[i+1]
continue
}
}
lowerdirs = append(lowerdirs, e)
}
if len(lowerdirs) != 2 ||
filepath.Base(lowerdirs[0]) != "MGWmEfjut2QE2xPJwTsmUzpff4BN_FEnQ7T0j7gvUCCiugJQNwqt9m151fm9D1yU" ||
filepath.Base(lowerdirs[1]) != "nY_CUdiaUM1OL4cPr5TS92FCJ3rCRV7Hm5oVTzAvMXwC03_QnTRfQ5PPs7mOU9fK" {
log.Fatalf("unexpected lowerdirs %s", strings.Join(lowerdirs, ", "))
} }
} }
} else { } else {
+6 -6
View File
@@ -7,10 +7,10 @@ func (t Toolchain) newAttr() (pkg.Artifact, string) {
version = "2.5.2" version = "2.5.2"
checksum = "YWEphrz6vg1sUMmHHVr1CRo53pFXRhq_pjN-AlG8UgwZK1y6m7zuDhxqJhD0SV0l" checksum = "YWEphrz6vg1sUMmHHVr1CRo53pFXRhq_pjN-AlG8UgwZK1y6m7zuDhxqJhD0SV0l"
) )
return t.NewPackage("attr", version, pkg.NewHTTPGetTar( return t.NewPackage("attr", version, newTar(
nil, "https://download.savannah.nongnu.org/releases/attr/"+ "https://download.savannah.nongnu.org/releases/attr/"+
"attr-"+version+".tar.gz", "attr-"+version+".tar.gz",
mustDecode(checksum), checksum,
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
Patches: []KV{ Patches: []KV{
@@ -81,10 +81,10 @@ func (t Toolchain) newACL() (pkg.Artifact, string) {
version = "2.3.2" version = "2.3.2"
checksum = "-fY5nwH4K8ZHBCRXrzLdguPkqjKI6WIiGu4dBtrZ1o0t6AIU73w8wwJz_UyjIS0P" checksum = "-fY5nwH4K8ZHBCRXrzLdguPkqjKI6WIiGu4dBtrZ1o0t6AIU73w8wwJz_UyjIS0P"
) )
return t.NewPackage("acl", version, pkg.NewHTTPGetTar( return t.NewPackage("acl", version, newTar(
nil, "https://download.savannah.nongnu.org/releases/acl/"+ "https://download.savannah.nongnu.org/releases/acl/"+
"acl-"+version+".tar.gz", "acl-"+version+".tar.gz",
mustDecode(checksum), checksum,
pkg.TarGzip, pkg.TarGzip,
), nil, &MakeHelper{ ), nil, &MakeHelper{
// makes assumptions about uid_map/gid_map // makes assumptions about uid_map/gid_map
+69 -7
View File
@@ -16,9 +16,7 @@ import (
type PArtifact int type PArtifact int
const ( const (
LLVMCompilerRT PArtifact = iota LLVM PArtifact = iota
LLVMRuntimes
LLVMClang
// EarlyInit is the Rosa OS init program. // EarlyInit is the Rosa OS init program.
EarlyInit EarlyInit
@@ -47,6 +45,7 @@ const (
Bison Bison
Bzip2 Bzip2
CMake CMake
Connman
Coreutils Coreutils
Curl Curl
DBus DBus
@@ -63,6 +62,7 @@ const (
GenInitCPIO GenInitCPIO
Gettext Gettext
Git Git
Glslang
GnuTLS GnuTLS
Go Go
Gperf Gperf
@@ -72,16 +72,22 @@ const (
HakureiDist HakureiDist
IPTables IPTables
Kmod Kmod
LIT
LibX11
LibXau LibXau
LibXext
Libbsd Libbsd
Libcap Libcap
Libdrm
Libev Libev
Libexpat Libexpat
Libffi Libffi
Libgd Libgd
Libglvnd
Libiconv Libiconv
Libmd Libmd
Libmnl Libmnl
Libpciaccess
Libnftnl Libnftnl
Libpsl Libpsl
Libseccomp Libseccomp
@@ -89,8 +95,10 @@ const (
Libtool Libtool
Libucontext Libucontext
Libunistring Libunistring
Libxshmfence
Libxml2 Libxml2
Libxslt Libxslt
Libxtrans
M4 M4
MPC MPC
MPFR MPFR
@@ -118,22 +126,35 @@ const (
PerlTermReadKey PerlTermReadKey
PerlTextCharWidth PerlTextCharWidth
PerlTextWrapI18N PerlTextWrapI18N
PerlUnicodeGCString PerlUnicodeLineBreak
PerlYAMLTiny PerlYAMLTiny
PkgConfig PkgConfig
Procps Procps
Python Python
PythonFlitCore
PythonHatchling
PythonIniConfig PythonIniConfig
PythonMako
PythonMarkupSafe
PythonPackaging PythonPackaging
PythonPathspec
PythonPluggy PythonPluggy
PythonPyTest PythonPyTest
PythonPyYAML
PythonPygments PythonPygments
PythonSetuptools
PythonSetuptoolsSCM
PythonTroveClassifiers
PythonVCSVersioning
PythonWheel
QEMU QEMU
Rdfind Rdfind
Readline Readline
Rsync Rsync
Sed Sed
Setuptools SPIRVHeaders
SPIRVLLVMTranslator
SPIRVTools
SquashfsTools SquashfsTools
Strace Strace
TamaGo TamaGo
@@ -147,19 +168,35 @@ const (
WaylandProtocols WaylandProtocols
XCB XCB
XCBProto XCBProto
Xproto XDGDBusProxy
XZ XZ
XorgProto
Zlib Zlib
Zstd Zstd
// PresetUnexportedStart is the first unexported preset. // PresetUnexportedStart is the first unexported preset.
PresetUnexportedStart PresetUnexportedStart
buildcatrust = iota - 1 llvmSource = iota - 1
// earlyCompilerRT is an early, standalone compiler-rt installation for the
// standalone runtimes build.
//
// earlyCompilerRT must only be loaded by [LLVM].
earlyCompilerRT
// earlyRuntimes is an early, standalone installation of LLVM runtimes to
// work around the cmake build system leaking the system LLVM installation
// when invoking the newly built toolchain.
//
// earlyRuntimes must only be loaded by [LLVM].
earlyRuntimes
buildcatrust
utilMacros utilMacros
// Musl is a standalone libc that does not depend on the toolchain. // Musl is a standalone libc that does not depend on the toolchain.
Musl Musl
// muslHeaders is a system installation of [Musl] headers.
muslHeaders
// gcc is a hacked-to-pieces GCC toolchain meant for use in intermediate // gcc is a hacked-to-pieces GCC toolchain meant for use in intermediate
// stages only. This preset and its direct output must never be exposed. // stages only. This preset and its direct output must never be exposed.
@@ -304,8 +341,33 @@ var (
} }
// artifactsOnce is for lazy initialisation of artifacts. // artifactsOnce is for lazy initialisation of artifacts.
artifactsOnce [_toolchainEnd][len(artifactsM)]sync.Once artifactsOnce [_toolchainEnd][len(artifactsM)]sync.Once
// presetOpts globally modifies behaviour of presets.
presetOpts int
) )
const (
// OptSkipCheck skips running all test suites.
OptSkipCheck = 1 << iota
// OptLLVMNoLTO disables LTO in all [LLVM] stages.
OptLLVMNoLTO
)
// Flags returns the current preset flags
func Flags() int { return presetOpts }
// zero zeros the value pointed to by p.
func zero[T any](p *T) { var v T; *p = v }
// DropCaches arranges for all cached [pkg.Artifact] to be freed some time after
// it returns. Must not be used concurrently with any other function from this
// package.
func DropCaches(flags int) {
presetOpts = flags
zero(&artifacts)
zero(&artifactsOnce)
}
// GetMetadata returns [Metadata] of a [PArtifact]. // GetMetadata returns [Metadata] of a [PArtifact].
func GetMetadata(p PArtifact) *Metadata { return &artifactsM[p] } func GetMetadata(p PArtifact) *Metadata { return &artifactsM[p] }
+15
View File
@@ -19,6 +19,21 @@ func TestLoad(t *testing.T) {
} }
} }
func BenchmarkAll(b *testing.B) {
flags := rosa.Flags()
b.Cleanup(func() { rosa.DropCaches(flags) })
for b.Loop() {
for i := range rosa.PresetEnd {
rosa.Std.Load(rosa.PArtifact(i))
}
b.StopTimer()
rosa.DropCaches(0)
b.StartTimer()
}
}
func TestResolveName(t *testing.T) { func TestResolveName(t *testing.T) {
t.Parallel() t.Parallel()
+3 -3
View File
@@ -7,10 +7,10 @@ func (t Toolchain) newArgpStandalone() (pkg.Artifact, string) {
version = "1.3" version = "1.3"
checksum = "vtW0VyO2pJ-hPyYmDI2zwSLS8QL0sPAUKC1t3zNYbwN2TmsaE-fADhaVtNd3eNFl" checksum = "vtW0VyO2pJ-hPyYmDI2zwSLS8QL0sPAUKC1t3zNYbwN2TmsaE-fADhaVtNd3eNFl"
) )
return t.NewPackage("argp-standalone", version, pkg.NewHTTPGetTar( return t.NewPackage("argp-standalone", version, newTar(
nil, "http://www.lysator.liu.se/~nisse/misc/"+ "http://www.lysator.liu.se/~nisse/misc/"+
"argp-standalone-"+version+".tar.gz", "argp-standalone-"+version+".tar.gz",
mustDecode(checksum), checksum,
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
Env: []string{ Env: []string{
+3 -3
View File
@@ -7,9 +7,9 @@ func (t Toolchain) newBzip2() (pkg.Artifact, string) {
version = "1.0.8" version = "1.0.8"
checksum = "cTLykcco7boom-s05H1JVsQi1AtChYL84nXkg_92Dm1Xt94Ob_qlMg_-NSguIK-c" checksum = "cTLykcco7boom-s05H1JVsQi1AtChYL84nXkg_92Dm1Xt94Ob_qlMg_-NSguIK-c"
) )
return t.NewPackage("bzip2", version, pkg.NewHTTPGetTar( return t.NewPackage("bzip2", version, newTar(
nil, "https://sourceware.org/pub/bzip2/bzip2-"+version+".tar.gz", "https://sourceware.org/pub/bzip2/bzip2-"+version+".tar.gz",
mustDecode(checksum), checksum,
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
Writable: true, Writable: true,
+36 -31
View File
@@ -10,13 +10,14 @@ import (
func (t Toolchain) newCMake() (pkg.Artifact, string) { func (t Toolchain) newCMake() (pkg.Artifact, string) {
const ( const (
version = "4.3.0" version = "4.3.2"
checksum = "amBtnY2eGsEdlrB-cTRuOESBTsIqtyaxWlEKNlnp2EWLwAKWINjssilo4KXE6El9" checksum = "6QylwRVKletndTSkZTV2YBRwgd_9rUVgav_QW23HpjUgV21AVYZOUOal8tdBDmO7"
) )
return t.NewPackage("cmake", version, pkg.NewHTTPGetTar( return t.NewPackage("cmake", version, newFromGitHubRelease(
nil, "https://github.com/Kitware/CMake/releases/download/"+ "Kitware/CMake",
"v"+version+"/cmake-"+version+".tar.gz", "v"+version,
mustDecode(checksum), "cmake-"+version+".tar.gz",
checksum,
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
// test suite expects writable source tree // test suite expects writable source tree
@@ -90,7 +91,7 @@ index 2ead810437..f85cbb8b1c 100644
ConfigureName: "/usr/src/cmake/bootstrap", ConfigureName: "/usr/src/cmake/bootstrap",
Configure: []KV{ Configure: []KV{
{"prefix", "/system"}, {"prefix", "/system"},
{"parallel", `"$(nproc)"`}, {"parallel", jobsE},
{"--"}, {"--"},
{"-DCMAKE_USE_OPENSSL", "OFF"}, {"-DCMAKE_USE_OPENSSL", "OFF"},
{"-DCMake_TEST_NO_NETWORK", "ON"}, {"-DCMake_TEST_NO_NETWORK", "ON"},
@@ -118,31 +119,27 @@ func init() {
// CMakeHelper is the [CMake] build system helper. // CMakeHelper is the [CMake] build system helper.
type CMakeHelper struct { type CMakeHelper struct {
// Joined with name with a dash if non-empty.
Variant string
// Path elements joined with source. // Path elements joined with source.
Append []string Append []string
// Value of CMAKE_BUILD_TYPE. The zero value is equivalent to "Release".
BuildType string
// CMake CACHE entries. // CMake CACHE entries.
Cache []KV Cache []KV
// Runs after install. // Runs after install.
Script string Script string
// Replaces the default test command.
Test string
// Whether to skip running tests.
SkipTest bool
// Whether to generate Makefile instead. // Whether to generate Makefile instead.
Make bool Make bool
} }
var _ Helper = new(CMakeHelper) var _ Helper = new(CMakeHelper)
// name returns its arguments and an optional variant string joined with '-'.
func (attr *CMakeHelper) name(name, version string) string {
if attr != nil && attr.Variant != "" {
name += "-" + attr.Variant
}
return name + "-" + version
}
// extra returns a hardcoded slice of [CMake] and [Ninja]. // extra returns a hardcoded slice of [CMake] and [Ninja].
func (attr *CMakeHelper) extra(int) P { func (attr *CMakeHelper) extra(int) P {
if attr != nil && attr.Make { if attr != nil && attr.Make {
@@ -169,22 +166,30 @@ func (*CMakeHelper) wantsDir() string { return "/cure/" }
// script generates the cure script. // script generates the cure script.
func (attr *CMakeHelper) script(name string) string { func (attr *CMakeHelper) script(name string) string {
if attr == nil { if attr == nil {
attr = &CMakeHelper{ attr = new(CMakeHelper)
Cache: []KV{
{"CMAKE_BUILD_TYPE", "Release"},
},
}
}
if len(attr.Cache) == 0 {
panic("CACHE must be non-empty")
} }
generate := "Ninja" generate := "Ninja"
jobs := "" test := "ninja " + jobsFlagE + " test"
if attr.Make { if attr.Make {
generate = "'Unix Makefiles'" generate = "'Unix Makefiles'"
jobs += ` "--parallel=$(nproc)"` test = "make " + jobsFlagE + " test"
} }
if attr.Test != "" {
test = attr.Test
}
script := attr.Script
if !attr.SkipTest && presetOpts&OptSkipCheck == 0 {
script += "\n" + test
}
cache := make([]KV, 1, 1+len(attr.Cache))
cache[0] = KV{"CMAKE_BUILD_TYPE", "Release"}
if attr.BuildType != "" {
cache[0][1] = attr.BuildType
}
cache = append(cache, attr.Cache...)
return ` return `
cmake -G ` + generate + ` \ cmake -G ` + generate + ` \
@@ -193,7 +198,7 @@ cmake -G ` + generate + ` \
-DCMAKE_ASM_COMPILER_TARGET="${ROSA_TRIPLE}" \ -DCMAKE_ASM_COMPILER_TARGET="${ROSA_TRIPLE}" \
-DCMAKE_INSTALL_LIBDIR=lib \ -DCMAKE_INSTALL_LIBDIR=lib \
` + strings.Join(slices.Collect(func(yield func(string) bool) { ` + strings.Join(slices.Collect(func(yield func(string) bool) {
for _, v := range attr.Cache { for _, v := range cache {
if !yield("-D" + v[0] + "=" + v[1]) { if !yield("-D" + v[0] + "=" + v[1]) {
return return
} }
@@ -201,7 +206,7 @@ cmake -G ` + generate + ` \
}), " \\\n\t") + ` \ }), " \\\n\t") + ` \
-DCMAKE_INSTALL_PREFIX=/system \ -DCMAKE_INSTALL_PREFIX=/system \
'/usr/src/` + name + `/` + filepath.Join(attr.Append...) + `' '/usr/src/` + name + `/` + filepath.Join(attr.Append...) + `'
cmake --build .` + jobs + ` cmake --build . --parallel=` + jobsE + `
cmake --install . --prefix=/work/system cmake --install . --prefix=/work/system
` + attr.Script ` + script
} }
+109
View File
@@ -0,0 +1,109 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newConnman() (pkg.Artifact, string) {
const (
version = "2.0"
checksum = "MhVTdJOhndnZn2SWd8URKo_Pj7Zvc14tntEbrVOf9L3yVWJvpb3v3Q6104tWJgtW"
)
return t.NewPackage("connman", version, newTar(
"https://git.kernel.org/pub/scm/network/connman/connman.git/"+
"snapshot/connman-"+version+".tar.gz",
checksum,
pkg.TarGzip,
), &PackageAttr{
Patches: []KV{
{"alpine-musl-res", `musl does not implement res_ninit
--- a/gweb/gresolv.c
+++ b/gweb/gresolv.c
@@ -877,8 +877,6 @@
resolv->index = index;
resolv->nameserver_list = NULL;
- res_ninit(&resolv->res);
-
return resolv;
}
@@ -918,8 +916,6 @@
flush_nameservers(resolv);
- res_nclose(&resolv->res);
-
g_free(resolv);
}
@@ -1022,24 +1018,19 @@
debug(resolv, "hostname %s", hostname);
if (!resolv->nameserver_list) {
- int i;
-
- for (i = 0; i < resolv->res.nscount; i++) {
- char buf[100];
- int family = resolv->res.nsaddr_list[i].sin_family;
- void *sa_addr = &resolv->res.nsaddr_list[i].sin_addr;
-
- if (family != AF_INET &&
- resolv->res._u._ext.nsaddrs[i]) {
- family = AF_INET6;
- sa_addr = &resolv->res._u._ext.nsaddrs[i]->sin6_addr;
+ FILE *f = fopen("/etc/resolv.conf", "r");
+ if (f) {
+ char line[256], *s;
+ int i;
+ while (fgets(line, sizeof(line), f)) {
+ if (strncmp(line, "nameserver", 10) || !isspace(line[10]))
+ continue;
+ for (s = &line[11]; isspace(s[0]); s++);
+ for (i = 0; s[i] && !isspace(s[i]); i++);
+ s[i] = 0;
+ g_resolv_add_nameserver(resolv, s, 53, 0);
}
-
- if (family != AF_INET && family != AF_INET6)
- continue;
-
- if (inet_ntop(family, sa_addr, buf, sizeof(buf)))
- g_resolv_add_nameserver(resolv, buf, 53, 0);
+ fclose(f);
}
if (!resolv->nameserver_list)
`},
},
}, &MakeHelper{
Generate: "./bootstrap",
},
Automake,
Libtool,
PkgConfig,
DBus,
IPTables,
GnuTLS,
Readline,
KernelHeaders,
), version
}
func init() {
artifactsM[Connman] = Metadata{
f: Toolchain.newConnman,
Name: "connman",
Description: "a daemon for managing Internet connections",
Website: "https://git.kernel.org/pub/scm/network/connman/connman.git/",
Dependencies: P{
DBus,
IPTables,
GnuTLS,
Readline,
},
ID: 337,
}
}
+5 -5
View File
@@ -7,15 +7,15 @@ func (t Toolchain) newCurl() (pkg.Artifact, string) {
version = "8.19.0" version = "8.19.0"
checksum = "YHuVLVVp8q_Y7-JWpID5ReNjq2Zk6t7ArHB6ngQXilp_R5l3cubdxu3UKo-xDByv" checksum = "YHuVLVVp8q_Y7-JWpID5ReNjq2Zk6t7ArHB6ngQXilp_R5l3cubdxu3UKo-xDByv"
) )
return t.NewPackage("curl", version, pkg.NewHTTPGetTar( return t.NewPackage("curl", version, newTar(
nil, "https://curl.se/download/curl-"+version+".tar.bz2", "https://curl.se/download/curl-"+version+".tar.bz2",
mustDecode(checksum), checksum,
pkg.TarBzip2, pkg.TarBzip2,
), &PackageAttr{ ), &PackageAttr{
// remove broken test // remove broken test
Writable: true, Writable: true,
ScriptEarly: ` ScriptEarly: `
chmod +w tests/data && rm tests/data/test459 chmod +w tests/data && rm -f tests/data/test459
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: []KV{ Configure: []KV{
@@ -25,7 +25,7 @@ chmod +w tests/data && rm tests/data/test459
{"disable-smb"}, {"disable-smb"},
}, },
Check: []string{ Check: []string{
`TFLAGS="-j$(expr "$(nproc)" '*' 2)"`, "TFLAGS=" + jobsLFlagE,
"test-nonflaky", "test-nonflaky",
}, },
}, },

Some files were not shown because too many files have changed in this diff Show More