11 Commits

Author SHA1 Message Date
dbb89dfb0f internal/pkg: buffer tar reader
All checks were successful
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 4m0s
Test / ShareFS (push) Successful in 4m10s
Test / Hpkg (push) Successful in 4m44s
Test / Sandbox (race detector) (push) Successful in 5m34s
Test / Hakurei (race detector) (push) Successful in 8m7s
Test / Flake checks (push) Successful in 1m49s
This significantly improves performance and is a good assumption since the primary use case of FileArtifact is over the network.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-11 02:08:44 +09:00
de06ea2be4 internal/pkg: read buffer free list
All checks were successful
Test / Create distribution (push) Successful in 1m0s
Test / Sandbox (push) Successful in 2m53s
Test / Hakurei (push) Successful in 4m30s
Test / ShareFS (push) Successful in 4m28s
Test / Hpkg (push) Successful in 5m6s
Test / Sandbox (race detector) (push) Successful in 5m18s
Test / Hakurei (race detector) (push) Successful in 6m35s
Test / Flake checks (push) Successful in 2m3s
Reader has a non-insignificant buffer that is worth saving as well.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-11 02:02:46 +09:00
1ef7bedfb5 internal/rosa/toybox: do not assume bash location
All checks were successful
Test / Create distribution (push) Successful in 59s
Test / Sandbox (push) Successful in 2m36s
Test / Hakurei (push) Successful in 3m52s
Test / ShareFS (push) Successful in 3m58s
Test / Hpkg (push) Successful in 4m27s
Test / Sandbox (race detector) (push) Successful in 5m7s
Test / Hakurei (race detector) (push) Successful in 5m54s
Test / Flake checks (push) Successful in 1m43s
For compatibility with Gentoo stage3 as bootstrap seed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-11 01:36:41 +09:00
05a828c474 internal/pkg: validate tar pathnames
All checks were successful
Test / Create distribution (push) Successful in 59s
Test / Sandbox (push) Successful in 2m35s
Test / ShareFS (push) Successful in 3m59s
Test / Hpkg (push) Successful in 4m46s
Test / Sandbox (race detector) (push) Successful in 5m2s
Test / Hakurei (race detector) (push) Successful in 5m57s
Test / Hakurei (push) Successful in 4m24s
Test / Flake checks (push) Successful in 1m50s
TContext no longer validates FileArtifact ahead of time, validation outcome is instead determined after consuming the reader to EOF. All data must therefore be treated as untrusted input until the reader is closed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-11 00:40:54 +09:00
0061d11f93 internal/rosa: use self-hosted stage0
All checks were successful
Test / Create distribution (push) Successful in 59s
Test / Sandbox (push) Successful in 2m42s
Test / Hakurei (push) Successful in 3m53s
Test / ShareFS (push) Successful in 4m3s
Test / Hpkg (push) Successful in 4m31s
Test / Sandbox (race detector) (push) Successful in 5m4s
Test / Hakurei (race detector) (push) Successful in 5m58s
Test / Flake checks (push) Successful in 3m10s
This removes the bootstrap dependency on Gentoo stage3 tarball.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-10 22:31:08 +09:00
fb101a02f2 internal/rosa: self-host stage0 tarball
All checks were successful
Test / Create distribution (push) Successful in 2m26s
Test / Sandbox (push) Successful in 5m22s
Test / Hakurei (push) Successful in 6m37s
Test / ShareFS (push) Successful in 6m39s
Test / Hpkg (push) Successful in 7m10s
Test / Sandbox (race detector) (push) Successful in 7m15s
Test / Hakurei (race detector) (push) Successful in 8m15s
Test / Flake checks (push) Successful in 1m51s
This replaces gentoo stage3 tarballs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-10 21:22:46 +09:00
3dbd67d113 internal/rosa: consistent stage0 paths
All checks were successful
Test / Create distribution (push) Successful in 2m2s
Test / Hakurei (push) Successful in 6m35s
Test / ShareFS (push) Successful in 6m33s
Test / Sandbox (race detector) (push) Successful in 6m52s
Test / Hpkg (push) Successful in 7m20s
Test / Hakurei (race detector) (push) Successful in 7m46s
Test / Sandbox (push) Successful in 1m37s
Test / Flake checks (push) Successful in 1m50s
This makes using the gentoo stage3 as our stage0 compatible with Rosa OS stage0 tarballs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-10 19:28:47 +09:00
f511f0a9e9 internal/rosa: bzip2 artifact
All checks were successful
Test / Create distribution (push) Successful in 30s
Test / Sandbox (push) Successful in 45s
Test / ShareFS (push) Successful in 2m44s
Test / Hakurei (push) Successful in 3m1s
Test / Hpkg (push) Successful in 3m31s
Test / Sandbox (race detector) (push) Successful in 5m10s
Test / Hakurei (race detector) (push) Successful in 3m13s
Test / Flake checks (push) Successful in 1m48s
For creating the stage0 tarball. Might be replaced by a custom artifact at some point.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-09 02:26:52 +09:00
47995137b3 internal/rosa/perl: skip installing manpages
All checks were successful
Test / Create distribution (push) Successful in 59s
Test / Sandbox (push) Successful in 2m48s
Test / Hakurei (push) Successful in 4m13s
Test / ShareFS (push) Successful in 4m10s
Test / Hpkg (push) Successful in 4m48s
Test / Sandbox (race detector) (push) Successful in 5m5s
Test / Hakurei (race detector) (push) Successful in 7m0s
Test / Flake checks (push) Successful in 1m48s
Perl manpages ignore prefix and gets installed to /. This change does not use the configure script because it is completely broken and specifying either "none" or a single space character (undocumented) has no effect.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-09 01:50:52 +09:00
e1b8607101 internal/rosa: rename stage0 toolchain
All checks were successful
Test / Create distribution (push) Successful in 1m2s
Test / Sandbox (push) Successful in 1m48s
Test / Sandbox (race detector) (push) Successful in 2m52s
Test / Hakurei (push) Successful in 3m1s
Test / ShareFS (push) Successful in 3m16s
Test / Hakurei (race detector) (push) Successful in 3m58s
Test / Hpkg (push) Successful in 3m57s
Test / Flake checks (push) Successful in 1m48s
This is stage0 relative to Rosa OS, and stage3 relative to the toolchain it is compiled on (Gentoo in this case). Referring to the toolchain itself as stage3 is counterintuitive and misleading.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-09 00:55:07 +09:00
3d3bd45b95 internal/rosa/hakurei: 0.3.4 to 0.3.5
All checks were successful
Test / Create distribution (push) Successful in 59s
Test / Sandbox (push) Successful in 2m37s
Test / ShareFS (push) Successful in 3m59s
Test / Hpkg (push) Successful in 4m33s
Test / Sandbox (race detector) (push) Successful in 5m0s
Test / Hakurei (race detector) (push) Successful in 6m16s
Test / Hakurei (push) Successful in 3m0s
Test / Flake checks (push) Successful in 2m7s
This removes all backport patches.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-02-09 00:27:45 +09:00
15 changed files with 340 additions and 341 deletions

View File

@@ -109,46 +109,92 @@ func main() {
)
}
c.NewCommand(
"stage3",
"Check for toolchain 3-stage non-determinism",
func(args []string) (err error) {
_, _, _, stage1 := (rosa.Std - 2).NewLLVM()
_, _, _, stage2 := (rosa.Std - 1).NewLLVM()
_, _, _, stage3 := rosa.Std.NewLLVM()
var (
pathname *check.Absolute
checksum [2]unique.Handle[pkg.Checksum]
)
{
var (
flagGentoo string
flagChecksum string
if pathname, _, err = cache.Cure(stage1); err != nil {
return err
}
log.Println("stage1:", pathname)
flagStage0 bool
)
c.NewCommand(
"stage3",
"Check for toolchain 3-stage non-determinism",
func(args []string) (err error) {
std := rosa.Std
if flagGentoo != "" {
std -= 3 // magic number to discourage misuse
if pathname, checksum[0], err = cache.Cure(stage2); err != nil {
return err
}
log.Println("stage2:", pathname)
if pathname, checksum[1], err = cache.Cure(stage3); err != nil {
return err
}
log.Println("stage3:", pathname)
if checksum[0] != checksum[1] {
err = &pkg.ChecksumMismatchError{
Got: checksum[0].Value(),
Want: checksum[1].Value(),
var checksum pkg.Checksum
if len(flagChecksum) != 0 {
if err = pkg.Decode(&checksum, flagChecksum); err != nil {
return
}
}
rosa.SetGentooStage3(flagGentoo, checksum)
}
} else {
log.Println(
"stage2 is identical to stage3",
"("+pkg.Encode(checksum[0].Value())+")",
_, _, _, stage1 := (std - 2).NewLLVM()
_, _, _, stage2 := (std - 1).NewLLVM()
_, _, _, stage3 := std.NewLLVM()
var (
pathname *check.Absolute
checksum [2]unique.Handle[pkg.Checksum]
)
}
return
},
)
if pathname, _, err = cache.Cure(stage1); err != nil {
return err
}
log.Println("stage1:", pathname)
if pathname, checksum[0], err = cache.Cure(stage2); err != nil {
return err
}
log.Println("stage2:", pathname)
if pathname, checksum[1], err = cache.Cure(stage3); err != nil {
return err
}
log.Println("stage3:", pathname)
if checksum[0] != checksum[1] {
err = &pkg.ChecksumMismatchError{
Got: checksum[0].Value(),
Want: checksum[1].Value(),
}
} else {
log.Println(
"stage2 is identical to stage3",
"("+pkg.Encode(checksum[0].Value())+")",
)
}
if flagStage0 {
if pathname, _, err = cache.Cure(
std.Load(rosa.Stage0),
); err != nil {
return err
}
log.Println(pathname)
}
return
},
).
Flag(
&flagGentoo,
"gentoo", command.StringFlag(""),
"Bootstrap from a Gentoo stage3 tarball",
).
Flag(
&flagChecksum,
"checksum", command.StringFlag(""),
"Checksum of Gentoo stage3 tarball",
).
Flag(
&flagStage0,
"stage0", command.BoolFlag(false),
"Create bootstrap stage0 tarball",
)
}
{
var (

View File

@@ -466,7 +466,7 @@ type Cache struct {
// Synchronises entry into exclusive artifacts for the cure method.
exclMu sync.Mutex
// Buffered I/O free list, must not be accessed directly.
bufioPool sync.Pool
brPool, bwPool sync.Pool
// Unlocks the on-filesystem cache. Must only be called from Close.
unlock func()
@@ -548,6 +548,26 @@ func (c *Cache) unsafeIdent(a Artifact, encodeKind bool) (
return
}
// getReader is like [bufio.NewReader] but for brPool.
func (c *Cache) getReader(r io.Reader) *bufio.Reader {
br := c.brPool.Get().(*bufio.Reader)
br.Reset(r)
return br
}
// putReader adds br to brPool.
func (c *Cache) putReader(br *bufio.Reader) { c.brPool.Put(br) }
// getWriter is like [bufio.NewWriter] but for bwPool.
func (c *Cache) getWriter(w io.Writer) *bufio.Writer {
bw := c.bwPool.Get().(*bufio.Writer)
bw.Reset(w)
return bw
}
// putWriter adds bw to bwPool.
func (c *Cache) putWriter(bw *bufio.Writer) { c.bwPool.Put(bw) }
// A ChecksumMismatchError describes an [Artifact] with unexpected content.
type ChecksumMismatchError struct {
// Actual and expected checksums.
@@ -1214,13 +1234,6 @@ func (c *Cache) exitCure(a Artifact, curesExempt bool) {
<-c.cures
}
// getWriter is like [bufio.NewWriter] but for bufioPool.
func (c *Cache) getWriter(w io.Writer) *bufio.Writer {
bw := c.bufioPool.Get().(*bufio.Writer)
bw.Reset(w)
return bw
}
// measuredReader implements [io.ReadCloser] and measures the checksum during
// Close. If the underlying reader is not read to EOF, Close blocks until all
// remaining data is consumed and validated.
@@ -1303,9 +1316,6 @@ func (r *RContext) NewMeasuredReader(
return r.cache.newMeasuredReader(rc, checksum)
}
// putWriter adds bw to bufioPool.
func (c *Cache) putWriter(bw *bufio.Writer) { c.bufioPool.Put(bw) }
// cure implements Cure without checking the full dependency graph.
func (c *Cache) cure(a Artifact, curesExempt bool) (
pathname *check.Absolute,
@@ -1713,13 +1723,16 @@ func open(
msg: msg,
base: base,
identPool: sync.Pool{New: func() any { return new(extIdent) }},
ident: make(map[unique.Handle[ID]]unique.Handle[Checksum]),
identErr: make(map[unique.Handle[ID]]error),
identPending: make(map[unique.Handle[ID]]<-chan struct{}),
brPool: sync.Pool{New: func() any { return new(bufio.Reader) }},
bwPool: sync.Pool{New: func() any { return new(bufio.Writer) }},
}
c.ctx, c.cancel = context.WithCancel(ctx)
c.identPool.New = func() any { return new(extIdent) }
c.bufioPool.New = func() any { return new(bufio.Writer) }
if lock || !testing.Testing() {
if unlock, err := lockedfile.MutexAt(

View File

@@ -10,8 +10,7 @@ import (
"io/fs"
"net/http"
"os"
"hakurei.app/container/check"
"path"
)
const (
@@ -100,7 +99,6 @@ func (e DisallowedTypeflagError) Error() string {
// Cure cures the [Artifact], producing a directory located at work.
func (a *tarArtifact) Cure(t *TContext) (err error) {
temp := t.GetTempDir()
var tr io.ReadCloser
if tr, err = t.Open(a.f); err != nil {
return
@@ -116,7 +114,9 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
err = closeErr
}
}(tr)
tr = io.NopCloser(tr)
br := t.cache.getReader(tr)
defer t.cache.putReader(br)
tr = io.NopCloser(br)
switch a.compression {
case TarUncompressed:
@@ -137,14 +137,24 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
}
type dirTargetPerm struct {
path *check.Absolute
path string
mode fs.FileMode
}
var madeDirectories []dirTargetPerm
if err = os.MkdirAll(temp.String(), 0700); err != nil {
if err = os.MkdirAll(t.GetTempDir().String(), 0700); err != nil {
return
}
var root *os.Root
if root, err = os.OpenRoot(t.GetTempDir().String()); err != nil {
return
}
defer func() {
closeErr := root.Close()
if err == nil {
err = closeErr
}
}()
var header *tar.Header
r := tar.NewReader(tr)
@@ -158,9 +168,8 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
}
}
pathname := temp.Append(header.Name)
if typeflag >= '0' && typeflag <= '9' && typeflag != tar.TypeDir {
if err = os.MkdirAll(pathname.Dir().String(), 0700); err != nil {
if err = root.MkdirAll(path.Dir(header.Name), 0700); err != nil {
return
}
}
@@ -168,8 +177,8 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
switch typeflag {
case tar.TypeReg:
var f *os.File
if f, err = os.OpenFile(
pathname.String(),
if f, err = root.OpenFile(
header.Name,
os.O_CREATE|os.O_EXCL|os.O_WRONLY,
header.FileInfo().Mode()&0500,
); err != nil {
@@ -184,26 +193,29 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
break
case tar.TypeLink:
if err = os.Link(
temp.Append(header.Linkname).String(),
pathname.String(),
if err = root.Link(
header.Linkname,
header.Name,
); err != nil {
return
}
break
case tar.TypeSymlink:
if err = os.Symlink(header.Linkname, pathname.String()); err != nil {
if err = root.Symlink(
header.Linkname,
header.Name,
); err != nil {
return
}
break
case tar.TypeDir:
madeDirectories = append(madeDirectories, dirTargetPerm{
path: pathname,
path: header.Name,
mode: header.FileInfo().Mode(),
})
if err = os.MkdirAll(pathname.String(), 0700); err != nil {
if err = root.MkdirAll(header.Name, 0700); err != nil {
return
}
break
@@ -220,7 +232,7 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
}
if err == nil {
for _, e := range madeDirectories {
if err = os.Chmod(e.path.String(), e.mode&0500); err != nil {
if err = root.Chmod(e.path, e.mode&0500); err != nil {
return
}
}
@@ -228,6 +240,7 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
return
}
temp := t.GetTempDir()
if err = os.Chmod(temp.String(), 0700); err != nil {
return
}

View File

@@ -16,6 +16,7 @@ const (
Automake
Bash
Binutils
Bzip2
CMake
Coreutils
Curl
@@ -82,6 +83,10 @@ const (
// stages only. This preset and its direct output must never be exposed.
gcc
// Stage0 is a tarball containing all compile-time dependencies of artifacts
// part of the [Std] toolchain.
Stage0
// _presetEnd is the total number of presets and does not denote a preset.
_presetEnd
)
@@ -113,6 +118,7 @@ func ResolveName(name string) (p PArtifact, ok bool) {
"automake": Automake,
"bash": Bash,
"binutils": Binutils,
"bzip2": Bzip2,
"cmake": CMake,
"coreutils": Coreutils,
"curl": Curl,

22
internal/rosa/bzip2.go Normal file
View File

@@ -0,0 +1,22 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newBzip2() pkg.Artifact {
const (
version = "1.0.8"
checksum = "cTLykcco7boom-s05H1JVsQi1AtChYL84nXkg_92Dm1Xt94Ob_qlMg_-NSguIK-c"
)
return t.New("bzip2-"+version, 0, []pkg.Artifact{
t.Load(Make),
}, nil, nil, `
cd /usr/src/bzip2
make CC=cc
make PREFIX=/work/system install
`, pkg.Path(AbsUsrSrc.Append("bzip2"), true, pkg.NewHTTPGetTar(
nil, "https://sourceware.org/pub/bzip2/bzip2-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
)))
}
func init() { artifactsF[Bzip2] = Toolchain.newBzip2 }

View File

@@ -97,7 +97,7 @@ func (t Toolchain) NewViaCMake(
}
sourcePath := AbsUsrSrc.Append(name)
return t.New(name+"-"+variant+"-"+version, attr.Flag, stage3Concat(t, extra,
return t.New(name+"-"+variant+"-"+version, attr.Flag, stage0Concat(t, extra,
t.Load(CMake),
t.Load(Ninja),
), nil, slices.Concat([]string{

View File

@@ -6,8 +6,8 @@ import (
func (t Toolchain) newHakurei(suffix, script string) pkg.Artifact {
const (
version = "0.3.4"
checksum = "wVwSLo75a2OnH5tgxNWXR_YhiOJUFnYM_9-sJtxAEOKhcPE0BJafs6PU8o5JzyCT"
version = "0.3.5"
checksum = "6Tn38NLezRD2d3aGdFg5qFfqn8_KvC6HwMKwJMPvaHmVw8xRgxn8B0PObswl2mOk"
)
return t.New("hakurei"+suffix+"-"+version, 0, []pkg.Artifact{
t.Load(Go),
@@ -44,213 +44,12 @@ chmod -R +w /usr/src/hakurei
cd /usr/src/hakurei
HAKUREI_VERSION='v`+version+`'
`+script, pkg.Path(AbsUsrSrc.Append("hakurei"), true, t.NewPatchedSource("hakurei", version, pkg.NewHTTPGetTar(
`+script, pkg.Path(AbsUsrSrc.Append("hakurei"), true, pkg.NewHTTPGetTar(
nil, "https://git.gensokyo.uk/security/hakurei/archive/"+
"v"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), true, [2]string{"dist-00-tests", `From 67e453f5c4de915de23ecbe5980e595758f0f2fb Mon Sep 17 00:00:00 2001
From: Ophestra <cat@gensokyo.uk>
Date: Tue, 27 Jan 2026 06:49:48 +0900
Subject: [PATCH] dist: run tests
This used to be impossible due to nix jank which has been addressed.
Signed-off-by: Ophestra <cat@gensokyo.uk>
---
dist/release.sh | 21 ++++++++++++++++-----
flake.nix | 32 ++++++++++++++++++++------------
internal/acl/acl_test.go | 2 +-
package.nix | 2 +-
4 files changed, 38 insertions(+), 19 deletions(-)
diff --git a/dist/release.sh b/dist/release.sh
index 4dcb278..0ba9104 100755
--- a/dist/release.sh
+++ b/dist/release.sh
@@ -2,19 +2,30 @@
cd "$(dirname -- "$0")/.."
VERSION="${HAKUREI_VERSION:-untagged}"
pname="hakurei-${VERSION}"
-out="dist/${pname}"
+out="${DESTDIR:-dist}/${pname}"
+echo '# Preparing distribution files.'
mkdir -p "${out}"
cp -v "README.md" "dist/hsurc.default" "dist/install.sh" "${out}"
cp -rv "dist/comp" "${out}"
+echo
+echo '# Building hakurei.'
go generate ./...
-go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w -buildid= -extldflags '-static'
+go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w
+ -buildid= -extldflags '-static'
-X hakurei.app/internal/info.buildVersion=${VERSION}
-X hakurei.app/internal/info.hakureiPath=/usr/bin/hakurei
-X hakurei.app/internal/info.hsuPath=/usr/bin/hsu
-X main.hakureiPath=/usr/bin/hakurei" ./...
+echo
-rm -f "./${out}.tar.gz" && tar -C dist -czf "${out}.tar.gz" "${pname}"
-rm -rf "./${out}"
-(cd dist && sha512sum "${pname}.tar.gz" > "${pname}.tar.gz.sha512")
+echo '# Testing hakurei.'
+go test -ldflags='-buildid= -extldflags=-static' ./...
+echo
+
+echo '# Creating distribution.'
+rm -f "${out}.tar.gz" && tar -C "${out}/.." -vczf "${out}.tar.gz" "${pname}"
+rm -rf "${out}"
+(cd "${out}/.." && sha512sum "${pname}.tar.gz" > "${pname}.tar.gz.sha512")
+echo
diff --git a/flake.nix b/flake.nix
index 9e09c61..2340b92 100644
--- a/flake.nix
+++ b/flake.nix
@@ -143,19 +143,27 @@
"bin/mount.fuse.sharefs" = "${hakurei}/libexec/sharefs";
};
- dist = pkgs.runCommand "${hakurei.name}-dist" { buildInputs = hakurei.targetPkgs ++ [ pkgs.pkgsStatic.musl ]; } ''
- # go requires XDG_CACHE_HOME for the build cache
- export XDG_CACHE_HOME="$(mktemp -d)"
+ dist =
+ pkgs.runCommand "${hakurei.name}-dist"
+ {
+ buildInputs = hakurei.targetPkgs ++ [
+ pkgs.pkgsStatic.musl
+ ];
+ }
+ ''
+ cd $(mktemp -d) \
+ && cp -r ${hakurei.src}/. . \
+ && chmod +w cmd && cp -r ${hsu.src}/. cmd/hsu/ \
+ && chmod -R +w .
- # get a different workdir as go does not like /build
- cd $(mktemp -d) \
- && cp -r ${hakurei.src}/. . \
- && chmod +w cmd && cp -r ${hsu.src}/. cmd/hsu/ \
- && chmod -R +w .
-
- export HAKUREI_VERSION="v${hakurei.version}"
- CC="clang -O3 -Werror" ./dist/release.sh && mkdir $out && cp -v "dist/hakurei-$HAKUREI_VERSION.tar.gz"* $out
- '';
+ CC="musl-clang -O3 -Werror -Qunused-arguments" \
+ GOCACHE="$(mktemp -d)" \
+ HAKUREI_TEST_SKIP_ACL=1 \
+ PATH="${pkgs.pkgsStatic.musl.bin}/bin:$PATH" \
+ DESTDIR="$out" \
+ HAKUREI_VERSION="v${hakurei.version}" \
+ ./dist/release.sh
+ '';
}
);
diff --git a/internal/acl/acl_test.go b/internal/acl/acl_test.go
index af6da55..19ce45a 100644
--- a/internal/acl/acl_test.go
+++ b/internal/acl/acl_test.go
@@ -24,7 +24,7 @@ var (
)
func TestUpdate(t *testing.T) {
- if os.Getenv("GO_TEST_SKIP_ACL") == "1" {
+ if os.Getenv("HAKUREI_TEST_SKIP_ACL") == "1" {
t.Skip("acl test skipped")
}
diff --git a/package.nix b/package.nix
index 00c4401..2eaa2ec 100644
--- a/package.nix
+++ b/package.nix
@@ -89,7 +89,7 @@ buildGoModule rec {
CC = "clang -O3 -Werror";
# nix build environment does not allow acls
- GO_TEST_SKIP_ACL = 1;
+ HAKUREI_TEST_SKIP_ACL = 1;
};
buildInputs = [`}, [2]string{"container-tests", `From bf14a412e47344fff2681f4b24d1ecc7415bfcb0 Mon Sep 17 00:00:00 2001
From: Ophestra <cat@gensokyo.uk>
Date: Sat, 31 Jan 2026 10:59:56 +0900
Subject: [PATCH] container: fix host-dependent test cases
These are not fully controlled by hakurei and may change depending on host configuration.
Signed-off-by: Ophestra <cat@gensokyo.uk>
---
container/container_test.go | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/container/container_test.go b/container/container_test.go
index d737a18..98713cb 100644
--- a/container/container_test.go
+++ b/container/container_test.go
@@ -275,12 +275,12 @@ var containerTestCases = []struct {
),
earlyMnt(
ent("/", "/dev", "ro,nosuid,nodev,relatime", "tmpfs", "devtmpfs", ignore),
- ent("/null", "/dev/null", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/zero", "/dev/zero", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/full", "/dev/full", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/random", "/dev/random", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/urandom", "/dev/urandom", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/tty", "/dev/tty", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
+ ent("/null", "/dev/null", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/zero", "/dev/zero", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/full", "/dev/full", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/random", "/dev/random", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/urandom", "/dev/urandom", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/tty", "/dev/tty", ignore, "devtmpfs", "devtmpfs", ignore),
ent("/", "/dev/pts", "rw,nosuid,noexec,relatime", "devpts", "devpts", "rw,mode=620,ptmxmode=666"),
ent("/", "/dev/mqueue", "rw,nosuid,nodev,noexec,relatime", "mqueue", "mqueue", "rw"),
ent("/", "/dev/shm", "rw,nosuid,nodev,relatime", "tmpfs", "tmpfs", ignore),
@@ -293,12 +293,12 @@ var containerTestCases = []struct {
),
earlyMnt(
ent("/", "/dev", "ro,nosuid,nodev,relatime", "tmpfs", "devtmpfs", ignore),
- ent("/null", "/dev/null", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/zero", "/dev/zero", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/full", "/dev/full", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/random", "/dev/random", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/urandom", "/dev/urandom", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
- ent("/tty", "/dev/tty", "rw,nosuid", "devtmpfs", "devtmpfs", ignore),
+ ent("/null", "/dev/null", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/zero", "/dev/zero", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/full", "/dev/full", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/random", "/dev/random", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/urandom", "/dev/urandom", ignore, "devtmpfs", "devtmpfs", ignore),
+ ent("/tty", "/dev/tty", ignore, "devtmpfs", "devtmpfs", ignore),
ent("/", "/dev/pts", "rw,nosuid,noexec,relatime", "devpts", "devpts", "rw,mode=620,ptmxmode=666"),
ent("/", "/dev/shm", "rw,nosuid,nodev,relatime", "tmpfs", "tmpfs", ignore),
),
@@ -696,6 +696,9 @@ func init() {
mnt[i].VfsOptstr = strings.TrimSuffix(mnt[i].VfsOptstr, ",relatime")
mnt[i].VfsOptstr = strings.TrimSuffix(mnt[i].VfsOptstr, ",noatime")
+ cur.FsOptstr = strings.Replace(cur.FsOptstr, ",seclabel", "", 1)
+ mnt[i].FsOptstr = strings.Replace(mnt[i].FsOptstr, ",seclabel", "", 1)
+
if !cur.EqualWithIgnore(mnt[i], "\x00") {
fail = true
log.Printf("[FAIL] %s", cur)`}, [2]string{"dist-01-tarball-name", `diff --git a/dist/release.sh b/dist/release.sh
index 0ba9104..2990ee1 100755
--- a/dist/release.sh
+++ b/dist/release.sh
@@ -1,7 +1,7 @@
#!/bin/sh -e
cd "$(dirname -- "$0")/.."
VERSION="${HAKUREI_VERSION:-untagged}"
-pname="hakurei-${VERSION}"
+pname="hakurei-${VERSION}-$(go env GOARCH)"
out="${DESTDIR:-dist}/${pname}"
echo '# Preparing distribution files.'
`}),
), pkg.Path(AbsUsrSrc.Append("hostname", "main.go"), false, pkg.NewFile(
)), pkg.Path(AbsUsrSrc.Append("hostname", "main.go"), false, pkg.NewFile(
"hostname.go",
[]byte(`
package main

View File

@@ -160,7 +160,7 @@ ln -s ld.lld /work/system/bin/ld
[2]string{"LIBCXX_USE_COMPILER_RT", "ON"},
)
if t > toolchainStage3 {
if t > toolchainStage0 {
// libcxxabi fails to compile if c++ headers not prefixed in /usr
// is found by the compiler, and doing this is easier than
// overriding CXXFLAGS; not using mv here to avoid chown failures
@@ -199,7 +199,7 @@ cp -r /system/include /usr/include && rm -rf /system/include
Paths: attr.paths,
Flag: TExclusive,
}, stage3Concat(t, attr.extra,
}, stage0Concat(t, attr.extra,
t.Load(Libffi),
t.Load(Python),
t.Load(Perl),
@@ -233,7 +233,7 @@ func (t Toolchain) newLLVM() (musl, compilerRT, runtimes, clang pkg.Artifact) {
}
compilerRT = t.newLLVMVariant("compiler-rt", &llvmAttr{
env: stage3ExclConcat(t, []string{},
env: stage0ExclConcat(t, []string{},
"LDFLAGS="+earlyLDFLAGS(false),
),
cmake: [][2]string{
@@ -276,7 +276,7 @@ ln -s \
musl = t.NewMusl(&MuslAttr{
Extra: []pkg.Artifact{compilerRT},
Env: stage3ExclConcat(t, []string{
Env: stage0ExclConcat(t, []string{
"CC=clang",
"LIBCC=/system/lib/clang/21/lib/" +
triplet() + "/libclang_rt.builtins.a",
@@ -288,7 +288,7 @@ ln -s \
})
runtimes = t.newLLVMVariant("runtimes", &llvmAttr{
env: stage3ExclConcat(t, []string{},
env: stage0ExclConcat(t, []string{},
"LDFLAGS="+earlyLDFLAGS(false),
),
flags: llvmRuntimeLibunwind | llvmRuntimeLibcxx | llvmRuntimeLibcxxABI,
@@ -308,7 +308,7 @@ ln -s \
clang = t.newLLVMVariant("clang", &llvmAttr{
flags: llvmProjectClang | llvmProjectLld,
env: stage3ExclConcat(t, []string{},
env: stage0ExclConcat(t, []string{},
"CFLAGS="+earlyCFLAGS,
"CXXFLAGS="+earlyCXXFLAGS(),
"LDFLAGS="+earlyLDFLAGS(false),

View File

@@ -36,8 +36,8 @@ type MakeAttr struct {
// Do not include default extras.
OmitDefaults bool
// Dependencies not provided by stage3.
NonStage3 []pkg.Artifact
// Dependencies not provided by stage0.
NonStage0 []pkg.Artifact
// Additional environment variables.
Env []string
@@ -148,8 +148,8 @@ func (t Toolchain) NewViaMake(
panic("cannot remain in root")
}
return t.New(name+"-"+version, attr.Flag, stage3Concat(t,
attr.NonStage3,
return t.New(name+"-"+version, attr.Flag, stage0Concat(t,
attr.NonStage0,
finalExtra...,
), nil, attr.Env, scriptEarly+`
/usr/src/`+name+`/configure \

View File

@@ -7,7 +7,7 @@ func (t Toolchain) newMksh() pkg.Artifact {
version = "59c"
checksum = "0Zj-k4nXEu3IuJY4lvwD2OrC2t27GdZj8SPy4DoaeuBRH1padWb7oREpYgwY8JNq"
)
return t.New("mksh-"+version, 0, stage3Concat(t, []pkg.Artifact{},
return t.New("mksh-"+version, 0, stage0Concat(t, []pkg.Artifact{},
t.Load(Perl),
t.Load(Coreutils),
), nil, []string{

View File

@@ -42,7 +42,7 @@ rmdir -v /work/lib
script = ""
}
return t.New("musl-"+version, 0, stage3Concat(t, attr.Extra,
return t.New("musl-"+version, 0, stage0Concat(t, attr.Extra,
t.Load(Make),
t.Load(Coreutils),
), nil, slices.Concat([]string{

View File

@@ -27,7 +27,7 @@ make \
"-j$(nproc)" \
TEST_JOBS=256 \
test_harness
make DESTDIR=/work install
./perl -Ilib -I. installperl --destdir=/work
`, pkg.Path(AbsUsrSrc.Append("perl"), true, t.NewPatchedSource(
"perl", version, pkg.NewHTTPGetTar(
nil, "https://www.cpan.org/src/5.0/perl-"+version+".tar.gz",

View File

@@ -2,6 +2,7 @@
package rosa
import (
"errors"
"log"
"runtime"
"slices"
@@ -82,11 +83,11 @@ func earlyLDFLAGS(static bool) string {
return s
}
// earlyCFLAGS is reference CFLAGS for the stage3 toolchain.
// earlyCFLAGS is reference CFLAGS for the stage0 toolchain.
const earlyCFLAGS = "-Qunused-arguments " +
"-isystem/system/include"
// earlyCXXFLAGS returns reference CXXFLAGS for the stage3 toolchain
// earlyCXXFLAGS returns reference CXXFLAGS for the stage0 toolchain
// corresponding to [runtime.GOARCH].
func earlyCXXFLAGS() string {
return "--start-no-unused-arguments " +
@@ -101,16 +102,30 @@ func earlyCXXFLAGS() string {
type Toolchain uintptr
const (
// toolchainBusybox denotes a busybox installation from the busyboxBin
// binary distribution. This is for decompressing unsupported formats.
toolchainBusybox Toolchain = iota
// _toolchainBusybox denotes a busybox installation from the busyboxBin
// binary distribution. This is defined as a toolchain to make use of the
// toolchain abstractions to preprocess toolchainGentoo and is not a real,
// functioning toolchain. It does not contain any compilers.
_toolchainBusybox Toolchain = iota
// toolchainStage3 denotes the Gentoo stage3 toolchain. Special care must be
// taken to compile correctly against this toolchain.
toolchainStage3
// toolchainGentoo denotes the toolchain in a Gentoo stage3 tarball. Special
// care must be taken to compile correctly against this toolchain.
toolchainGentoo
// toolchainIntermediateGentoo is like to toolchainIntermediate, but
// compiled against toolchainGentoo.
toolchainIntermediateGentoo
// toolchainStdGentoo is like Std, but bootstrapped from toolchainGentoo.
// This toolchain creates the first [Stage0] distribution.
toolchainStdGentoo
// toolchainStage0 denotes the stage0 toolchain. Special care must be taken
// to compile correctly against this toolchain.
toolchainStage0
// toolchainIntermediate denotes the intermediate toolchain compiled against
// toolchainStage3. This toolchain should be functionally identical to [Std]
// toolchainStage0. This toolchain should be functionally identical to [Std]
// and is used to bootstrap [Std].
toolchainIntermediate
@@ -122,19 +137,19 @@ const (
_toolchainEnd
)
// stage3Concat concatenates s and values. If the current toolchain is
// toolchainStage3, stage3Concat returns s as is.
func stage3Concat[S ~[]E, E any](t Toolchain, s S, values ...E) S {
if t == toolchainStage3 {
// stage0Concat concatenates s and values. If the current toolchain is
// toolchainStage0, stage0Concat returns s as is.
func stage0Concat[S ~[]E, E any](t Toolchain, s S, values ...E) S {
if t == toolchainStage0 || t == toolchainGentoo {
return s
}
return slices.Concat(s, values)
}
// stage3ExclConcat concatenates s and values. If the current toolchain is not
// toolchainStage3, stage3ExclConcat returns s as is.
func stage3ExclConcat[S ~[]E, E any](t Toolchain, s S, values ...E) S {
if t == toolchainStage3 {
// stage0ExclConcat concatenates s and values. If the current toolchain is not
// toolchainStage0, stage0ExclConcat returns s as is.
func stage0ExclConcat[S ~[]E, E any](t Toolchain, s S, values ...E) S {
if t == toolchainStage0 || t == toolchainGentoo {
return slices.Concat(s, values)
}
return s
@@ -173,7 +188,7 @@ func fixupEnviron(env, extras []string, paths ...string) []string {
// absCureScript is the absolute pathname [Toolchain.New] places the fixed-up
// build script under.
var absCureScript = fhs.AbsUsrBin.Append(".cure-script")
var absCureScript = AbsSystem.Append("bin", ".cure-script")
const (
// TExclusive denotes an exclusive [pkg.Artifact].
@@ -182,12 +197,31 @@ const (
TEarly
)
var (
// gentooStage3 is the url of a Gentoo stage3 tarball.
gentooStage3 string
// gentooStage3Checksum is the expected checksum of gentooStage3.
gentooStage3Checksum pkg.Checksum
)
// SetGentooStage3 sets the Gentoo stage3 tarball url and checksum. It panics
// if given zero values or if these values have already been set.
func SetGentooStage3(url string, checksum pkg.Checksum) {
if gentooStage3 != "" {
panic(errors.New("attempting to set Gentoo stage3 url twice"))
}
if url == "" {
panic(errors.New("attempting to set Gentoo stage3 url to the zero value"))
}
gentooStage3, gentooStage3Checksum = url, checksum
}
// New returns a [pkg.Artifact] compiled on this toolchain.
func (t Toolchain) New(
name string,
flag int,
extra []pkg.Artifact,
checksum *pkg.Checksum,
knownChecksum *pkg.Checksum,
env []string,
script string,
@@ -195,57 +229,47 @@ func (t Toolchain) New(
) pkg.Artifact {
const lcMessages = "LC_MESSAGES=C.UTF-8"
var (
path = AbsSystem.Append("bin", "sh")
args = []string{"sh", absCureScript.String()}
support []pkg.Artifact
)
var support []pkg.Artifact
switch t {
case toolchainBusybox:
case _toolchainBusybox:
name += "-early"
support = slices.Concat([]pkg.Artifact{newBusyboxBin()}, extra)
path = AbsSystem.Append("bin", "busybox")
args[0] = "hush"
env = fixupEnviron(env, nil, "/system/bin")
case toolchainStage3:
case toolchainGentoo, toolchainStage0:
name += "-boot"
var seed string
switch runtime.GOARCH {
case "amd64":
seed = "c5_FwMnRN8RZpTdBLGYkL4RR8ampdaZN2JbkgrFLe8-QHQAVQy08APVvIL6eT7KW"
case "arm64":
seed = "79uRbRI44PyknQQ9RlFUQrwqplup7vImiIk6klefL8TN-fT42TXMS_v4XszwexCb"
default:
panic("unsupported target " + runtime.GOARCH)
}
path = fhs.AbsRoot.Append("bin", "bash")
args[0] = "bash"
support = slices.Concat([]pkg.Artifact{
cureEtc{},
toolchainBusybox.New("stage3", 0, nil, nil, nil, `
support = append(support, cureEtc{})
if t == toolchainStage0 {
support = append(support, NewStage0())
} else {
support = append(support, _toolchainBusybox.New("gentoo", 0, nil, nil, nil, `
tar -C /work -xf /usr/src/stage3.tar.xz
rm -rf /work/dev/ /work/proc/
ln -vs ../usr/bin /work/bin
mkdir -vp /work/system/bin
(cd /work/system/bin && ln -vs \
../../bin/sh \
../../usr/lib/llvm/*/bin/* \
.)
`, pkg.Path(AbsUsrSrc.Append("stage3.tar.xz"), false,
pkg.NewHTTPGet(
nil, "https://basement.gensokyo.uk/seed/"+seed,
mustDecode(seed),
nil, gentooStage3,
gentooStage3Checksum,
),
)),
}, extra)
)))
}
support = slices.Concat(support, extra)
env = fixupEnviron(env, []string{
EnvTriplet + "=" + triplet(),
lcMessages,
"LDFLAGS=" + earlyLDFLAGS(true),
}, "/system/bin",
"/usr/bin",
"/usr/lib/llvm/21/bin",
)
case toolchainIntermediate, Std:
if t < Std {
case toolchainIntermediateGentoo, toolchainStdGentoo,
toolchainIntermediate, Std:
if t == toolchainIntermediateGentoo || t == toolchainIntermediate {
name += "-std"
}
@@ -279,9 +303,10 @@ ln -vs ../usr/bin /work/bin
}
return pkg.NewExec(
name, checksum, pkg.ExecTimeoutMax, flag&TExclusive != 0,
name, knownChecksum, pkg.ExecTimeoutMax, flag&TExclusive != 0,
fhs.AbsRoot, env,
path, args,
AbsSystem.Append("bin", "sh"),
[]string{"sh", absCureScript.String()},
slices.Concat([]pkg.ExecPath{pkg.Path(
fhs.AbsRoot, true,
@@ -328,7 +353,7 @@ cat /usr/src/` + name + `-patches/* | \
`
aname += "-patched"
}
return t.New(aname, 0, stage3Concat(t, []pkg.Artifact{},
return t.New(aname, 0, stage0Concat(t, []pkg.Artifact{},
t.Load(Patch),
), nil, nil, script, paths...)
}

73
internal/rosa/stage0.go Normal file
View File

@@ -0,0 +1,73 @@
package rosa
import (
"runtime"
"sync"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newStage0() pkg.Artifact {
musl, compilerRT, runtimes, clang := t.NewLLVM()
return t.New("rosa-stage0", 0, []pkg.Artifact{
musl,
compilerRT,
runtimes,
clang,
t.Load(Bzip2),
t.Load(Patch),
t.Load(Make),
t.Load(CMake),
t.Load(Ninja),
t.Load(Libffi),
t.Load(Python),
t.Load(Perl),
t.Load(Diffutils),
t.Load(Bash),
t.Load(Gawk),
t.Load(Coreutils),
t.Load(Findutils),
t.Load(KernelHeaders),
}, nil, nil, `
umask 377
tar \
-vjc \
-C / \
-f /work/stage0-`+triplet()+`.tar.bz2 \
system bin usr/bin/env
`)
}
func init() { artifactsF[Stage0] = Toolchain.newStage0 }
var (
// stage0 stores the tarball unpack artifact.
stage0 pkg.Artifact
// stage0Once is for lazy initialisation of stage0.
stage0Once sync.Once
)
// NewStage0 returns a stage0 distribution created from curing [Stage0].
func NewStage0() pkg.Artifact {
stage0Once.Do(func() {
var seed string
switch runtime.GOARCH {
case "amd64":
seed = "tqM1Li15BJ-uFG8zU-XjgFxoN_kuzh1VxrSDVUVa0vGmo-NeWapSftH739sY8EAg"
default:
panic("unsupported target " + runtime.GOARCH)
}
stage0 = pkg.NewHTTPGetTar(
nil, "https://hakurei.app/seed/20260210/"+
"stage0-"+triplet()+".tar.bz2",
mustDecode(seed),
pkg.TarBzip2,
)
})
return stage0
}

View File

@@ -7,16 +7,18 @@ func (t Toolchain) newToybox(suffix, script string) pkg.Artifact {
version = "0.8.13"
checksum = "rZ1V1ATDte2WeQZanxLVoiRGdfPXhMlEo5-exX-e-ml8cGn9qOv0ABEUVZpX3wTI"
)
return t.New("toybox-"+version+suffix, TEarly, stage3Concat(t, []pkg.Artifact{},
return t.New("toybox-"+version+suffix, TEarly, stage0Concat(t, []pkg.Artifact{},
t.Load(Make),
t.Load(Bash),
t.Load(Gzip),
t.Load(KernelHeaders),
), nil, stage3Concat(t, []string{},
), nil, stage0Concat(t, []string{},
"ROSA_CHECK=make USER=cure tests",
), `
ln -s ../system/bin/bash /bin/ || true
chmod +w /bin/
ln -rs "$(which bash)" /bin/ || true
cd /usr/src/toybox
chmod +w kconfig tests
rm \