1
0
forked from rosa/hakurei

97 Commits

Author SHA1 Message Date
33a0e6c01b hst: conditionally skip root remount
This enables the writable root overlay use case.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 14:04:21 +09:00
d58f5c7590 dist: destroy workdir on exist
This no longer relies on the hermetic build system to clean up.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:32:16 +09:00
1da992e342 dist: prefix from environment
These are baked in, so make them configurable for the build.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 13:11:44 +09:00
9641805ec2 container/init: ignore finished process
This is not considered an error, if the process finishes while the signal is being delivered.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:38:33 +09:00
0738f4889a internal/rosa/gnu: fetch mpc source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:35:15 +09:00
7de3cfe221 internal/rosa/netfilter: fetch iptables source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:19:40 +09:00
8b0648dd5d internal/rosa/netfilter: fetch libnftnl source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:14:45 +09:00
4667fac76c internal/rosa/libbsd: fetch source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:05:56 +09:00
52e5443b0e internal/rosa/libbsd: fetch libmd source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-05 00:02:56 +09:00
130e470b60 internal/rosa/libxslt: fetch source via git
Eliminates the xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:05:10 +09:00
ba5ee8e3ee internal/rosa/libxml2: fetch source via git
Eliminates the xz dependency. This also switches to meson to avoid pulling in autotools.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:58:29 +09:00
d1cef30877 internal/rosa/gtk: fetch glib source via git
This eliminates xz dependency.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:47:10 +09:00
0188a3f0c7 internal/rosa/gnu: gnutls disable arm64 hardware acceleration
Hardware on arm64 is quite messy, this miscompiles.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 23:37:18 +09:00
04fe3b24ce internal/rosa/gnu: gnutls configure trust store
The test suite is somehow happy on amd64 but fails on arm64.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 22:33:51 +09:00
93ad551054 internal/rosa/git: shallow clone
The .git directory is destroyed anyway, so no point fetching more than the bare minimum.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-04 20:14:40 +09:00
3d54d1f176 internal/rosa: drop caches
This enables accurate benchmarking of the toolchain abstraction.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:58:23 +09:00
9feac7738f internal/pkg: optionally suppress via assumed checksum
This is quite error-prone and causes cache inconsistency similar to the store inconsistency seen on nix when a similar condition happens. Keep this behind a flag in case it is ever beneficial.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 20:33:59 +09:00
591a60bac9 internal/pkg: per-cache SCHED_IDLE
This is cleaner than setting it globally, and is impossible to race.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:49:34 +09:00
5093a06026 internal/pkg: cache flags
This is cleaner for extending the API.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:16:55 +09:00
50c1d7f880 internal/rosa/kernel: 6.12.78 to 6.12.80
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 16:14:16 +09:00
9e63633fbc container: remove test timeouts
These timeouts are no longer useful, and causes spurious test failures under load.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:51:37 +09:00
61f981a34a internal/rosa/perl: 5.42.1 to 5.42.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:25 +09:00
d717c41bbe internal/rosa/cmake: 4.3.0 to 4.3.1
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:45:07 +09:00
b896eec9b7 internal/rosa/gnu: parallel 20260222 to 20260322
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:48 +09:00
8ab99e5e40 internal/rosa/util-linux: 2.41.3 to 2.42
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:25 +09:00
2b6160ef7d internal/rosa/wayland: wayland-protocols 1.47 to 1.48
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:44:04 +09:00
4dcac7f133 internal/rosa/xz: 5.8.2 to 5.8.3
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 10:43:37 +09:00
966fd4df9e internal/rosa: connman artifact
Will be gradually replaced with a native implementation.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:17:10 +09:00
a2cf59b989 internal/rosa/ncurses: also build dynamic library
GNU readline breaks without this.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 01:13:04 +09:00
e87f59c4e4 internal/rosa/gnu: readline artifact
Nice to have library for command line programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:42:59 +09:00
3b221c3e77 internal/rosa/gnu: gnutls artifact
Incredibly ugly and expensive package, but unfortunately required by some packages.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:36:28 +09:00
ff3b385b12 internal/rosa: libunistring artifact
Required by GnuTLS.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-03 00:00:13 +09:00
c6920e6ab7 cmd/mbf: pick up $TERM
This improves behaviour of some programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:59:15 +09:00
59b25d45fe internal/pkg: pick up $TERM if attaching stdin
This improves behaviour of some programs.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:50:07 +09:00
9b99650eb1 internal/rosa: libev artifact
Required by gnutls.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 23:35:01 +09:00
15bff9e1a6 internal/rosa/git: determine reporting name from url
This is generally correct, and is a lot cleaner to call.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:42:16 +09:00
b948525c07 internal/rosa: nettle3 artifact
Removed after all packages upgrade for nettle 4.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:12:39 +09:00
9acbd16e9a internal/rosa/p11: explicitly enable libffi
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-02 22:05:07 +09:00
64e5a1068b internal/rosa: libtasn1 artifact
Optional dependency of p11-kit.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 17:20:12 +09:00
b6cbd49d8c internal/rosa: p11-kit artifact
Another package distributed in xz only. This is fetched from the git remote directly to avoid XZ Utils.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 16:34:30 +09:00
6913b9224a internal/rosa/git: recursively clone submodules
There is generally no reason to disable this, so it was not made optional.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 16:21:46 +09:00
9584958ecc internal/rosa/pkg-config: generate build system
This unfortunately pulls automake, libtool and their dependencies into stage2.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-04-01 15:31:24 +09:00
389844b1ea internal/rosa/gnu: mpc 1.3.1 to 1.4.0
This package now unfortunately switched to xz as well.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 23:56:20 +09:00
5b7ab35633 internal/rosa: iptables artifact
This also pulls in netlink libraries from netfilter project.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 23:47:16 +09:00
52b1a5a725 internal/rosa: use type P in helper interface
This is easier to type and serialises correctly.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 23:45:01 +09:00
6b78df8714 internal/rosa: libmd and libbsd artifacts
These provide headers that are provided by glibc but not musl.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 22:57:08 +09:00
dadf170a46 internal/rosa: dbus artifact
Unfortunate ugly indirect dependency we cannot yet get rid of.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 21:47:05 +09:00
9594832302 internal/rosa/meson: disallow download
This will fail and waste time on KindExec, and cause nondeterminism in KindExecNet.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 21:38:09 +09:00
91a2d4d6e1 internal/uevent: integrate error handling in event loop
There are many subtleties when recovering from errors in the event loop, and coldboot requires internals to drain the receive buffer as synthetic uevents are being arranged.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-31 00:10:14 +09:00
a854719b9f internal/netlink: optional recvmsg without netpoll
For draining the socket receive buffer.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 23:18:43 +09:00
f03c0fb249 internal/uevent: synthetic events for coldboot
This causes the kernel to regenerate events that happened before earlyinit started.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 23:01:08 +09:00
a6600be34a all: use filepath
This makes package check portable, and removes nonportable behaviour from package pkg, pipewire, and system. All other packages remain nonportable due to their nature. No latency increase was observed due to this change on amd64 and arm64 linux.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 18:24:53 +09:00
b5592633f5 internal/uevent: separate recvmsg helper
This enables messages to be received separately.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 02:53:26 +09:00
584e302168 internal/netlink: set receive buffer size
This is done by both systemd sd-device and AOSP ueventd to improve robustness. Rosa OS will still handle ENOBUFS via coldboot but a big buffer should mitigate this as well.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 02:14:47 +09:00
141958656f internal/uevent: handle state divergence
This requires the caller to arrange for a coldboot to happen, some time after this error is encountered, and to resume event processing.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-30 01:50:09 +09:00
648079f42c internal/netlink: switch to recvmsg/sendmsg
These are more flexible than recvfrom/sendto.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-29 23:36:00 +09:00
19c76e0831 cmd: document Rosa OS programs
The earlyinit and mbf program are not covered by the compatibility promise, so specify that here.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 22:43:25 +09:00
71fcc972ba cmd/hsu: alternative hsurc path for Rosa OS
Rosa OS does not have /etc.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 18:22:55 +09:00
62002efd08 cmd/hsu: document hsurc format and internals
This was previously only documented via an unexported function.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 18:17:31 +09:00
e33294db9c cmd/hakurei: document stable behaviour
These are undocumented anywhere else and is required by tools invoking hakurei.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 17:31:46 +09:00
b1ea3b4acf cmd/hakurei: rename app to run
The run command was a legacy holdover from very early days and is only useful for testing and demonstration these days. This change also renames it to exec.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 16:48:26 +09:00
2c254c70b8 cmd/hakurei: remove linkname directive
This used to be a function that did much more, and was later relocated to another package and exported.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 16:20:02 +09:00
ea014d6af2 internal/uevent: consume kernel-originated events
These are not possible to cover outside integration vm. Extreme care is required when dealing with this method, so keep it simple.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 15:39:16 +09:00
1b48484c16 internal/uevent: exclusive socket access
This is a much simplified mutex, since blocking is not required.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 01:01:06 +09:00
713bff3eb0 internal/uevent: decode uevent messages
The wire format and behaviour is entirely undocumented. This is implemented by reading lib/kobject_uevent.c, with testdata collected from the internal/rosa kernel.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 00:49:34 +09:00
30f459e690 internal/uevent: nontrivial errors
These errors are best represented as JSON.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-28 00:07:56 +09:00
8766fddcb3 internal/uevent: recoverable errors
This runs in the Rosa OS init, so recover as much as possible, as otherwise it is likely to require a full system reboot to resume event processing. The caller is responsible for reporting the error.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-27 22:58:16 +09:00
2745602be3 internal/uevent: wrap netlink socket
Unfortunately these messages do not have the same format as rtnetlink.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-27 22:46:18 +09:00
ee22847dde internal/uevent: kobject_action lookup
This is encoded as part of kobject uevent message headers.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-27 22:39:43 +09:00
c61188649b internal/netlink: export generic connection
This enables abstractions around some families to be implemented in a separate package.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-27 19:08:48 +09:00
6a87a96838 internal/rosa/kernel: 6.12.77 to 6.12.78
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-27 10:40:27 +09:00
2548a681e9 internal/rosa: key-value type
This type is used very frequently. The new type is much easier to type and can receive helper methods eventually if needed.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 16:23:15 +09:00
d514d0679f internal/rosa: set PYTHONUNBUFFERED=1
Some python tools try to be clever and buffers output. This makes the build process appear to hang and is quite frustrating. Instead of trying to address this on a case-by-case basis, this is turned off globally for the interpreter.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 15:29:29 +09:00
4407892632 cmd/mbf: optionally enter cure container
This is very useful for troubleshooting failing tests and such. The ephemeral state is cleaned up by internal/pkg.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 15:10:11 +09:00
e661260607 internal/pkg: enter exec container
This enables much easier troubleshooting of failing cures.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 15:05:04 +09:00
044490e0a5 cmd/mbf: retain session by default
This almost never make sense to be turned off.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 14:59:17 +09:00
af038c89ff internal/pkg: collection helper-artifact
This was moved from internal/rosa because it is considered generally useful.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 14:11:10 +09:00
d2f30173cd internal/pkg: isolate container params
This enables exporting container params for interactive troubleshooting within the cure container.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 14:02:58 +09:00
5319ea994c internal/rosa/libseccomp: fix upstream out-of-bounds read
This was revealed by optimisation changes in the latest toolchain.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 10:43:11 +09:00
bbe178be3e internal/rosa/llvm: 22.1.1 to 22.1.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 09:56:34 +09:00
ca28e9936b internal/rosa/musl: 1.2.5 to 1.2.6
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 09:56:06 +09:00
f61c6ade56 internal/rosa/nss: 3.121 to 3.122
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 09:55:27 +09:00
fce3d63823 internal/rosa/gnu: autoconf 2.72 to 2.73
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-26 09:54:44 +09:00
722c3cc54f internal/netlink: optional check header as reply
Not every received message is a reply.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 19:33:01 +09:00
372d509e5c internal/netlink: expose multicast groups
This also gets rid of the cached pid value for port since that prevents multiple sockets from being open at once.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 17:55:35 +09:00
d62516ed1e internal/netlink: enlarge recvfrom buffer
This also uses an array type for the buffer since its size now uses the hardcoded value found in the kernel.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 17:18:56 +09:00
d2b635eb55 cmd/mbf: correctly describe --with-toolchain
The behaviour of this was changed to include the stage2 toolchain instead, but the help text was never updated.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 15:41:28 +09:00
50403e9d60 internal/netlink: wrap netpoll via context
This removes netpoll boilerplate for the most common use case.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 15:39:29 +09:00
b98c5f2e21 internal/netlink: nonblocking socket I/O
This enables use with blocking calls like when used with NETLINK_KOBJECT_UEVENT.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-25 14:06:59 +09:00
d972cffe5a internal/netlink: make full response available
The previous API makes it impossible to retrieve remaining messages in the current iteration.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-23 16:39:25 +09:00
d8648304bb internal/netlink: isolate receive method
This enables use with epoll for receiving events only.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-23 15:03:15 +09:00
f7bfa9a6c2 internal/rosa/go: disable go1.25.7 smtp test
This uses certs that had just expired.

Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 17:52:54 +09:00
7035b4b598 internal/rosa/cmake: 4.2.3 to 4.3.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 16:39:57 +09:00
094b8400dd internal/rosa/qemu: 10.2.1 to 10.2.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 16:13:51 +09:00
4652d921d8 internal/rosa/wayland: 1.24.91 to 1.25.0
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 16:13:28 +09:00
066213c245 internal/rosa/libexpat: 2.7.4 to 2.7.5
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 16:00:50 +09:00
98832c21ee internal/rosa/fuse: 3.18.1 to 3.18.2
Signed-off-by: Ophestra <cat@gensokyo.uk>
2026-03-20 15:57:49 +09:00
123 changed files with 3313 additions and 755 deletions

View File

@@ -5,7 +5,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"path" "path/filepath"
"slices" "slices"
"strings" "strings"
"syscall" "syscall"
@@ -61,7 +61,7 @@ func (a *Absolute) Is(v *Absolute) bool {
// NewAbs checks pathname and returns a new [Absolute] if pathname is absolute. // NewAbs checks pathname and returns a new [Absolute] if pathname is absolute.
func NewAbs(pathname string) (*Absolute, error) { func NewAbs(pathname string) (*Absolute, error) {
if !path.IsAbs(pathname) { if !filepath.IsAbs(pathname) {
return nil, AbsoluteError(pathname) return nil, AbsoluteError(pathname)
} }
return unsafeAbs(pathname), nil return unsafeAbs(pathname), nil
@@ -76,13 +76,13 @@ func MustAbs(pathname string) *Absolute {
} }
} }
// Append calls [path.Join] with [Absolute] as the first element. // Append calls [filepath.Join] with [Absolute] as the first element.
func (a *Absolute) Append(elem ...string) *Absolute { func (a *Absolute) Append(elem ...string) *Absolute {
return unsafeAbs(path.Join(append([]string{a.String()}, elem...)...)) return unsafeAbs(filepath.Join(append([]string{a.String()}, elem...)...))
} }
// Dir calls [path.Dir] with [Absolute] as its argument. // Dir calls [filepath.Dir] with [Absolute] as its argument.
func (a *Absolute) Dir() *Absolute { return unsafeAbs(path.Dir(a.String())) } func (a *Absolute) Dir() *Absolute { return unsafeAbs(filepath.Dir(a.String())) }
// GobEncode returns the checked pathname. // GobEncode returns the checked pathname.
func (a *Absolute) GobEncode() ([]byte, error) { func (a *Absolute) GobEncode() ([]byte, error) {
@@ -92,7 +92,7 @@ func (a *Absolute) GobEncode() ([]byte, error) {
// GobDecode stores data if it represents an absolute pathname. // GobDecode stores data if it represents an absolute pathname.
func (a *Absolute) GobDecode(data []byte) error { func (a *Absolute) GobDecode(data []byte) error {
pathname := string(data) pathname := string(data)
if !path.IsAbs(pathname) { if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname) return AbsoluteError(pathname)
} }
a.pathname = unique.Make(pathname) a.pathname = unique.Make(pathname)
@@ -110,7 +110,7 @@ func (a *Absolute) UnmarshalJSON(data []byte) error {
if err := json.Unmarshal(data, &pathname); err != nil { if err := json.Unmarshal(data, &pathname); err != nil {
return err return err
} }
if !path.IsAbs(pathname) { if !filepath.IsAbs(pathname) {
return AbsoluteError(pathname) return AbsoluteError(pathname)
} }
a.pathname = unique.Make(pathname) a.pathname = unique.Make(pathname)

View File

@@ -1,3 +1,7 @@
// The earlyinit is part of the Rosa OS initramfs and serves as the system init.
//
// This program is an internal detail of Rosa OS and is not usable on its own.
// It is not covered by the compatibility promise.
package main package main
import ( import (

View File

@@ -2,6 +2,7 @@ package main
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"io" "io"
"log" "log"
@@ -11,7 +12,6 @@ import (
"strconv" "strconv"
"sync" "sync"
"time" "time"
_ "unsafe" // for go:linkname
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/command" "hakurei.app/command"
@@ -27,9 +27,14 @@ import (
// optionalErrorUnwrap calls [errors.Unwrap] and returns the resulting value // optionalErrorUnwrap calls [errors.Unwrap] and returns the resulting value
// if it is not nil, or the original value if it is. // if it is not nil, or the original value if it is.
// func optionalErrorUnwrap(err error) error {
//go:linkname optionalErrorUnwrap hakurei.app/container.optionalErrorUnwrap if underlyingErr := errors.Unwrap(err); underlyingErr != nil {
func optionalErrorUnwrap(err error) error return underlyingErr
}
return err
}
var errSuccess = errors.New("success")
func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErrs, out io.Writer) command.Command { func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErrs, out io.Writer) command.Command {
var ( var (
@@ -60,9 +65,9 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
var ( var (
flagIdentifierFile int flagIdentifierFile int
) )
c.NewCommand("app", "Load and start container from configuration file", func(args []string) error { c.NewCommand("run", "Load and start container from configuration file", func(args []string) error {
if len(args) < 1 { if len(args) < 1 {
log.Fatal("app requires at least 1 argument") log.Fatal("run requires at least 1 argument")
} }
config := tryPath(msg, args[0]) config := tryPath(msg, args[0])
@@ -98,7 +103,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
flagWayland, flagX11, flagDBus, flagPipeWire, flagPulse bool flagWayland, flagX11, flagDBus, flagPipeWire, flagPulse bool
) )
c.NewCommand("run", "Configure and start a permissive container", func(args []string) error { c.NewCommand("exec", "Configure and start a permissive container", func(args []string) error {
if flagIdentity < hst.IdentityStart || flagIdentity > hst.IdentityEnd { if flagIdentity < hst.IdentityStart || flagIdentity > hst.IdentityEnd {
log.Fatalf("identity %d out of range", flagIdentity) log.Fatalf("identity %d out of range", flagIdentity)
} }
@@ -323,7 +328,7 @@ func buildCommand(ctx context.Context, msg message.Msg, early *earlyHardeningErr
flagShort bool flagShort bool
flagNoStore bool flagNoStore bool
) )
c.NewCommand("show", "Show live or local app configuration", func(args []string) error { c.NewCommand("show", "Show live or local instance configuration", func(args []string) error {
switch len(args) { switch len(args) {
case 0: // system case 0: // system
printShowSystem(os.Stdout, flagShort, flagJSON) printShowSystem(os.Stdout, flagShort, flagJSON)

View File

@@ -23,9 +23,9 @@ func TestHelp(t *testing.T) {
Usage: hakurei [-h | --help] [-v] [--json] COMMAND [OPTIONS] Usage: hakurei [-h | --help] [-v] [--json] COMMAND [OPTIONS]
Commands: Commands:
app Load and start container from configuration file run Load and start container from configuration file
run Configure and start a permissive container exec Configure and start a permissive container
show Show live or local app configuration show Show live or local instance configuration
ps List active instances ps List active instances
version Display version information version Display version information
license Show full license text license Show full license text
@@ -35,8 +35,8 @@ Commands:
`, `,
}, },
{ {
"run", []string{"run", "-h"}, ` "exec", []string{"exec", "-h"}, `
Usage: hakurei run [-h | --help] [--dbus-config <value>] [--dbus-system <value>] [--mpris] [--dbus-log] [--id <value>] [-a <int>] [-g <value>] [-d <value>] [-u <value>] [--policy <value>] [--priority <int>] [--private-runtime] [--private-tmpdir] [--wayland] [-X] [--dbus] [--pipewire] [--pulse] COMMAND [OPTIONS] Usage: hakurei exec [-h | --help] [--dbus-config <value>] [--dbus-system <value>] [--mpris] [--dbus-log] [--id <value>] [-a <int>] [-g <value>] [-d <value>] [-u <value>] [--policy <value>] [--priority <int>] [--private-runtime] [--private-tmpdir] [--wayland] [-X] [--dbus] [--pipewire] [--pulse] COMMAND [OPTIONS]
Flags: Flags:
-X Enable direct connection to X11 -X Enable direct connection to X11

View File

@@ -1,8 +1,42 @@
// Hakurei runs user-specified containers as subordinate users.
//
// This program is generally invoked by another, higher level program, which
// creates container configuration via package [hst] or an implementation of it.
//
// The parent may leave files open and specify their file descriptor for various
// uses. In these cases, standard streams and netpoll files are treated as
// invalid file descriptors and rejected. All string representations must be in
// decimal.
//
// When specifying a [hst.Config] JSON stream or file to the run subcommand, the
// argument "-" is equivalent to stdin. Otherwise, file descriptor rules
// described above applies. Invalid file descriptors are treated as file names
// in their string representation, with the exception that if a netpoll file
// descriptor is attempted, the program fails.
//
// The flag --identifier-fd can be optionally specified to the run subcommand to
// receive the identifier of the newly started instance. File descriptor rules
// described above applies, and the file must be writable. This is sent after
// its state is made available, so the client must not attempt to poll for it.
// This uses the internal binary format of [hst.ID].
//
// For the show and ps subcommands, the flag --json can be applied to the main
// hakurei command to serialise output in JSON when applicable. Additionally,
// the flag --short targeting each subcommand is used to omit some information
// in both JSON and user-facing output. Only JSON-encoded output is covered
// under the compatibility promise.
//
// A template for [hst.Config] demonstrating all available configuration fields
// is returned by [hst.Template]. The JSON-encoded equivalent of this can be
// obtained via the template subcommand. Fields left unpopulated in the template
// (the direct_* family of fields, which are insecure under any configuration if
// enabled) are unsupported.
//
// For simple (but insecure) testing scenarios, the exec subcommand can be used
// to generate a simple, permissive configuration in-memory. See its help
// message for all available options.
package main package main
// this works around go:embed '..' limitation
//go:generate cp ../../LICENSE .
import ( import (
"context" "context"
_ "embed" _ "embed"
@@ -17,12 +51,9 @@ import (
"hakurei.app/message" "hakurei.app/message"
) )
var ( //go:generate cp ../../LICENSE .
errSuccess = errors.New("success")
//go:embed LICENSE //go:embed LICENSE
license string var license string
)
// earlyHardeningErrs are errors collected while setting up early hardening feature. // earlyHardeningErrs are errors collected while setting up early hardening feature.
type earlyHardeningErrs struct{ yamaLSM, dumpable error } type earlyHardeningErrs struct{ yamaLSM, dumpable error }
@@ -31,8 +62,8 @@ func main() {
// early init path, skips root check and duplicate PR_SET_DUMPABLE // early init path, skips root check and duplicate PR_SET_DUMPABLE
container.TryArgv0(nil) container.TryArgv0(nil)
log.SetPrefix("hakurei: ")
log.SetFlags(0) log.SetFlags(0)
log.SetPrefix("hakurei: ")
msg := message.New(log.Default()) msg := message.New(log.Default())
early := earlyHardeningErrs{ early := earlyHardeningErrs{

View File

@@ -17,8 +17,9 @@ import (
) )
// tryPath attempts to read [hst.Config] from multiple sources. // tryPath attempts to read [hst.Config] from multiple sources.
// tryPath reads from [os.Stdin] if name has value "-". //
// Otherwise, name is passed to tryFd, and if that returns nil, name is passed to [os.Open]. // tryPath reads from [os.Stdin] if name has value "-". Otherwise, name is
// passed to tryFd, and if that returns nil, name is passed to [os.Open].
func tryPath(msg message.Msg, name string) (config *hst.Config) { func tryPath(msg message.Msg, name string) (config *hst.Config) {
var r io.ReadCloser var r io.ReadCloser
config = new(hst.Config) config = new(hst.Config)
@@ -46,7 +47,8 @@ func tryPath(msg message.Msg, name string) (config *hst.Config) {
return return
} }
// tryFd returns a [io.ReadCloser] if name represents an integer corresponding to a valid file descriptor. // tryFd returns a [io.ReadCloser] if name represents an integer corresponding
// to a valid file descriptor.
func tryFd(msg message.Msg, name string) io.ReadCloser { func tryFd(msg message.Msg, name string) io.ReadCloser {
if v, err := strconv.Atoi(name); err != nil { if v, err := strconv.Atoi(name); err != nil {
if !errors.Is(err, strconv.ErrSyntax) { if !errors.Is(err, strconv.ErrSyntax) {
@@ -60,7 +62,12 @@ func tryFd(msg message.Msg, name string) io.ReadCloser {
msg.Verbosef("trying config stream from %d", v) msg.Verbosef("trying config stream from %d", v)
fd := uintptr(v) fd := uintptr(v)
if _, _, errno := syscall.Syscall(syscall.SYS_FCNTL, fd, syscall.F_GETFD, 0); errno != 0 { if _, _, errno := syscall.Syscall(
syscall.SYS_FCNTL,
fd,
syscall.F_GETFD,
0,
); errno != 0 {
if errors.Is(errno, syscall.EBADF) { // reject bad fd if errors.Is(errno, syscall.EBADF) { // reject bad fd
return nil return nil
} }
@@ -75,10 +82,12 @@ func tryFd(msg message.Msg, name string) io.ReadCloser {
} }
} }
// shortLengthMin is the minimum length a short form identifier can have and still be interpreted as an identifier. // shortLengthMin is the minimum length a short form identifier can have and
// still be interpreted as an identifier.
const shortLengthMin = 1 << 3 const shortLengthMin = 1 << 3
// shortIdentifier returns an eight character short representation of [hst.ID] from its random bytes. // shortIdentifier returns an eight character short representation of [hst.ID]
// from its random bytes.
func shortIdentifier(id *hst.ID) string { func shortIdentifier(id *hst.ID) string {
return shortIdentifierString(id.String()) return shortIdentifierString(id.String())
} }
@@ -88,7 +97,8 @@ func shortIdentifierString(s string) string {
return s[len(hst.ID{}) : len(hst.ID{})+shortLengthMin] return s[len(hst.ID{}) : len(hst.ID{})+shortLengthMin]
} }
// tryIdentifier attempts to match [hst.State] from a [hex] representation of [hst.ID] or a prefix of its lower half. // tryIdentifier attempts to match [hst.State] from a [hex] representation of
// [hst.ID] or a prefix of its lower half.
func tryIdentifier(msg message.Msg, name string, s *store.Store) *hst.State { func tryIdentifier(msg message.Msg, name string, s *store.Store) *hst.State {
const ( const (
likeShort = 1 << iota likeShort = 1 << iota
@@ -96,7 +106,8 @@ func tryIdentifier(msg message.Msg, name string, s *store.Store) *hst.State {
) )
var likely uintptr var likely uintptr
if len(name) >= shortLengthMin && len(name) <= len(hst.ID{}) { // half the hex representation // half the hex representation
if len(name) >= shortLengthMin && len(name) <= len(hst.ID{}) {
// cannot safely decode here due to unknown alignment // cannot safely decode here due to unknown alignment
for _, c := range name { for _, c := range name {
if c >= '0' && c <= '9' { if c >= '0' && c <= '9' {

7
cmd/hsu/conf.go Normal file
View File

@@ -0,0 +1,7 @@
//go:build !rosa
package main
// hsuConfPath is an absolute pathname to the hsu configuration file. Its
// contents are interpreted by parseConfig.
const hsuConfPath = "/etc/hsurc"

7
cmd/hsu/config_rosa.go Normal file
View File

@@ -0,0 +1,7 @@
//go:build rosa
package main
// hsuConfPath is the pathname to the hsu configuration file, specific to
// Rosa OS. Its contents are interpreted by parseConfig.
const hsuConfPath = "/system/etc/hsurc"

View File

@@ -1,6 +1,6 @@
package main package main
/* copied from hst and must never be changed */ /* keep in sync with hst */
const ( const (
userOffset = 100000 userOffset = 100000

View File

@@ -1,13 +1,64 @@
// hsu starts the hakurei shim as the target subordinate user.
//
// The hsu program must be installed with the setuid and setgid bit set, and
// owned by root. A configuration file must be installed at /etc/hsurc with
// permission bits 0400, and owned by root. Each line of the file specifies a
// hakurei userid to kernel uid mapping. A line consists of the decimal string
// representation of the uid of the user wishing to start hakurei containers,
// followed by a space, followed by the decimal string representation of its
// userid. Duplicate uid entries are ignored, with the first occurrence taking
// effect.
//
// For example, to map the kernel uid 1000 to the hakurei user id 0:
//
// 1000 0
//
// # Internals
//
// Hakurei and hsu holds pathnames pointing to each other set at link time. For
// this reason, a distribution of hakurei has fixed installation prefix. Since
// this program is never invoked by the user, behaviour described in the
// following paragraphs are considered an internal detail and not covered by the
// compatibility promise.
//
// After checking credentials, hsu checks via /proc/ the absolute pathname of
// its parent process, and fails if it does not match the hakurei pathname set
// at link time. This is not a security feature: the priv-side is considered
// trusted, and this feature makes no attempt to address the racy nature of
// querying /proc/, or debuggers attached to the parent process. Instead, this
// aims to discourage misuse and reduce confusion if the user accidentally
// stumbles upon this program. It also prevents accidental use of the incorrect
// installation of hsu in some environments.
//
// Since target container environment variables are set up in shim via the
// [container] infrastructure, the environment is used for parameters from the
// parent process.
//
// HAKUREI_SHIM specifies a single byte between '3' and '9' representing the
// setup pipe file descriptor. It is passed as is to the shim process and is the
// only value in the environment of the shim process. Since hsurc is not
// accessible to the parent process, leaving this unset causes hsu to print the
// corresponding hakurei user id of the parent and terminate.
//
// HAKUREI_IDENTITY specifies the identity of the instance being started and is
// used to produce the kernel uid alongside hakurei user id looked up from hsurc.
//
// HAKUREI_GROUPS specifies supplementary groups to inherit from the credentials
// of the parent process in a ' ' separated list of decimal string
// representations of gid. This has the unfortunate consequence of allowing
// users mapped via hsurc to effectively drop group membership, so special care
// must be taken to ensure this does not lead to an increase in access. This is
// not applicable to Rosa OS since unsigned code execution is not permitted
// outside hakurei containers, and is generally nonapplicable to the security
// model of hakurei, where all untrusted code runs within containers.
package main package main
// minimise imports to avoid inadvertently calling init or global variable functions
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"log" "log"
"os" "os"
"path" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strconv" "strconv"
@@ -16,10 +67,13 @@ import (
) )
const ( const (
// envIdentity is the name of the environment variable holding a // envShim is the name of the environment variable holding a single byte
// single byte representing the shim setup pipe file descriptor. // representing the shim setup pipe file descriptor.
envShim = "HAKUREI_SHIM" envShim = "HAKUREI_SHIM"
// envGroups holds a ' ' separated list of string representations of // envIdentity is the name of the environment variable holding a decimal
// string representation of the current application identity.
envIdentity = "HAKUREI_IDENTITY"
// envGroups holds a ' ' separated list of decimal string representations of
// supplementary group gid. Membership requirements are enforced. // supplementary group gid. Membership requirements are enforced.
envGroups = "HAKUREI_GROUPS" envGroups = "HAKUREI_GROUPS"
) )
@@ -35,7 +89,6 @@ func main() {
log.SetFlags(0) log.SetFlags(0)
log.SetPrefix("hsu: ") log.SetPrefix("hsu: ")
log.SetOutput(os.Stderr)
if os.Geteuid() != 0 { if os.Geteuid() != 0 {
log.Fatal("this program must be owned by uid 0 and have the setuid bit set") log.Fatal("this program must be owned by uid 0 and have the setuid bit set")
@@ -49,13 +102,13 @@ func main() {
log.Fatal("this program must not be started by root") log.Fatal("this program must not be started by root")
} }
if !path.IsAbs(hakureiPath) { if !filepath.IsAbs(hakureiPath) {
log.Fatal("this program is compiled incorrectly") log.Fatal("this program is compiled incorrectly")
return return
} }
var toolPath string var toolPath string
pexe := path.Join("/proc", strconv.Itoa(os.Getppid()), "exe") pexe := filepath.Join("/proc", strconv.Itoa(os.Getppid()), "exe")
if p, err := os.Readlink(pexe); err != nil { if p, err := os.Readlink(pexe); err != nil {
log.Fatalf("cannot read parent executable path: %v", err) log.Fatalf("cannot read parent executable path: %v", err)
} else if strings.HasSuffix(p, " (deleted)") { } else if strings.HasSuffix(p, " (deleted)") {
@@ -99,8 +152,6 @@ func main() {
// last possible uid outcome // last possible uid outcome
uidEnd = 999919999 uidEnd = 999919999
) )
// cast to int for use with library functions
uid := int(toUser(userid, identity)) uid := int(toUser(userid, identity))
// final bounds check to catch any bugs // final bounds check to catch any bugs
@@ -136,7 +187,6 @@ func main() {
} }
// careful! users in the allowlist is effectively allowed to drop groups via hsu // careful! users in the allowlist is effectively allowed to drop groups via hsu
if err := syscall.Setresgid(uid, uid, uid); err != nil { if err := syscall.Setresgid(uid, uid, uid); err != nil {
log.Fatalf("cannot set gid: %v", err) log.Fatalf("cannot set gid: %v", err)
} }
@@ -146,10 +196,21 @@ func main() {
if err := syscall.Setresuid(uid, uid, uid); err != nil { if err := syscall.Setresuid(uid, uid, uid); err != nil {
log.Fatalf("cannot set uid: %v", err) log.Fatalf("cannot set uid: %v", err)
} }
if _, _, errno := syscall.AllThreadsSyscall(syscall.SYS_PRCTL, PR_SET_NO_NEW_PRIVS, 1, 0); errno != 0 {
if _, _, errno := syscall.AllThreadsSyscall(
syscall.SYS_PRCTL,
PR_SET_NO_NEW_PRIVS, 1,
0,
); errno != 0 {
log.Fatalf("cannot set no_new_privs flag: %s", errno.Error()) log.Fatalf("cannot set no_new_privs flag: %s", errno.Error())
} }
if err := syscall.Exec(toolPath, []string{"hakurei", "shim"}, []string{envShim + "=" + shimSetupFd}); err != nil {
if err := syscall.Exec(toolPath, []string{
"hakurei",
"shim",
}, []string{
envShim + "=" + shimSetupFd,
}); err != nil {
log.Fatalf("cannot start shim: %v", err) log.Fatalf("cannot start shim: %v", err)
} }

View File

@@ -18,8 +18,9 @@ const (
useridEnd = useridStart + rangeSize - 1 useridEnd = useridStart + rangeSize - 1
) )
// parseUint32Fast parses a string representation of an unsigned 32-bit integer value // parseUint32Fast parses a string representation of an unsigned 32-bit integer
// using the fast path only. This limits the range of values it is defined in. // value using the fast path only. This limits the range of values it is defined
// in but is perfectly adequate for this use case.
func parseUint32Fast(s string) (uint32, error) { func parseUint32Fast(s string) (uint32, error) {
sLen := len(s) sLen := len(s)
if sLen < 1 { if sLen < 1 {
@@ -40,12 +41,14 @@ func parseUint32Fast(s string) (uint32, error) {
return n, nil return n, nil
} }
// parseConfig reads a list of allowed users from r until it encounters puid or [io.EOF]. // parseConfig reads a list of allowed users from r until it encounters puid or
// [io.EOF].
// //
// Each line of the file specifies a hakurei userid to kernel uid mapping. A line consists // Each line of the file specifies a hakurei userid to kernel uid mapping. A
// of the string representation of the uid of the user wishing to start hakurei containers, // line consists of the string representation of the uid of the user wishing to
// followed by a space, followed by the string representation of its userid. Duplicate uid // start hakurei containers, followed by a space, followed by the string
// entries are ignored, with the first occurrence taking effect. // representation of its userid. Duplicate uid entries are ignored, with the
// first occurrence taking effect.
// //
// All string representations are parsed by calling parseUint32Fast. // All string representations are parsed by calling parseUint32Fast.
func parseConfig(r io.Reader, puid uint32) (userid uint32, ok bool, err error) { func parseConfig(r io.Reader, puid uint32) (userid uint32, ok bool, err error) {
@@ -81,10 +84,6 @@ func parseConfig(r io.Reader, puid uint32) (userid uint32, ok bool, err error) {
return useridEnd + 1, false, s.Err() return useridEnd + 1, false, s.Err()
} }
// hsuConfPath is an absolute pathname to the hsu configuration file.
// Its contents are interpreted by parseConfig.
const hsuConfPath = "/etc/hsurc"
// mustParseConfig calls parseConfig to interpret the contents of hsuConfPath, // mustParseConfig calls parseConfig to interpret the contents of hsuConfPath,
// terminating the program if an error is encountered, the syntax is incorrect, // terminating the program if an error is encountered, the syntax is incorrect,
// or the current user is not authorised to use hsu because its uid is missing. // or the current user is not authorised to use hsu because its uid is missing.
@@ -112,10 +111,6 @@ func mustParseConfig(puid int) (userid uint32) {
return return
} }
// envIdentity is the name of the environment variable holding a
// string representation of the current application identity.
var envIdentity = "HAKUREI_IDENTITY"
// mustReadIdentity calls parseUint32Fast to interpret the value stored in envIdentity, // mustReadIdentity calls parseUint32Fast to interpret the value stored in envIdentity,
// terminating the program if the value is not set, malformed, or out of bounds. // terminating the program if the value is not set, malformed, or out of bounds.
func mustReadIdentity() uint32 { func mustReadIdentity() uint32 {

View File

@@ -1,3 +1,15 @@
// The mbf program is a frontend for [hakurei.app/internal/rosa].
//
// This program is not covered by the compatibility promise. The command line
// interface, available packages and their behaviour, and even the on-disk
// format, may change at any time.
//
// # Name
//
// The name mbf stands for maiden's best friend, as a tribute to the DOOM source
// port of [the same name]. This name is a placeholder and is subject to change.
//
// [the same name]: https://www.doomwiki.org/wiki/MBF
package main package main
import ( import (
@@ -60,7 +72,6 @@ func main() {
flagQuiet bool flagQuiet bool
flagCures int flagCures int
flagBase string flagBase string
flagTShift int
flagIdle bool flagIdle bool
) )
c := command.New(os.Stderr, log.Printf, "mbf", func([]string) (err error) { c := command.New(os.Stderr, log.Printf, "mbf", func([]string) (err error) {
@@ -77,19 +88,12 @@ func main() {
} else if base, err = check.NewAbs(flagBase); err != nil { } else if base, err = check.NewAbs(flagBase); err != nil {
return return
} }
if cache, err = pkg.Open(ctx, msg, flagCures, base); err == nil {
if flagTShift < 0 {
cache.SetThreshold(0)
} else if flagTShift > 31 {
cache.SetThreshold(1 << 31)
} else {
cache.SetThreshold(1 << flagTShift)
}
}
var flags int
if flagIdle { if flagIdle {
pkg.SetSchedIdle = true flags &= pkg.CSchedIdle
} }
cache, err = pkg.Open(ctx, msg, flags, flagCures, base)
return return
}).Flag( }).Flag(
@@ -104,10 +108,6 @@ func main() {
&flagBase, &flagBase,
"d", command.StringFlag("$MBF_CACHE_DIR"), "d", command.StringFlag("$MBF_CACHE_DIR"),
"Directory to store cured artifacts", "Directory to store cured artifacts",
).Flag(
&flagTShift,
"tshift", command.IntFlag(-1),
"Dependency graph size exponent, to the power of 2",
).Flag( ).Flag(
&flagIdle, &flagIdle,
"sched-idle", command.BoolFlag(false), "sched-idle", command.BoolFlag(false),
@@ -436,6 +436,7 @@ func main() {
{ {
var ( var (
flagDump string flagDump string
flagEnter bool
flagExport string flagExport string
) )
c.NewCommand( c.NewCommand(
@@ -445,9 +446,13 @@ func main() {
if len(args) != 1 { if len(args) != 1 {
return errors.New("cure requires 1 argument") return errors.New("cure requires 1 argument")
} }
if p, ok := rosa.ResolveName(args[0]); !ok { p, ok := rosa.ResolveName(args[0])
if !ok {
return fmt.Errorf("unknown artifact %q", args[0]) return fmt.Errorf("unknown artifact %q", args[0])
} else if flagDump == "" { }
switch {
default:
pathname, _, err := cache.Cure(rosa.Std.Load(p)) pathname, _, err := cache.Cure(rosa.Std.Load(p))
if err != nil { if err != nil {
return err return err
@@ -477,7 +482,8 @@ func main() {
} }
return nil return nil
} else {
case flagDump != "":
f, err := os.OpenFile( f, err := os.OpenFile(
flagDump, flagDump,
os.O_WRONLY|os.O_CREATE|os.O_EXCL, os.O_WRONLY|os.O_CREATE|os.O_EXCL,
@@ -493,6 +499,15 @@ func main() {
} }
return f.Close() return f.Close()
case flagEnter:
return cache.EnterExec(
ctx,
rosa.Std.Load(p),
true, os.Stdin, os.Stdout, os.Stderr,
rosa.AbsSystem.Append("bin", "mksh"),
"sh",
)
} }
}, },
). ).
@@ -505,6 +520,11 @@ func main() {
&flagExport, &flagExport,
"export", command.StringFlag(""), "export", command.StringFlag(""),
"Export cured artifact to specified pathname", "Export cured artifact to specified pathname",
).
Flag(
&flagEnter,
"enter", command.BoolFlag(false),
"Enter cure container with an interactive shell",
) )
} }
@@ -527,7 +547,7 @@ func main() {
} }
presets[i] = p presets[i] = p
} }
root := make(rosa.Collect, 0, 6+len(args)) root := make(pkg.Collect, 0, 6+len(args))
root = rosa.Std.AppendPresets(root, presets...) root = rosa.Std.AppendPresets(root, presets...)
if flagWithToolchain { if flagWithToolchain {
@@ -543,7 +563,7 @@ func main() {
if _, _, err := cache.Cure(&root); err == nil { if _, _, err := cache.Cure(&root); err == nil {
return errors.New("unreachable") return errors.New("unreachable")
} else if !errors.Is(err, rosa.Collected{}) { } else if !pkg.IsCollected(err) {
return err return err
} }
@@ -586,6 +606,9 @@ func main() {
z.Hostname = "localhost" z.Hostname = "localhost"
z.Uid, z.Gid = (1<<10)-1, (1<<10)-1 z.Uid, z.Gid = (1<<10)-1, (1<<10)-1
z.Stdin, z.Stdout, z.Stderr = os.Stdin, os.Stdout, os.Stderr z.Stdin, z.Stdout, z.Stderr = os.Stdin, os.Stdout, os.Stderr
if s, ok := os.LookupEnv("TERM"); ok {
z.Env = append(z.Env, "TERM="+s)
}
var tempdir *check.Absolute var tempdir *check.Absolute
if s, err := filepath.Abs(os.TempDir()); err != nil { if s, err := filepath.Abs(os.TempDir()); err != nil {
@@ -636,13 +659,13 @@ func main() {
). ).
Flag( Flag(
&flagSession, &flagSession,
"session", command.BoolFlag(false), "session", command.BoolFlag(true),
"Retain session", "Retain session",
). ).
Flag( Flag(
&flagWithToolchain, &flagWithToolchain,
"with-toolchain", command.BoolFlag(false), "with-toolchain", command.BoolFlag(false),
"Include the stage3 LLVM toolchain", "Include the stage2 LLVM toolchain",
) )
} }

View File

@@ -24,7 +24,7 @@ import (
"os" "os"
"os/exec" "os/exec"
"os/signal" "os/signal"
"path" "path/filepath"
"runtime" "runtime"
"runtime/cgo" "runtime/cgo"
"strconv" "strconv"
@@ -85,7 +85,10 @@ func destroySetup(private_data unsafe.Pointer) (ok bool) {
} }
//export sharefs_init //export sharefs_init
func sharefs_init(_ *C.struct_fuse_conn_info, cfg *C.struct_fuse_config) unsafe.Pointer { func sharefs_init(
_ *C.struct_fuse_conn_info,
cfg *C.struct_fuse_config,
) unsafe.Pointer {
ctx := C.fuse_get_context() ctx := C.fuse_get_context()
priv := (*C.struct_sharefs_private)(ctx.private_data) priv := (*C.struct_sharefs_private)(ctx.private_data)
setup := cgo.Handle(priv.setup).Value().(*setupState) setup := cgo.Handle(priv.setup).Value().(*setupState)
@@ -103,7 +106,11 @@ func sharefs_init(_ *C.struct_fuse_conn_info, cfg *C.struct_fuse_config) unsafe.
cfg.negative_timeout = 0 cfg.negative_timeout = 0
// all future filesystem operations happen through this dirfd // all future filesystem operations happen through this dirfd
if fd, err := syscall.Open(setup.Source.String(), syscall.O_DIRECTORY|syscall.O_RDONLY|syscall.O_CLOEXEC, 0); err != nil { if fd, err := syscall.Open(
setup.Source.String(),
syscall.O_DIRECTORY|syscall.O_RDONLY|syscall.O_CLOEXEC,
0,
); err != nil {
log.Printf("cannot open %q: %v", setup.Source, err) log.Printf("cannot open %q: %v", setup.Source, err)
goto fail goto fail
} else if err = syscall.Fchdir(fd); err != nil { } else if err = syscall.Fchdir(fd); err != nil {
@@ -138,9 +145,9 @@ func sharefs_destroy(private_data unsafe.Pointer) {
func showHelp(args *fuseArgs) { func showHelp(args *fuseArgs) {
executableName := sharefsName executableName := sharefsName
if args.argc > 0 { if args.argc > 0 {
executableName = path.Base(C.GoString(*args.argv)) executableName = filepath.Base(C.GoString(*args.argv))
} else if name, err := os.Executable(); err == nil { } else if name, err := os.Executable(); err == nil {
executableName = path.Base(name) executableName = filepath.Base(name)
} }
fmt.Printf("usage: %s [options] <mountpoint>\n\n", executableName) fmt.Printf("usage: %s [options] <mountpoint>\n\n", executableName)
@@ -169,8 +176,11 @@ func parseOpts(args *fuseArgs, setup *setupState, log *log.Logger) (ok bool) {
// Decimal string representation of gid to set when running as root. // Decimal string representation of gid to set when running as root.
setgid *C.char setgid *C.char
// Decimal string representation of open file descriptor to read setupState from. // Decimal string representation of open file descriptor to read
// This is an internal detail for containerisation and must not be specified directly. // setupState from.
//
// This is an internal detail for containerisation and must not be
// specified directly.
setup *C.char setup *C.char
} }
@@ -253,7 +263,8 @@ func parseOpts(args *fuseArgs, setup *setupState, log *log.Logger) (ok bool) {
return true return true
} }
// copyArgs returns a heap allocated copy of an argument slice in fuse_args representation. // copyArgs returns a heap allocated copy of an argument slice in fuse_args
// representation.
func copyArgs(s ...string) fuseArgs { func copyArgs(s ...string) fuseArgs {
if len(s) == 0 { if len(s) == 0 {
return fuseArgs{argc: 0, argv: nil, allocated: 0} return fuseArgs{argc: 0, argv: nil, allocated: 0}
@@ -269,6 +280,7 @@ func copyArgs(s ...string) fuseArgs {
func freeArgs(args *fuseArgs) { C.fuse_opt_free_args(args) } func freeArgs(args *fuseArgs) { C.fuse_opt_free_args(args) }
// unsafeAddArgument adds an argument to fuseArgs via fuse_opt_add_arg. // unsafeAddArgument adds an argument to fuseArgs via fuse_opt_add_arg.
//
// The last byte of arg must be 0. // The last byte of arg must be 0.
func unsafeAddArgument(args *fuseArgs, arg string) { func unsafeAddArgument(args *fuseArgs, arg string) {
C.fuse_opt_add_arg(args, (*C.char)(unsafe.Pointer(unsafe.StringData(arg)))) C.fuse_opt_add_arg(args, (*C.char)(unsafe.Pointer(unsafe.StringData(arg))))
@@ -288,8 +300,8 @@ func _main(s ...string) (exitCode int) {
args := copyArgs(s...) args := copyArgs(s...)
defer freeArgs(&args) defer freeArgs(&args)
// this causes the kernel to enforce access control based on // this causes the kernel to enforce access control based on struct stat
// struct stat populated by sharefs_getattr // populated by sharefs_getattr
unsafeAddArgument(&args, "-odefault_permissions\x00") unsafeAddArgument(&args, "-odefault_permissions\x00")
var priv C.struct_sharefs_private var priv C.struct_sharefs_private
@@ -453,7 +465,10 @@ func _main(s ...string) (exitCode int) {
z.Stdin, z.Stdout, z.Stderr = os.Stdin, os.Stdout, os.Stderr z.Stdin, z.Stdout, z.Stderr = os.Stdin, os.Stdout, os.Stderr
} }
z.Bind(z.Path, z.Path, 0) z.Bind(z.Path, z.Path, 0)
setup.Fuse = int(proc.ExtraFileSlice(&z.ExtraFiles, os.NewFile(uintptr(C.fuse_session_fd(se)), "fuse"))) setup.Fuse = int(proc.ExtraFileSlice(
&z.ExtraFiles,
os.NewFile(uintptr(C.fuse_session_fd(se)), "fuse"),
))
var setupWriter io.WriteCloser var setupWriter io.WriteCloser
if fd, w, err := container.Setup(&z.ExtraFiles); err != nil { if fd, w, err := container.Setup(&z.ExtraFiles); err != nil {

View File

@@ -1,3 +1,10 @@
// The sharefs FUSE filesystem is a permissionless shared filesystem.
//
// This filesystem is the primary means of file sharing between hakurei
// application containers. It serves the same purpose in Rosa OS as /sdcard
// does in AOSP.
//
// See help message for all available options.
package main package main
import ( import (

View File

@@ -16,7 +16,6 @@ import (
"strings" "strings"
"syscall" "syscall"
"testing" "testing"
"time"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/command" "hakurei.app/command"
@@ -436,11 +435,8 @@ func TestContainer(t *testing.T) {
wantOps, wantOpsCtx := tc.ops(t) wantOps, wantOpsCtx := tc.ops(t)
wantMnt := tc.mnt(t, wantOpsCtx) wantMnt := tc.mnt(t, wantOpsCtx)
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout)
defer cancel()
var libPaths []*check.Absolute var libPaths []*check.Absolute
c := helperNewContainerLibPaths(ctx, &libPaths, "container", strconv.Itoa(i)) c := helperNewContainerLibPaths(t.Context(), &libPaths, "container", strconv.Itoa(i))
c.Uid = tc.uid c.Uid = tc.uid
c.Gid = tc.gid c.Gid = tc.gid
c.Hostname = hostnameFromTestCase(tc.name) c.Hostname = hostnameFromTestCase(tc.name)
@@ -450,7 +446,6 @@ func TestContainer(t *testing.T) {
} else { } else {
c.Stdout, c.Stderr = os.Stdout, os.Stderr c.Stdout, c.Stderr = os.Stdout, os.Stderr
} }
c.WaitDelay = helperDefaultTimeout
*c.Ops = append(*c.Ops, *wantOps...) *c.Ops = append(*c.Ops, *wantOps...)
c.SeccompRules = tc.rules c.SeccompRules = tc.rules
c.SeccompFlags = tc.flags | seccomp.AllowMultiarch c.SeccompFlags = tc.flags | seccomp.AllowMultiarch
@@ -553,11 +548,10 @@ func testContainerCancel(
) func(t *testing.T) { ) func(t *testing.T) {
return func(t *testing.T) { return func(t *testing.T) {
t.Parallel() t.Parallel()
ctx, cancel := context.WithTimeout(t.Context(), helperDefaultTimeout) ctx, cancel := context.WithCancel(t.Context())
c := helperNewContainer(ctx, "block") c := helperNewContainer(ctx, "block")
c.Stdout, c.Stderr = os.Stdout, os.Stderr c.Stdout, c.Stderr = os.Stdout, os.Stderr
c.WaitDelay = helperDefaultTimeout
if containerExtra != nil { if containerExtra != nil {
containerExtra(c) containerExtra(c)
} }
@@ -738,7 +732,6 @@ func init() {
const ( const (
envDoCheck = "HAKUREI_TEST_DO_CHECK" envDoCheck = "HAKUREI_TEST_DO_CHECK"
helperDefaultTimeout = 5 * time.Second
helperInnerPath = "/usr/bin/helper" helperInnerPath = "/usr/bin/helper"
) )

View File

@@ -1,6 +1,7 @@
package container package container
import ( import (
"context"
"io" "io"
"io/fs" "io/fs"
"net" "net"
@@ -66,7 +67,7 @@ type syscallDispatcher interface {
// ensureFile provides ensureFile. // ensureFile provides ensureFile.
ensureFile(name string, perm, pperm os.FileMode) error ensureFile(name string, perm, pperm os.FileMode) error
// mustLoopback provides mustLoopback. // mustLoopback provides mustLoopback.
mustLoopback(msg message.Msg) mustLoopback(ctx context.Context, msg message.Msg)
// seccompLoad provides [seccomp.Load]. // seccompLoad provides [seccomp.Load].
seccompLoad(rules []std.NativeRule, flags seccomp.ExportFlag) error seccompLoad(rules []std.NativeRule, flags seccomp.ExportFlag) error
@@ -170,7 +171,7 @@ func (k direct) mountTmpfs(fsname, target string, flags uintptr, size int, perm
func (direct) ensureFile(name string, perm, pperm os.FileMode) error { func (direct) ensureFile(name string, perm, pperm os.FileMode) error {
return ensureFile(name, perm, pperm) return ensureFile(name, perm, pperm)
} }
func (direct) mustLoopback(msg message.Msg) { func (direct) mustLoopback(ctx context.Context, msg message.Msg) {
var lo int var lo int
if ifi, err := net.InterfaceByName("lo"); err != nil { if ifi, err := net.InterfaceByName("lo"); err != nil {
msg.GetLogger().Fatalln(err) msg.GetLogger().Fatalln(err)
@@ -178,7 +179,7 @@ func (direct) mustLoopback(msg message.Msg) {
lo = ifi.Index lo = ifi.Index
} }
c, err := netlink.DialRoute() c, err := netlink.DialRoute(0)
if err != nil { if err != nil {
msg.GetLogger().Fatalln(err) msg.GetLogger().Fatalln(err)
} }
@@ -199,11 +200,14 @@ func (direct) mustLoopback(msg message.Msg) {
msg.GetLogger().Fatalf("RTNETLINK answers: %v", err) msg.GetLogger().Fatalf("RTNETLINK answers: %v", err)
default: default:
msg.GetLogger().Fatalf("RTNETLINK answers with malformed message") if err == context.DeadlineExceeded || err == context.Canceled {
msg.GetLogger().Fatalf("interrupted RTNETLINK operation")
}
msg.GetLogger().Fatal("RTNETLINK answers with malformed message")
} }
} }
must(c.SendNewaddrLo(uint32(lo))) must(c.SendNewaddrLo(ctx, uint32(lo)))
must(c.SendIfInfomsg(syscall.RTM_NEWLINK, 0, &syscall.IfInfomsg{ must(c.SendIfInfomsg(ctx, syscall.RTM_NEWLINK, 0, &syscall.IfInfomsg{
Family: syscall.AF_UNSPEC, Family: syscall.AF_UNSPEC,
Index: int32(lo), Index: int32(lo),
Flags: syscall.IFF_UP, Flags: syscall.IFF_UP,

View File

@@ -2,6 +2,7 @@ package container
import ( import (
"bytes" "bytes"
"context"
"fmt" "fmt"
"io" "io"
"io/fs" "io/fs"
@@ -468,7 +469,7 @@ func (k *kstub) ensureFile(name string, perm, pperm os.FileMode) error {
stub.CheckArg(k.Stub, "pperm", pperm, 2)) stub.CheckArg(k.Stub, "pperm", pperm, 2))
} }
func (*kstub) mustLoopback(message.Msg) { /* noop */ } func (*kstub) mustLoopback(context.Context, message.Msg) { /* noop */ }
func (k *kstub) seccompLoad(rules []std.NativeRule, flags seccomp.ExportFlag) error { func (k *kstub) seccompLoad(rules []std.NativeRule, flags seccomp.ExportFlag) error {
k.Helper() k.Helper()

View File

@@ -7,7 +7,8 @@ import (
"log" "log"
"os" "os"
"os/exec" "os/exec"
"path" "os/signal"
"path/filepath"
"slices" "slices"
"strconv" "strconv"
"sync" "sync"
@@ -175,7 +176,11 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
} }
if !params.HostNet { if !params.HostNet {
k.mustLoopback(msg) ctx, cancel := signal.NotifyContext(context.Background(), CancelSignal,
os.Interrupt, SIGTERM, SIGQUIT)
defer cancel() // for panics
k.mustLoopback(ctx, msg)
cancel()
} }
// write uid/gid map here so parent does not need to set dumpable // write uid/gid map here so parent does not need to set dumpable
@@ -490,7 +495,7 @@ func initEntrypoint(k syscallDispatcher, msg message.Msg) {
case s := <-sig: case s := <-sig:
if s == CancelSignal && params.ForwardCancel && cmd.Process != nil { if s == CancelSignal && params.ForwardCancel && cmd.Process != nil {
msg.Verbose("forwarding context cancellation") msg.Verbose("forwarding context cancellation")
if err := k.signal(cmd, os.Interrupt); err != nil { if err := k.signal(cmd, os.Interrupt); err != nil && !errors.Is(err, os.ErrProcessDone) {
k.printf(msg, "cannot forward cancellation: %v", err) k.printf(msg, "cannot forward cancellation: %v", err)
} }
continue continue
@@ -564,7 +569,7 @@ func TryArgv0(msg message.Msg) {
msg = message.New(log.Default()) msg = message.New(log.Default())
} }
if len(os.Args) > 0 && path.Base(os.Args[0]) == initName { if len(os.Args) > 0 && filepath.Base(os.Args[0]) == initName {
Init(msg) Init(msg)
msg.BeforeExit() msg.BeforeExit()
os.Exit(0) os.Exit(0)

View File

@@ -3,7 +3,7 @@ package container
import ( import (
"encoding/gob" "encoding/gob"
"fmt" "fmt"
"path" "path/filepath"
. "syscall" . "syscall"
"hakurei.app/check" "hakurei.app/check"
@@ -46,7 +46,7 @@ func (d *MountDevOp) apply(state *setupState, k syscallDispatcher) error {
} }
for _, name := range []string{"null", "zero", "full", "random", "urandom", "tty"} { for _, name := range []string{"null", "zero", "full", "random", "urandom", "tty"} {
targetPath := path.Join(target, name) targetPath := filepath.Join(target, name)
if err := k.ensureFile(targetPath, 0444, state.ParentPerm); err != nil { if err := k.ensureFile(targetPath, 0444, state.ParentPerm); err != nil {
return err return err
} }
@@ -62,7 +62,7 @@ func (d *MountDevOp) apply(state *setupState, k syscallDispatcher) error {
for i, name := range []string{"stdin", "stdout", "stderr"} { for i, name := range []string{"stdin", "stdout", "stderr"} {
if err := k.symlink( if err := k.symlink(
fhs.Proc+"self/fd/"+string(rune(i+'0')), fhs.Proc+"self/fd/"+string(rune(i+'0')),
path.Join(target, name), filepath.Join(target, name),
); err != nil { ); err != nil {
return err return err
} }
@@ -72,13 +72,13 @@ func (d *MountDevOp) apply(state *setupState, k syscallDispatcher) error {
{fhs.Proc + "kcore", "core"}, {fhs.Proc + "kcore", "core"},
{"pts/ptmx", "ptmx"}, {"pts/ptmx", "ptmx"},
} { } {
if err := k.symlink(pair[0], path.Join(target, pair[1])); err != nil { if err := k.symlink(pair[0], filepath.Join(target, pair[1])); err != nil {
return err return err
} }
} }
devShmPath := path.Join(target, "shm") devShmPath := filepath.Join(target, "shm")
devPtsPath := path.Join(target, "pts") devPtsPath := filepath.Join(target, "pts")
for _, name := range []string{devShmPath, devPtsPath} { for _, name := range []string{devShmPath, devPtsPath} {
if err := k.mkdir(name, state.ParentPerm); err != nil { if err := k.mkdir(name, state.ParentPerm); err != nil {
return err return err
@@ -92,7 +92,7 @@ func (d *MountDevOp) apply(state *setupState, k syscallDispatcher) error {
if state.RetainSession { if state.RetainSession {
if k.isatty(Stdout) { if k.isatty(Stdout) {
consolePath := path.Join(target, "console") consolePath := filepath.Join(target, "console")
if err := k.ensureFile(consolePath, 0444, state.ParentPerm); err != nil { if err := k.ensureFile(consolePath, 0444, state.ParentPerm); err != nil {
return err return err
} }
@@ -110,7 +110,7 @@ func (d *MountDevOp) apply(state *setupState, k syscallDispatcher) error {
} }
if d.Mqueue { if d.Mqueue {
mqueueTarget := path.Join(target, "mqueue") mqueueTarget := filepath.Join(target, "mqueue")
if err := k.mkdir(mqueueTarget, state.ParentPerm); err != nil { if err := k.mkdir(mqueueTarget, state.ParentPerm); err != nil {
return err return err
} }

View File

@@ -3,7 +3,7 @@ package container
import ( import (
"encoding/gob" "encoding/gob"
"fmt" "fmt"
"path" "path/filepath"
"hakurei.app/check" "hakurei.app/check"
) )
@@ -30,7 +30,7 @@ func (l *SymlinkOp) Valid() bool { return l != nil && l.Target != nil && l.LinkN
func (l *SymlinkOp) early(_ *setupState, k syscallDispatcher) error { func (l *SymlinkOp) early(_ *setupState, k syscallDispatcher) error {
if l.Dereference { if l.Dereference {
if !path.IsAbs(l.LinkName) { if !filepath.IsAbs(l.LinkName) {
return check.AbsoluteError(l.LinkName) return check.AbsoluteError(l.LinkName)
} }
if name, err := k.readlink(l.LinkName); err != nil { if name, err := k.readlink(l.LinkName); err != nil {
@@ -44,7 +44,7 @@ func (l *SymlinkOp) early(_ *setupState, k syscallDispatcher) error {
func (l *SymlinkOp) apply(state *setupState, k syscallDispatcher) error { func (l *SymlinkOp) apply(state *setupState, k syscallDispatcher) error {
target := toSysroot(l.Target.String()) target := toSysroot(l.Target.String())
if err := k.mkdirAll(path.Dir(target), state.ParentPerm); err != nil { if err := k.mkdirAll(filepath.Dir(target), state.ParentPerm); err != nil {
return err return err
} }
return k.symlink(l.LinkName, target) return k.symlink(l.LinkName, target)

View File

@@ -4,7 +4,7 @@ import (
"errors" "errors"
"io/fs" "io/fs"
"os" "os"
"path" "path/filepath"
"strconv" "strconv"
"strings" "strings"
"syscall" "syscall"
@@ -29,16 +29,16 @@ const (
func toSysroot(name string) string { func toSysroot(name string) string {
name = strings.TrimLeftFunc(name, func(r rune) bool { return r == '/' }) name = strings.TrimLeftFunc(name, func(r rune) bool { return r == '/' })
return path.Join(sysrootPath, name) return filepath.Join(sysrootPath, name)
} }
func toHost(name string) string { func toHost(name string) string {
name = strings.TrimLeftFunc(name, func(r rune) bool { return r == '/' }) name = strings.TrimLeftFunc(name, func(r rune) bool { return r == '/' })
return path.Join(hostPath, name) return filepath.Join(hostPath, name)
} }
func createFile(name string, perm, pperm os.FileMode, content []byte) error { func createFile(name string, perm, pperm os.FileMode, content []byte) error {
if err := os.MkdirAll(path.Dir(name), pperm); err != nil { if err := os.MkdirAll(filepath.Dir(name), pperm); err != nil {
return err return err
} }
f, err := os.OpenFile(name, syscall.O_CREAT|syscall.O_EXCL|syscall.O_WRONLY, perm) f, err := os.OpenFile(name, syscall.O_CREAT|syscall.O_EXCL|syscall.O_WRONLY, perm)

View File

@@ -4,7 +4,7 @@ import (
"io" "io"
"math" "math"
"os" "os"
"path" "path/filepath"
"reflect" "reflect"
"syscall" "syscall"
"testing" "testing"
@@ -61,7 +61,7 @@ func TestCreateFile(t *testing.T) {
Path: "/proc/nonexistent", Path: "/proc/nonexistent",
Err: syscall.ENOENT, Err: syscall.ENOENT,
} }
if err := createFile(path.Join(Nonexistent, ":3"), 0644, 0755, nil); !reflect.DeepEqual(err, wantErr) { if err := createFile(filepath.Join(Nonexistent, ":3"), 0644, 0755, nil); !reflect.DeepEqual(err, wantErr) {
t.Errorf("createFile: error = %#v, want %#v", err, wantErr) t.Errorf("createFile: error = %#v, want %#v", err, wantErr)
} }
}) })
@@ -72,7 +72,7 @@ func TestCreateFile(t *testing.T) {
Path: "/proc/nonexistent", Path: "/proc/nonexistent",
Err: syscall.ENOENT, Err: syscall.ENOENT,
} }
if err := createFile(path.Join(Nonexistent), 0644, 0755, nil); !reflect.DeepEqual(err, wantErr) { if err := createFile(filepath.Join(Nonexistent), 0644, 0755, nil); !reflect.DeepEqual(err, wantErr) {
t.Errorf("createFile: error = %#v, want %#v", err, wantErr) t.Errorf("createFile: error = %#v, want %#v", err, wantErr)
} }
}) })
@@ -80,7 +80,7 @@ func TestCreateFile(t *testing.T) {
t.Run("touch", func(t *testing.T) { t.Run("touch", func(t *testing.T) {
tempDir := t.TempDir() tempDir := t.TempDir()
pathname := path.Join(tempDir, "empty") pathname := filepath.Join(tempDir, "empty")
if err := createFile(pathname, 0644, 0755, nil); err != nil { if err := createFile(pathname, 0644, 0755, nil); err != nil {
t.Fatalf("createFile: error = %v", err) t.Fatalf("createFile: error = %v", err)
} }
@@ -93,7 +93,7 @@ func TestCreateFile(t *testing.T) {
t.Run("write", func(t *testing.T) { t.Run("write", func(t *testing.T) {
tempDir := t.TempDir() tempDir := t.TempDir()
pathname := path.Join(tempDir, "zero") pathname := filepath.Join(tempDir, "zero")
if err := createFile(pathname, 0644, 0755, []byte{0}); err != nil { if err := createFile(pathname, 0644, 0755, []byte{0}); err != nil {
t.Fatalf("createFile: error = %v", err) t.Fatalf("createFile: error = %v", err)
} }
@@ -107,7 +107,7 @@ func TestCreateFile(t *testing.T) {
func TestEnsureFile(t *testing.T) { func TestEnsureFile(t *testing.T) {
t.Run("create", func(t *testing.T) { t.Run("create", func(t *testing.T) {
if err := ensureFile(path.Join(t.TempDir(), "ensure"), 0644, 0755); err != nil { if err := ensureFile(filepath.Join(t.TempDir(), "ensure"), 0644, 0755); err != nil {
t.Errorf("ensureFile: error = %v", err) t.Errorf("ensureFile: error = %v", err)
} }
}) })
@@ -115,7 +115,7 @@ func TestEnsureFile(t *testing.T) {
t.Run("stat", func(t *testing.T) { t.Run("stat", func(t *testing.T) {
t.Run("inaccessible", func(t *testing.T) { t.Run("inaccessible", func(t *testing.T) {
tempDir := t.TempDir() tempDir := t.TempDir()
pathname := path.Join(tempDir, "inaccessible") pathname := filepath.Join(tempDir, "inaccessible")
if f, err := os.Create(pathname); err != nil { if f, err := os.Create(pathname); err != nil {
t.Fatalf("Create: error = %v", err) t.Fatalf("Create: error = %v", err)
} else { } else {
@@ -150,7 +150,7 @@ func TestEnsureFile(t *testing.T) {
t.Run("ensure", func(t *testing.T) { t.Run("ensure", func(t *testing.T) {
tempDir := t.TempDir() tempDir := t.TempDir()
pathname := path.Join(tempDir, "ensure") pathname := filepath.Join(tempDir, "ensure")
if f, err := os.Create(pathname); err != nil { if f, err := os.Create(pathname); err != nil {
t.Fatalf("Create: error = %v", err) t.Fatalf("Create: error = %v", err)
} else { } else {
@@ -195,12 +195,12 @@ func TestProcPaths(t *testing.T) {
t.Run("sample", func(t *testing.T) { t.Run("sample", func(t *testing.T) {
tempDir := t.TempDir() tempDir := t.TempDir()
if err := os.MkdirAll(path.Join(tempDir, "proc/self"), 0755); err != nil { if err := os.MkdirAll(filepath.Join(tempDir, "proc/self"), 0755); err != nil {
t.Fatalf("MkdirAll: error = %v", err) t.Fatalf("MkdirAll: error = %v", err)
} }
t.Run("clean", func(t *testing.T) { t.Run("clean", func(t *testing.T) {
if err := os.WriteFile(path.Join(tempDir, "proc/self/mountinfo"), []byte(`15 20 0:3 / /proc rw,relatime - proc /proc rw if err := os.WriteFile(filepath.Join(tempDir, "proc/self/mountinfo"), []byte(`15 20 0:3 / /proc rw,relatime - proc /proc rw
16 20 0:15 / /sys rw,relatime - sysfs /sys rw 16 20 0:15 / /sys rw,relatime - sysfs /sys rw
17 20 0:5 / /dev rw,relatime - devtmpfs udev rw,size=1983516k,nr_inodes=495879,mode=755`), 0644); err != nil { 17 20 0:5 / /dev rw,relatime - devtmpfs udev rw,size=1983516k,nr_inodes=495879,mode=755`), 0644); err != nil {
t.Fatalf("WriteFile: error = %v", err) t.Fatalf("WriteFile: error = %v", err)
@@ -243,8 +243,8 @@ func TestProcPaths(t *testing.T) {
}) })
t.Run("malformed", func(t *testing.T) { t.Run("malformed", func(t *testing.T) {
path.Join(tempDir, "proc/self/mountinfo") filepath.Join(tempDir, "proc/self/mountinfo")
if err := os.WriteFile(path.Join(tempDir, "proc/self/mountinfo"), []byte{0}, 0644); err != nil { if err := os.WriteFile(filepath.Join(tempDir, "proc/self/mountinfo"), []byte{0}, 0644); err != nil {
t.Fatalf("WriteFile: error = %v", err) t.Fatalf("WriteFile: error = %v", err)
} }

10
dist/comp/_hakurei vendored
View File

@@ -1,11 +1,11 @@
#compdef hakurei #compdef hakurei
_hakurei_app() { _hakurei_run() {
__hakurei_files __hakurei_files
return $? return $?
} }
_hakurei_run() { _hakurei_exec() {
_arguments \ _arguments \
'--id[Reverse-DNS style Application identifier, leave empty to inherit instance identifier]:id' \ '--id[Reverse-DNS style Application identifier, leave empty to inherit instance identifier]:id' \
'-a[Application identity]: :_numbers' \ '-a[Application identity]: :_numbers' \
@@ -57,9 +57,9 @@ __hakurei_instances() {
{ {
local -a _hakurei_cmds local -a _hakurei_cmds
_hakurei_cmds=( _hakurei_cmds=(
"app:Load and start container from configuration file" "run:Load and start container from configuration file"
"run:Configure and start a permissive container" "exec:Configure and start a permissive container"
"show:Show live or local app configuration" "show:Show live or local instance configuration"
"ps:List active instances" "ps:List active instances"
"version:Display version information" "version:Display version information"
"license:Show full license text" "license:Show full license text"

12
dist/release.sh vendored
View File

@@ -1,9 +1,15 @@
#!/bin/sh -e #!/bin/sh -e
cd "$(dirname -- "$0")/.." cd "$(dirname -- "$0")/.."
VERSION="${HAKUREI_VERSION:-untagged}" VERSION="${HAKUREI_VERSION:-untagged}"
prefix="${PREFIX:-/usr}"
pname="hakurei-${VERSION}-$(go env GOARCH)" pname="hakurei-${VERSION}-$(go env GOARCH)"
out="${DESTDIR:-dist}/${pname}" out="${DESTDIR:-dist}/${pname}"
destroy_workdir() {
rm -rf "${out}"
}
trap destroy_workdir EXIT
echo '# Preparing distribution files.' echo '# Preparing distribution files.'
mkdir -p "${out}" mkdir -p "${out}"
cp -v "README.md" "dist/hsurc.default" "dist/install.sh" "${out}" cp -v "README.md" "dist/hsurc.default" "dist/install.sh" "${out}"
@@ -15,9 +21,9 @@ go generate ./...
go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w go build -trimpath -v -o "${out}/bin/" -ldflags "-s -w
-buildid= -linkmode external -extldflags=-static -buildid= -linkmode external -extldflags=-static
-X hakurei.app/internal/info.buildVersion=${VERSION} -X hakurei.app/internal/info.buildVersion=${VERSION}
-X hakurei.app/internal/info.hakureiPath=/usr/bin/hakurei -X hakurei.app/internal/info.hakureiPath=${prefix}/bin/hakurei
-X hakurei.app/internal/info.hsuPath=/usr/bin/hsu -X hakurei.app/internal/info.hsuPath=${prefix}/bin/hsu
-X main.hakureiPath=/usr/bin/hakurei" ./... -X main.hakureiPath=${prefix}/bin/hakurei" ./...
echo echo
echo '# Testing hakurei.' echo '# Testing hakurei.'

View File

@@ -56,8 +56,10 @@ type Ops interface {
// ApplyState holds the address of [Ops] and any relevant application state. // ApplyState holds the address of [Ops] and any relevant application state.
type ApplyState struct { type ApplyState struct {
// AutoEtcPrefix is the prefix for [FSBind] in autoetc [FSBind.Special] condition. // Prefix for [FSBind] in autoetc [FSBind.Special] condition.
AutoEtcPrefix string AutoEtcPrefix string
// Whether to skip remounting root.
NoRemountRoot bool
Ops Ops
} }

View File

@@ -2,7 +2,7 @@ package hst
import ( import (
"encoding/gob" "encoding/gob"
"path" "path/filepath"
"hakurei.app/check" "hakurei.app/check"
) )
@@ -28,7 +28,7 @@ func (l *FSLink) Valid() bool {
if l == nil || l.Target == nil || l.Linkname == "" { if l == nil || l.Target == nil || l.Linkname == "" {
return false return false
} }
return !l.Dereference || path.IsAbs(l.Linkname) return !l.Dereference || filepath.IsAbs(l.Linkname)
} }
func (l *FSLink) Path() *check.Absolute { func (l *FSLink) Path() *check.Absolute {

View File

@@ -5,6 +5,7 @@ import (
"strings" "strings"
"hakurei.app/check" "hakurei.app/check"
"hakurei.app/fhs"
) )
func init() { gob.Register(new(FSOverlay)) } func init() { gob.Register(new(FSOverlay)) }
@@ -69,9 +70,12 @@ func (o *FSOverlay) Apply(z *ApplyState) {
return return
} }
if o.Upper != nil && o.Work != nil { // rw if o.Upper != nil && o.Work != nil {
z.Overlay(o.Target, o.Upper, o.Work, o.Lower...) z.Overlay(o.Target, o.Upper, o.Work, o.Lower...)
} else { // ro if o.Target.Is(fhs.AbsRoot) {
z.NoRemountRoot = true
}
} else {
z.OverlayReadonly(o.Target, o.Lower...) z.OverlayReadonly(o.Target, o.Lower...)
} }
} }

View File

@@ -49,5 +49,18 @@ func TestFSOverlay(t *testing.T) {
Lower: ms("/tmp/.src0", "/tmp/.src1"), Lower: ms("/tmp/.src0", "/tmp/.src1"),
}}, m("/mnt/src"), ms("/tmp/.src0", "/tmp/.src1"), }}, m("/mnt/src"), ms("/tmp/.src0", "/tmp/.src1"),
"*/mnt/src:/tmp/.src0:/tmp/.src1"}, "*/mnt/src:/tmp/.src0:/tmp/.src1"},
{"no remount root", &hst.FSOverlay{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}, true, container.Ops{&container.MountOverlayOp{
Target: m("/"),
Lower: ms("/tmp/.src0", "/tmp/.src1"),
Upper: m("/tmp/upper"),
Work: m("/tmp/work"),
}}, m("/"), ms("/tmp/upper", "/tmp/work", "/tmp/.src0", "/tmp/.src1"),
"w*/:/tmp/upper:/tmp/work:/tmp/.src0:/tmp/.src1"},
}) })
} }

View File

@@ -8,7 +8,7 @@ import (
"io" "io"
"os" "os"
"os/exec" "os/exec"
"path" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
"testing" "testing"
@@ -28,7 +28,7 @@ func TestUpdate(t *testing.T) {
t.Skip("acl test skipped") t.Skip("acl test skipped")
} }
testFilePath := path.Join(t.TempDir(), testFileName) testFilePath := filepath.Join(t.TempDir(), testFileName)
if f, err := os.Create(testFilePath); err != nil { if f, err := os.Create(testFilePath); err != nil {
t.Fatalf("Create: error = %v", err) t.Fatalf("Create: error = %v", err)

View File

@@ -2,29 +2,32 @@
package netlink package netlink
import ( import (
"context"
"fmt" "fmt"
"os" "os"
"sync"
"syscall" "syscall"
"time"
"unsafe" "unsafe"
) )
// AF_NETLINK socket is never shared // net/netlink/af_netlink.c
var ( const maxRecvmsgLen = 32768
nlPid uint32
nlPidOnce sync.Once const (
// stateOpen denotes an open conn.
stateOpen uint32 = 1 << iota
) )
// getpid returns a cached pid value. // A Conn represents resources associated to a netlink socket.
func getpid() uint32 { type Conn struct {
nlPidOnce.Do(func() { nlPid = uint32(os.Getpid()) })
return nlPid
}
// A conn represents resources associated to a netlink socket.
type conn struct {
// AF_NETLINK socket. // AF_NETLINK socket.
fd int f *os.File
// For using runtime polling via f.
raw syscall.RawConn
// Port ID assigned by the kernel.
port uint32
// Internal connection status.
state uint32
// Kernel module or netlink group to communicate with. // Kernel module or netlink group to communicate with.
family int family int
// Message sequence number. // Message sequence number.
@@ -33,40 +36,193 @@ type conn struct {
typ, flags uint16 typ, flags uint16
// Outgoing position in buf. // Outgoing position in buf.
pos int pos int
// A page holding incoming and outgoing messages. // Pages holding incoming and outgoing messages.
buf []byte buf [maxRecvmsgLen]byte
// An instant some time after conn was established, but before the first
// I/O operation on f through raw. This serves as a cached deadline to
// cancel blocking I/O.
t time.Time
} }
// dial returns the address of a newly connected conn of specified family. // Dial returns the address of a newly connected generic netlink connection of
func dial(family int) (*conn, error) { // specified family and groups.
var c conn //
// For a nonzero rcvbuf, the socket receive buffer size is set to its absolute
// value via SO_RCVBUF for a positive value, or SO_RCVBUFFORCE for a negative
// value.
func Dial(family int, groups uint32, rcvbuf int64) (*Conn, error) {
var c Conn
if fd, err := syscall.Socket( if fd, err := syscall.Socket(
syscall.AF_NETLINK, syscall.AF_NETLINK,
syscall.SOCK_RAW|syscall.SOCK_CLOEXEC, syscall.SOCK_RAW|syscall.SOCK_NONBLOCK|syscall.SOCK_CLOEXEC,
family, family,
); err != nil { ); err != nil {
return nil, os.NewSyscallError("socket", err) return nil, os.NewSyscallError("socket", err)
} else if err = syscall.Bind(fd, &syscall.SockaddrNetlink{ } else if err = syscall.Bind(fd, &syscall.SockaddrNetlink{
Family: syscall.AF_NETLINK, Family: syscall.AF_NETLINK,
Pid: getpid(), Groups: groups,
}); err != nil { }); err != nil {
_ = syscall.Close(fd) _ = syscall.Close(fd)
return nil, os.NewSyscallError("bind", err) return nil, os.NewSyscallError("bind", err)
} else { } else {
c.fd, c.family = fd, family var addr syscall.Sockaddr
if addr, err = syscall.Getsockname(fd); err != nil {
_ = syscall.Close(fd)
return nil, os.NewSyscallError("getsockname", err)
} }
switch a := addr.(type) {
case *syscall.SockaddrNetlink:
c.port = a.Pid
default: // unreachable
_ = syscall.Close(fd)
return nil, syscall.ENOTRECOVERABLE
}
if rcvbuf != 0 {
opt := syscall.SO_RCVBUF
if rcvbuf < 0 {
opt = syscall.SO_RCVBUFFORCE
rcvbuf = -rcvbuf
}
if err = syscall.SetsockoptInt(
fd,
syscall.SOL_SOCKET,
opt,
int(rcvbuf),
); err != nil {
_ = syscall.Close(fd)
return nil, os.NewSyscallError("setsockopt", err)
}
}
c.family = family
c.f = os.NewFile(uintptr(fd), "netlink")
if c.raw, err = c.f.SyscallConn(); err != nil {
_ = c.f.Close()
return nil, err
}
c.state |= stateOpen
}
c.pos = syscall.NLMSG_HDRLEN c.pos = syscall.NLMSG_HDRLEN
c.buf = make([]byte, os.Getpagesize()) c.t = time.Now().UTC()
return &c, nil return &c, nil
} }
// ok returns whether conn is still open.
func (c *Conn) ok() bool { return c.state&stateOpen != 0 }
// Close closes the underlying socket. // Close closes the underlying socket.
func (c *conn) Close() error { func (c *Conn) Close() error {
if c.buf == nil { if !c.ok() {
return syscall.EINVAL return syscall.EINVAL
} }
c.buf = nil c.state &= ^stateOpen
return syscall.Close(c.fd) return c.f.Close()
}
// Recvmsg wraps recv(2) with nonblocking behaviour via the runtime network poller.
//
// The returned slice is valid until the next call to Recvmsg.
func (c *Conn) Recvmsg(
ctx context.Context,
flags int,
) (data []byte, recvflags int, from syscall.Sockaddr, err error) {
if err = c.f.SetReadDeadline(time.Time{}); err != nil {
return
}
var n int
data = c.buf[:]
if ctx == nil {
rcErr := c.raw.Control(func(fd uintptr) {
n, _, recvflags, from, err = syscall.Recvmsg(int(fd), data, nil, flags)
})
if n >= 0 {
data = data[:n]
}
if err != nil {
err = os.NewSyscallError("recvmsg", err)
} else {
err = rcErr
}
return
}
done := make(chan error, 1)
go func() {
rcErr := c.raw.Read(func(fd uintptr) (done bool) {
n, _, recvflags, from, err = syscall.Recvmsg(int(fd), data, nil, flags)
return err != syscall.EWOULDBLOCK
})
if n >= 0 {
data = data[:n]
}
done <- rcErr
}()
select {
case rcErr := <-done:
if err != nil {
err = os.NewSyscallError("recvmsg", err)
} else {
err = rcErr
}
return
case <-ctx.Done():
cancelErr := c.f.SetReadDeadline(c.t)
<-done
if cancelErr != nil {
err = cancelErr
} else {
err = ctx.Err()
}
return
}
}
// Sendmsg wraps send(2) with nonblocking behaviour via the runtime network poller.
func (c *Conn) Sendmsg(
ctx context.Context,
p []byte,
to syscall.Sockaddr,
flags int,
) (err error) {
if err = c.f.SetWriteDeadline(time.Time{}); err != nil {
return
}
done := make(chan error, 1)
go func() {
done <- c.raw.Write(func(fd uintptr) (done bool) {
err = syscall.Sendmsg(int(fd), p, nil, to, flags)
return err != syscall.EWOULDBLOCK
})
}()
select {
case rcErr := <-done:
if err != nil {
err = os.NewSyscallError("sendmsg", err)
} else {
err = rcErr
}
return
case <-ctx.Done():
cancelErr := c.f.SetWriteDeadline(c.t)
<-done
if cancelErr != nil {
err = cancelErr
} else {
err = ctx.Err()
}
return
}
} }
// Msg is type constraint for types sent over the wire via netlink. // Msg is type constraint for types sent over the wire via netlink.
@@ -88,7 +244,7 @@ func As[M Msg](data []byte) *M {
} }
// add queues a value to be sent by conn. // add queues a value to be sent by conn.
func add[M Msg](c *conn, p *M) bool { func add[M Msg](c *Conn, p *M) bool {
pos := c.pos pos := c.pos
c.pos += int(unsafe.Sizeof(*p)) c.pos += int(unsafe.Sizeof(*p))
if c.pos > len(c.buf) { if c.pos > len(c.buf) {
@@ -122,8 +278,16 @@ func (e *InconsistentError) Error() string {
return s return s
} }
// checkReply checks the message header of a reply from the kernel.
func (c *Conn) checkReply(header *syscall.NlMsghdr) error {
if header.Seq != c.seq || header.Pid != c.port {
return &InconsistentError{*header, c.seq, c.port}
}
return nil
}
// pending returns the valid slice of buf and initialises pos. // pending returns the valid slice of buf and initialises pos.
func (c *conn) pending() []byte { func (c *Conn) pending() []byte {
buf := c.buf[:c.pos] buf := c.buf[:c.pos]
c.pos = syscall.NLMSG_HDRLEN c.pos = syscall.NLMSG_HDRLEN
@@ -132,7 +296,7 @@ func (c *conn) pending() []byte {
Type: c.typ, Type: c.typ,
Flags: c.flags, Flags: c.flags,
Seq: c.seq, Seq: c.seq,
Pid: getpid(), Pid: c.port,
} }
return buf return buf
} }
@@ -143,39 +307,24 @@ type Complete struct{}
// Error returns a hardcoded string that should never be displayed to the user. // Error returns a hardcoded string that should never be displayed to the user.
func (Complete) Error() string { return "returning from roundtrip" } func (Complete) Error() string { return "returning from roundtrip" }
// Roundtrip sends the pending message and handles the reply. // HandlerFunc handles [syscall.NetlinkMessage] and returns a non-nil error to
func (c *conn) Roundtrip(f func(msg *syscall.NetlinkMessage) error) error { // discontinue the receiving of more messages.
if c.buf == nil { type HandlerFunc func(resp []syscall.NetlinkMessage) error
return syscall.EINVAL
}
defer func() { c.seq++ }()
if err := syscall.Sendto(c.fd, c.pending(), 0, &syscall.SockaddrNetlink{
Family: syscall.AF_NETLINK,
}); err != nil {
return os.NewSyscallError("sendto", err)
}
// receive receives from a socket with specified flags until a non-nil error is
// returned by f. An error of type [Complete] is returned as nil.
func (c *Conn) receive(ctx context.Context, f HandlerFunc, flags int) error {
for { for {
buf := c.buf var resp []syscall.NetlinkMessage
if n, _, err := syscall.Recvfrom(c.fd, buf, 0); err != nil { if data, _, _, err := c.Recvmsg(ctx, flags); err != nil {
return os.NewSyscallError("recvfrom", err) return err
} else if n < syscall.NLMSG_HDRLEN { } else if len(data) < syscall.NLMSG_HDRLEN {
return syscall.EBADE return syscall.EBADE
} else { } else if resp, err = syscall.ParseNetlinkMessage(data); err != nil {
buf = buf[:n]
}
msgs, err := syscall.ParseNetlinkMessage(buf)
if err != nil {
return err return err
} }
for _, msg := range msgs { if err := f(resp); err != nil {
if msg.Header.Seq != c.seq || msg.Header.Pid != getpid() {
return &InconsistentError{msg.Header, c.seq, getpid()}
}
if err = f(&msg); err != nil {
if err == (Complete{}) { if err == (Complete{}) {
return nil return nil
} }
@@ -183,4 +332,19 @@ func (c *conn) Roundtrip(f func(msg *syscall.NetlinkMessage) error) error {
} }
} }
} }
// Roundtrip sends the pending message and handles the reply.
func (c *Conn) Roundtrip(ctx context.Context, f HandlerFunc) error {
if !c.ok() {
return syscall.EINVAL
}
defer func() { c.seq++ }()
if err := c.Sendmsg(ctx, c.pending(), &syscall.SockaddrNetlink{
Family: syscall.AF_NETLINK,
}, 0); err != nil {
return err
}
return c.receive(ctx, f, 0)
} }

View File

@@ -1,16 +1,13 @@
package netlink package netlink
import ( import (
"os"
"syscall" "syscall"
"testing" "testing"
) )
func init() { nlPidOnce.Do(func() {}); nlPid = 1 }
type payloadTestCase struct { type payloadTestCase struct {
name string name string
f func(c *conn) f func(c *Conn)
want []byte want []byte
} }
@@ -22,11 +19,9 @@ func checkPayload(t *testing.T, testCases []payloadTestCase) {
for _, tc := range testCases { for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
t.Helper()
c := conn{ c := Conn{port: 1, pos: syscall.NLMSG_HDRLEN}
pos: syscall.NLMSG_HDRLEN,
buf: make([]byte, os.Getpagesize()),
}
tc.f(&c) tc.f(&c)
if got := c.pending(); string(got) != string(tc.want) { if got := c.pending(); string(got) != string(tc.want) {
t.Errorf("pending: %#v, want %#v", got, tc.want) t.Errorf("pending: %#v, want %#v", got, tc.want)

View File

@@ -1,16 +1,20 @@
package netlink package netlink
import ( import (
"context"
"syscall" "syscall"
"unsafe" "unsafe"
) )
// RouteConn represents a NETLINK_ROUTE socket. // RouteConn represents a NETLINK_ROUTE socket.
type RouteConn struct{ *conn } type RouteConn struct{ conn *Conn }
// Close closes the underlying socket.
func (c *RouteConn) Close() error { return c.conn.Close() }
// DialRoute returns the address of a newly connected [RouteConn]. // DialRoute returns the address of a newly connected [RouteConn].
func DialRoute() (*RouteConn, error) { func DialRoute(rcvbuf int64) (*RouteConn, error) {
c, err := dial(syscall.NETLINK_ROUTE) c, err := Dial(syscall.NETLINK_ROUTE, 0, rcvbuf)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -18,24 +22,28 @@ func DialRoute() (*RouteConn, error) {
} }
// rtnlConsume consumes a message from rtnetlink. // rtnlConsume consumes a message from rtnetlink.
func rtnlConsume(msg *syscall.NetlinkMessage) error { func (c *RouteConn) rtnlConsume(resp []syscall.NetlinkMessage) error {
switch msg.Header.Type { for i := range resp {
if err := c.conn.checkReply(&resp[i].Header); err != nil {
return err
}
switch resp[i].Header.Type {
case syscall.NLMSG_DONE: case syscall.NLMSG_DONE:
return Complete{} return Complete{}
case syscall.NLMSG_ERROR: case syscall.NLMSG_ERROR:
if e := As[syscall.NlMsgerr](msg.Data); e != nil { if e := As[syscall.NlMsgerr](resp[i].Data); e != nil {
if e.Error == 0 { if e.Error == 0 {
return Complete{} return Complete{}
} }
return syscall.Errno(-e.Error) return syscall.Errno(-e.Error)
} }
return syscall.EBADE return syscall.EBADE
default:
return nil
} }
} }
return nil
}
// InAddr is equivalent to struct in_addr. // InAddr is equivalent to struct in_addr.
type InAddr [4]byte type InAddr [4]byte
@@ -57,7 +65,7 @@ func (c *RouteConn) writeIfAddrmsg(
msg *syscall.IfAddrmsg, msg *syscall.IfAddrmsg,
attrs ...RtAttrMsg[InAddr], attrs ...RtAttrMsg[InAddr],
) bool { ) bool {
c.typ, c.flags = typ, syscall.NLM_F_REQUEST|syscall.NLM_F_ACK|flags c.conn.typ, c.conn.flags = typ, syscall.NLM_F_REQUEST|syscall.NLM_F_ACK|flags
if !add(c.conn, msg) { if !add(c.conn, msg) {
return false return false
} }
@@ -72,6 +80,7 @@ func (c *RouteConn) writeIfAddrmsg(
// SendIfAddrmsg sends an ifaddrmsg structure to rtnetlink. // SendIfAddrmsg sends an ifaddrmsg structure to rtnetlink.
func (c *RouteConn) SendIfAddrmsg( func (c *RouteConn) SendIfAddrmsg(
ctx context.Context,
typ, flags uint16, typ, flags uint16,
msg *syscall.IfAddrmsg, msg *syscall.IfAddrmsg,
attrs ...RtAttrMsg[InAddr], attrs ...RtAttrMsg[InAddr],
@@ -79,7 +88,7 @@ func (c *RouteConn) SendIfAddrmsg(
if !c.writeIfAddrmsg(typ, flags, msg, attrs...) { if !c.writeIfAddrmsg(typ, flags, msg, attrs...) {
return syscall.ENOMEM return syscall.ENOMEM
} }
return c.Roundtrip(rtnlConsume) return c.conn.Roundtrip(ctx, c.rtnlConsume)
} }
// writeNewaddrLo writes a RTM_NEWADDR message for the loopback address. // writeNewaddrLo writes a RTM_NEWADDR message for the loopback address.
@@ -104,11 +113,11 @@ func (c *RouteConn) writeNewaddrLo(lo uint32) bool {
} }
// SendNewaddrLo sends a RTM_NEWADDR message for the loopback address to the kernel. // SendNewaddrLo sends a RTM_NEWADDR message for the loopback address to the kernel.
func (c *RouteConn) SendNewaddrLo(lo uint32) error { func (c *RouteConn) SendNewaddrLo(ctx context.Context, lo uint32) error {
if !c.writeNewaddrLo(lo) { if !c.writeNewaddrLo(lo) {
return syscall.ENOMEM return syscall.ENOMEM
} }
return c.Roundtrip(rtnlConsume) return c.conn.Roundtrip(ctx, c.rtnlConsume)
} }
// writeIfInfomsg writes an ifinfomsg structure to conn. // writeIfInfomsg writes an ifinfomsg structure to conn.
@@ -116,17 +125,18 @@ func (c *RouteConn) writeIfInfomsg(
typ, flags uint16, typ, flags uint16,
msg *syscall.IfInfomsg, msg *syscall.IfInfomsg,
) bool { ) bool {
c.typ, c.flags = typ, syscall.NLM_F_REQUEST|syscall.NLM_F_ACK|flags c.conn.typ, c.conn.flags = typ, syscall.NLM_F_REQUEST|syscall.NLM_F_ACK|flags
return add(c.conn, msg) return add(c.conn, msg)
} }
// SendIfInfomsg sends an ifinfomsg structure to rtnetlink. // SendIfInfomsg sends an ifinfomsg structure to rtnetlink.
func (c *RouteConn) SendIfInfomsg( func (c *RouteConn) SendIfInfomsg(
ctx context.Context,
typ, flags uint16, typ, flags uint16,
msg *syscall.IfInfomsg, msg *syscall.IfInfomsg,
) error { ) error {
if !c.writeIfInfomsg(typ, flags, msg) { if !c.writeIfInfomsg(typ, flags, msg) {
return syscall.ENOMEM return syscall.ENOMEM
} }
return c.Roundtrip(rtnlConsume) return c.conn.Roundtrip(ctx, c.rtnlConsume)
} }

View File

@@ -9,7 +9,7 @@ func TestPayloadRTNETLINK(t *testing.T) {
t.Parallel() t.Parallel()
checkPayload(t, []payloadTestCase{ checkPayload(t, []payloadTestCase{
{"RTM_NEWADDR lo", func(c *conn) { {"RTM_NEWADDR lo", func(c *Conn) {
(&RouteConn{c}).writeNewaddrLo(1) (&RouteConn{c}).writeNewaddrLo(1)
}, []byte{ }, []byte{
/* Len */ 0x28, 0, 0, 0, /* Len */ 0x28, 0, 0, 0,
@@ -33,7 +33,7 @@ func TestPayloadRTNETLINK(t *testing.T) {
/* in_addr */ 127, 0, 0, 1, /* in_addr */ 127, 0, 0, 1,
}}, }},
{"RTM_NEWLINK", func(c *conn) { {"RTM_NEWLINK", func(c *Conn) {
c.seq++ c.seq++
(&RouteConn{c}).writeIfInfomsg( (&RouteConn{c}).writeIfInfomsg(
syscall.RTM_NEWLINK, 0, syscall.RTM_NEWLINK, 0,

View File

@@ -5,7 +5,7 @@ import (
"errors" "errors"
"io/fs" "io/fs"
"os" "os"
"path" "path/filepath"
"slices" "slices"
"strconv" "strconv"
"syscall" "syscall"
@@ -165,9 +165,9 @@ func (s *spFilesystemOp) toSystem(state *outcomeStateSys) error {
} }
for _, pair := range entry.Values { for _, pair := range entry.Values {
if pair[0] == "path" { if pair[0] == "path" {
if path.IsAbs(pair[1]) { if filepath.IsAbs(pair[1]) {
// get parent dir of socket // get parent dir of socket
dir := path.Dir(pair[1]) dir := filepath.Dir(pair[1])
if dir == "." || dir == fhs.Root { if dir == "." || dir == fhs.Root {
state.msg.Verbosef("dbus socket %q is in an unusual location", pair[1]) state.msg.Verbosef("dbus socket %q is in an unusual location", pair[1])
} }
@@ -290,7 +290,9 @@ func (s *spFilesystemOp) toContainer(state *outcomeStateParams) error {
if state.Container.Flags&hst.FDevice == 0 { if state.Container.Flags&hst.FDevice == 0 {
state.params.Remount(fhs.AbsDev, syscall.MS_RDONLY) state.params.Remount(fhs.AbsDev, syscall.MS_RDONLY)
} }
if !state.as.NoRemountRoot {
state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY) state.params.Remount(fhs.AbsRoot, syscall.MS_RDONLY)
}
state.params.Env = make([]string, 0, len(state.env)) state.params.Env = make([]string, 0, len(state.env))
for key, value := range state.env { for key, value := range state.env {

View File

@@ -20,7 +20,7 @@ import (
"fmt" "fmt"
"io" "io"
"os" "os"
"path" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strconv" "strconv"
@@ -973,23 +973,23 @@ func connectName(name string, manager bool) (conn Conn, err error) {
return connectName(name+"-manager", false) return connectName(name+"-manager", false)
} }
if path.IsAbs(name) || (len(name) > 0 && name[0] == '@') { if filepath.IsAbs(name) || (len(name) > 0 && name[0] == '@') {
return Dial(name) return Dial(name)
} else { } else {
runtimeDir, ok := os.LookupEnv("PIPEWIRE_RUNTIME_DIR") runtimeDir, ok := os.LookupEnv("PIPEWIRE_RUNTIME_DIR")
if !ok || !path.IsAbs(runtimeDir) { if !ok || !filepath.IsAbs(runtimeDir) {
runtimeDir, ok = os.LookupEnv("XDG_RUNTIME_DIR") runtimeDir, ok = os.LookupEnv("XDG_RUNTIME_DIR")
} }
if !ok || !path.IsAbs(runtimeDir) { if !ok || !filepath.IsAbs(runtimeDir) {
// this is cargo culted from windows stuff and has no effect normally; // this is cargo culted from windows stuff and has no effect normally;
// keeping it to maintain compatibility in case someone sets this // keeping it to maintain compatibility in case someone sets this
runtimeDir, ok = os.LookupEnv("USERPROFILE") runtimeDir, ok = os.LookupEnv("USERPROFILE")
} }
if !ok || !path.IsAbs(runtimeDir) { if !ok || !filepath.IsAbs(runtimeDir) {
runtimeDir = DEFAULT_SYSTEM_RUNTIME_DIR runtimeDir = DEFAULT_SYSTEM_RUNTIME_DIR
} }
return Dial(path.Join(runtimeDir, name)) return Dial(filepath.Join(runtimeDir, name))
} }
} }

View File

@@ -27,6 +27,31 @@ func TestFlatten(t *testing.T) {
fs.ModeCharDevice | 0400, fs.ModeCharDevice | 0400,
)}, )},
{"coldboot", fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},
"devices": {Mode: fs.ModeDir | 0700},
"devices/uevent": {Mode: 0600, Data: []byte("add")},
"devices/empty": {Mode: fs.ModeDir | 0700},
"devices/sub": {Mode: fs.ModeDir | 0700},
"devices/sub/uevent": {Mode: 0600, Data: []byte("add")},
"block": {Mode: fs.ModeDir | 0700},
"block/uevent": {Mode: 0600, Data: []byte{}},
}, []pkg.FlatEntry{
{Mode: fs.ModeDir | 0700, Path: "."},
{Mode: fs.ModeDir | 0700, Path: "block"},
{Mode: 0600, Path: "block/uevent", Data: []byte{}},
{Mode: fs.ModeDir | 0700, Path: "devices"},
{Mode: fs.ModeDir | 0700, Path: "devices/empty"},
{Mode: fs.ModeDir | 0700, Path: "devices/sub"},
{Mode: 0600, Path: "devices/sub/uevent", Data: []byte("add")},
{Mode: 0600, Path: "devices/uevent", Data: []byte("add")},
}, pkg.MustDecode("mEy_Lf5KotThm7OwMx7yTKZh5HCCyaB41pVAvI9uDMgVQFM91iosBLYsRm8bDsX8"), nil},
{"empty", fstest.MapFS{ {"empty", fstest.MapFS{
".": {Mode: fs.ModeDir | 0700}, ".": {Mode: fs.ModeDir | 0700},
"checksum": {Mode: fs.ModeDir | 0700}, "checksum": {Mode: fs.ModeDir | 0700},
@@ -159,6 +184,32 @@ func TestFlatten(t *testing.T) {
{Mode: fs.ModeDir | 0700, Path: "work"}, {Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d"), nil}, }, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d"), nil},
{"sample no assume checksum", fstest.MapFS{
".": {Mode: fs.ModeDir | 0700},
"checksum": {Mode: fs.ModeDir | 0700},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M": {Mode: fs.ModeDir | 0500},
"checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check": {Mode: 0400, Data: []byte{}},
"identifier": {Mode: fs.ModeDir | 0700},
"identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA": {Mode: fs.ModeSymlink | 0777, Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
"work": {Mode: fs.ModeDir | 0700},
}, []pkg.FlatEntry{
{Mode: fs.ModeDir | 0700, Path: "."},
{Mode: fs.ModeDir | 0700, Path: "checksum"},
{Mode: fs.ModeDir | 0500, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M"},
{Mode: 0400, Path: "checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M/check", Data: []byte{}},
{Mode: fs.ModeDir | 0700, Path: "identifier"},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeSymlink | 0777, Path: "identifier/_wEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", Data: []byte("../checksum/Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")},
{Mode: fs.ModeDir | 0700, Path: "work"},
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w"), nil},
{"sample tar step unpack", fstest.MapFS{ {"sample tar step unpack", fstest.MapFS{
".": {Mode: fs.ModeDir | 0500}, ".": {Mode: fs.ModeDir | 0500},

View File

@@ -8,7 +8,7 @@ import (
"io" "io"
"os" "os"
"os/exec" "os/exec"
"path" "path/filepath"
"slices" "slices"
"strconv" "strconv"
"syscall" "syscall"
@@ -40,14 +40,14 @@ type ExecPath struct {
W bool W bool
} }
// SetSchedIdle is whether to set [std.SCHED_IDLE] scheduling priority. // GetArtifactFunc is the function signature of [FContext.GetArtifact].
var SetSchedIdle bool type GetArtifactFunc func(Artifact) (*check.Absolute, unique.Handle[Checksum])
// PromoteLayers returns artifacts with identical-by-content layers promoted to // PromoteLayers returns artifacts with identical-by-content layers promoted to
// the highest priority instance, as if mounted via [ExecPath]. // the highest priority instance, as if mounted via [ExecPath].
func PromoteLayers( func PromoteLayers(
artifacts []Artifact, artifacts []Artifact,
getArtifact func(Artifact) (*check.Absolute, unique.Handle[Checksum]), getArtifact GetArtifactFunc,
report func(i int, d Artifact), report func(i int, d Artifact),
) []*check.Absolute { ) []*check.Absolute {
layers := make([]*check.Absolute, 0, len(artifacts)) layers := make([]*check.Absolute, 0, len(artifacts))
@@ -67,14 +67,14 @@ func PromoteLayers(
} }
// layers returns pathnames collected from A deduplicated via [PromoteLayers]. // layers returns pathnames collected from A deduplicated via [PromoteLayers].
func (p *ExecPath) layers(f *FContext) []*check.Absolute { func (p *ExecPath) layers(
msg := f.GetMessage() msg message.Msg,
return PromoteLayers(p.A, f.GetArtifact, func(i int, d Artifact) { getArtifact GetArtifactFunc,
ident func(a Artifact) unique.Handle[ID],
) []*check.Absolute {
return PromoteLayers(p.A, getArtifact, func(i int, d Artifact) {
if msg.IsVerbose() { if msg.IsVerbose() {
msg.Verbosef( msg.Verbosef("promoted layer %d as %s", i, reportName(d, ident(d)))
"promoted layer %d as %s",
i, reportName(d, f.cache.Ident(d)),
)
} }
}) })
} }
@@ -186,7 +186,7 @@ func NewExec(
paths ...ExecPath, paths ...ExecPath,
) Artifact { ) Artifact {
if name == "" { if name == "" {
name = "exec-" + path.Base(pathname.String()) name = "exec-" + filepath.Base(pathname.String())
} }
if timeout <= 0 { if timeout <= 0 {
timeout = ExecTimeoutDefault timeout = ExecTimeoutDefault
@@ -382,17 +382,30 @@ func scanVerbose(
} }
} }
var (
// ErrInvalidPaths is returned for an [Artifact] of [KindExec] or
// [KindExecNet] specified with invalid paths.
ErrInvalidPaths = errors.New("invalid mount point")
)
// SeccompPresets is the [seccomp] presets used by exec artifacts. // SeccompPresets is the [seccomp] presets used by exec artifacts.
const SeccompPresets = std.PresetStrict & const SeccompPresets = std.PresetStrict &
^(std.PresetDenyNS | std.PresetDenyDevel) ^(std.PresetDenyNS | std.PresetDenyDevel)
// cure is like Cure but allows optional host net namespace. This is used for // makeContainer sets up the specified temp and work directories and returns the
// the [KnownChecksum] variant where networking is allowed. // corresponding [container.Container] that would have run for cure.
func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) { func (a *execArtifact) makeContainer(
ctx context.Context,
msg message.Msg,
hostNet bool,
temp, work *check.Absolute,
getArtifact GetArtifactFunc,
ident func(a Artifact) unique.Handle[ID],
) (z *container.Container, err error) {
overlayWorkIndex := -1 overlayWorkIndex := -1
for i, p := range a.paths { for i, p := range a.paths {
if p.P == nil || len(p.A) == 0 { if p.P == nil || len(p.A) == 0 {
return os.ErrInvalid return nil, ErrInvalidPaths
} }
if p.P.Is(AbsWork) { if p.P.Is(AbsWork) {
overlayWorkIndex = i overlayWorkIndex = i
@@ -404,29 +417,202 @@ func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
artifactCount += len(p.A) artifactCount += len(p.A)
} }
ctx, cancel := context.WithTimeout(f.Unwrap(), a.timeout) z = container.New(ctx, msg)
defer cancel()
z := container.New(ctx, f.GetMessage())
z.WaitDelay = execWaitDelay z.WaitDelay = execWaitDelay
z.SeccompPresets = SeccompPresets z.SeccompPresets = SeccompPresets
z.SeccompFlags |= seccomp.AllowMultiarch z.SeccompFlags |= seccomp.AllowMultiarch
z.ParentPerm = 0700 z.ParentPerm = 0700
z.HostNet = hostNet z.HostNet = hostNet
z.Hostname = "cure" z.Hostname = "cure"
z.SetScheduler = SetSchedIdle
z.SchedPolicy = ext.SCHED_IDLE z.SchedPolicy = ext.SCHED_IDLE
if z.HostNet { if z.HostNet {
z.Hostname = "cure-net" z.Hostname = "cure-net"
} }
z.Uid, z.Gid = (1<<10)-1, (1<<10)-1 z.Uid, z.Gid = (1<<10)-1, (1<<10)-1
z.Dir, z.Env, z.Path, z.Args = a.dir, a.env, a.path, a.args
z.Grow(len(a.paths) + 4)
for i, b := range a.paths {
if i == overlayWorkIndex {
if err = os.MkdirAll(work.String(), 0700); err != nil {
return
}
tempWork := temp.Append(".work")
if err = os.MkdirAll(tempWork.String(), 0700); err != nil {
return
}
z.Overlay(
AbsWork,
work,
tempWork,
b.layers(msg, getArtifact, ident)...,
)
continue
}
if a.paths[i].W {
tempUpper, tempWork := temp.Append(
".upper", strconv.Itoa(i),
), temp.Append(
".work", strconv.Itoa(i),
)
if err = os.MkdirAll(tempUpper.String(), 0700); err != nil {
return
}
if err = os.MkdirAll(tempWork.String(), 0700); err != nil {
return
}
z.Overlay(b.P, tempUpper, tempWork, b.layers(msg, getArtifact, ident)...)
} else if len(b.A) == 1 {
pathname, _ := getArtifact(b.A[0])
z.Bind(pathname, b.P, 0)
} else {
z.OverlayReadonly(b.P, b.layers(msg, getArtifact, ident)...)
}
}
if overlayWorkIndex < 0 {
z.Bind(
work,
AbsWork,
std.BindWritable|std.BindEnsure,
)
}
z.Bind(
temp,
fhs.AbsTmp,
std.BindWritable|std.BindEnsure,
)
z.Proc(fhs.AbsProc).Dev(fhs.AbsDev, true)
return
}
var (
// ErrExecBusy is returned entering [Cache.EnterExec] while another
// goroutine has not yet returned from it.
ErrExecBusy = errors.New("scratch directories in use")
// ErrNotExec is returned for unsupported implementations of [Artifact]
// passed to [Cache.EnterExec].
ErrNotExec = errors.New("attempting to run a non-exec artifact")
)
// EnterExec runs the container of an [Artifact] of [KindExec] or [KindExecNet]
// with its entry point, argument, and standard streams replaced with values
// supplied by the caller.
func (c *Cache) EnterExec(
ctx context.Context,
a Artifact,
retainSession bool,
stdin io.Reader,
stdout, stderr io.Writer,
path *check.Absolute,
args ...string,
) (err error) {
if !c.inExec.CompareAndSwap(false, true) {
return ErrExecBusy
}
defer c.inExec.Store(false)
var hostNet bool
var e *execArtifact
switch f := a.(type) {
case *execArtifact:
e = f
case *execNetArtifact:
e = &f.execArtifact
hostNet = true
default:
return ErrNotExec
}
deps := Collect(a.Dependencies())
if _, _, err = c.Cure(&deps); err == nil {
return errors.New("unreachable")
} else if !IsCollected(err) {
return
}
dm := make(map[Artifact]cureRes)
for i, p := range deps {
var res cureRes
res.pathname, res.checksum, err = c.Cure(p)
if err != nil {
return
}
dm[deps[i]] = res
}
scratch := c.base.Append(dirExecScratch)
temp, work := scratch.Append("temp"), scratch.Append("work")
// work created during makeContainer
if err = os.MkdirAll(temp.String(), 0700); err != nil {
return
}
defer func() {
if chmodErr, removeErr := removeAll(scratch); chmodErr != nil || removeErr != nil {
err = errors.Join(err, chmodErr, removeErr)
}
}()
var z *container.Container
z, err = e.makeContainer(
ctx, c.msg,
hostNet,
temp, work,
func(a Artifact) (*check.Absolute, unique.Handle[Checksum]) {
if res, ok := dm[a]; ok {
return res.pathname, res.checksum
}
panic(InvalidLookupError(c.Ident(a).Value()))
},
c.Ident,
)
if err != nil {
return
}
z.Stdin, z.Stdout, z.Stderr = stdin, stdout, stderr
z.Path, z.Args = path, args
z.RetainSession = retainSession
if stdin == os.Stdin {
if s, ok := os.LookupEnv("TERM"); ok {
z.Env = append(z.Env, "TERM="+s)
}
}
if err = z.Start(); err != nil {
return
}
if err = z.Serve(); err != nil {
return
}
return z.Wait()
}
// cure is like Cure but allows optional host net namespace.
func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
ctx, cancel := context.WithTimeout(f.Unwrap(), a.timeout)
defer cancel()
msg := f.GetMessage()
var z *container.Container
if z, err = a.makeContainer(
ctx, msg, hostNet,
f.GetTempDir(), f.GetWorkDir(),
f.GetArtifact,
f.cache.Ident,
); err != nil {
return
}
z.SetScheduler = f.cache.flags&CSchedIdle != 0
var status io.Writer var status io.Writer
if status, err = f.GetStatusWriter(); err != nil { if status, err = f.GetStatusWriter(); err != nil {
return return
} }
if msg := f.GetMessage(); msg.IsVerbose() { if msg.IsVerbose() {
var stdout, stderr io.ReadCloser var stdout, stderr io.ReadCloser
if stdout, err = z.StdoutPipe(); err != nil { if stdout, err = z.StdoutPipe(); err != nil {
return return
@@ -464,62 +650,6 @@ func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
z.Stdout, z.Stderr = status, status z.Stdout, z.Stderr = status, status
} }
z.Dir, z.Env, z.Path, z.Args = a.dir, a.env, a.path, a.args
z.Grow(len(a.paths) + 4)
temp, work := f.GetTempDir(), f.GetWorkDir()
for i, b := range a.paths {
if i == overlayWorkIndex {
if err = os.MkdirAll(work.String(), 0700); err != nil {
return
}
tempWork := temp.Append(".work")
if err = os.MkdirAll(tempWork.String(), 0700); err != nil {
return
}
z.Overlay(
AbsWork,
work,
tempWork,
b.layers(f)...,
)
continue
}
if a.paths[i].W {
tempUpper, tempWork := temp.Append(
".upper", strconv.Itoa(i),
), temp.Append(
".work", strconv.Itoa(i),
)
if err = os.MkdirAll(tempUpper.String(), 0700); err != nil {
return
}
if err = os.MkdirAll(tempWork.String(), 0700); err != nil {
return
}
z.Overlay(b.P, tempUpper, tempWork, b.layers(f)...)
} else if len(b.A) == 1 {
pathname, _ := f.GetArtifact(b.A[0])
z.Bind(pathname, b.P, 0)
} else {
z.OverlayReadonly(b.P, b.layers(f)...)
}
}
if overlayWorkIndex < 0 {
z.Bind(
work,
AbsWork,
std.BindWritable|std.BindEnsure,
)
}
z.Bind(
f.GetTempDir(),
fhs.AbsTmp,
std.BindWritable|std.BindEnsure,
)
z.Proc(fhs.AbsProc).Dev(fhs.AbsDev, true)
if err = z.Start(); err != nil { if err = z.Start(); err != nil {
return return
} }
@@ -532,7 +662,7 @@ func (a *execArtifact) cure(f *FContext, hostNet bool) (err error) {
// do not allow empty directories to succeed // do not allow empty directories to succeed
for { for {
err = syscall.Rmdir(work.String()) err = syscall.Rmdir(f.GetWorkDir().String())
if err != syscall.EINTR { if err != syscall.EINTR {
break break
} }

View File

@@ -33,8 +33,7 @@ func TestExec(t *testing.T) {
) )
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"offline", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"offline", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -92,7 +91,7 @@ func TestExec(t *testing.T) {
[]string{"testtool"}, []string{"testtool"},
pkg.ExecPath{}, pkg.ExecPath{},
), nil, pkg.Checksum{}, os.ErrInvalid}, ), nil, pkg.Checksum{}, pkg.ErrInvalidPaths},
}) })
// check init failure passthrough // check init failure passthrough
@@ -111,8 +110,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("Q5DluWQCAeohLoiGRImurwFp3vdz9IfQCoj7Fuhh73s4KQPRHpEQEnHTdNHmB8Fx")}, }, pkg.MustDecode("Q5DluWQCAeohLoiGRImurwFp3vdz9IfQCoj7Fuhh73s4KQPRHpEQEnHTdNHmB8Fx")},
{"net", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"net", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
wantChecksum := pkg.MustDecode( wantChecksum := pkg.MustDecode(
@@ -146,8 +144,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("bPYvvqxpfV7xcC1EptqyKNK1klLJgYHMDkzBcoOyK6j_Aj5hb0mXNPwTwPSK5F6Z")}, }, pkg.MustDecode("bPYvvqxpfV7xcC1EptqyKNK1klLJgYHMDkzBcoOyK6j_Aj5hb0mXNPwTwPSK5F6Z")},
{"overlay root", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay root", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -172,8 +169,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("PO2DSSCa4yoSgEYRcCSZfQfwow1yRigL3Ry-hI0RDI4aGuFBha-EfXeSJnG_5_Rl")}, }, pkg.MustDecode("PO2DSSCa4yoSgEYRcCSZfQfwow1yRigL3Ry-hI0RDI4aGuFBha-EfXeSJnG_5_Rl")},
{"overlay work", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay work", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -203,8 +199,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("iaRt6l_Wm2n-h5UsDewZxQkCmjZjyL8r7wv32QT2kyV55-Lx09Dq4gfg9BiwPnKs")}, }, pkg.MustDecode("iaRt6l_Wm2n-h5UsDewZxQkCmjZjyL8r7wv32QT2kyV55-Lx09Dq4gfg9BiwPnKs")},
{"multiple layers", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"multiple layers", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
@@ -256,8 +251,7 @@ func TestExec(t *testing.T) {
testtoolDestroy(t, base, c) testtoolDestroy(t, base, c)
}, pkg.MustDecode("O2YzyR7IUGU5J2CADy0hUZ3A5NkP_Vwzs4UadEdn2oMZZVWRtH0xZGJ3HXiimTnZ")}, }, pkg.MustDecode("O2YzyR7IUGU5J2CADy0hUZ3A5NkP_Vwzs4UadEdn2oMZZVWRtH0xZGJ3HXiimTnZ")},
{"overlay layer promotion", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"overlay layer promotion", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
testtool, testtoolDestroy := newTesttool() testtool, testtoolDestroy := newTesttool()
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{

View File

@@ -11,9 +11,7 @@ func TestFile(t *testing.T) {
t.Parallel() t.Parallel()
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"file", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
{"short", pkg.NewFile("null", []byte{0}), base.Append( {"short", pkg.NewFile("null", []byte{0}), base.Append(
"identifier", "identifier",

View File

@@ -85,7 +85,7 @@ func TestIRRoundtrip(t *testing.T) {
testCasesCache := make([]cacheTestCase, len(testCases)) testCasesCache := make([]cacheTestCase, len(testCases))
for i, tc := range testCases { for i, tc := range testCases {
want := tc.a want := tc.a
testCasesCache[i] = cacheTestCase{tc.name, nil, testCasesCache[i] = cacheTestCase{tc.name, 0, nil,
func(t *testing.T, base *check.Absolute, c *pkg.Cache) { func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
r, w := io.Pipe() r, w := io.Pipe()

View File

@@ -32,7 +32,7 @@ func TestHTTPGet(t *testing.T) {
})) }))
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"direct", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"direct", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext var r pkg.RContext
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache") rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt( reflect.NewAt(
@@ -94,7 +94,7 @@ func TestHTTPGet(t *testing.T) {
} }
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")}, }, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"cure", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"cure", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
var r pkg.RContext var r pkg.RContext
rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache") rCacheVal := reflect.ValueOf(&r).Elem().FieldByName("cache")
reflect.NewAt( reflect.NewAt(

View File

@@ -13,15 +13,14 @@ import (
"hash" "hash"
"io" "io"
"io/fs" "io/fs"
"iter"
"maps" "maps"
"os" "os"
"path"
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strings" "strings"
"sync" "sync"
"sync/atomic"
"syscall" "syscall"
"testing" "testing"
"unique" "unique"
@@ -331,23 +330,6 @@ type FloodArtifact interface {
Artifact Artifact
} }
// Flood returns an iterator over the dependency tree of an [Artifact].
func Flood(a Artifact) iter.Seq[Artifact] {
return func(yield func(Artifact) bool) {
for _, d := range a.Dependencies() {
if !yield(d) {
return
}
for d0 := range Flood(d) {
if !yield(d0) {
return
}
}
}
}
}
// TrivialArtifact refers to an [Artifact] that cures without requiring that // TrivialArtifact refers to an [Artifact] that cures without requiring that
// any other [Artifact] is cured before it. Its dependency tree is ignored after // any other [Artifact] is cured before it. Its dependency tree is ignored after
// computing its identifier. // computing its identifier.
@@ -366,7 +348,7 @@ type TrivialArtifact interface {
} }
// KnownIdent is optionally implemented by [Artifact] and is used instead of // KnownIdent is optionally implemented by [Artifact] and is used instead of
// [Kind.Ident] when it is available. // [Cache.Ident] when it is available.
// //
// This is very subtle to use correctly. The implementation must ensure that // This is very subtle to use correctly. The implementation must ensure that
// this value is globally unique, otherwise [Cache] can enter an inconsistent // this value is globally unique, otherwise [Cache] can enter an inconsistent
@@ -439,6 +421,11 @@ const (
KindCustomOffset = 1 << 31 KindCustomOffset = 1 << 31
) )
const (
// kindCollection is the kind of [Collect]. It never cures successfully.
kindCollection Kind = KindCustomOffset - 1 - iota
)
const ( const (
// fileLock is the file name appended to Cache.base for guaranteeing // fileLock is the file name appended to Cache.base for guaranteeing
// exclusive access to the cache directory. // exclusive access to the cache directory.
@@ -461,6 +448,11 @@ const (
// pathnames allocated during [Cache.Cure]. // pathnames allocated during [Cache.Cure].
dirTemp = "temp" dirTemp = "temp"
// dirExecScratch is the directory name appended to Cache.base for scratch
// space setting up the container started by [Cache.EnterExec]. Exclusivity
// via Cache.inExec.
dirExecScratch = "scratch"
// checksumLinknamePrefix is prepended to the encoded [Checksum] value // checksumLinknamePrefix is prepended to the encoded [Checksum] value
// of an [Artifact] when creating a symbolic link to dirChecksum. // of an [Artifact] when creating a symbolic link to dirChecksum.
checksumLinknamePrefix = "../" + dirChecksum + "/" checksumLinknamePrefix = "../" + dirChecksum + "/"
@@ -476,7 +468,7 @@ type cureRes struct {
// subject to the cures limit. Values pointed to by result addresses are safe // subject to the cures limit. Values pointed to by result addresses are safe
// to access after the [sync.WaitGroup] associated with this pendingArtifactDep // to access after the [sync.WaitGroup] associated with this pendingArtifactDep
// is done. pendingArtifactDep must not be reused or modified after it is sent // is done. pendingArtifactDep must not be reused or modified after it is sent
// to Cache.cureDep. // to cure.
type pendingArtifactDep struct { type pendingArtifactDep struct {
// Dependency artifact populated during [Cache.Cure]. // Dependency artifact populated during [Cache.Cure].
a Artifact a Artifact
@@ -496,6 +488,41 @@ type pendingArtifactDep struct {
*sync.WaitGroup *sync.WaitGroup
} }
const (
// CValidateKnown arranges for [KnownChecksum] outcomes to be validated to
// match its intended checksum.
//
// A correct implementation of [KnownChecksum] does not successfully cure
// with output not matching its intended checksum. When an implementation
// fails to perform this validation correctly, the on-disk format enters
// an inconsistent state (correctable by [Cache.Scrub]).
//
// This flag causes [Cache.Cure] to always compute the checksum, and reject
// a cure if it does not match the intended checksum.
//
// This behaviour significantly reduces performance and is not recommended
// outside of testing a custom [Artifact] implementation.
CValidateKnown = 1 << iota
// CSchedIdle arranges for the [ext.SCHED_IDLE] scheduling priority to be
// set for [KindExec] and [KindExecNet] containers.
CSchedIdle
// CAssumeChecksum enables the use of [KnownChecksum] for duplicate function
// call suppression via the on-disk cache.
//
// This may cause incorrect cure outcome if an impossible checksum is
// specified that matches an output already present in the on-disk cache.
// This may be avoided by purposefully specifying a statistically
// unattainable checksum, like the zero value.
//
// While this optimisation might seem appealing, it is almost never
// applicable in real world use. Almost every time this path was taken, it
// was caused by an incorrect checksum accidentally left behind while
// bumping a package. Only enable this if you are really sure you need it.
CAssumeChecksum
)
// Cache is a support layer that implementations of [Artifact] can use to store // Cache is a support layer that implementations of [Artifact] can use to store
// cured [Artifact] data in a content addressed fashion. // cured [Artifact] data in a content addressed fashion.
type Cache struct { type Cache struct {
@@ -515,12 +542,8 @@ type Cache struct {
// Directory where all [Cache] related files are placed. // Directory where all [Cache] related files are placed.
base *check.Absolute base *check.Absolute
// Immutable cure options set by [Open].
// Whether to validate [FileArtifact.Cure] for a [KnownChecksum] file. This flags int
// significantly reduces performance.
strict bool
// Maximum size of a dependency graph.
threshold uintptr
// Artifact to [unique.Handle] of identifier cache. // Artifact to [unique.Handle] of identifier cache.
artifact sync.Map artifact sync.Map
@@ -548,24 +571,11 @@ type Cache struct {
unlock func() unlock func()
// Synchronises calls to Close. // Synchronises calls to Close.
closeOnce sync.Once closeOnce sync.Once
// Whether EnterExec has not yet returned.
inExec atomic.Bool
} }
// IsStrict returns whether the [Cache] strictly verifies checksums.
func (c *Cache) IsStrict() bool { return c.strict }
// SetStrict sets whether the [Cache] strictly verifies checksums, even when
// the implementation promises to validate them internally. This significantly
// reduces performance and is not recommended outside of testing.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetStrict(strict bool) { c.strict = strict }
// SetThreshold imposes a maximum size on the dependency graph, checked on every
// call to Cure. The zero value disables this check entirely.
//
// This method is not safe for concurrent use with any other method.
func (c *Cache) SetThreshold(threshold uintptr) { c.threshold = threshold }
// extIdent is a [Kind] concatenated with [ID]. // extIdent is a [Kind] concatenated with [ID].
type extIdent [wordSize + len(ID{})]byte type extIdent [wordSize + len(ID{})]byte
@@ -880,7 +890,7 @@ func (c *Cache) Scrub(checks int) error {
se.DanglingIdentifiers = append(se.DanglingIdentifiers, *want) se.DanglingIdentifiers = append(se.DanglingIdentifiers, *want)
seMu.Unlock() seMu.Unlock()
return false return false
} else if err = Decode(got, path.Base(linkname)); err != nil { } else if err = Decode(got, filepath.Base(linkname)); err != nil {
seMu.Lock() seMu.Lock()
lnp := dir.Append(linkname) lnp := dir.Append(linkname)
se.Errs[lnp.Handle()] = append(se.Errs[lnp.Handle()], err) se.Errs[lnp.Handle()] = append(se.Errs[lnp.Handle()], err)
@@ -1045,7 +1055,7 @@ func (c *Cache) finaliseIdent(
// [FileArtifact] to the filesystem. If err is nil, the caller is responsible // [FileArtifact] to the filesystem. If err is nil, the caller is responsible
// for closing the resulting [io.ReadCloser]. // for closing the resulting [io.ReadCloser].
func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) { func (c *Cache) openFile(f FileArtifact) (r io.ReadCloser, err error) {
if kc, ok := f.(KnownChecksum); ok { if kc, ok := f.(KnownChecksum); c.flags&CAssumeChecksum != 0 && ok {
c.checksumMu.RLock() c.checksumMu.RLock()
r, err = os.Open(c.base.Append( r, err = os.Open(c.base.Append(
dirChecksum, dirChecksum,
@@ -1216,14 +1226,6 @@ func (e InvalidArtifactError) Error() string {
return "artifact " + Encode(e) + " cannot be cured" return "artifact " + Encode(e) + " cannot be cured"
} }
// DependencyError refers to an artifact with a dependency tree larger than the
// threshold specified by a previous call to [Cache.SetThreshold].
type DependencyError struct{ A Artifact }
func (e DependencyError) Error() string {
return "artifact has too many dependencies"
}
// Cure cures the [Artifact] and returns its pathname and [Checksum]. Direct // Cure cures the [Artifact] and returns its pathname and [Checksum]. Direct
// calls to Cure are not subject to the cures limit. // calls to Cure are not subject to the cures limit.
func (c *Cache) Cure(a Artifact) ( func (c *Cache) Cure(a Artifact) (
@@ -1239,18 +1241,6 @@ func (c *Cache) Cure(a Artifact) (
default: default:
} }
if c.threshold > 0 {
var n uintptr
for range Flood(a) {
if n == c.threshold {
err = DependencyError{a}
return
}
n++
}
c.msg.Verbosef("visited %d artifacts", n)
}
return c.cure(a, true) return c.cure(a, true)
} }
@@ -1474,7 +1464,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
return return
} }
buf := c.getIdentBuf() buf := c.getIdentBuf()
err = Decode((*Checksum)(buf[:]), path.Base(name)) err = Decode((*Checksum)(buf[:]), filepath.Base(name))
if err == nil { if err == nil {
checksum = unique.Make(Checksum(buf[:])) checksum = unique.Make(Checksum(buf[:]))
} }
@@ -1508,6 +1498,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
checksums, checksums,
) )
if c.flags&CAssumeChecksum != 0 {
c.checksumMu.RLock() c.checksumMu.RLock()
checksumFi, err = os.Stat(checksumPathname.String()) checksumFi, err = os.Stat(checksumPathname.String())
c.checksumMu.RUnlock() c.checksumMu.RUnlock()
@@ -1520,6 +1511,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
checksumFi, err = nil, nil checksumFi, err = nil, nil
} }
} }
}
if c.msg.IsVerbose() { if c.msg.IsVerbose() {
rn := reportName(a, id) rn := reportName(a, id)
@@ -1573,7 +1565,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
} }
r, err = f.Cure(&RContext{common{c}}) r, err = f.Cure(&RContext{common{c}})
if err == nil { if err == nil {
if checksumPathname == nil || c.IsStrict() { if checksumPathname == nil || c.flags&CValidateKnown != 0 {
h := sha512.New384() h := sha512.New384()
hbw := c.getWriter(h) hbw := c.getWriter(h)
_, err = io.Copy(w, io.TeeReader(r, hbw)) _, err = io.Copy(w, io.TeeReader(r, hbw))
@@ -1590,7 +1582,7 @@ func (c *Cache) cure(a Artifact, curesExempt bool) (
if checksumPathname == nil { if checksumPathname == nil {
checksum = unique.Make(Checksum(buf[:])) checksum = unique.Make(Checksum(buf[:]))
checksums = Encode(Checksum(buf[:])) checksums = Encode(Checksum(buf[:]))
} else if c.IsStrict() { } else if c.flags&CValidateKnown != 0 {
if got := Checksum(buf[:]); got != checksum.Value() { if got := Checksum(buf[:]); got != checksum.Value() {
err = &ChecksumMismatchError{ err = &ChecksumMismatchError{
Got: got, Got: got,
@@ -1828,10 +1820,10 @@ func (c *Cache) Close() {
func Open( func Open(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures int,
base *check.Absolute, base *check.Absolute,
) (*Cache, error) { ) (*Cache, error) {
return open(ctx, msg, cures, base, true) return open(ctx, msg, flags, cures, base, true)
} }
// open implements Open but allows omitting the [lockedfile] lock when called // open implements Open but allows omitting the [lockedfile] lock when called
@@ -1839,7 +1831,7 @@ func Open(
func open( func open(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures int,
base *check.Absolute, base *check.Absolute,
lock bool, lock bool,
) (*Cache, error) { ) (*Cache, error) {
@@ -1861,6 +1853,7 @@ func open(
c := Cache{ c := Cache{
cures: make(chan struct{}, cures), cures: make(chan struct{}, cures),
flags: flags,
msg: msg, msg: msg,
base: base, base: base,
@@ -1890,3 +1883,33 @@ func open(
return &c, nil return &c, nil
} }
// Collected is returned by [Collect.Cure] to indicate a successful collection.
type Collected struct{}
// Error returns a constant string to satisfy error, but should never be seen
// by the user.
func (Collected) Error() string { return "artifacts successfully collected" }
// IsCollected returns whether the underlying error contains that of the result
// of curing a [Collect] helper.
func IsCollected(err error) bool { return errors.As(err, new(Collected)) }
// Collect implements [pkg.FloodArtifact] to concurrently cure multiple
// [pkg.Artifact]. It returns [Collected].
type Collect []Artifact
// Cure returns [Collected].
func (*Collect) Cure(*FContext) error { return Collected{} }
// Kind returns the hardcoded [pkg.Kind] value.
func (*Collect) Kind() Kind { return kindCollection }
// Params is a noop: dependencies are already represented in the header.
func (*Collect) Params(*IContext) {}
// Dependencies returns [Collect] as is.
func (c *Collect) Dependencies() []Artifact { return *c }
// IsExclusive returns false: Cure is a noop.
func (*Collect) IsExclusive() bool { return false }

View File

@@ -33,7 +33,7 @@ import (
func unsafeOpen( func unsafeOpen(
ctx context.Context, ctx context.Context,
msg message.Msg, msg message.Msg,
cures int, flags, cures int,
base *check.Absolute, base *check.Absolute,
lock bool, lock bool,
) (*pkg.Cache, error) ) (*pkg.Cache, error)
@@ -228,7 +228,7 @@ func TestIdent(t *testing.T) {
var cache *pkg.Cache var cache *pkg.Cache
if a, err := check.NewAbs(t.TempDir()); err != nil { if a, err := check.NewAbs(t.TempDir()); err != nil {
t.Fatal(err) t.Fatal(err)
} else if cache, err = pkg.Open(t.Context(), msg, 0, a); err != nil { } else if cache, err = pkg.Open(t.Context(), msg, 0, 0, a); err != nil {
t.Fatal(err) t.Fatal(err)
} }
t.Cleanup(cache.Close) t.Cleanup(cache.Close)
@@ -252,6 +252,7 @@ func TestIdent(t *testing.T) {
// on test completion. // on test completion.
type cacheTestCase struct { type cacheTestCase struct {
name string name string
flags int
early func(t *testing.T, base *check.Absolute) early func(t *testing.T, base *check.Absolute)
f func(t *testing.T, base *check.Absolute, c *pkg.Cache) f func(t *testing.T, base *check.Absolute, c *pkg.Cache)
want pkg.Checksum want pkg.Checksum
@@ -289,7 +290,7 @@ func checkWithCache(t *testing.T, testCases []cacheTestCase) {
msg.SwapVerbose(testing.Verbose()) msg.SwapVerbose(testing.Verbose())
var scrubFunc func() error // scrub after hashing var scrubFunc func() error // scrub after hashing
if c, err := pkg.Open(t.Context(), msg, 1<<4, base); err != nil { if c, err := pkg.Open(t.Context(), msg, tc.flags, 1<<4, base); err != nil {
t.Fatalf("Open: error = %v", err) t.Fatalf("Open: error = %v", err)
} else { } else {
t.Cleanup(c.Close) t.Cleanup(c.Close)
@@ -468,9 +469,7 @@ func TestCache(t *testing.T) {
}() }()
testCases := []cacheTestCase{ testCases := []cacheTestCase{
{"file", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"file", pkg.CValidateKnown | pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
identifier := (pkg.ID)(bytes.Repeat([]byte{ identifier := (pkg.ID)(bytes.Repeat([]byte{
0x75, 0xe6, 0x9d, 0x6d, 0xe7, 0x9f, 0x75, 0xe6, 0x9d, 0x6d, 0xe7, 0x9f,
}, 8)) }, 8))
@@ -593,7 +592,7 @@ func TestCache(t *testing.T) {
if c0, err := unsafeOpen( if c0, err := unsafeOpen(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, base, false, 0, 0, base, false,
); err != nil { ); err != nil {
t.Fatalf("open: error = %v", err) t.Fatalf("open: error = %v", err)
} else { } else {
@@ -627,7 +626,7 @@ func TestCache(t *testing.T) {
} }
}, pkg.MustDecode("St9rlE-mGZ5gXwiv_hzQ_B8bZP-UUvSNmf4nHUZzCMOumb6hKnheZSe0dmnuc4Q2")}, }, pkg.MustDecode("St9rlE-mGZ5gXwiv_hzQ_B8bZP-UUvSNmf4nHUZzCMOumb6hKnheZSe0dmnuc4Q2")},
{"directory", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"directory", pkg.CAssumeChecksum, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
id := pkg.MustDecode( id := pkg.MustDecode(
"HnySzeLQvSBZuTUcvfmLEX_OmH4yJWWH788NxuLuv7kVn8_uPM6Ks4rqFWM2NZJY", "HnySzeLQvSBZuTUcvfmLEX_OmH4yJWWH788NxuLuv7kVn8_uPM6Ks4rqFWM2NZJY",
) )
@@ -804,9 +803,7 @@ func TestCache(t *testing.T) {
}) })
}, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d")}, }, pkg.MustDecode("WVpvsVqVKg9Nsh744x57h51AuWUoUR2nnh8Md-EYBQpk6ziyTuUn6PLtF2e0Eu_d")},
{"pending", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"pending", pkg.CValidateKnown, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
c.SetStrict(true)
wantErr := stub.UniqueError(0xcafe) wantErr := stub.UniqueError(0xcafe)
n, ready := make(chan struct{}), make(chan struct{}) n, ready := make(chan struct{}), make(chan struct{})
go func() { go func() {
@@ -876,7 +873,54 @@ func TestCache(t *testing.T) {
<-wCureDone <-wCureDone
}, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")}, }, pkg.MustDecode("E4vEZKhCcL2gPZ2Tt59FS3lDng-d_2SKa2i5G_RbDfwGn6EemptFaGLPUDiOa94C")},
{"scrub", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"no assume checksum", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
makeGarbage := func(work *check.Absolute, wantErr error) error {
if err := os.Mkdir(work.String(), 0700); err != nil {
return err
}
if err := os.WriteFile(work.Append(
"check",
).String(), nil, 0400); err != nil {
return err
}
return wantErr
}
wantChecksum := pkg.MustDecode("Aubi5EG4_Y8DhL9bQ3Q4HFBhLRF7X5gt9D3CNCQfT-TeBtlRXc7Zi_JYZEMoCC7M")
cureMany(t, c, []cureStep{
{"create", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 0}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 0}),
), wantChecksum, nil},
{"reject", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xfe, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), stub.UniqueError(0xbad))
},
}}}, nil, pkg.Checksum{}, stub.UniqueError(0xbad)},
{"match", overrideChecksum{wantChecksum, overrideIdent{pkg.ID{0xff, 1}, &stubArtifact{
kind: pkg.KindTar,
cure: func(t *pkg.TContext) error {
return makeGarbage(t.GetWorkDir(), nil)
},
}}}, base.Append(
"identifier",
pkg.Encode(pkg.ID{0xff, 1}),
), wantChecksum, nil},
})
}, pkg.MustDecode("OC290t23aimNo2Rp2pPwan5GI2KRLRdOwYxXQMD9jw0QROgHnNXWodoWdV0hwu2w")},
{"scrub", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
cureMany(t, c, []cureStep{ cureMany(t, c, []cureStep{
{"bad measured file", newStubFile( {"bad measured file", newStubFile(
pkg.KindHTTPGet, pkg.KindHTTPGet,
@@ -1182,7 +1226,7 @@ func (a earlyFailureF) Cure(*pkg.FContext) error {
func TestDependencyCureErrorEarly(t *testing.T) { func TestDependencyCureErrorEarly(t *testing.T) {
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"early", nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) { {"early", 0, nil, func(t *testing.T, _ *check.Absolute, c *pkg.Cache) {
_, _, err := c.Cure(earlyFailureF(8)) _, _, err := c.Cure(earlyFailureF(8))
if !errors.Is(err, stub.UniqueError(0xcafe)) { if !errors.Is(err, stub.UniqueError(0xcafe)) {
t.Fatalf("Cure: error = %v", err) t.Fatalf("Cure: error = %v", err)
@@ -1205,7 +1249,7 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open( if _, err := pkg.Open(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, check.MustAbs(container.Nonexistent), 0, 0, check.MustAbs(container.Nonexistent),
); !reflect.DeepEqual(err, wantErr) { ); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr) t.Errorf("Open: error = %#v, want %#v", err, wantErr)
} }
@@ -1233,7 +1277,7 @@ func TestNew(t *testing.T) {
if _, err := pkg.Open( if _, err := pkg.Open(
t.Context(), t.Context(),
message.New(nil), message.New(nil),
0, tempDir.Append("cache"), 0, 0, tempDir.Append("cache"),
); !reflect.DeepEqual(err, wantErr) { ); !reflect.DeepEqual(err, wantErr) {
t.Errorf("Open: error = %#v, want %#v", err, wantErr) t.Errorf("Open: error = %#v, want %#v", err, wantErr)
} }

View File

@@ -10,7 +10,7 @@ import (
"io/fs" "io/fs"
"net/http" "net/http"
"os" "os"
"path" "path/filepath"
) )
const ( const (
@@ -169,7 +169,7 @@ func (a *tarArtifact) Cure(t *TContext) (err error) {
} }
if typeflag >= '0' && typeflag <= '9' && typeflag != tar.TypeDir { if typeflag >= '0' && typeflag <= '9' && typeflag != tar.TypeDir {
if err = root.MkdirAll(path.Dir(header.Name), 0700); err != nil { if err = root.MkdirAll(filepath.Dir(header.Name), 0700); err != nil {
return return
} }
} }

View File

@@ -21,7 +21,7 @@ func TestTar(t *testing.T) {
t.Parallel() t.Parallel()
checkWithCache(t, []cacheTestCase{ checkWithCache(t, []cacheTestCase{
{"http", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"http", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{ checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700}, ".": {Mode: fs.ModeDir | 0700},
@@ -42,7 +42,7 @@ func TestTar(t *testing.T) {
)) ))
}, pkg.MustDecode("NQTlc466JmSVLIyWklm_u8_g95jEEb98PxJU-kjwxLpfdjwMWJq0G8ze9R4Vo1Vu")}, }, pkg.MustDecode("NQTlc466JmSVLIyWklm_u8_g95jEEb98PxJU-kjwxLpfdjwMWJq0G8ze9R4Vo1Vu")},
{"http expand", nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) { {"http expand", 0, nil, func(t *testing.T, base *check.Absolute, c *pkg.Cache) {
checkTarHTTP(t, base, c, fstest.MapFS{ checkTarHTTP(t, base, c, fstest.MapFS{
".": {Mode: fs.ModeDir | 0700}, ".": {Mode: fs.ModeDir | 0700},

View File

@@ -7,7 +7,7 @@ import (
"log" "log"
"net" "net"
"os" "os"
"path" "path/filepath"
"reflect" "reflect"
"slices" "slices"
"strings" "strings"
@@ -68,7 +68,7 @@ func main() {
if got, err := os.Executable(); err != nil { if got, err := os.Executable(); err != nil {
log.Fatalf("Executable: error = %v", err) log.Fatalf("Executable: error = %v", err)
} else { } else {
iftPath = path.Join(path.Dir(path.Dir(got)), "ift") iftPath = filepath.Join(filepath.Dir(filepath.Dir(got)), "ift")
if got != wantExec { if got != wantExec {
switch got { switch got {
@@ -161,7 +161,7 @@ func main() {
} }
} }
if !layers { if !layers {
if path.Base(lowerdir) != checksumEmptyDir { if filepath.Base(lowerdir) != checksumEmptyDir {
log.Fatal("unexpected artifact checksum") log.Fatal("unexpected artifact checksum")
} }
} else { } else {
@@ -187,8 +187,8 @@ func main() {
} }
if len(lowerdirs) != 2 || if len(lowerdirs) != 2 ||
path.Base(lowerdirs[0]) != "MGWmEfjut2QE2xPJwTsmUzpff4BN_FEnQ7T0j7gvUCCiugJQNwqt9m151fm9D1yU" || filepath.Base(lowerdirs[0]) != "MGWmEfjut2QE2xPJwTsmUzpff4BN_FEnQ7T0j7gvUCCiugJQNwqt9m151fm9D1yU" ||
path.Base(lowerdirs[1]) != "nY_CUdiaUM1OL4cPr5TS92FCJ3rCRV7Hm5oVTzAvMXwC03_QnTRfQ5PPs7mOU9fK" { filepath.Base(lowerdirs[1]) != "nY_CUdiaUM1OL4cPr5TS92FCJ3rCRV7Hm5oVTzAvMXwC03_QnTRfQ5PPs7mOU9fK" {
log.Fatalf("unexpected lowerdirs %s", strings.Join(lowerdirs, ", ")) log.Fatalf("unexpected lowerdirs %s", strings.Join(lowerdirs, ", "))
} }
} }
@@ -202,12 +202,12 @@ func main() {
} }
next() next()
if path.Base(m.Root) != "OLBgp1GsljhM2TJ-sbHjaiH9txEUvgdDTAzHv2P24donTt6_529l-9Ua0vFImLlb" { if filepath.Base(m.Root) != "OLBgp1GsljhM2TJ-sbHjaiH9txEUvgdDTAzHv2P24donTt6_529l-9Ua0vFImLlb" {
log.Fatal("unexpected file artifact checksum") log.Fatal("unexpected file artifact checksum")
} }
next() next()
if path.Base(m.Root) != checksumEmptyDir { if filepath.Base(m.Root) != checksumEmptyDir {
log.Fatal("unexpected artifact checksum") log.Fatal("unexpected artifact checksum")
} }
} }
@@ -226,13 +226,13 @@ func main() {
log.Fatal("unexpected work mount entry") log.Fatal("unexpected work mount entry")
} }
} else { } else {
if path.Base(m.Root) != ident || m.Target != "/work" { if filepath.Base(m.Root) != ident || m.Target != "/work" {
log.Fatal("unexpected work mount entry") log.Fatal("unexpected work mount entry")
} }
} }
next() next()
if path.Base(m.Root) != ident || m.Target != "/tmp" { if filepath.Base(m.Root) != ident || m.Target != "/tmp" {
log.Fatal("unexpected temp mount entry") log.Fatal("unexpected temp mount entry")
} }

View File

@@ -13,7 +13,7 @@ func (t Toolchain) newAttr() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
Patches: [][2]string{ Patches: []KV{
{"libgen-basename", `From 8a80d895dfd779373363c3a4b62ecce5a549efb2 Mon Sep 17 00:00:00 2001 {"libgen-basename", `From 8a80d895dfd779373363c3a4b62ecce5a549efb2 Mon Sep 17 00:00:00 2001
From: "Haelwenn (lanodan) Monnier" <contact@hacktivis.me> From: "Haelwenn (lanodan) Monnier" <contact@hacktivis.me>
Date: Sat, 30 Mar 2024 10:17:10 +0100 Date: Sat, 30 Mar 2024 10:17:10 +0100

View File

@@ -47,8 +47,10 @@ const (
Bison Bison
Bzip2 Bzip2
CMake CMake
Connman
Coreutils Coreutils
Curl Curl
DBus
DTC DTC
Diffutils Diffutils
Elfutils Elfutils
@@ -62,23 +64,32 @@ const (
GenInitCPIO GenInitCPIO
Gettext Gettext
Git Git
GnuTLS
Go Go
Gperf Gperf
Grep Grep
Gzip Gzip
Hakurei Hakurei
HakureiDist HakureiDist
IPTables
Kmod Kmod
LibXau LibXau
Libbsd
Libcap Libcap
Libev
Libexpat Libexpat
Libiconv
Libpsl
Libffi Libffi
Libgd Libgd
Libtool Libiconv
Libmd
Libmnl
Libnftnl
Libpsl
Libseccomp Libseccomp
Libtasn1
Libtool
Libucontext Libucontext
Libunistring
Libxml2 Libxml2
Libxslt Libxslt
M4 M4
@@ -95,6 +106,7 @@ const (
Nettle Nettle
Ninja Ninja
OpenSSL OpenSSL
P11Kit
PCRE2 PCRE2
Parallel Parallel
Patch Patch
@@ -119,6 +131,7 @@ const (
PythonPygments PythonPygments
QEMU QEMU
Rdfind Rdfind
Readline
Rsync Rsync
Sed Sed
Setuptools Setuptools
@@ -153,6 +166,9 @@ const (
// stages only. This preset and its direct output must never be exposed. // stages only. This preset and its direct output must never be exposed.
gcc gcc
// nettle3 is an older version of [Nettle].
nettle3
// Stage0 is a tarball containing all compile-time dependencies of artifacts // Stage0 is a tarball containing all compile-time dependencies of artifacts
// part of the [Std] toolchain. // part of the [Std] toolchain.
Stage0 Stage0
@@ -291,6 +307,17 @@ var (
artifactsOnce [_toolchainEnd][len(artifactsM)]sync.Once artifactsOnce [_toolchainEnd][len(artifactsM)]sync.Once
) )
// zero zeros the value pointed to by p.
func zero[T any](p *T) { var v T; *p = v }
// DropCaches arranges for all cached [pkg.Artifact] to be freed some time after
// it returns. Must not be used concurrently with any other function from this
// package.
func DropCaches() {
zero(&artifacts)
zero(&artifactsOnce)
}
// GetMetadata returns [Metadata] of a [PArtifact]. // GetMetadata returns [Metadata] of a [PArtifact].
func GetMetadata(p PArtifact) *Metadata { return &artifactsM[p] } func GetMetadata(p PArtifact) *Metadata { return &artifactsM[p] }

View File

@@ -19,6 +19,18 @@ func TestLoad(t *testing.T) {
} }
} }
func BenchmarkAll(b *testing.B) {
for b.Loop() {
for i := range rosa.PresetEnd {
rosa.Std.Load(rosa.PArtifact(i))
}
b.StopTimer()
rosa.DropCaches()
b.StartTimer()
}
}
func TestResolveName(t *testing.T) { func TestResolveName(t *testing.T) {
t.Parallel() t.Parallel()

View File

@@ -1,7 +1,7 @@
package rosa package rosa
import ( import (
"path" "path/filepath"
"slices" "slices"
"strings" "strings"
@@ -10,8 +10,8 @@ import (
func (t Toolchain) newCMake() (pkg.Artifact, string) { func (t Toolchain) newCMake() (pkg.Artifact, string) {
const ( const (
version = "4.2.3" version = "4.3.1"
checksum = "Y4uYGnLrDQX78UdzH7fMzfok46Nt_1taDIHSmqgboU1yFi6f0iAXBDegMCu4eS-J" checksum = "RHpzZiM1kJ5bwLjo9CpXSeHJJg3hTtV9QxBYpQoYwKFtRh5YhGWpShrqZCSOzQN6"
) )
return t.NewPackage("cmake", version, pkg.NewHTTPGetTar( return t.NewPackage("cmake", version, pkg.NewHTTPGetTar(
nil, "https://github.com/Kitware/CMake/releases/download/"+ nil, "https://github.com/Kitware/CMake/releases/download/"+
@@ -25,7 +25,7 @@ func (t Toolchain) newCMake() (pkg.Artifact, string) {
// expected to be writable in the copy made during bootstrap // expected to be writable in the copy made during bootstrap
Chmod: true, Chmod: true,
Patches: [][2]string{ Patches: []KV{
{"bootstrap-test-no-openssl", `diff --git a/Tests/BootstrapTest.cmake b/Tests/BootstrapTest.cmake {"bootstrap-test-no-openssl", `diff --git a/Tests/BootstrapTest.cmake b/Tests/BootstrapTest.cmake
index 137de78bc1..b4da52e664 100644 index 137de78bc1..b4da52e664 100644
--- a/Tests/BootstrapTest.cmake --- a/Tests/BootstrapTest.cmake
@@ -88,7 +88,7 @@ index 2ead810437..f85cbb8b1c 100644
OmitDefaults: true, OmitDefaults: true,
ConfigureName: "/usr/src/cmake/bootstrap", ConfigureName: "/usr/src/cmake/bootstrap",
Configure: [][2]string{ Configure: []KV{
{"prefix", "/system"}, {"prefix", "/system"},
{"parallel", `"$(nproc)"`}, {"parallel", `"$(nproc)"`},
{"--"}, {"--"},
@@ -125,7 +125,7 @@ type CMakeHelper struct {
Append []string Append []string
// CMake CACHE entries. // CMake CACHE entries.
Cache [][2]string Cache []KV
// Runs after install. // Runs after install.
Script string Script string
@@ -144,11 +144,11 @@ func (attr *CMakeHelper) name(name, version string) string {
} }
// extra returns a hardcoded slice of [CMake] and [Ninja]. // extra returns a hardcoded slice of [CMake] and [Ninja].
func (attr *CMakeHelper) extra(int) []PArtifact { func (attr *CMakeHelper) extra(int) P {
if attr != nil && attr.Make { if attr != nil && attr.Make {
return []PArtifact{CMake, Make} return P{CMake, Make}
} }
return []PArtifact{CMake, Ninja} return P{CMake, Ninja}
} }
// wantsChmod returns false. // wantsChmod returns false.
@@ -170,7 +170,7 @@ func (*CMakeHelper) wantsDir() string { return "/cure/" }
func (attr *CMakeHelper) script(name string) string { func (attr *CMakeHelper) script(name string) string {
if attr == nil { if attr == nil {
attr = &CMakeHelper{ attr = &CMakeHelper{
Cache: [][2]string{ Cache: []KV{
{"CMAKE_BUILD_TYPE", "Release"}, {"CMAKE_BUILD_TYPE", "Release"},
}, },
} }
@@ -200,7 +200,7 @@ cmake -G ` + generate + ` \
} }
}), " \\\n\t") + ` \ }), " \\\n\t") + ` \
-DCMAKE_INSTALL_PREFIX=/system \ -DCMAKE_INSTALL_PREFIX=/system \
'/usr/src/` + name + `/` + path.Join(attr.Append...) + `' '/usr/src/` + name + `/` + filepath.Join(attr.Append...) + `'
cmake --build .` + jobs + ` cmake --build .` + jobs + `
cmake --install . --prefix=/work/system cmake --install . --prefix=/work/system
` + attr.Script ` + attr.Script

109
internal/rosa/connman.go Normal file
View File

@@ -0,0 +1,109 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newConnman() (pkg.Artifact, string) {
const (
version = "2.0"
checksum = "MhVTdJOhndnZn2SWd8URKo_Pj7Zvc14tntEbrVOf9L3yVWJvpb3v3Q6104tWJgtW"
)
return t.NewPackage("connman", version, pkg.NewHTTPGetTar(
nil, "https://git.kernel.org/pub/scm/network/connman/connman.git/"+
"snapshot/connman-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), &PackageAttr{
Patches: []KV{
{"alpine-musl-res", `musl does not implement res_ninit
--- a/gweb/gresolv.c
+++ b/gweb/gresolv.c
@@ -877,8 +877,6 @@
resolv->index = index;
resolv->nameserver_list = NULL;
- res_ninit(&resolv->res);
-
return resolv;
}
@@ -918,8 +916,6 @@
flush_nameservers(resolv);
- res_nclose(&resolv->res);
-
g_free(resolv);
}
@@ -1022,24 +1018,19 @@
debug(resolv, "hostname %s", hostname);
if (!resolv->nameserver_list) {
- int i;
-
- for (i = 0; i < resolv->res.nscount; i++) {
- char buf[100];
- int family = resolv->res.nsaddr_list[i].sin_family;
- void *sa_addr = &resolv->res.nsaddr_list[i].sin_addr;
-
- if (family != AF_INET &&
- resolv->res._u._ext.nsaddrs[i]) {
- family = AF_INET6;
- sa_addr = &resolv->res._u._ext.nsaddrs[i]->sin6_addr;
+ FILE *f = fopen("/etc/resolv.conf", "r");
+ if (f) {
+ char line[256], *s;
+ int i;
+ while (fgets(line, sizeof(line), f)) {
+ if (strncmp(line, "nameserver", 10) || !isspace(line[10]))
+ continue;
+ for (s = &line[11]; isspace(s[0]); s++);
+ for (i = 0; s[i] && !isspace(s[i]); i++);
+ s[i] = 0;
+ g_resolv_add_nameserver(resolv, s, 53, 0);
}
-
- if (family != AF_INET && family != AF_INET6)
- continue;
-
- if (inet_ntop(family, sa_addr, buf, sizeof(buf)))
- g_resolv_add_nameserver(resolv, buf, 53, 0);
+ fclose(f);
}
if (!resolv->nameserver_list)
`},
},
}, &MakeHelper{
Generate: "./bootstrap",
},
Automake,
Libtool,
PkgConfig,
DBus,
IPTables,
GnuTLS,
Readline,
KernelHeaders,
), version
}
func init() {
artifactsM[Connman] = Metadata{
f: Toolchain.newConnman,
Name: "connman",
Description: "a daemon for managing Internet connections",
Website: "https://git.kernel.org/pub/scm/network/connman/connman.git/",
Dependencies: P{
DBus,
IPTables,
GnuTLS,
Readline,
},
ID: 337,
}
}

View File

@@ -18,7 +18,7 @@ func (t Toolchain) newCurl() (pkg.Artifact, string) {
chmod +w tests/data && rm tests/data/test459 chmod +w tests/data && rm tests/data/test459
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"with-openssl"}, {"with-openssl"},
{"with-ca-bundle", "/system/etc/ssl/certs/ca-bundle.crt"}, {"with-ca-bundle", "/system/etc/ssl/certs/ca-bundle.crt"},

46
internal/rosa/dbus.go Normal file
View File

@@ -0,0 +1,46 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newDBus() (pkg.Artifact, string) {
const (
version = "1.16.2"
checksum = "INwOuNdrDG7XW5ilW_vn8JSxEa444rRNc5ho97i84I1CNF09OmcFcV-gzbF4uCyg"
)
return t.NewPackage("dbus", version, pkg.NewHTTPGetTar(
nil, "https://gitlab.freedesktop.org/dbus/dbus/-/archive/"+
"dbus-"+version+"/dbus-dbus-"+version+".tar.bz2",
mustDecode(checksum),
pkg.TarBzip2,
), &PackageAttr{
// OSError: [Errno 30] Read-only file system: '/usr/src/dbus/subprojects/packagecache'
Writable: true,
// PermissionError: [Errno 13] Permission denied: '/usr/src/dbus/subprojects/packagecache'
Chmod: true,
}, &MesonHelper{
Setup: []KV{
{"Depoll", "enabled"},
{"Dinotify", "enabled"},
{"Dx11_autolaunch", "disabled"},
},
},
GLib,
Libexpat,
), version
}
func init() {
artifactsM[DBus] = Metadata{
f: Toolchain.newDBus,
Name: "dbus",
Description: "a message bus system",
Website: "https://www.freedesktop.org/wiki/Software/dbus/",
Dependencies: P{
GLib,
Libexpat,
},
ID: 5356,
}
}

View File

@@ -18,7 +18,7 @@ func (t Toolchain) newDTC() (pkg.Artifact, string) {
Writable: true, Writable: true,
Chmod: true, Chmod: true,
}, &MesonHelper{ }, &MesonHelper{
Setup: [][2]string{ Setup: []KV{
{"Dyaml", "disabled"}, {"Dyaml", "disabled"},
{"Dstatic-build", "true"}, {"Dstatic-build", "true"},
}, },

View File

@@ -22,7 +22,7 @@ func (t Toolchain) newElfutils() (pkg.Artifact, string) {
// nonstandard glibc extension // nonstandard glibc extension
SkipCheck: true, SkipCheck: true,
Configure: [][2]string{ Configure: []KV{
{"enable-deterministic-archives"}, {"enable-deterministic-archives"},
}, },
}, },

View File

@@ -25,7 +25,7 @@ func (a cureEtc) Cure(t *pkg.FContext) (err error) {
if err = os.MkdirAll(etc.String(), 0700); err != nil { if err = os.MkdirAll(etc.String(), 0700); err != nil {
return return
} }
for _, f := range [][2]string{ for _, f := range []KV{
{"hosts", "127.0.0.1 localhost cure cure-net\n"}, {"hosts", "127.0.0.1 localhost cure cure-net\n"},
{"passwd", `root:x:0:0:System administrator:/proc/nonexistent:/bin/sh {"passwd", `root:x:0:0:System administrator:/proc/nonexistent:/bin/sh
cure:x:1023:1023:Cure:/usr/src:/bin/sh cure:x:1023:1023:Cure:/usr/src:/bin/sh

View File

@@ -13,7 +13,7 @@ func (t Toolchain) newFakeroot() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarBzip2, pkg.TarBzip2,
), &PackageAttr{ ), &PackageAttr{
Patches: [][2]string{ Patches: []KV{
{"remove-broken-docs", `diff --git a/doc/Makefile.am b/doc/Makefile.am {"remove-broken-docs", `diff --git a/doc/Makefile.am b/doc/Makefile.am
index f135ad9..85c784c 100644 index f135ad9..85c784c 100644
--- a/doc/Makefile.am --- a/doc/Makefile.am

View File

@@ -4,8 +4,8 @@ import "hakurei.app/internal/pkg"
func (t Toolchain) newFuse() (pkg.Artifact, string) { func (t Toolchain) newFuse() (pkg.Artifact, string) {
const ( const (
version = "3.18.1" version = "3.18.2"
checksum = "COb-BgJRWXLbt9XUkNeuiroQizpMifXqxgieE1SlkMXhs_WGSyJStrmyewAw2hd6" checksum = "iL-7b7eUtmlVSf5cSq0dzow3UiqSjBmzV3cI_ENPs1tXcHdktkG45j1V12h-4jZe"
) )
return t.NewPackage("fuse", version, pkg.NewHTTPGetTar( return t.NewPackage("fuse", version, pkg.NewHTTPGetTar(
nil, "https://github.com/libfuse/libfuse/releases/download/"+ nil, "https://github.com/libfuse/libfuse/releases/download/"+
@@ -13,7 +13,7 @@ func (t Toolchain) newFuse() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarGzip,
), nil, &MesonHelper{ ), nil, &MesonHelper{
Setup: [][2]string{ Setup: []KV{
{"Ddefault_library", "both"}, {"Ddefault_library", "both"},
{"Dtests", "true"}, {"Dtests", "true"},
{"Duseroot", "false"}, {"Duseroot", "false"},

View File

@@ -1,6 +1,11 @@
package rosa package rosa
import "hakurei.app/internal/pkg" import (
"path"
"strings"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newGit() (pkg.Artifact, string) { func (t Toolchain) newGit() (pkg.Artifact, string) {
const ( const (
@@ -87,17 +92,23 @@ func init() {
// NewViaGit returns a [pkg.Artifact] for cloning a git repository. // NewViaGit returns a [pkg.Artifact] for cloning a git repository.
func (t Toolchain) NewViaGit( func (t Toolchain) NewViaGit(
name, url, rev string, url, rev string,
checksum pkg.Checksum, checksum pkg.Checksum,
) pkg.Artifact { ) pkg.Artifact {
return t.New(name+"-"+rev, 0, t.AppendPresets(nil, return t.New(strings.TrimSuffix(
path.Base(url),
".git",
)+"-src-"+path.Base(rev), 0, t.AppendPresets(nil,
NSSCACert, NSSCACert,
Git, Git,
), &checksum, nil, ` ), &checksum, nil, `
git \ git \
-c advice.detachedHead=false \ -c advice.detachedHead=false \
clone \ clone \
--depth=1 \
--revision=`+rev+` \ --revision=`+rev+` \
--shallow-submodules \
--recurse-submodules \
`+url+` \ `+url+` \
/work /work
rm -rf /work/.git rm -rf /work/.git

View File

@@ -1,6 +1,10 @@
package rosa package rosa
import "hakurei.app/internal/pkg" import (
"runtime"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newM4() (pkg.Artifact, string) { func (t Toolchain) newM4() (pkg.Artifact, string) {
const ( const (
@@ -88,8 +92,8 @@ func init() {
func (t Toolchain) newAutoconf() (pkg.Artifact, string) { func (t Toolchain) newAutoconf() (pkg.Artifact, string) {
const ( const (
version = "2.72" version = "2.73"
checksum = "-c5blYkC-xLDer3TWEqJTyh1RLbOd1c5dnRLKsDnIrg_wWNOLBpaqMY8FvmUFJ33" checksum = "yGabDTeOfaCUB0JX-h3REYLYzMzvpDwFmFFzHNR7QilChCUNE4hR6q7nma4viDYg"
) )
return t.NewPackage("autoconf", version, pkg.NewHTTPGetTar( return t.NewPackage("autoconf", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/autoconf/autoconf-"+version+".tar.gz", nil, "https://ftpmirror.gnu.org/gnu/autoconf/autoconf-"+version+".tar.gz",
@@ -351,7 +355,7 @@ func (t Toolchain) newBash() (pkg.Artifact, string) {
Flag: TEarly, Flag: TEarly,
}, &MakeHelper{ }, &MakeHelper{
Script: "ln -s bash /work/system/bin/sh\n", Script: "ln -s bash /work/system/bin/sh\n",
Configure: [][2]string{ Configure: []KV{
{"without-bash-malloc"}, {"without-bash-malloc"},
}, },
}), version }), version
@@ -390,7 +394,7 @@ test_disable 'int main(){return 0;}' gnulib-tests/test-fchownat.c
test_disable 'int main(){return 0;}' gnulib-tests/test-lchown.c test_disable 'int main(){return 0;}' gnulib-tests/test-lchown.c
`, `,
Patches: [][2]string{ Patches: []KV{
{"tests-fix-job-control", `From 21d287324aa43aa3a31f39619ade0deac7fd6013 Mon Sep 17 00:00:00 2001 {"tests-fix-job-control", `From 21d287324aa43aa3a31f39619ade0deac7fd6013 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com> From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com>
Date: Tue, 24 Feb 2026 15:44:41 +0000 Date: Tue, 24 Feb 2026 15:44:41 +0000
@@ -485,7 +489,7 @@ index 9a395416b..fbb043312 100755
Flag: TEarly, Flag: TEarly,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"enable-single-binary", "symlinks"}, {"enable-single-binary", "symlinks"},
}, },
}, },
@@ -720,7 +724,7 @@ func (t Toolchain) newTar() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarGzip,
), nil, &MakeHelper{ ), nil, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"disable-acl"}, {"disable-acl"},
{"without-posix-acls"}, {"without-posix-acls"},
{"without-xattrs"}, {"without-xattrs"},
@@ -754,8 +758,8 @@ func init() {
func (t Toolchain) newParallel() (pkg.Artifact, string) { func (t Toolchain) newParallel() (pkg.Artifact, string) {
const ( const (
version = "20260222" version = "20260322"
checksum = "4wxjMi3G2zMxr9hvLcIn6D7_12A3e5UNObeTPhzn7mDAYwsZApmmkxfGPyllQQ7E" checksum = "gHoPmFkOO62ev4xW59HqyMlodhjp8LvTsBOwsVKHUUdfrt7KwB8koXmSVqQ4VOrB"
) )
return t.NewPackage("parallel", version, pkg.NewHTTPGetTar( return t.NewPackage("parallel", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/parallel/parallel-"+version+".tar.bz2", nil, "https://ftpmirror.gnu.org/gnu/parallel/parallel-"+version+".tar.bz2",
@@ -781,6 +785,278 @@ func init() {
} }
} }
func (t Toolchain) newLibunistring() (pkg.Artifact, string) {
const (
version = "1.4.2"
checksum = "iW9BbfLoVlXjWoLTZ4AekQSu4cFBnLcZ4W8OHWbv0AhJNgD3j65_zqaLMzFKylg2"
)
return t.NewPackage("libunistring", version, pkg.NewHTTPGetTar(
nil, "https://ftp.gnu.org/gnu/libunistring/libunistring-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), &PackageAttr{
Writable: true,
ScriptEarly: `
test_disable() { chmod +w "$2" && echo "$1" > "$2"; }
test_disable '#!/bin/sh' tests/test-c32ispunct.sh
test_disable 'int main(){return 0;}' tests/test-c32ispunct.c
`,
}, (*MakeHelper)(nil),
Diffutils,
), version
}
func init() {
artifactsM[Libunistring] = Metadata{
f: Toolchain.newLibunistring,
Name: "libunistring",
Description: "provides functions for manipulating Unicode strings",
Website: "https://www.gnu.org/software/libunistring/",
ID: 1747,
}
}
func (t Toolchain) newLibtasn1() (pkg.Artifact, string) {
const (
version = "4.21.0"
checksum = "9DYI3UYbfYLy8JsKUcY6f0irskbfL0fHZA91Q-JEOA3kiUwpodyjemRsYRjUpjuq"
)
return t.NewPackage("libtasn1", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/libtasn1/libtasn1-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), nil, (*MakeHelper)(nil)), version
}
func init() {
artifactsM[Libtasn1] = Metadata{
f: Toolchain.newLibtasn1,
Name: "libtasn1",
Description: "the ASN.1 library used by GnuTLS, p11-kit and some other packages",
Website: "https://www.gnu.org/software/libtasn1/",
ID: 1734,
}
}
func (t Toolchain) newReadline() (pkg.Artifact, string) {
const (
version = "8.3"
checksum = "r-lcGRJq_MvvBpOq47Z2Y1OI2iqrmtcqhTLVXR0xWo37ZpC2uT_md7gKq5o_qTMV"
)
return t.NewPackage("readline", version, pkg.NewHTTPGetTar(
nil, "https://ftp.gnu.org/gnu/readline/readline-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), nil, &MakeHelper{
Configure: []KV{
{"with-curses"},
{"with-shared-termcap-library"},
},
},
Ncurses,
), version
}
func init() {
artifactsM[Readline] = Metadata{
f: Toolchain.newReadline,
Name: "readline",
Description: "provides a set of functions for use by applications that allow users to edit command lines as they are typed in",
Website: "https://tiswww.cwru.edu/php/chet/readline/rltop.html",
Dependencies: P{
Ncurses,
},
ID: 4173,
}
}
func (t Toolchain) newGnuTLS() (pkg.Artifact, string) {
const (
version = "3.8.12"
checksum = "VPdP-nRydQQRJcnma-YA7CJYA_kzTJ2rb3QFeP6D27emSyInJ8sQ-Wzn518I38dl"
)
var configureExtra []KV
switch runtime.GOARCH {
case "arm64":
configureExtra = []KV{
{"disable-hardware-acceleration"},
}
}
return t.NewPackage("gnutls", version, t.NewViaGit(
"https://gitlab.com/gnutls/gnutls.git",
"refs/tags/"+version,
mustDecode(checksum),
), &PackageAttr{
Patches: []KV{
{"bootstrap-remove-gtk-doc", `diff --git a/bootstrap.conf b/bootstrap.conf
index 1c3cc61e6..32bae9387 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -50,7 +50,6 @@ bison 2.4
gettext 0.17
git 1.4.4
gperf -
-gtkdocize -
perl 5.5
wget -
"
diff --git a/configure.ac b/configure.ac
index 5057536e5..731558a15 100644
--- a/configure.ac
+++ b/configure.ac
@@ -403,11 +403,6 @@ if test "$enable_fuzzer_target" != "no";then
AC_DEFINE([FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION], 1, [Enable fuzzer target -not for production])
fi
-dnl
-dnl check for gtk-doc
-dnl
-GTK_DOC_CHECK([1.14],[--flavour no-tmpl])
-
AM_GNU_GETTEXT([external])
AM_GNU_GETTEXT_VERSION([0.19])
m4_ifdef([AM_GNU_GET][TEXT_REQUIRE_VERSION],[
diff --git a/doc/Makefile.am b/doc/Makefile.am
index fb1390d70..52f0ad9af 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -33,9 +33,6 @@ IMAGES = \
pkcs11-vision.png
SUBDIRS = examples scripts credentials latex
-if ENABLE_GTK_DOC
-SUBDIRS += reference
-endif
-include $(top_srcdir)/doc/doc.mk
diff --git a/doc/reference/Makefile.am b/doc/reference/Makefile.am
index f10c8ed3c..b711b58ec 100644
--- a/doc/reference/Makefile.am
+++ b/doc/reference/Makefile.am
@@ -82,13 +82,4 @@ include $(top_srcdir)/gtk-doc.make
# e.g. EXTRA_DIST += version.xml.in
EXTRA_DIST += version.xml.in
-# Comment this out if you want 'make check' to test you doc status
-# and run some sanity checks
-if ENABLE_GTK_DOC
-TESTS_ENVIRONMENT = \
- DOC_MODULE=$(DOC_MODULE) DOC_MAIN_SGML_FILE=$(DOC_MAIN_SGML_FILE) \
- SRCDIR=$(abs_srcdir) BUILDDIR=$(abs_builddir)
-#TESTS = $(GTKDOC_CHECK)
-endif
-
-include $(top_srcdir)/git.mk
`},
{"alpine-tests-certtool", `I think this tests is simply wrong.
When a PIN is given, the program should run in batch mode.
So the question for "Enter password" should _not_ be present.
DO NOT REMOVE UNLESS VERIFIED IT'S NOT ACTUALLY NECESSARY ANYMORE.
--- a/tests/cert-tests/certtool.sh 2019-02-07 07:33:45.960887338 +0000
+++ b/tests/cert-tests/certtool.sh 2019-02-07 07:36:14.550955051 +0000
@@ -49,7 +49,7 @@
#check whether password is being honoured
#some CI runners need GNUTLS_PIN (GNUTLS_PIN=${PASS})
- ${SETSID} "${CERTTOOL}" --generate-self-signed --load-privkey ${TMPFILE1} --template ${srcdir}/templates/template-test.tmpl --ask-pass >${TMPFILE2} 2>&1 <<EOF
+ GNUTLS_PIN=${PASS} ${SETSID} "${CERTTOOL}" --generate-self-signed --load-privkey ${TMPFILE1} --template ${srcdir}/templates/template-test.tmpl --ask-pass >${TMPFILE2} 2>&1 <<EOF
$PASS
EOF
if test $? != 0;then
@@ -59,7 +59,7 @@
fi
grep "Enter password" ${TMPFILE2} >/dev/null 2>&1
- if test $? != 0;then
+ if test $? != 1; then
cat ${TMPFILE2}
echo "No password was asked"
exit 1
`},
{"test-kernel-version-ksh", `diff --git a/tests/scripts/common.sh b/tests/scripts/common.sh
index 1b78b8cf1..350156a86 100644
--- a/tests/scripts/common.sh
+++ b/tests/scripts/common.sh
@@ -279,10 +279,6 @@ kernel_version_check() {
kernel_major=$(echo $kernel_version | cut -d. -f1 2>/dev/null)
kernel_minor=$(echo $kernel_version | cut -d. -f2 2>/dev/null)
- if ! [[ "$kernel_major" =~ ^[0-9]+$ ]] || ! [[ "$kernel_minor" =~ ^[0-9]+$ ]]; then
- return 1
- fi
-
if [ "$kernel_major" -lt "$required_major" ]; then
return 1
fi
`},
},
}, &MakeHelper{
Generate: "./bootstrap --skip-po --no-git --gnulib-srcdir=gnulib",
Configure: append([]KV{
{"disable-doc"},
{"disable-openssl-compatibility"},
{"with-default-trust-store-file", "/system/etc/ssl/certs/ca-bundle.crt"},
{"with-default-trust-store-pkcs11", "pkcs11:"},
{"with-zlib", "link"},
{"with-zstd", "link"},
}, configureExtra...),
},
Gzip,
Automake,
Libtool,
Bison,
Gettext,
Gperf,
PkgConfig,
Python,
Texinfo,
Diffutils,
NSSCACert,
Libev,
Zlib,
Zstd,
P11Kit,
nettle3,
Libunistring,
), version
}
func init() {
artifactsM[GnuTLS] = Metadata{
f: Toolchain.newGnuTLS,
Name: "gnutls",
Description: "a secure communications library implementing the SSL, TLS and DTLS protocols",
Website: "https://gnutls.org",
Dependencies: P{
Zlib,
Zstd,
P11Kit,
nettle3,
Libunistring,
},
ID: 1221,
}
}
func (t Toolchain) newBinutils() (pkg.Artifact, string) { func (t Toolchain) newBinutils() (pkg.Artifact, string) {
const ( const (
version = "2.46.0" version = "2.46.0"
@@ -864,15 +1140,24 @@ func init() {
func (t Toolchain) newMPC() (pkg.Artifact, string) { func (t Toolchain) newMPC() (pkg.Artifact, string) {
const ( const (
version = "1.3.1" version = "1.4.0"
checksum = "o8r8K9R4x7PuRx0-JE3-bC5jZQrtxGV2nkB773aqJ3uaxOiBDCID1gKjPaaDxX4V" checksum = "TbrxLiE3ipQrHz_F3Xzz4zqBAnkMWyjhNwIK6wh9360RZ39xMt8rxfW3LxA9SnvU"
) )
return t.NewPackage("mpc", version, pkg.NewHTTPGetTar( return t.NewPackage("mpc", version, t.NewViaGit(
nil, "https://gcc.gnu.org/pub/gcc/infrastructure/"+ "https://gitlab.inria.fr/mpc/mpc.git",
"mpc-"+version+".tar.gz", "refs/tags/"+version,
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, ), &PackageAttr{
), nil, (*MakeHelper)(nil), // does not find mpc-impl.h otherwise
EnterSource: true,
}, &MakeHelper{
InPlace: true,
Generate: "autoreconf -vfi",
},
Automake,
Libtool,
Texinfo,
MPFR, MPFR,
), version ), version
} }
@@ -903,7 +1188,7 @@ func (t Toolchain) newGCC() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarGzip,
), &PackageAttr{ ), &PackageAttr{
Patches: [][2]string{ Patches: []KV{
{"musl-off64_t-loff_t", `diff --git a/libgo/sysinfo.c b/libgo/sysinfo.c {"musl-off64_t-loff_t", `diff --git a/libgo/sysinfo.c b/libgo/sysinfo.c
index 180f5c31d74..44d7ea73f7d 100644 index 180f5c31d74..44d7ea73f7d 100644
--- a/libgo/sysinfo.c --- a/libgo/sysinfo.c
@@ -1062,7 +1347,7 @@ ln -s system/lib /work/
// it also saturates the CPU for a consequential amount of time. // it also saturates the CPU for a consequential amount of time.
Flag: TExclusive, Flag: TExclusive,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"disable-multilib"}, {"disable-multilib"},
{"with-multilib-list", `""`}, {"with-multilib-list", `""`},
{"enable-default-pie"}, {"enable-default-pie"},

View File

@@ -135,7 +135,8 @@ sed -i \
cmd/link/internal/`+runtime.GOARCH+`/obj.go cmd/link/internal/`+runtime.GOARCH+`/obj.go
rm \ rm \
os/root_unix_test.go os/root_unix_test.go \
net/smtp/smtp_test.go
`, go123, `, go123,
) )

View File

@@ -1,8 +1,6 @@
package rosa package rosa
import ( import (
"strings"
"hakurei.app/fhs" "hakurei.app/fhs"
"hakurei.app/internal/pkg" "hakurei.app/internal/pkg"
) )
@@ -10,16 +8,13 @@ import (
func (t Toolchain) newGLib() (pkg.Artifact, string) { func (t Toolchain) newGLib() (pkg.Artifact, string) {
const ( const (
version = "2.88.0" version = "2.88.0"
checksum = "bCLkAmp1o_Po4cXDbC06AyjLyxkBxyNJnflwBpSdf4W8K6dc9xKj6Pm3JYbHPdDf" checksum = "T79Cg4z6j-sDZ2yIwvbY4ccRv2-fbwbqgcw59F5NQ6qJT6z4v261vbYp3dHO6Ma3"
) )
return t.NewPackage("glib", version, pkg.NewHTTPGet( return t.NewPackage("glib", version, t.NewViaGit(
nil, "https://download.gnome.org/sources/glib/"+ "https://gitlab.gnome.org/GNOME/glib.git",
strings.Join(strings.SplitN(version, ".", 3)[:2], ".")+ "refs/tags/"+version,
"/glib-"+version+".tar.xz",
mustDecode(checksum), mustDecode(checksum),
), &PackageAttr{ ), &PackageAttr{
SourceKind: SourceKindTarXZ,
Paths: []pkg.ExecPath{ Paths: []pkg.ExecPath{
pkg.Path(fhs.AbsEtc.Append( pkg.Path(fhs.AbsEtc.Append(
"machine-id", "machine-id",
@@ -35,11 +30,10 @@ func (t Toolchain) newGLib() (pkg.Artifact, string) {
)), )),
}, },
}, &MesonHelper{ }, &MesonHelper{
Setup: [][2]string{ Setup: []KV{
{"Ddefault_library", "both"}, {"Ddefault_library", "both"},
}, },
}, },
XZ,
PythonPackaging, PythonPackaging,
Bash, Bash,

View File

@@ -66,7 +66,7 @@ mkdir -p /work/system/libexec/hakurei/
echo '# Building hakurei.' echo '# Building hakurei.'
go generate -v ./... go generate -v ./...
go build -trimpath -v -o /work/system/libexec/hakurei -ldflags="-s -w go build -trimpath -v -tags=rosa -o /work/system/libexec/hakurei -ldflags="-s -w
-buildid= -buildid=
-linkmode external -linkmode external
-extldflags=-static -extldflags=-static

View File

@@ -23,4 +23,4 @@ var hakureiSource = pkg.NewTar(pkg.NewFile(
), pkg.TarGzip) ), pkg.TarGzip)
// hakureiPatches are patches applied against the compile-time source tree. // hakureiPatches are patches applied against the compile-time source tree.
var hakureiPatches [][2]string var hakureiPatches []KV

View File

@@ -15,4 +15,4 @@ var hakureiSource = pkg.NewHTTPGetTar(
) )
// hakureiPatches are patches applied against a hakurei release. // hakureiPatches are patches applied against a hakurei release.
var hakureiPatches [][2]string var hakureiPatches []KV

View File

@@ -2,12 +2,12 @@ package rosa
import "hakurei.app/internal/pkg" import "hakurei.app/internal/pkg"
const kernelVersion = "6.12.77" const kernelVersion = "6.12.80"
var kernelSource = pkg.NewHTTPGetTar( var kernelSource = pkg.NewHTTPGetTar(
nil, "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/"+ nil, "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/"+
"snapshot/linux-"+kernelVersion+".tar.gz", "snapshot/linux-"+kernelVersion+".tar.gz",
mustDecode("_MyFL0MqqNwAJx4fP8L9FkUayXIqEJto5trAPr_9UJvaT5TK1tvlU8leS82Hw2uw"), mustDecode("_iJEAYoQISJxefuWZYfv0RPWUmHHIjHQw33Fapix-irXrEIREP5ruK37UJW4uMZO"),
pkg.TarGzip, pkg.TarGzip,
) )
@@ -90,7 +90,7 @@ exec /system/sbin/depmod -m /lib/modules "$@"
`))), `))),
}, },
Patches: [][2]string{ Patches: []KV{
{"f54a91f5337cd918eb86cf600320d25b6cfd8209", `From f54a91f5337cd918eb86cf600320d25b6cfd8209 Mon Sep 17 00:00:00 2001 {"f54a91f5337cd918eb86cf600320d25b6cfd8209", `From f54a91f5337cd918eb86cf600320d25b6cfd8209 Mon Sep 17 00:00:00 2001
From: Nathan Chancellor <nathan@kernel.org> From: Nathan Chancellor <nathan@kernel.org>
Date: Sat, 13 Dec 2025 19:58:10 +0900 Date: Sat, 13 Dec 2025 19:58:10 +0900

View File

@@ -1,16 +1,16 @@
# #
# Automatically generated file; DO NOT EDIT. # Automatically generated file; DO NOT EDIT.
# Linux/x86 6.12.76 Kernel Configuration # Linux/x86 6.12.80 Kernel Configuration
# #
CONFIG_CC_VERSION_TEXT="clang version 22.1.1" CONFIG_CC_VERSION_TEXT="clang version 22.1.2"
CONFIG_GCC_VERSION=0 CONFIG_GCC_VERSION=0
CONFIG_CC_IS_CLANG=y CONFIG_CC_IS_CLANG=y
CONFIG_CLANG_VERSION=220101 CONFIG_CLANG_VERSION=220102
CONFIG_AS_IS_LLVM=y CONFIG_AS_IS_LLVM=y
CONFIG_AS_VERSION=220101 CONFIG_AS_VERSION=220102
CONFIG_LD_VERSION=0 CONFIG_LD_VERSION=0
CONFIG_LD_IS_LLD=y CONFIG_LD_IS_LLD=y
CONFIG_LLD_VERSION=220101 CONFIG_LLD_VERSION=220102
CONFIG_RUSTC_VERSION=0 CONFIG_RUSTC_VERSION=0
CONFIG_RUSTC_LLVM_VERSION=0 CONFIG_RUSTC_LLVM_VERSION=0
CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y

View File

@@ -1,16 +1,16 @@
# #
# Automatically generated file; DO NOT EDIT. # Automatically generated file; DO NOT EDIT.
# Linux/arm64 6.12.76 Kernel Configuration # Linux/arm64 6.12.80 Kernel Configuration
# #
CONFIG_CC_VERSION_TEXT="clang version 22.1.1" CONFIG_CC_VERSION_TEXT="clang version 21.1.8"
CONFIG_GCC_VERSION=0 CONFIG_GCC_VERSION=0
CONFIG_CC_IS_CLANG=y CONFIG_CC_IS_CLANG=y
CONFIG_CLANG_VERSION=220101 CONFIG_CLANG_VERSION=210108
CONFIG_AS_IS_LLVM=y CONFIG_AS_IS_LLVM=y
CONFIG_AS_VERSION=220101 CONFIG_AS_VERSION=210108
CONFIG_LD_VERSION=0 CONFIG_LD_VERSION=0
CONFIG_LD_IS_LLD=y CONFIG_LD_IS_LLD=y
CONFIG_LLD_VERSION=220101 CONFIG_LLD_VERSION=210108
CONFIG_RUSTC_VERSION=0 CONFIG_RUSTC_VERSION=0
CONFIG_RUSTC_LLVM_VERSION=0 CONFIG_RUSTC_LLVM_VERSION=0
CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y
@@ -73,6 +73,7 @@ CONFIG_IRQ_MSI_IOMMU=y
CONFIG_IRQ_FORCED_THREADING=y CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y CONFIG_SPARSE_IRQ=y
# CONFIG_GENERIC_IRQ_DEBUGFS is not set # CONFIG_GENERIC_IRQ_DEBUGFS is not set
CONFIG_GENERIC_IRQ_KEXEC_CLEAR_VM_FORWARD=y
# end of IRQ subsystem # end of IRQ subsystem
CONFIG_GENERIC_TIME_VSYSCALL=y CONFIG_GENERIC_TIME_VSYSCALL=y

View File

@@ -13,7 +13,7 @@ func (t Toolchain) newKmod() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarGzip,
), nil, &MesonHelper{ ), nil, &MesonHelper{
Setup: [][2]string{ Setup: []KV{
{"Dmoduledir", "/system/lib/modules"}, {"Dmoduledir", "/system/lib/modules"},
{"Dsysconfdir", "/system/etc"}, {"Dsysconfdir", "/system/etc"},
{"Dbashcompletiondir", "no"}, {"Dbashcompletiondir", "no"},

64
internal/rosa/libbsd.go Normal file
View File

@@ -0,0 +1,64 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newLibmd() (pkg.Artifact, string) {
const (
version = "1.1.0"
checksum = "9apYqPPZm0j5HQT8sCsVIhnVIqRD7XgN7kPIaTwTqnTuUq5waUAMq4M7ev8CODJ1"
)
return t.NewPackage("libmd", version, t.NewViaGit(
"https://git.hadrons.org/git/libmd.git",
"refs/tags/"+version,
mustDecode(checksum),
), nil, &MakeHelper{
Generate: "echo '" + version + "' > .dist-version && ./autogen",
ScriptMakeEarly: `
install -D /usr/src/libmd/src/helper.c src/helper.c
`,
},
Automake,
Libtool,
), version
}
func init() {
artifactsM[Libmd] = Metadata{
f: Toolchain.newLibmd,
Name: "libmd",
Description: "Message Digest functions from BSD systems",
Website: "https://www.hadrons.org/software/libmd/",
ID: 15525,
}
}
func (t Toolchain) newLibbsd() (pkg.Artifact, string) {
const (
version = "0.12.2"
checksum = "NVS0xFLTwSP8JiElEftsZ-e1_C-IgJhHrHE77RwKt5178M7r087waO-zYx2_dfGX"
)
return t.NewPackage("libbsd", version, t.NewViaGit(
"https://gitlab.freedesktop.org/libbsd/libbsd.git",
"refs/tags/"+version,
mustDecode(checksum),
), nil, &MakeHelper{
Generate: "echo '" + version + "' > .dist-version && ./autogen",
},
Automake,
Libtool,
Libmd,
), version
}
func init() {
artifactsM[Libbsd] = Metadata{
f: Toolchain.newLibbsd,
Name: "libbsd",
Description: "provides useful functions commonly found on BSD systems",
Website: "https://libbsd.freedesktop.org/",
ID: 1567,
}
}

26
internal/rosa/libev.go Normal file
View File

@@ -0,0 +1,26 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newLibev() (pkg.Artifact, string) {
const (
version = "4.33"
checksum = "774eSXV_4k8PySRprUDChbEwsw-kzjIFnJ3MpNOl5zDpamBRvC3BqPyRxvkwcL6_"
)
return t.NewPackage("libev", version, pkg.NewHTTPGetTar(
nil, "https://dist.schmorp.de/libev/Attic/libev-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), nil, (*MakeHelper)(nil)), version
}
func init() {
artifactsM[Libev] = Metadata{
f: Toolchain.newLibev,
Name: "libev",
Description: "a full-featured and high-performance event loop",
Website: "http://libev.schmorp.de/",
ID: 1605,
}
}

View File

@@ -8,8 +8,8 @@ import (
func (t Toolchain) newLibexpat() (pkg.Artifact, string) { func (t Toolchain) newLibexpat() (pkg.Artifact, string) {
const ( const (
version = "2.7.4" version = "2.7.5"
checksum = "W6NI2FESBjrTqRPcvs15fK5c3nwF6f9RT8U-XHKQKblXVzJB3nt_ez5B5jO0ZVDG" checksum = "vTRUjjg-qbHSXUBYKXgzVHkUO7UNyuhrkSYrE7ikApQm0g-OvQ8tspw4w55M-1Tp"
) )
return t.NewPackage("libexpat", version, pkg.NewHTTPGetTar( return t.NewPackage("libexpat", version, pkg.NewHTTPGetTar(
nil, "https://github.com/libexpat/libexpat/releases/download/"+ nil, "https://github.com/libexpat/libexpat/releases/download/"+

View File

@@ -16,6 +16,23 @@ func (t Toolchain) newLibseccomp() (pkg.Artifact, string) {
ScriptEarly: ` ScriptEarly: `
ln -s ../system/bin/bash /bin/ ln -s ../system/bin/bash /bin/
`, `,
Patches: []KV{
{"fix-export-oob-read", `diff --git a/src/api.c b/src/api.c
index adccef3..65a277a 100644
--- a/src/api.c
+++ b/src/api.c
@@ -786,7 +786,7 @@ API int seccomp_export_bpf_mem(const scmp_filter_ctx ctx, void *buf,
if (BPF_PGM_SIZE(program) > *len)
rc = _rc_filter(-ERANGE);
else
- memcpy(buf, program->blks, *len);
+ memcpy(buf, program->blks, BPF_PGM_SIZE(program));
}
*len = BPF_PGM_SIZE(program);
`},
},
}, (*MakeHelper)(nil), }, (*MakeHelper)(nil),
Bash, Bash,
Diffutils, Diffutils,

View File

@@ -1,26 +1,22 @@
package rosa package rosa
import ( import "hakurei.app/internal/pkg"
"strings"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newLibxml2() (pkg.Artifact, string) { func (t Toolchain) newLibxml2() (pkg.Artifact, string) {
const ( const (
version = "2.15.2" version = "2.15.2"
checksum = "xba8VCofMsbWmQypA2__M9_RXNq9HDEuccjib6-tOni6OPngplRoAsYdY3NdYf8o" checksum = "zwQvCIBnjzUFY-inX5ckfNT3mIezsCRV55C_Iztde5OnRTB3u33lfO5h03g7DK_8"
) )
return t.NewPackage("libxml2", version, pkg.NewHTTPGet( return t.NewPackage("libxml2", version, t.NewViaGit(
nil, "https://download.gnome.org/sources/libxml2/"+ "https://gitlab.gnome.org/GNOME/libxml2.git",
strings.Join(strings.Split(version, ".")[:2], ".")+ "refs/tags/v"+version,
"/libxml2-"+version+".tar.xz",
mustDecode(checksum), mustDecode(checksum),
), &PackageAttr{ ), &PackageAttr{
SourceKind: SourceKindTarXZ, // can't create shell.out: Read-only file system
}, (*MakeHelper)(nil), Writable: true,
}, (*MesonHelper)(nil),
Git,
Diffutils, Diffutils,
XZ,
), version ), version
} }
func init() { func init() {

View File

@@ -1,28 +1,24 @@
package rosa package rosa
import ( import "hakurei.app/internal/pkg"
"strings"
"hakurei.app/internal/pkg"
)
func (t Toolchain) newLibxslt() (pkg.Artifact, string) { func (t Toolchain) newLibxslt() (pkg.Artifact, string) {
const ( const (
version = "1.1.45" version = "1.1.45"
checksum = "vw72UbREQnA3YDYuZ9-93hDr9BYCaKV6oh_U4Kt4n1Js_na4E-nFj-ksZnZ0kvEK" checksum = "MZc_dyUWpHChkWDKa5iycrECxBsRd4ZMbYfL4VojTbung593mlH2tHGmxYB6NFYT"
) )
return t.NewPackage("libxslt", version, pkg.NewHTTPGet( return t.NewPackage("libxslt", version, t.NewViaGit(
nil, "https://download.gnome.org/sources/libxslt/"+ "https://gitlab.gnome.org/GNOME/libxslt.git",
strings.Join(strings.Split(version, ".")[:2], ".")+ "refs/tags/v"+version,
"/libxslt-"+version+".tar.xz",
mustDecode(checksum), mustDecode(checksum),
), &PackageAttr{ ), nil, &MakeHelper{
SourceKind: SourceKindTarXZ, Generate: "NOCONFIGURE=1 ./autogen.sh",
}, &MakeHelper{
// python libxml2 cyclic dependency // python libxml2 cyclic dependency
SkipCheck: true, SkipCheck: true,
}, },
XZ, Automake,
Libtool,
Python, Python,
PkgConfig, PkgConfig,

View File

@@ -19,7 +19,7 @@ type llvmAttr struct {
// Concatenated with default environment for PackageAttr.Env. // Concatenated with default environment for PackageAttr.Env.
env []string env []string
// Concatenated with generated entries for CMakeHelper.Cache. // Concatenated with generated entries for CMakeHelper.Cache.
cmake [][2]string cmake []KV
// Override CMakeHelper.Append. // Override CMakeHelper.Append.
append []string append []string
// Passed through to PackageAttr.NonStage0. // Passed through to PackageAttr.NonStage0.
@@ -30,7 +30,7 @@ type llvmAttr struct {
script string script string
// Patch name and body pairs. // Patch name and body pairs.
patches [][2]string patches []KV
} }
const ( const (
@@ -94,43 +94,45 @@ func (t Toolchain) newLLVMVariant(variant string, attr *llvmAttr) pkg.Artifact {
var script string var script string
cache := [][2]string{ cache := []KV{
{"CMAKE_BUILD_TYPE", "Release"}, {"CMAKE_BUILD_TYPE", "Release"},
{"LLVM_HOST_TRIPLE", `"${ROSA_TRIPLE}"`}, {"LLVM_HOST_TRIPLE", `"${ROSA_TRIPLE}"`},
{"LLVM_DEFAULT_TARGET_TRIPLE", `"${ROSA_TRIPLE}"`}, {"LLVM_DEFAULT_TARGET_TRIPLE", `"${ROSA_TRIPLE}"`},
} }
if len(projects) > 0 { if len(projects) > 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LLVM_ENABLE_PROJECTS", `"${ROSA_LLVM_PROJECTS}"`}) {"LLVM_ENABLE_PROJECTS", `"${ROSA_LLVM_PROJECTS}"`},
}...)
} }
if len(runtimes) > 0 { if len(runtimes) > 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LLVM_ENABLE_RUNTIMES", `"${ROSA_LLVM_RUNTIMES}"`}) {"LLVM_ENABLE_RUNTIMES", `"${ROSA_LLVM_RUNTIMES}"`},
}...)
} }
cmakeAppend := []string{"llvm"} cmakeAppend := []string{"llvm"}
if attr.append != nil { if attr.append != nil {
cmakeAppend = attr.append cmakeAppend = attr.append
} else { } else {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LLVM_ENABLE_LIBCXX", "ON"}, {"LLVM_ENABLE_LIBCXX", "ON"},
[2]string{"LLVM_USE_LINKER", "lld"}, {"LLVM_USE_LINKER", "lld"},
[2]string{"LLVM_INSTALL_BINUTILS_SYMLINKS", "ON"}, {"LLVM_INSTALL_BINUTILS_SYMLINKS", "ON"},
[2]string{"LLVM_INSTALL_CCTOOLS_SYMLINKS", "ON"}, {"LLVM_INSTALL_CCTOOLS_SYMLINKS", "ON"},
[2]string{"LLVM_LIT_ARGS", "'--verbose'"}, {"LLVM_LIT_ARGS", "'--verbose'"},
) }...)
} }
if attr.flags&llvmProjectClang != 0 { if attr.flags&llvmProjectClang != 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"CLANG_DEFAULT_LINKER", "lld"}, {"CLANG_DEFAULT_LINKER", "lld"},
[2]string{"CLANG_DEFAULT_CXX_STDLIB", "libc++"}, {"CLANG_DEFAULT_CXX_STDLIB", "libc++"},
[2]string{"CLANG_DEFAULT_RTLIB", "compiler-rt"}, {"CLANG_DEFAULT_RTLIB", "compiler-rt"},
[2]string{"CLANG_DEFAULT_UNWINDLIB", "libunwind"}, {"CLANG_DEFAULT_UNWINDLIB", "libunwind"},
) }...)
} }
if attr.flags&llvmProjectLld != 0 { if attr.flags&llvmProjectLld != 0 {
script += ` script += `
@@ -139,25 +141,27 @@ ln -s ld.lld /work/system/bin/ld
} }
if attr.flags&llvmRuntimeCompilerRT != 0 { if attr.flags&llvmRuntimeCompilerRT != 0 {
if attr.append == nil { if attr.append == nil {
cache = append(cache, cache = append(cache, []KV{
[2]string{"COMPILER_RT_USE_LLVM_UNWINDER", "ON"}) {"COMPILER_RT_USE_LLVM_UNWINDER", "ON"},
}...)
} }
} }
if attr.flags&llvmRuntimeLibunwind != 0 { if attr.flags&llvmRuntimeLibunwind != 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LIBUNWIND_USE_COMPILER_RT", "ON"}) {"LIBUNWIND_USE_COMPILER_RT", "ON"},
}...)
} }
if attr.flags&llvmRuntimeLibcxx != 0 { if attr.flags&llvmRuntimeLibcxx != 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LIBCXX_HAS_MUSL_LIBC", "ON"}, {"LIBCXX_HAS_MUSL_LIBC", "ON"},
[2]string{"LIBCXX_USE_COMPILER_RT", "ON"}, {"LIBCXX_USE_COMPILER_RT", "ON"},
) }...)
} }
if attr.flags&llvmRuntimeLibcxxABI != 0 { if attr.flags&llvmRuntimeLibcxxABI != 0 {
cache = append(cache, cache = append(cache, []KV{
[2]string{"LIBCXXABI_USE_COMPILER_RT", "ON"}, {"LIBCXXABI_USE_COMPILER_RT", "ON"},
[2]string{"LIBCXXABI_USE_LLVM_UNWINDER", "ON"}, {"LIBCXXABI_USE_LLVM_UNWINDER", "ON"},
) }...)
} }
return t.NewPackage("llvm", llvmVersion, pkg.NewHTTPGetTar( return t.NewPackage("llvm", llvmVersion, pkg.NewHTTPGetTar(
@@ -208,7 +212,7 @@ func (t Toolchain) newLLVM() (musl, compilerRT, runtimes, clang pkg.Artifact) {
panic("unsupported target " + runtime.GOARCH) panic("unsupported target " + runtime.GOARCH)
} }
minimalDeps := [][2]string{ minimalDeps := []KV{
{"LLVM_ENABLE_ZLIB", "OFF"}, {"LLVM_ENABLE_ZLIB", "OFF"},
{"LLVM_ENABLE_ZSTD", "OFF"}, {"LLVM_ENABLE_ZSTD", "OFF"},
{"LLVM_ENABLE_LIBXML2", "OFF"}, {"LLVM_ENABLE_LIBXML2", "OFF"},
@@ -222,7 +226,7 @@ func (t Toolchain) newLLVM() (musl, compilerRT, runtimes, clang pkg.Artifact) {
env: stage0ExclConcat(t, []string{}, env: stage0ExclConcat(t, []string{},
"LDFLAGS="+earlyLDFLAGS(false), "LDFLAGS="+earlyLDFLAGS(false),
), ),
cmake: [][2]string{ cmake: []KV{
// libc++ not yet available // libc++ not yet available
{"CMAKE_CXX_COMPILER_TARGET", ""}, {"CMAKE_CXX_COMPILER_TARGET", ""},
@@ -272,7 +276,7 @@ ln -s \
"LDFLAGS="+earlyLDFLAGS(false), "LDFLAGS="+earlyLDFLAGS(false),
), ),
flags: llvmRuntimeLibunwind | llvmRuntimeLibcxx | llvmRuntimeLibcxxABI, flags: llvmRuntimeLibunwind | llvmRuntimeLibcxx | llvmRuntimeLibcxxABI,
cmake: slices.Concat([][2]string{ cmake: slices.Concat([]KV{
// libc++ not yet available // libc++ not yet available
{"CMAKE_CXX_COMPILER_WORKS", "ON"}, {"CMAKE_CXX_COMPILER_WORKS", "ON"},
@@ -293,7 +297,7 @@ ln -s \
"CXXFLAGS="+earlyCXXFLAGS(), "CXXFLAGS="+earlyCXXFLAGS(),
"LDFLAGS="+earlyLDFLAGS(false), "LDFLAGS="+earlyLDFLAGS(false),
), ),
cmake: slices.Concat([][2]string{ cmake: slices.Concat([]KV{
{"LLVM_TARGETS_TO_BUILD", target}, {"LLVM_TARGETS_TO_BUILD", target},
{"CMAKE_CROSSCOMPILING", "OFF"}, {"CMAKE_CROSSCOMPILING", "OFF"},
{"CXX_SUPPORTS_CUSTOM_LINKER", "ON"}, {"CXX_SUPPORTS_CUSTOM_LINKER", "ON"},
@@ -310,7 +314,7 @@ ln -s clang++ /work/system/bin/c++
ninja check-all ninja check-all
`, `,
patches: slices.Concat([][2]string{ patches: slices.Concat([]KV{
{"add-rosa-vendor", `diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h {"add-rosa-vendor", `diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h
index 9c83abeeb3b1..5acfe5836a23 100644 index 9c83abeeb3b1..5acfe5836a23 100644
--- a/llvm/include/llvm/TargetParser/Triple.h --- a/llvm/include/llvm/TargetParser/Triple.h

View File

@@ -1,4 +1,4 @@
package rosa package rosa
// clangPatches are patches applied to the LLVM source tree for building clang. // clangPatches are patches applied to the LLVM source tree for building clang.
var clangPatches [][2]string var clangPatches []KV

View File

@@ -1,7 +1,7 @@
package rosa package rosa
// clangPatches are patches applied to the LLVM source tree for building clang. // clangPatches are patches applied to the LLVM source tree for building clang.
var clangPatches [][2]string var clangPatches []KV
// one version behind, latest fails 5 tests with 2 flaky on arm64 // one version behind, latest fails 5 tests with 2 flaky on arm64
const ( const (

View File

@@ -5,7 +5,7 @@ package rosa
// latest version of LLVM, conditional to temporarily avoid broken new releases // latest version of LLVM, conditional to temporarily avoid broken new releases
const ( const (
llvmVersionMajor = "22" llvmVersionMajor = "22"
llvmVersion = llvmVersionMajor + ".1.1" llvmVersion = llvmVersionMajor + ".1.2"
llvmChecksum = "bQvV6D8AZvQykg7-uQb_saTbVavnSo1ykNJ3g57F5iE-evU3HuOYtcRnVIXTK76e" llvmChecksum = "FwsmurWDVyYYQlOowowFjekwIGSB5__aKTpW_VGP3eWoZGXvBny-bOn1DuQ1U5xE"
) )

View File

@@ -60,7 +60,7 @@ type MakeHelper struct {
// Alternative name for the configure script. // Alternative name for the configure script.
ConfigureName string ConfigureName string
// Flags passed to the configure script. // Flags passed to the configure script.
Configure [][2]string Configure []KV
// Host target triple, zero value is equivalent to the Rosa OS triple. // Host target triple, zero value is equivalent to the Rosa OS triple.
Host string Host string
// Target triple, zero value is equivalent to the Rosa OS triple. // Target triple, zero value is equivalent to the Rosa OS triple.
@@ -84,8 +84,8 @@ func (*MakeHelper) name(name, version string) string {
} }
// extra returns make and other optional dependencies. // extra returns make and other optional dependencies.
func (attr *MakeHelper) extra(flag int) []PArtifact { func (attr *MakeHelper) extra(flag int) P {
extra := []PArtifact{Make} extra := P{Make}
if (attr == nil || !attr.OmitDefaults) && flag&TEarly == 0 { if (attr == nil || !attr.OmitDefaults) && flag&TEarly == 0 {
extra = append(extra, extra = append(extra,
Gawk, Gawk,

View File

@@ -59,7 +59,7 @@ type MesonHelper struct {
Script string Script string
// Flags passed to the setup command. // Flags passed to the setup command.
Setup [][2]string Setup []KV
// Whether to skip meson test. // Whether to skip meson test.
SkipTest bool SkipTest bool
} }
@@ -72,9 +72,7 @@ func (*MesonHelper) name(name, version string) string {
} }
// extra returns hardcoded meson runtime dependencies. // extra returns hardcoded meson runtime dependencies.
func (*MesonHelper) extra(int) []PArtifact { func (*MesonHelper) extra(int) P { return P{Meson} }
return []PArtifact{Meson}
}
// wantsChmod returns false. // wantsChmod returns false.
func (*MesonHelper) wantsChmod() bool { return false } func (*MesonHelper) wantsChmod() bool { return false }
@@ -113,7 +111,8 @@ meson test \
cd "$(mktemp -d)" cd "$(mktemp -d)"
meson setup \ meson setup \
` + strings.Join(slices.Collect(func(yield func(string) bool) { ` + strings.Join(slices.Collect(func(yield func(string) bool) {
for _, v := range append([][2]string{ for _, v := range append([]KV{
{"wrap-mode", "nodownload"},
{"prefix", "/system"}, {"prefix", "/system"},
{"buildtype", "release"}, {"buildtype", "release"},
}, attr.Setup...) { }, attr.Setup...) {

View File

@@ -8,8 +8,8 @@ func (t Toolchain) newMusl(
extra ...pkg.Artifact, extra ...pkg.Artifact,
) (pkg.Artifact, string) { ) (pkg.Artifact, string) {
const ( const (
version = "1.2.5" version = "1.2.6"
checksum = "y6USdIeSdHER_Fw2eT2CNjqShEye85oEg2jnOur96D073ukmIpIqDOLmECQroyDb" checksum = "WtWb_OV_XxLDAB5NerOL9loLlHVadV00MmGk65PPBU1evaolagoMHfvpZp_vxEzS"
) )
name := "musl" name := "musl"

View File

@@ -15,9 +15,11 @@ func (t Toolchain) newNcurses() (pkg.Artifact, string) {
// "tests" are actual demo programs, not a test suite. // "tests" are actual demo programs, not a test suite.
SkipCheck: true, SkipCheck: true,
Configure: [][2]string{ Configure: []KV{
{"with-pkg-config"}, {"with-pkg-config"},
{"enable-pc-files"}, {"enable-pc-files"},
{"with-shared"},
{"with-cxx-shared"},
}, },
}, },
PkgConfig, PkgConfig,

149
internal/rosa/netfilter.go Normal file
View File

@@ -0,0 +1,149 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newLibmnl() (pkg.Artifact, string) {
const (
version = "1.0.5"
checksum = "DN-vbbvQDpxXJm0TJ6xlluILvfrB86avrCTX50XyE9SEFSAZ_o8nuKc5Gu0Am7-u"
)
return t.NewPackage("libmnl", version, pkg.NewHTTPGetTar(
nil, "https://www.netfilter.org/projects/libmnl/files/"+
"libmnl-"+version+".tar.bz2",
mustDecode(checksum),
pkg.TarBzip2,
), &PackageAttr{
Patches: []KV{
{"libbsd-sys-queue", `diff --git a/examples/netfilter/nfct-daemon.c b/examples/netfilter/nfct-daemon.c
index d223ac2..a7878d0 100644
--- a/examples/netfilter/nfct-daemon.c
+++ b/examples/netfilter/nfct-daemon.c
@@ -20,7 +20,7 @@
#include <linux/netfilter/nfnetlink.h>
#include <linux/netfilter/nfnetlink_conntrack.h>
-#include <sys/queue.h>
+#include <bsd/sys/queue.h>
struct nstats {
LIST_ENTRY(nstats) list;
`},
},
}, &MakeHelper{
Configure: []KV{
{"enable-static"},
},
},
Libbsd,
KernelHeaders,
), version
}
func init() {
artifactsM[Libmnl] = Metadata{
f: Toolchain.newLibmnl,
Name: "libmnl",
Description: "a minimalistic user-space library oriented to Netlink developers",
Website: "https://www.netfilter.org/projects/libmnl/",
ID: 1663,
}
}
func (t Toolchain) newLibnftnl() (pkg.Artifact, string) {
const (
version = "1.3.1"
checksum = "91ou66K-I17iX6DB6hiQkhhC_v4DFW5iDGzwjVRNbJNEmKqowLZBlh3FY-ZDO0r9"
)
return t.NewPackage("libnftnl", version, t.NewViaGit(
"https://git.netfilter.org/libnftnl",
"refs/tags/libnftnl-"+version,
mustDecode(checksum),
), &PackageAttr{
Env: []string{
"CFLAGS=-D_GNU_SOURCE",
},
}, &MakeHelper{
Generate: "./autogen.sh",
Configure: []KV{
{"enable-static"},
},
},
Automake,
Libtool,
PkgConfig,
Libmnl,
KernelHeaders,
), version
}
func init() {
artifactsM[Libnftnl] = Metadata{
f: Toolchain.newLibnftnl,
Name: "libnftnl",
Description: "a userspace library providing a low-level netlink API to the in-kernel nf_tables subsystem",
Website: "https://www.netfilter.org/projects/libnftnl/",
Dependencies: P{
Libmnl,
},
ID: 1681,
}
}
func (t Toolchain) newIPTables() (pkg.Artifact, string) {
const (
version = "1.8.13"
checksum = "TUA-cFIAsiMvtRR-XzQvXzoIhJUOc9J2gQDJCbBRjmgmVfGfPTCf58wL7e-cUKVQ"
)
return t.NewPackage("iptables", version, t.NewViaGit(
"https://git.netfilter.org/iptables",
"refs/tags/v"+version,
mustDecode(checksum),
), &PackageAttr{
ScriptEarly: `
rm \
extensions/libxt_connlabel.txlate \
extensions/libxt_conntrack.txlate
sed -i \
's/de:ad:0:be:ee:ff/DE:AD:00:BE:EE:FF/g' \
extensions/libebt_dnat.txlate \
extensions/libebt_snat.txlate
`,
}, &MakeHelper{
Generate: "./autogen.sh",
Configure: []KV{
{"enable-static"},
},
ScriptCheckEarly: `
ln -s ../system/bin/bash /bin/
chmod +w /etc/ && ln -s ../usr/src/iptables/etc/ethertypes /etc/
`,
},
Automake,
Libtool,
PkgConfig,
Bash,
Python,
Libnftnl,
KernelHeaders,
), version
}
func init() {
artifactsM[IPTables] = Metadata{
f: Toolchain.newIPTables,
Name: "iptables",
Description: "the userspace command line program used to configure the Linux 2.4.x and later packet filtering ruleset",
Website: "https://www.netfilter.org/projects/iptables/",
Dependencies: P{
Libnftnl,
},
ID: 1394,
}
}

33
internal/rosa/nettle3.go Normal file
View File

@@ -0,0 +1,33 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newNettle3() (pkg.Artifact, string) {
const (
version = "3.10.2"
checksum = "07aXlj10X5llf67jIqRQAA1pgLSgb0w_JYggZVPuKNoc-B-_usb5Kr8FrfBe7g1S"
)
return t.NewPackage("nettle", version, pkg.NewHTTPGetTar(
nil, "https://ftpmirror.gnu.org/gnu/nettle/nettle-"+version+".tar.gz",
mustDecode(checksum),
pkg.TarGzip,
), nil, (*MakeHelper)(nil),
M4,
Diffutils,
GMP,
), version
}
func init() {
artifactsM[nettle3] = Metadata{
f: Toolchain.newNettle3,
Name: "nettle3",
Description: "a low-level cryptographic library",
Website: "https://www.lysator.liu.se/~nisse/nettle/",
Dependencies: P{
GMP,
},
}
}

View File

@@ -8,8 +8,8 @@ import (
func (t Toolchain) newNSS() (pkg.Artifact, string) { func (t Toolchain) newNSS() (pkg.Artifact, string) {
const ( const (
version = "3.121" version = "3.122"
checksum = "MTS4Eg-1vBN3T7gdUAdNO0y_e9x9BE3f_k_DHdM_BIovc7y57vhsZTfB5f6BeQfi" checksum = "QvC6TBO4BAUEh6wmgUrb1hwH5podQAN-QdcAaWL32cWEppmZs6oKkZpD9GvZf59S"
version0 = "4_38_2" version0 = "4_38_2"
checksum0 = "25x2uJeQnOHIiq_zj17b4sYqKgeoU8-IsySUptoPcdHZ52PohFZfGuIisBreWzx0" checksum0 = "25x2uJeQnOHIiq_zj17b4sYqKgeoU8-IsySUptoPcdHZ52PohFZfGuIisBreWzx0"

View File

@@ -20,7 +20,7 @@ func (t Toolchain) newOpenSSL() (pkg.Artifact, string) {
OmitDefaults: true, OmitDefaults: true,
ConfigureName: "/usr/src/openssl/Configure", ConfigureName: "/usr/src/openssl/Configure",
Configure: [][2]string{ Configure: []KV{
{"prefix", "/system"}, {"prefix", "/system"},
{"libdir", "lib"}, {"libdir", "lib"},
{"openssldir", "etc/ssl"}, {"openssldir", "etc/ssl"},

40
internal/rosa/p11.go Normal file
View File

@@ -0,0 +1,40 @@
package rosa
import "hakurei.app/internal/pkg"
func (t Toolchain) newP11Kit() (pkg.Artifact, string) {
const (
version = "0.26.2"
checksum = "3ei-6DUVtYzrRVe-SubtNgRlweXd6H2qHmUu-_5qVyIn6gSTvZbGS2u79Y8IFb2N"
)
return t.NewPackage("p11-kit", version, t.NewViaGit(
"https://github.com/p11-glue/p11-kit.git",
"refs/tags/"+version, mustDecode(checksum),
), nil, &MesonHelper{
Setup: []KV{
{"Dsystemd", "disabled"},
{"Dlibffi", "enabled"},
},
},
Coreutils,
Diffutils,
Libtasn1,
), version
}
func init() {
artifactsM[P11Kit] = Metadata{
f: Toolchain.newP11Kit,
Name: "p11-kit",
Description: "provides a way to load and enumerate PKCS#11 modules",
Website: "https://p11-glue.freedesktop.org/p11-kit.html",
Dependencies: P{
Libffi,
Libtasn1,
},
ID: 2582,
}
}

View File

@@ -20,7 +20,7 @@ func (t Toolchain) newPCRE2() (pkg.Artifact, string) {
ln -s ../system/bin/toybox /bin/echo ln -s ../system/bin/toybox /bin/echo
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"enable-jit"}, {"enable-jit"},
{"enable-pcre2-8"}, {"enable-pcre2-8"},
{"enable-pcre2-16"}, {"enable-pcre2-16"},

View File

@@ -8,8 +8,8 @@ import (
func (t Toolchain) newPerl() (pkg.Artifact, string) { func (t Toolchain) newPerl() (pkg.Artifact, string) {
const ( const (
version = "5.42.1" version = "5.42.2"
checksum = "FsJVq5CZFA7nZklfUl1eC6z2ECEu02XaB1pqfHSKtRLZWpnaBjlB55QOhjKpjkQ2" checksum = "Me_xFfgkRnVyG0sE6a74TktK2OUq9Z1LVJNEu_9RdZG3S2fbjfzNiuk2SJqHAgbm"
) )
return t.NewPackage("perl", version, pkg.NewHTTPGetTar( return t.NewPackage("perl", version, pkg.NewHTTPGetTar(
nil, "https://www.cpan.org/src/5.0/perl-"+version+".tar.gz", nil, "https://www.cpan.org/src/5.0/perl-"+version+".tar.gz",
@@ -31,7 +31,7 @@ rm -f /system/bin/ps # perl does not like toybox ps
InPlace: true, InPlace: true,
ConfigureName: "./Configure", ConfigureName: "./Configure",
Configure: [][2]string{ Configure: []KV{
{"-des"}, {"-des"},
{"Dprefix", "/system"}, {"Dprefix", "/system"},
{"Dcc", "clang"}, {"Dcc", "clang"},
@@ -67,7 +67,7 @@ func init() {
func (t Toolchain) newViaPerlModuleBuild( func (t Toolchain) newViaPerlModuleBuild(
name, version string, name, version string,
source pkg.Artifact, source pkg.Artifact,
patches [][2]string, patches []KV,
extra ...PArtifact, extra ...PArtifact,
) pkg.Artifact { ) pkg.Artifact {
if name == "" || version == "" { if name == "" || version == "" {
@@ -116,7 +116,7 @@ func init() {
func (t Toolchain) newViaPerlMakeMaker( func (t Toolchain) newViaPerlMakeMaker(
name, version string, name, version string,
source pkg.Artifact, source pkg.Artifact,
patches [][2]string, patches []KV,
extra ...PArtifact, extra ...PArtifact,
) pkg.Artifact { ) pkg.Artifact {
return t.NewPackage("perl-"+name, version, source, &PackageAttr{ return t.NewPackage("perl-"+name, version, source, &PackageAttr{
@@ -131,11 +131,11 @@ func (t Toolchain) newViaPerlMakeMaker(
InPlace: true, InPlace: true,
ConfigureName: "perl Makefile.PL", ConfigureName: "perl Makefile.PL",
Configure: [][2]string{ Configure: []KV{
{"PREFIX", "/system"}, {"PREFIX", "/system"},
}, },
Check: []string{"test"}, Check: []string{"test"},
}, slices.Concat(extra, []PArtifact{ }, slices.Concat(extra, P{
Perl, Perl,
})...) })...)
} }

View File

@@ -5,19 +5,23 @@ import "hakurei.app/internal/pkg"
func (t Toolchain) newPkgConfig() (pkg.Artifact, string) { func (t Toolchain) newPkgConfig() (pkg.Artifact, string) {
const ( const (
version = "0.29.2" version = "0.29.2"
checksum = "gi7yAvkwo20Inys1tHbeYZ3Wjdm5VPkrnO0Q6_QZPCAwa1zrA8F4a63cdZDd-717" checksum = "6UsGqEMA8EER_5b9N0b32UCqiRy39B6_RnPfvuslWhtFV1qYD4DfS10crGZN_TP2"
) )
return t.NewPackage("pkg-config", version, pkg.NewHTTPGetTar( return t.NewPackage("pkg-config", version, pkg.NewHTTPGetTar(
nil, "https://pkgconfig.freedesktop.org/releases/"+ nil, "https://gitlab.freedesktop.org/pkg-config/pkg-config/-/archive"+
"pkg-config-"+version+".tar.gz", "/pkg-config-"+version+"/pkg-config-pkg-config-"+version+".tar.bz2",
mustDecode(checksum), mustDecode(checksum),
pkg.TarGzip, pkg.TarBzip2,
), nil, &MakeHelper{ ), nil, &MakeHelper{
Configure: [][2]string{ Generate: "./autogen.sh --no-configure",
Configure: []KV{
{"CFLAGS", "'-Wno-int-conversion'"}, {"CFLAGS", "'-Wno-int-conversion'"},
{"with-internal-glib"}, {"with-internal-glib"},
}, },
}), version },
Automake,
Libtool,
), version
} }
func init() { func init() {
artifactsM[PkgConfig] = Metadata{ artifactsM[PkgConfig] = Metadata{

View File

@@ -14,7 +14,7 @@ func (t Toolchain) newProcps() (pkg.Artifact, string) {
pkg.TarBzip2, pkg.TarBzip2,
), nil, &MakeHelper{ ), nil, &MakeHelper{
Generate: "./autogen.sh", Generate: "./autogen.sh",
Configure: [][2]string{ Configure: []KV{
{"without-ncurses"}, {"without-ncurses"},
}, },
}, },

View File

@@ -4,15 +4,15 @@ import "hakurei.app/internal/pkg"
func (t Toolchain) newQEMU() (pkg.Artifact, string) { func (t Toolchain) newQEMU() (pkg.Artifact, string) {
const ( const (
version = "10.2.1" version = "10.2.2"
checksum = "rjLTSgHJd3X3Vgpxrsus_ZZiaYLiNix1YhcHaGbLd_odYixwZjCcAIt8CVQPJGdZ" checksum = "uNzRxlrVoLWe-EmZmBp75SezymgE512iE5XN90Bl7wi6CjE_oQGQB-9ocs7E16QG"
) )
return t.NewPackage("qemu", version, pkg.NewHTTPGetTar( return t.NewPackage("qemu", version, pkg.NewHTTPGetTar(
nil, "https://download.qemu.org/qemu-"+version+".tar.bz2", nil, "https://download.qemu.org/qemu-"+version+".tar.bz2",
mustDecode(checksum), mustDecode(checksum),
pkg.TarBzip2, pkg.TarBzip2,
), &PackageAttr{ ), &PackageAttr{
Patches: [][2]string{ Patches: []KV{
{"disable-mcast-test", `diff --git a/tests/qtest/netdev-socket.c b/tests/qtest/netdev-socket.c {"disable-mcast-test", `diff --git a/tests/qtest/netdev-socket.c b/tests/qtest/netdev-socket.c
index b731af0ad9..b5cbed4801 100644 index b731af0ad9..b5cbed4801 100644
--- a/tests/qtest/netdev-socket.c --- a/tests/qtest/netdev-socket.c
@@ -58,7 +58,7 @@ _notrun 'appears to spuriously fail on zfs'
EOF EOF
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"disable-download"}, {"disable-download"},
{"disable-docs"}, {"disable-docs"},

View File

@@ -3,7 +3,7 @@ package rosa_test
import ( import (
"errors" "errors"
"os" "os"
"path" "path/filepath"
"syscall" "syscall"
"testing" "testing"
"unique" "unique"
@@ -13,7 +13,7 @@ import (
) )
func TestReportZeroLength(t *testing.T) { func TestReportZeroLength(t *testing.T) {
report := path.Join(t.TempDir(), "report") report := filepath.Join(t.TempDir(), "report")
if err := os.WriteFile(report, nil, 0400); err != nil { if err := os.WriteFile(report, nil, 0400); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -24,7 +24,7 @@ func TestReportZeroLength(t *testing.T) {
} }
func TestReportSIGSEGV(t *testing.T) { func TestReportSIGSEGV(t *testing.T) {
report := path.Join(t.TempDir(), "report") report := filepath.Join(t.TempDir(), "report")
if err := os.WriteFile(report, make([]byte, 64), 0400); err != nil { if err := os.WriteFile(report, make([]byte, 64), 0400); err != nil {
t.Fatal(err) t.Fatal(err)
} }

View File

@@ -20,9 +20,6 @@ const (
// kindBusyboxBin is the kind of [pkg.Artifact] of busyboxBin. // kindBusyboxBin is the kind of [pkg.Artifact] of busyboxBin.
kindBusyboxBin kindBusyboxBin
// kindCollection is the kind of [Collect]. It never cures successfully.
kindCollection
) )
// mustDecode is like [pkg.MustDecode], but replaces the zero value and prints // mustDecode is like [pkg.MustDecode], but replaces the zero value and prints
@@ -40,6 +37,9 @@ func mustDecode(s string) pkg.Checksum {
return pkg.MustDecode(s) return pkg.MustDecode(s)
} }
// KV is a key-value pair of strings.
type KV [2]string
var ( var (
// AbsUsrSrc is the conventional directory to place source code under. // AbsUsrSrc is the conventional directory to place source code under.
AbsUsrSrc = fhs.AbsUsr.Append("src") AbsUsrSrc = fhs.AbsUsr.Append("src")
@@ -202,6 +202,10 @@ func lastIndexFunc[S ~[]E, E any](s S, f func(E) bool) (i int) {
// fixupEnviron fixes up PATH, prepends extras and returns the resulting slice. // fixupEnviron fixes up PATH, prepends extras and returns the resulting slice.
func fixupEnviron(env, extras []string, paths ...string) []string { func fixupEnviron(env, extras []string, paths ...string) []string {
// some python tools try to be clever and buffers their output, making the
// build process appear to hang
env = append(env, "PYTHONUNBUFFERED=1")
const pathPrefix = "PATH=" const pathPrefix = "PATH="
pathVal := strings.Join(paths, ":") pathVal := strings.Join(paths, ":")
@@ -363,7 +367,7 @@ func (t Toolchain) NewPatchedSource(
name, version string, name, version string,
source pkg.Artifact, source pkg.Artifact,
passthrough bool, passthrough bool,
patches ...[2]string, patches ...KV,
) pkg.Artifact { ) pkg.Artifact {
if passthrough && len(patches) == 0 { if passthrough && len(patches) == 0 {
return source return source
@@ -405,7 +409,7 @@ type Helper interface {
// name returns the value passed to the name argument of [Toolchain.New]. // name returns the value passed to the name argument of [Toolchain.New].
name(name, version string) string name(name, version string) string
// extra returns helper-specific dependencies. // extra returns helper-specific dependencies.
extra(flag int) []PArtifact extra(flag int) P
// wantsChmod returns whether the source directory should be made writable. // wantsChmod returns whether the source directory should be made writable.
wantsChmod() bool wantsChmod() bool
@@ -445,7 +449,7 @@ type PackageAttr struct {
ScriptEarly string ScriptEarly string
// Passed to [Toolchain.NewPatchedSource]. // Passed to [Toolchain.NewPatchedSource].
Patches [][2]string Patches []KV
// Kind of source artifact. // Kind of source artifact.
SourceKind int SourceKind int
@@ -591,29 +595,3 @@ cd '/usr/src/` + name + `/'
})..., })...,
) )
} }
// Collected is returned by [Collect.Cure] to indicate a successful collection.
type Collected struct{}
// Error returns a constant string to satisfy error, but should never be seen
// by the user.
func (Collected) Error() string { return "artifacts successfully collected" }
// Collect implements [pkg.FloodArtifact] to concurrently cure multiple
// [pkg.Artifact]. It returns [Collected].
type Collect []pkg.Artifact
// Cure returns [Collected].
func (*Collect) Cure(*pkg.FContext) error { return Collected{} }
// Kind returns the hardcoded [pkg.Kind] value.
func (*Collect) Kind() pkg.Kind { return kindCollection }
// Params does not write anything, dependencies are already represented in the header.
func (*Collect) Params(*pkg.IContext) {}
// Dependencies returns [Collect] as is.
func (c *Collect) Dependencies() []pkg.Artifact { return *c }
// IsExclusive returns false: Cure is a noop.
func (*Collect) IsExclusive() bool { return false }

View File

@@ -60,7 +60,7 @@ func getCache(t *testing.T) *pkg.Cache {
msg := message.New(log.New(os.Stderr, "rosa: ", 0)) msg := message.New(log.New(os.Stderr, "rosa: ", 0))
msg.SwapVerbose(true) msg.SwapVerbose(true)
if buildTestCache, err = pkg.Open(ctx, msg, 0, a); err != nil { if buildTestCache, err = pkg.Open(ctx, msg, 0, 0, a); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
@@ -91,3 +91,13 @@ func TestCureAll(t *testing.T) {
}) })
} }
} }
func BenchmarkStage3(b *testing.B) {
for b.Loop() {
rosa.Std.Load(rosa.LLVMClang)
b.StopTimer()
rosa.DropCaches()
b.StartTimer()
}
}

View File

@@ -15,7 +15,7 @@ func (t Toolchain) newRsync() (pkg.Artifact, string) {
), &PackageAttr{ ), &PackageAttr{
Flag: TEarly, Flag: TEarly,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"disable-openssl"}, {"disable-openssl"},
{"disable-xxhash"}, {"disable-xxhash"},
{"disable-zstd"}, {"disable-zstd"},

View File

@@ -23,7 +23,7 @@ sed -i 's/unsigned int msg_len;$/uint32_t msg_len;/g' \
tests/nlattr.c tests/nlattr.c
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
// tests broken on clang // tests broken on clang
{"disable-gcc-Werror"}, {"disable-gcc-Werror"},

View File

@@ -8,8 +8,8 @@ import (
func (t Toolchain) newUtilLinux() (pkg.Artifact, string) { func (t Toolchain) newUtilLinux() (pkg.Artifact, string) {
const ( const (
version = "2.41.3" version = "2.42"
checksum = "gPTd5JJ2ho_Rd0qainuogcLiiWwKSXEZPXN3yCCRl0m0KBgMaqwFuMjYgu9z8zCH" checksum = "Uy8Nxg9DsW5YwDoeaZeZTyQJ2YmnaaL_fSsQXsLUiFFUd7wnZeD_3SEaVO7ClJlk"
) )
return t.NewPackage("util-linux", version, pkg.NewHTTPGetTar( return t.NewPackage("util-linux", version, pkg.NewHTTPGetTar(
nil, "https://www.kernel.org/pub/linux/utils/util-linux/"+ nil, "https://www.kernel.org/pub/linux/utils/util-linux/"+
@@ -22,7 +22,7 @@ func (t Toolchain) newUtilLinux() (pkg.Artifact, string) {
ln -s ../system/bin/bash /bin/ ln -s ../system/bin/bash /bin/
`, `,
}, &MakeHelper{ }, &MakeHelper{
Configure: [][2]string{ Configure: []KV{
{"disable-use-tty-group"}, {"disable-use-tty-group"},
{"disable-makeinstall-setuid"}, {"disable-makeinstall-setuid"},
{"disable-makeinstall-chown"}, {"disable-makeinstall-chown"},

View File

@@ -4,8 +4,8 @@ import "hakurei.app/internal/pkg"
func (t Toolchain) newWayland() (pkg.Artifact, string) { func (t Toolchain) newWayland() (pkg.Artifact, string) {
const ( const (
version = "1.24.91" version = "1.25.0"
checksum = "SQkjYShk2TutoBOfmeJcdLU9iDExVKOg0DZhLeL8U_qjc9olLTC7h3vuUBvVtx9w" checksum = "q-4dYXme46JPgLGtXAxyZGTy7udll9RfT0VXtcW2YRR1WWViUhvdZXZneXzLqpCg"
) )
return t.NewPackage("wayland", version, pkg.NewHTTPGetTar( return t.NewPackage("wayland", version, pkg.NewHTTPGetTar(
nil, "https://gitlab.freedesktop.org/wayland/wayland/"+ nil, "https://gitlab.freedesktop.org/wayland/wayland/"+
@@ -20,7 +20,7 @@ chmod +w tests tests/sanity-test.c
echo 'int main(){}' > tests/sanity-test.c echo 'int main(){}' > tests/sanity-test.c
`, `,
}, &MesonHelper{ }, &MesonHelper{
Setup: [][2]string{ Setup: []KV{
{"Ddefault_library", "both"}, {"Ddefault_library", "both"},
{"Ddocumentation", "false"}, {"Ddocumentation", "false"},
{"Dtests", "true"}, {"Dtests", "true"},
@@ -54,8 +54,8 @@ func init() {
func (t Toolchain) newWaylandProtocols() (pkg.Artifact, string) { func (t Toolchain) newWaylandProtocols() (pkg.Artifact, string) {
const ( const (
version = "1.47" version = "1.48"
checksum = "B_NodZ7AQfCstcx7kgbaVjpkYOzbAQq0a4NOk-SA8bQixAE20FY3p1-6gsbPgHn9" checksum = "xvfHCBIzXGwOqOu9b8dfhGw_U29Pd-g4JBwpYIaxee8SwEbxi6NaVU-Y1Q7wY4jK"
) )
return t.NewPackage("wayland-protocols", version, pkg.NewHTTPGetTar( return t.NewPackage("wayland-protocols", version, pkg.NewHTTPGetTar(
nil, "https://gitlab.freedesktop.org/wayland/wayland-protocols/"+ nil, "https://gitlab.freedesktop.org/wayland/wayland-protocols/"+
@@ -63,7 +63,7 @@ func (t Toolchain) newWaylandProtocols() (pkg.Artifact, string) {
mustDecode(checksum), mustDecode(checksum),
pkg.TarBzip2, pkg.TarBzip2,
), &PackageAttr{ ), &PackageAttr{
Patches: [][2]string{ Patches: []KV{
{"build-only", `From 8b4c76275fa1b6e0a99a53494151d9a2c907144d Mon Sep 17 00:00:00 2001 {"build-only", `From 8b4c76275fa1b6e0a99a53494151d9a2c907144d Mon Sep 17 00:00:00 2001
From: "A. Wilcox" <AWilcox@Wilcox-Tech.com> From: "A. Wilcox" <AWilcox@Wilcox-Tech.com>
Date: Fri, 8 Nov 2024 11:27:25 -0600 Date: Fri, 8 Nov 2024 11:27:25 -0600

Some files were not shown because too many files have changed in this diff Show More