0ab6c13c77
internal/pkg: consistency check for on-disk cache
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m32s
Test / ShareFS (push) Successful in 3m43s
Test / Hpkg (push) Successful in 4m29s
Test / Sandbox (race detector) (push) Successful in 4m57s
Test / Hakurei (race detector) (push) Successful in 5m45s
Test / Hakurei (push) Successful in 2m31s
Test / Flake checks (push) Successful in 1m44s
This change adds a method to check on-disk cache consistency and destroy inconsistent entries as they are encountered. This primarily helps verify artifact implementation correctness, but can also repair a cache that got into an inconsistent state from curing a misbehaving artifact, without having to destroy the entire cache.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 05:30:29 +09:00
834cb0d40b
internal/pkg: override "." for directory checksum
...
Test / Create distribution (push) Successful in 46s
Test / Sandbox (push) Successful in 2m45s
Test / ShareFS (push) Successful in 3m51s
Test / Hpkg (push) Successful in 4m34s
Test / Sandbox (race detector) (push) Successful in 5m0s
Test / Hakurei (race detector) (push) Successful in 6m0s
Test / Hakurei (push) Successful in 2m34s
Test / Flake checks (push) Successful in 1m50s
This makes the checksum consistent with the final resting state of artifact directories without incurring the cost of an extra pair of chown syscalls.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 04:00:13 +09:00
7548a627e5
internal/pkg: delete stale done channels
...
Test / Create distribution (push) Successful in 44s
Test / Sandbox (push) Successful in 2m31s
Test / Hakurei (push) Successful in 3m37s
Test / ShareFS (push) Successful in 3m43s
Test / Hpkg (push) Successful in 4m21s
Test / Sandbox (race detector) (push) Successful in 4m57s
Test / Hakurei (race detector) (push) Successful in 5m50s
Test / Flake checks (push) Successful in 1m41s
There is no reason to keep these around.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 02:12:34 +09:00
f3aa31e401
internal/pkg: temporary scratch space for cure
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m30s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m21s
Test / Sandbox (race detector) (push) Successful in 5m4s
Test / Hakurei (race detector) (push) Successful in 5m52s
Test / Hakurei (push) Successful in 2m35s
Test / Flake checks (push) Successful in 1m52s
This allows for more flexibility during implementation. The use case that required this was for expanding single directory tarballs.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 01:39:18 +09:00
4da26681b5
internal/pkg: compute http identifier from url
...
Test / Create distribution (push) Successful in 44s
Test / Sandbox (push) Successful in 2m30s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m24s
Test / Sandbox (race detector) (push) Successful in 4m46s
Test / Hakurei (race detector) (push) Successful in 5m51s
Test / Hakurei (push) Successful in 2m28s
Test / Flake checks (push) Successful in 1m41s
The previous implementation exposes arbitrary user input to the cache as an identifier, which is highly error-prone and can cause the cache to enter an inconsistent state if the user is not careful. This change replaces the implementation to compute identifier late, using url string as params.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 00:43:21 +09:00
4897b0259e
internal/pkg: improve artifact interface
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m35s
Test / Hakurei (push) Successful in 3m36s
Test / ShareFS (push) Successful in 3m41s
Test / Hpkg (push) Successful in 4m19s
Test / Sandbox (race detector) (push) Successful in 4m52s
Test / Hakurei (race detector) (push) Successful in 5m52s
Test / Flake checks (push) Successful in 1m53s
This moves all cache I/O code to Cache. Artifact now only contains methods for constructing their actual contents.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-05 00:01:23 +09:00
3eb927823f
internal/pkg: create symlinks for files
...
Test / Create distribution (push) Successful in 42s
Test / Sandbox (push) Successful in 2m45s
Test / Hakurei (push) Successful in 3m36s
Test / ShareFS (push) Successful in 3m42s
Test / Hpkg (push) Successful in 4m17s
Test / Sandbox (race detector) (push) Successful in 4m49s
Test / Hakurei (race detector) (push) Successful in 5m48s
Test / Flake checks (push) Successful in 1m44s
These are much easier to handle than hard links and should be just as transparent for this use case.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-04 01:48:53 +09:00
fa93476896
internal/pkg: override working directory perms
...
Test / Create distribution (push) Successful in 42s
Test / Sandbox (push) Successful in 2m31s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m20s
Test / Sandbox (race detector) (push) Successful in 4m51s
Test / Hakurei (race detector) (push) Successful in 5m52s
Test / Hakurei (push) Successful in 2m32s
Test / Flake checks (push) Successful in 1m42s
This must be writable to enable renaming, and the final result is conventionally read-only alongside the entire directory contents. This change overrides the permission bits as part of Store.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-04 00:55:52 +09:00
bd0ef086b1
internal/pkg: enable cache access during store
...
Test / Create distribution (push) Successful in 45s
Test / Sandbox (push) Successful in 2m37s
Test / Hakurei (push) Successful in 3m37s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m32s
Test / Sandbox (race detector) (push) Successful in 4m57s
Test / Hakurei (race detector) (push) Successful in 5m53s
Test / Flake checks (push) Successful in 1m47s
This is still not ideal as it makes entry into Store sequential. This will be improved after more usage code is written.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-04 00:39:14 +09:00
40081e7a06
internal/pkg: implement caching for directories
...
Test / Create distribution (push) Successful in 45s
Test / Sandbox (push) Successful in 2m30s
Test / Hakurei (push) Successful in 3m38s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m29s
Test / Sandbox (race detector) (push) Successful in 4m50s
Test / Hakurei (race detector) (push) Successful in 5m49s
Test / Flake checks (push) Successful in 1m43s
This works on any directories and should be robust against any bad state the artifact curing process might have failed at.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-03 22:54:46 +09:00
863d3dcf9f
internal/pkg: wrap checksum string encoding
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m36s
Test / ShareFS (push) Successful in 3m46s
Test / Hpkg (push) Successful in 4m24s
Test / Sandbox (race detector) (push) Successful in 4m53s
Test / Hakurei (race detector) (push) Successful in 5m48s
Test / Hakurei (push) Successful in 2m30s
Test / Flake checks (push) Successful in 1m45s
This wraps base64.URLEncoding.EncodeToString for cleaner call site.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-03 22:03:25 +09:00
8ad9909065
internal/pkg: compute identifier from deps
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m31s
Test / Hakurei (push) Successful in 3m34s
Test / ShareFS (push) Successful in 3m40s
Test / Hpkg (push) Successful in 4m21s
Test / Sandbox (race detector) (push) Successful in 4m51s
Test / Hakurei (race detector) (push) Successful in 5m50s
Test / Flake checks (push) Successful in 1m46s
This provides infrastructure for computing a deterministic identifier based on current artifact kind, opaque parameters data, and optional dependency kind and identifiers.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-03 21:26:25 +09:00
deda16da38
internal/pkg: create work directory
...
Test / Create distribution (push) Successful in 43s
Test / Sandbox (push) Successful in 2m39s
Test / Hakurei (push) Successful in 3m35s
Test / ShareFS (push) Successful in 3m38s
Test / Hpkg (push) Successful in 4m15s
Test / Sandbox (race detector) (push) Successful in 4m50s
Test / Hakurei (race detector) (push) Successful in 5m50s
Test / Flake checks (push) Successful in 1m43s
This is used for artifacts that cure into directories.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-03 20:56:11 +09:00
e9de5d3aca
internal/pkg: implement caching for files
...
Test / Create distribution (push) Successful in 45s
Test / Sandbox (push) Successful in 2m30s
Test / Hakurei (push) Successful in 3m30s
Test / ShareFS (push) Successful in 3m27s
Test / Hpkg (push) Successful in 4m19s
Test / Sandbox (race detector) (push) Successful in 4m40s
Test / Hakurei (race detector) (push) Successful in 5m37s
Test / Flake checks (push) Successful in 1m40s
This change contains primitives for validating and caching single-file artifacts.
Signed-off-by: Ophestra <cat@gensokyo.uk >
2026-01-02 12:57:19 +09:00