frankenbit/release-toolkit

Fork 0

chore(bake): does BUILD_BAKED_TOOLKIT_REF have the same placeholder hole as the digest? #1214

New issue

Closed

opened 2026-09-05 22:53:38 +02:00 by bosun · 3 comments

bosun commented

2026-09-05 22:53:38 +02:00

Owner

BUILD_BAKED_TOOLKIT_REF is the deliberate sibling of the image digest and may have the same placeholder-shaped hole, but nobody has tested it.

Requested by @shipwright while building #1205, explicitly as a hypothesis rather than a finding.

The shape to test

#1205 established that digestShapeRE accepts the all-zeros placeholder because 0 is valid hex and there are exactly 64 — the validator grades SHAPE and cannot distinguish "not yet rewritten" from "rewritten". Twelve published tags shipped dangling as a result.

BUILD_BAKED_TOOLKIT_REF has the same lifecycle: a placeholder on main between cuts, rewritten at bake time, with its own validator. Its between-cuts value is 'main'.

image digest              placeholder sha256:0000…  shape-valid, ACCEPTED   -> 12 dangling tags
BUILD_BAKED_TOOLKIT_REF   placeholder 'main'         shape-valid?  UNTESTED

⚠️ 'main' is a legitimate ref, so a shape check cannot reject it — and unlike the digest there may be no shape to check at all. That is what makes this a real question rather than a copy of #1205.

Scope

Establish whether the marker can ship un-rewritten at a tag, the way the digest did
If it can: the same identity-not-shape treatment as #1205, and a census across published tags
~~If it cannot: record WHY, so the asymmetry with the digest is documented rather than assumed~~ — RETIRED (it CAN): measured un-rewritten at v1.0.0-alpha.0, so this branch of the AC is unreachable.

🔑 #1173 is prior evidence that this family goes wrong: reusable-recover-pending-cut.yml sat pinned at v0.57.0 for five releases because it was absent from canonicalFiles — a different mechanism reaching the same outcome, an un-rewritten marker at a tag.

Explicitly a hypothesis

📌 @shipwright: "the sibling has the same defect is a hypothesis I have not tested." He requested this rather than widening #1209, and the tracker should not be closed by asserting the sibling is fine — it closes on a measurement either way.

#1205 / #1209 (the digest gate), #1173 (an un-rewritten marker from a different cause), #1180 (the Baker writing a twin-dirty tree — same file family)

Anchor

@shipwright, 2026-09-05, while building #1209. Filed by @bosun on his request.

`BUILD_BAKED_TOOLKIT_REF` is the deliberate sibling of the image digest and may have the same placeholder-shaped hole, but nobody has tested it. Requested by @shipwright while building `#1205`, explicitly as a hypothesis rather than a finding. ## The shape to test `#1205` established that `digestShapeRE` accepts the all-zeros placeholder because `0` is valid hex and there are exactly 64 — **the validator grades SHAPE and cannot distinguish "not yet rewritten" from "rewritten".** Twelve published tags shipped dangling as a result. `BUILD_BAKED_TOOLKIT_REF` has the same lifecycle: **a placeholder on `main` between cuts, rewritten at bake time, with its own validator.** Its between-cuts value is `'main'`. ``` image digest placeholder sha256:0000… shape-valid, ACCEPTED -> 12 dangling tags BUILD_BAKED_TOOLKIT_REF placeholder 'main' shape-valid? UNTESTED ``` ⚠️ **`'main'` is a legitimate ref, so a shape check cannot reject it — and unlike the digest there may be no shape to check at all.** That is what makes this a real question rather than a copy of `#1205`. ## Scope - [x] Establish whether the marker can ship un-rewritten at a tag, the way the digest did - [x] If it can: the same identity-not-shape treatment as `#1205`, and a census across published tags - [x] ~~If it cannot: record WHY, so the asymmetry with the digest is documented rather than assumed~~ — **RETIRED (it CAN):** measured un-rewritten at `v1.0.0-alpha.0`, so this branch of the AC is unreachable. 🔑 **`#1173` is prior evidence that this family goes wrong**: `reusable-recover-pending-cut.yml` sat pinned at `v0.57.0` for five releases because it was absent from `canonicalFiles` — **a different mechanism reaching the same outcome, an un-rewritten marker at a tag.** ## Explicitly a hypothesis 📌 **@shipwright: *"the sibling has the same defect is a hypothesis I have not tested."*** He requested this rather than widening `#1209`, and the tracker should not be closed by asserting the sibling is fine — **it closes on a measurement either way.** ## Related `#1205` / `#1209` (the digest gate), `#1173` (an un-rewritten marker from a different cause), `#1180` (the Baker writing a twin-dirty tree — same file family) ## Anchor @shipwright, 2026-09-05, while building `#1209`. Filed by @bosun on his request.

bosun added the

priority/medium

size/S

kind/chore

labels

2026-09-05 22:53:38 +02:00

bosun referenced this issue

2026-09-05 23:04:11 +02:00

Campaign: close every open release-toolkit issue, in five staged drains #1159

bosun commented

2026-09-06 09:42:38 +02:00

Author

Owner

🔴 MEASURED. @shipwright's HYPOTHESIS IS CONFIRMED: THE MARKER CAN SHIP UN-REWRITTEN, AND IT HAS.

The census — every published tag, `reusable-release.yml`

marker == the tag                 73
marker != the tag                  1     <- v1.0.0-alpha.0, value 'main'
file absent at that tag           40     (predates the file; checked, not assumed —
                                          zero tags have the file WITHOUT the marker)

v1.0.0-alpha.0   BUILD_BAKED_TOOLKIT_REF: 'main'      <- the placeholder, PUBLISHED
                 release draft=false prerelease=true  assets=2  2026-07-25T00:13:54
                 tagged commit 271cc01, NOT an ancestor of main (the v2/next
                 bootstrap lineage, ADR-0009 §3.3)

✅ So AC1 is answered YES, with an instance rather than an argument, and the census AC is done.

🔴 And the part that makes this worth a remedy rather than a note: THERE IS NO VALIDATOR

digestShapeRE exists for the image digest. There is no equivalent for this marker. Every match in internal/bake and cmd/rt/bake_digest.go is a comment or a test constant — nothing checks that the baked ref is the tag being cut.

⚠️ And the tracker anticipated why a shape check cannot be the answer: 'main' is a legitimate ref. The digest's placeholder was catchable in principle because sha256:0{64} is a recognisable shape; 'main' is indistinguishable from a deliberate float. ✅ The remedy has to be IDENTITY, not shape: at bake time, assert the marker equals the tag being cut, and refuse otherwise.

What the measurement does NOT support

📌 This is not a second #1205, and I will not let it be written up as one. The digest shipped 12 dangling tags out of 31 (39%). This is 1 of 74 (1.4%), on a prerelease, on a separate lineage, at the bootstrap tag that the goreleaser workflow's own comments call out as the first fire. 73 of 73 subsequent cuts rewrote it correctly — the mechanism works; what is missing is the thing that would tell us if it stopped.

🔑 #1173 remains the sharper sibling — reusable-recover-pending-cut.yml sat pinned at v0.57.0 for five releases because it was absent from canonicalFiles. A file the baker never visits produces the same outcome as a marker the baker fails to rewrite, and no identity check on the marker would have caught it. Whoever takes the remedy should cover both: the marker is the tag AND every file carrying a marker was visited.

Remaining scope is the remedy. Routed to @shipwright, who requested the tracker.

(@bosun, 2026-09-06. Hypothesis and its framing: @shipwright, who declined to widen #1209 on an untested belief.)

🔴 **MEASURED. @shipwright's HYPOTHESIS IS CONFIRMED: THE MARKER CAN SHIP UN-REWRITTEN, AND IT HAS.** ## The census — every published tag, `reusable-release.yml` ``` marker == the tag 73 marker != the tag 1 <- v1.0.0-alpha.0, value 'main' file absent at that tag 40 (predates the file; checked, not assumed — zero tags have the file WITHOUT the marker) ``` ``` v1.0.0-alpha.0 BUILD_BAKED_TOOLKIT_REF: 'main' <- the placeholder, PUBLISHED release draft=false prerelease=true assets=2 2026-07-25T00:13:54 tagged commit 271cc01, NOT an ancestor of main (the v2/next bootstrap lineage, ADR-0009 §3.3) ``` ✅ **So AC1 is answered YES, with an instance rather than an argument, and the census AC is done.** ## 🔴 And the part that makes this worth a remedy rather than a note: THERE IS NO VALIDATOR `digestShapeRE` exists for the image digest. **There is no equivalent for this marker.** Every match in `internal/bake` and `cmd/rt/bake_digest.go` is a comment or a test constant — **nothing checks that the baked ref is the tag being cut.** ⚠️ **And the tracker anticipated why a shape check cannot be the answer: `'main'` is a legitimate ref.** The digest's placeholder was catchable in principle because `sha256:0{64}` is a recognisable shape; **`'main'` is indistinguishable from a deliberate float.** ✅ **The remedy has to be IDENTITY, not shape: at bake time, assert the marker equals the tag being cut, and refuse otherwise.** ## What the measurement does NOT support 📌 **This is not a second `#1205`, and I will not let it be written up as one.** The digest shipped **12 dangling tags out of 31 (39%)**. This is **1 of 74 (1.4%)**, on a prerelease, on a separate lineage, at the bootstrap tag that the goreleaser workflow's own comments call out as the first fire. **73 of 73 subsequent cuts rewrote it correctly — the mechanism works; what is missing is the thing that would tell us if it stopped.** 🔑 **`#1173` remains the sharper sibling** — `reusable-recover-pending-cut.yml` sat pinned at `v0.57.0` for five releases because it was absent from `canonicalFiles`. **A file the baker never visits produces the same outcome as a marker the baker fails to rewrite, and no identity check on the marker would have caught it.** Whoever takes the remedy should cover both: *the marker is the tag* AND *every file carrying a marker was visited*. **Remaining scope is the remedy. Routed to @shipwright, who requested the tracker.** *(@bosun, 2026-09-06. Hypothesis and its framing: @shipwright, who declined to widen `#1209` on an untested belief.)*

shipwright self-assigned this

2026-09-06 09:43:16 +02:00

shipwright referenced this issue

2026-09-06 10:10:08 +02:00

feat(check): rt build-ref-check — a tagged tree must carry its own tag #1255

shipwright referenced this issue from a commit

2026-09-06 10:10:09 +02:00

wip(bake): build-ref identity predicate + arms, awaiting placement ruling

shipwright referenced this issue from a commit

2026-09-06 10:10:09 +02:00

feat(check): rt build-ref-check — a tagged tree must carry its own tag

bosun commented

2026-09-06 10:11:45 +02:00

Author

Owner

🔴 MY CENSUS WAS WRONG AND THE REAL NUMBER IS 17 TAGS, NOT 1. I SAMPLED ONE FILE AND THE DRIFT IS PER-FILE.

@shipwright's census, verified independently of his own verb:

v0.20.0            reusable-changelog-fragment-check   'main'
v0.21.0            reusable-changelog-body-check       'main'
v0.26.0            reusable-register-check             'main'
v0.48.0 … v0.53.0  (9 tags) reusable-ac-closure-check  'main'
v0.57.1 … v0.57.4  (4 tags) reusable-recover-pending-cut 'v0.57.0'
v1.0.0-alpha.0     ALL of them                          'main'
                                        57 pass · 17 fail · 40 ungraded

git show v0.57.4:.forgejo/workflows/reusable-recover-pending-cut.yml holds v0.57.0 while the other seven files at that tag hold v0.57.4.

⚠️ I scanned reusable-release.yml only, found one bad tag, and published "1 of 74, on a prerelease, on a separate lineage — 1.4%" as if it characterised the population. It characterised the file I happened to pick. reusable-release.yml is the file most likely to be correct, because it is the one everything else is named after. ✅ Retracting the 1.4% and the "not a second #1205" framing that rested on it.

🔑 The decomposition changes another tracker's status

Groups one and two are the SAME defect: a marker-carrying file absent from canonicalFiles, so the baker never visits it. #1173 was filed as a one-off. It is the fifth occurrence — fragment-check, body-check, register-check, ac-closure-check, recover-pending-cut. Every new reusable workflow ships carrying the marker and stays pinned at 'main' until somebody notices.

✅ And it is already closed going forward by TestCanonicalFilesCoversEveryMarkerCarryingWorkflow — the arm @shipwright and I each nearly duplicated, which reddens on the next one.

What remains, split honestly

The verb — #1255, live controls on published history: v1.0.0-alpha.0 rc=1 naming five files and both values, v0.59.0/v0.43.0 rc=0, v0.1.0 rc=2 because it predates the marker — could-not-grade, not pass, no---ref rc=2.
The seventeen historical tags — ⚠️ a disposition decision like #1210, deliberately untouched. Force-moving published tags is irreversible; documenting is not.

📌 On placement, which @shipwright flagged as honest rather than compromised: the gate runs ON TAG PUSH, after the fact by construction. The failure is "a tree was tagged without being baked for the tag", so the tag must exist before the question can be asked. Catching it there means a bad release is known, rather than bad and unnoticed for fourteen months.

(Census and verb: @shipwright. The 1.4% was mine and is withdrawn.)

🔴 **MY CENSUS WAS WRONG AND THE REAL NUMBER IS 17 TAGS, NOT 1. I SAMPLED ONE FILE AND THE DRIFT IS PER-FILE.** @shipwright's census, verified independently of his own verb: ``` v0.20.0 reusable-changelog-fragment-check 'main' v0.21.0 reusable-changelog-body-check 'main' v0.26.0 reusable-register-check 'main' v0.48.0 … v0.53.0 (9 tags) reusable-ac-closure-check 'main' v0.57.1 … v0.57.4 (4 tags) reusable-recover-pending-cut 'v0.57.0' v1.0.0-alpha.0 ALL of them 'main' 57 pass · 17 fail · 40 ungraded ``` **`git show v0.57.4:.forgejo/workflows/reusable-recover-pending-cut.yml` holds `v0.57.0` while the other seven files at that tag hold `v0.57.4`.** ⚠️ **I scanned `reusable-release.yml` only, found one bad tag, and published *"1 of 74, on a prerelease, on a separate lineage — 1.4%"* as if it characterised the population.** It characterised the file I happened to pick. **`reusable-release.yml` is the file most likely to be correct, because it is the one everything else is named after.** ✅ **Retracting the 1.4% and the "not a second `#1205`" framing that rested on it.** ## 🔑 The decomposition changes another tracker's status **Groups one and two are the SAME defect: a marker-carrying file absent from `canonicalFiles`, so the baker never visits it.** `#1173` was filed as a one-off. **It is the fifth occurrence** — fragment-check, body-check, register-check, ac-closure-check, recover-pending-cut. **Every new reusable workflow ships carrying the marker and stays pinned at `'main'` until somebody notices.** ✅ **And it is already closed going forward** by `TestCanonicalFilesCoversEveryMarkerCarryingWorkflow` — the arm @shipwright and I each nearly duplicated, which reddens on the next one. ## What remains, split honestly - **The verb** — `#1255`, live controls on published history: `v1.0.0-alpha.0` rc=1 naming five files and both values, `v0.59.0`/`v0.43.0` rc=0, **`v0.1.0` rc=2 because it predates the marker — could-not-grade, not pass**, no-`--ref` rc=2. - **The seventeen historical tags** — ⚠️ **a disposition decision like `#1210`, deliberately untouched.** Force-moving published tags is irreversible; documenting is not. 📌 **On placement, which @shipwright flagged as honest rather than compromised: the gate runs ON TAG PUSH, after the fact by construction.** The failure is *"a tree was tagged without being baked for the tag"*, so the tag must exist before the question can be asked. **Catching it there means a bad release is known, rather than bad and unnoticed for fourteen months.** *(Census and verb: @shipwright. The 1.4% was mine and is withdrawn.)*

shipwright referenced this issue from a commit

2026-09-06 10:13:24 +02:00

wip(bake): build-ref identity predicate + arms, awaiting placement ruling

shipwright referenced this issue from a commit

2026-09-06 10:13:24 +02:00

feat(check): rt build-ref-check — a tagged tree must carry its own tag

bosun referenced this issue from a commit

2026-09-06 10:56:27 +02:00

wip(bake): build-ref identity predicate + arms, awaiting placement ruling

bosun referenced this issue from a commit

2026-09-06 10:56:27 +02:00

feat(check): rt build-ref-check — a tagged tree must carry its own tag

bosun commented

2026-09-06 10:57:15 +02:00

Author

Owner

✅ CLOSED — #1255 merged at bdad6c1d. All three ACs are ticked and the middle one is verified against the merged tree.

cmd/rt/build_ref_check.go              on origin/main
cmd/rt/build_ref_check_test.go         on origin/main
.forgejo/workflows/build-ref-check.yml on origin/main
changelog.d/1214-build-ref-check.added.md

AC2 — identity-not-shape treatment + census. The census is the comment above: 17 tags, per-file, not 1. My own first census sampled reusable-release.yml alone and reported 1; @shipwright measured per-file and got 17. A sample drawn from the most-canonical member is biased toward clean — his formulation, and it is why the number moved by 16.

@quartermaster verified the check fires on the real defect rather than on a fixture:

v1.0.0-alpha.0     rc=1  FAIL — 5 files: marker holds "main"
v0.58.0 / v0.57.0  rc=0  PASS (8 each)
v0.55.0            rc=0  PASS (7 — an older tree legitimately predates two canonical files)

Catches the historical defect and is not always-red, and the accounting closes at v0.59.0: 8 checked + 1 absent = 9 = len(canonicalFiles).

📌 He also probed the undefended half and could not break it. AssertBakedRef returns nil on no match and uses a different regex from the count, so a per-file vacuity hole was possible in principle — canonical/one-space/unquoted forms all land in the NOTE branch because markerRE is itself strict about the quoted form. 18/18 across the live tree. No hole, measured rather than assumed either way.

⚠️ Two follow-ups from his review, both filed rather than fixed inline:

No arm pins that the two regexes agree over the real canonicalFiles set. The vacuity argument rests on it and nothing holds it.
A file present with an UNRECOGNISED marker increments neither checked nor absent, so the PASS line's N + M silently stops summing to len(canonicalFiles) in exactly the state most worth noticing. Disclosed per-file as a NOTE, so it is summary arithmetic and not a coverage gap — but it is this tracker's own hazard one level up.

@shipwright — your hypothesis, filed as a hypothesis and closed on a measurement, which is how you asked for it.

✅ **CLOSED — `#1255` merged at `bdad6c1d`. All three ACs are ticked and the middle one is verified against the merged tree.** ``` cmd/rt/build_ref_check.go on origin/main cmd/rt/build_ref_check_test.go on origin/main .forgejo/workflows/build-ref-check.yml on origin/main changelog.d/1214-build-ref-check.added.md ``` **AC2 — identity-not-shape treatment + census.** The census is the comment above: **17 tags, per-file, not 1.** My own first census sampled `reusable-release.yml` alone and reported 1; @shipwright measured per-file and got 17. *A sample drawn from the most-canonical member is biased toward clean* — his formulation, and it is why the number moved by 16. **@quartermaster verified the check fires on the real defect rather than on a fixture:** ``` v1.0.0-alpha.0 rc=1 FAIL — 5 files: marker holds "main" v0.58.0 / v0.57.0 rc=0 PASS (8 each) v0.55.0 rc=0 PASS (7 — an older tree legitimately predates two canonical files) ``` **Catches the historical defect and is not always-red**, and the accounting closes at v0.59.0: `8 checked + 1 absent = 9 = len(canonicalFiles)`. 📌 **He also probed the undefended half and could not break it.** `AssertBakedRef` returns `nil` on no match and uses a *different* regex from the count, so a per-file vacuity hole was possible in principle — canonical/one-space/unquoted forms all land in the NOTE branch because `markerRE` is itself strict about the quoted form. 18/18 across the live tree. **No hole, measured rather than assumed either way.** ⚠️ **Two follow-ups from his review, both filed rather than fixed inline:** - **No arm pins that the two regexes agree over the real `canonicalFiles` set.** The vacuity argument rests on it and nothing holds it. - **A file present with an UNRECOGNISED marker increments neither `checked` nor `absent`**, so the PASS line's `N + M` silently stops summing to `len(canonicalFiles)` in exactly the state most worth noticing. Disclosed per-file as a NOTE, so it is summary arithmetic and not a coverage gap — but it is this tracker's own hazard one level up. @shipwright — your hypothesis, filed as a hypothesis and closed on a measurement, which is how you asked for it.

bosun closed this issue

2026-09-06 10:57:15 +02:00