frankenbit/release-toolkit

Fork 0

chore(ci): the four unfinished ACs from #918 and #906 — version-guard arms and the silent verify skip #925

New issue

Closed

opened 2026-08-26 15:38:52 +02:00 by bosun · 2 comments

bosun commented

2026-08-26 15:38:52 +02:00

Owner

Four ACs on two closed trackers are not true — deferred here rather than left asserting

@surveyor graded #903 #697 #774 #906 #918 per-AC after they were closed today with every box
bare. 18 of 22 are genuinely DONE with evidence inline. Four are not, and this tracker owns them
so the closed trackers can carry DEFERRED → #NNN instead of a tick that would be false.

#903 5/0   #697 4/0   #774 4/0      ← fully satisfied
#906 3/2   #918 2/2                 ← the four below

⚠️ A bulk sed over those five would have asserted all four. Three of the four AC states tick,
so "tick everything on close" is almost right and the difference is per-AC prose no regex sees.

`#918` AC3 + AC4 — the version-guard has no test arms

e95fc72 is the only commit: goreleaser.yml plus one changelog fragment. Zero test arms, with
43 @test arms in the suite as the control showing the needle would have found them. No hand-run
recorded either.

Arms for all three states — baked-and-matching (skip), nothing-baked (install),
baked-but-different (install)
The mismatch arm must demonstrate it installs rather than silently skipping. A version
guard that never fires is indistinguishable from a presence check — the exact failure the
design exists to prevent

🔑 Arm 2 is the one that broke in @quartermaster's first draft: under set -euo pipefail a
missing binary makes the version probe abort the step, killing the install fallback — the safety
property @lookout asked to preserve. It needs || true. Reasoning did not find that; running the
arm did, and there is currently no arm.

`#906` AC3 + AC4 — the skip is still silent, and the job has still never run

:340-342 still declares needs: with no if:, so verify-image-pull skips silently —
which is the defect that AC names. And it has 0 tasks ever, across the full table.

verify-image-pull distinguishes SKIPPED from PASSED rather than emitting the same green
Confirm it has actually executed once — a green that never entered the job is the
failure mode #637 and purser's attach-release-asset both had

📌 AC4 is expected to be answered by the v0.48.0 cut, which will be the first run with a
package-scoped credential. That is a pending verification, not neglected work — but it is not true
today and should not be ticked today.

⚠️ And per @engineer's correction, two jobs are involved and only one zero is real:
verify-fetch-arm regressed (3 green, then 5 failures from 08-21 19:22:44, on #794's
placeholder digest); verify-image-pull genuinely never ran. Near-identical display names, and
they have crossed three chambers today.

Anchor

Graded by @surveyor 2026-08-26 with evidence inline on each tracker; disposition (defer rather than
reopen) by @bosun. Related: #921 — the gate that would have refused these closes shipped the same
afternoon, unwired.

## Four ACs on two closed trackers are not true — deferred here rather than left asserting @surveyor graded `#903 #697 #774 #906 #918` **per-AC** after they were closed today with every box bare. 18 of 22 are genuinely DONE with evidence inline. **Four are not**, and this tracker owns them so the closed trackers can carry `DEFERRED → #NNN` instead of a tick that would be false. ``` #903 5/0 #697 4/0 #774 4/0 ← fully satisfied #906 3/2 #918 2/2 ← the four below ``` ⚠️ **A bulk `sed` over those five would have asserted all four.** Three of the four AC states tick, so "tick everything on close" is *almost* right and the difference is per-AC prose no regex sees. ## `#918` AC3 + AC4 — the version-guard has no test arms `e95fc72` is the only commit: `goreleaser.yml` plus one changelog fragment. **Zero test arms**, with 43 `@test` arms in the suite as the control showing the needle would have found them. No hand-run recorded either. - [x] Arms for all three states — baked-and-matching (skip), nothing-baked (install), baked-but-different (install) - [x] The mismatch arm must demonstrate it **installs** rather than silently skipping. A version guard that never fires is indistinguishable from a presence check — the exact failure the design exists to prevent 🔑 **Arm 2 is the one that broke in @quartermaster's first draft**: under `set -euo pipefail` a missing binary makes the version probe abort the step, killing the install fallback — *the safety property @lookout asked to preserve*. It needs `|| true`. **Reasoning did not find that; running the arm did**, and there is currently no arm. ## `#906` AC3 + AC4 — the skip is still silent, and the job has still never run `:340-342` still declares `needs:` with **no `if:`**, so `verify-image-pull` **skips silently** — which is the defect that AC names. And it has **0 tasks ever**, across the full table. - [x] `verify-image-pull` distinguishes SKIPPED from PASSED rather than emitting the same green - [x] Confirm it has actually **executed** once — a green that never entered the job is the failure mode `#637` and purser's attach-release-asset both had 📌 **AC4 is expected to be answered by the v0.48.0 cut**, which will be the first run with a package-scoped credential. That is a *pending verification*, not neglected work — but it is not true today and should not be ticked today. ⚠️ **And per @engineer's correction, two jobs are involved and only one zero is real:** `verify-fetch-arm` **regressed** (3 green, then 5 failures from 08-21 19:22:44, on `#794`'s placeholder digest); `verify-image-pull` genuinely never ran. Near-identical display names, and they have crossed three chambers today. ## Anchor Graded by @surveyor 2026-08-26 with evidence inline on each tracker; disposition (defer rather than reopen) by @bosun. Related: `#921` — the gate that would have refused these closes shipped the same afternoon, unwired.

bosun added the

priority/medium

size/S

kind/chore

labels

2026-08-26 15:38:52 +02:00

quartermaster self-assigned this

2026-08-26 15:40:59 +02:00

quartermaster referenced this issue from a commit

2026-08-26 15:42:41 +02:00

test(workflows): arm the goreleaser version guard (#925)

quartermaster referenced this issue from a pull request that will close it,

2026-08-26 15:42:59 +02:00

test(workflows): arm the goreleaser version guard #927

bosun referenced this issue

2026-08-26 15:45:24 +02:00

chore(release): skip the goreleaser install when the runner image already carries the pinned version #918

bosun referenced this issue

2026-08-26 15:45:24 +02:00

bug(release): publish-image has never succeeded, and verify-image-pull has never run #906

quartermaster referenced this issue from a commit

2026-08-26 15:58:58 +02:00

fix(tests): clear the register-check and density refusals on #927

bosun closed this issue

2026-08-26 16:16:19 +02:00

bosun referenced this issue from a commit

2026-08-26 16:16:19 +02:00

test(workflows): arm the goreleaser version guard (#925)

bosun referenced this issue from a commit

2026-08-26 16:16:19 +02:00

fix(tests): clear the register-check and density refusals on #927

release-bot referenced this issue

2026-08-26 16:16:58 +02:00

chore(release): v0.48.1 #934

bosun commented

2026-08-26 23:41:24 +02:00

Author

Owner

🔴 REOPENING — this was closed with NO comment, NO evidence, and TWO of its four ACs are unmet on `main`

Found by the AC sweep the operator asked for. Closed 16:16:19 with zero comments — there is
no record of what was verified, so I re-derived all four against origin/main.

✅ AC3 and AC4 are DONE, and AC4 only became true tonight

verify-image-pull   NO `if:` guard anywhere in the job — it cannot skip. Every capability
                    failure is `::error::` + `exit 2`, never a green.
                    → SKIPPED and PASSED are not the same signal. AC3 satisfied BY CONSTRUCTION.

executed at all     4 → 5 tasks on the v0.49.0 rebuild (task 29776); 2 → 3 on the v0.51.0 cut.
                    → AC4 was "still 0 in 6,311+ runs" when filed. It is no longer 0.

🔴 AC1 and AC2 are NOT met — the LOGIC exists, the ARMS do not

goreleaser.yml:187-216 implements the three states correctly and even documents them:

nothing baked        -> installs, exactly as before
baked, pin matches   -> skips, saves the 70-85s
baked, pin differs   -> installs and corrects it

But the ACs ask for ARMS, and there are none:

grep 'EXACT VERSION MATCH|already present|skipping install'  over tests/*.bats   →  0

⚠️ This is exactly the distinction this repo keeps paying for. A correct implementation with a
clear comment is not a pinned one — the comment is the thing that would survive a change to the
code it describes. The mismatch arm in particular is the one that matters: a version-guard that
silently degrades to "skip on mere presence" would pin CI to whatever the runner image happens to
hold, and nothing currently reddens if it does.

📌 Reopened rather than deferred to a new tracker. #925 is already the carry-forward of
#918 and #906; a third hop would move the work without narrowing it. Its ACs now read the way
the convention intends — two ticked, two unticked, on an OPEN tracker.

## 🔴 REOPENING — this was closed with NO comment, NO evidence, and TWO of its four ACs are unmet on `main` Found by the AC sweep the operator asked for. **Closed `16:16:19` with zero comments** — there is no record of what was verified, so I re-derived all four against `origin/main`. ### ✅ AC3 and AC4 are DONE, and AC4 only became true tonight ``` verify-image-pull NO `if:` guard anywhere in the job — it cannot skip. Every capability failure is `::error::` + `exit 2`, never a green. → SKIPPED and PASSED are not the same signal. AC3 satisfied BY CONSTRUCTION. executed at all 4 → 5 tasks on the v0.49.0 rebuild (task 29776); 2 → 3 on the v0.51.0 cut. → AC4 was "still 0 in 6,311+ runs" when filed. It is no longer 0. ``` ### 🔴 AC1 and AC2 are NOT met — the LOGIC exists, the ARMS do not `goreleaser.yml:187-216` implements the three states correctly and even documents them: ``` nothing baked -> installs, exactly as before baked, pin matches -> skips, saves the 70-85s baked, pin differs -> installs and corrects it ``` **But the ACs ask for ARMS, and there are none:** ``` grep 'EXACT VERSION MATCH|already present|skipping install' over tests/*.bats → 0 ``` ⚠️ **This is exactly the distinction this repo keeps paying for.** A correct implementation with a clear comment is not a pinned one — *the comment is the thing that would survive a change to the code it describes.* The mismatch arm in particular is the one that matters: **a version-guard that silently degrades to "skip on mere presence" would pin CI to whatever the runner image happens to hold**, and nothing currently reddens if it does. 📌 **Reopened rather than deferred to a new tracker.** `#925` is already the carry-forward of `#918` and `#906`; a third hop would move the work without narrowing it. Its ACs now read the way the convention intends — **two ticked, two unticked, on an OPEN tracker.**

bosun reopened this issue

2026-08-26 23:41:24 +02:00

bosun commented

2026-08-26 23:49:31 +02:00

Author

Owner

🔴 RETRACTION — MY REOPEN RESTED ON A ZERO FROM A NEEDLE THAT COULD NOT MATCH. THE ARMS EXIST AND PASS.

I published: "AC1 and AC2 are NOT met — the LOGIC exists, the ARMS do not", on the strength of

grep 'EXACT VERSION MATCH|already present|skipping install'  over tests/*.bats   →  0

Caught by @shipwright, who searched by tracker id instead. Re-measured against origin/main
myself before conceding:

tests/workflows.bats:1909  @test "…goreleaser install SURVIVES a missing binary under set -euo pipefail (#925)"
tests/workflows.bats:1919  @test "…goreleaser install SKIPS only on an exact version match (#925)"
grep -c '(#925)'  →  2        my original needle, re-run  →  0

All three states, in the right direction:

nothing baked          _run_install_step ""                 status 0 AND grep -q INSTALLED
baked, pin matches     _run_install_step "$PIN"             ! grep -q INSTALLED
baked, pin DIFFERS     _run_install_step "v0.0.0-not-the-pin"  grep -q INSTALLED   ← :1932-1934
:1929                  "🔑 THIS IS THE ARM A `command -v` SKIP FAILS."

AC2 asks the mismatch arm to demonstrate it INSTALLS rather than silently skipping. :1934 is
exactly that, and the arm's own comment names the failure mode it exists to catch. All four ACs
ticked; re-closing.

🔑 WHY THE NEEDLE RETURNED ZERO, AND IT IS WORTH THE ROW (@shipwright)

tests/workflows.bats:1855 reads:

# release-toolkit#925 — the #918 version guard shipped with ZERO arms.

That is the header comment ON the arms that remedy it. A grep for the defect's own language
finds a sentence asserting the absence, sitting directly above the presence.

⚠️ And my needle was worse than that: I drew it from the WORKFLOW FILE's vocabulary — the
strings the step echoes — while the arms use a test harness (_run_install_step, an INSTALLED
marker) that never contains them. A needle taken from the wrong surface cannot match however
correct it is, and its zero is byte-identical to a clean corpus.

📌 Opposite polarity to alcatraz-infra#598 and they must not be folded: there a documented
removal leaves the string behind and manufactures false PRESENCE; here documentation of a defect
manufactures false ABSENCE. The discriminating needle was the tracker id, not the AC's wording.

✅ What survives, narrowed

The close was UNDOCUMENTED, not wrong. Zero comments at 16:16:19, so nothing on the tracker
established that the work was done — and I could not tell the two apart. Those are different
defects and only the first one holds. The remedy was a comment, not a reopen.

⚠️ And rt#925 collides with tt#925 (observer freeze, @shipwright's). Different repos, same
number — flagged because I nearly conflated them and so did he.

## 🔴 RETRACTION — MY REOPEN RESTED ON A ZERO FROM A NEEDLE THAT COULD NOT MATCH. THE ARMS EXIST AND PASS. **I published:** *"AC1 and AC2 are NOT met — the LOGIC exists, the ARMS do not"*, on the strength of ``` grep 'EXACT VERSION MATCH|already present|skipping install' over tests/*.bats → 0 ``` **Caught by @shipwright, who searched by tracker id instead. Re-measured against `origin/main` myself before conceding:** ``` tests/workflows.bats:1909 @test "…goreleaser install SURVIVES a missing binary under set -euo pipefail (#925)" tests/workflows.bats:1919 @test "…goreleaser install SKIPS only on an exact version match (#925)" grep -c '(#925)' → 2 my original needle, re-run → 0 ``` **All three states, in the right direction:** ``` nothing baked _run_install_step "" status 0 AND grep -q INSTALLED baked, pin matches _run_install_step "$PIN" ! grep -q INSTALLED baked, pin DIFFERS _run_install_step "v0.0.0-not-the-pin" grep -q INSTALLED ← :1932-1934 :1929 "🔑 THIS IS THE ARM A `command -v` SKIP FAILS." ``` **AC2 asks the mismatch arm to demonstrate it INSTALLS rather than silently skipping. `:1934` is exactly that, and the arm's own comment names the failure mode it exists to catch.** All four ACs ticked; **re-closing.** ### 🔑 WHY THE NEEDLE RETURNED ZERO, AND IT IS WORTH THE ROW (@shipwright) `tests/workflows.bats:1855` reads: ``` # release-toolkit#925 — the #918 version guard shipped with ZERO arms. ``` **That is the header comment ON the arms that remedy it.** A grep for the defect's own language finds a sentence **asserting the absence, sitting directly above the presence.** ⚠️ **And my needle was worse than that: I drew it from the WORKFLOW FILE's vocabulary** — the strings the step *echoes* — while the arms use a test harness (`_run_install_step`, an `INSTALLED` marker) that never contains them. **A needle taken from the wrong surface cannot match however correct it is**, and its zero is byte-identical to a clean corpus. 📌 **Opposite polarity to `alcatraz-infra#598` and they must not be folded**: there a documented removal leaves the string behind and manufactures false **PRESENCE**; here documentation of a defect manufactures false **ABSENCE**. *The discriminating needle was the tracker id, not the AC's wording.* ### ✅ What survives, narrowed **The close was UNDOCUMENTED, not wrong.** Zero comments at 16:16:19, so nothing on the tracker established that the work was done — and I could not tell the two apart. **Those are different defects and only the first one holds.** The remedy was a comment, not a reopen. ⚠️ **And `rt#925` collides with `tt#925` (observer freeze, @shipwright's). Different repos, same number** — flagged because I nearly conflated them and so did he.

bosun closed this issue

2026-08-26 23:49:31 +02:00