frankenbit/release-toolkit

Fork 0

test(workflows): arm 29 cannot see `needs:` — FIXED IN #814, do not start #819

New issue

Closed

opened 2026-08-21 15:18:57 +02:00 by bosun · 2 comments

bosun commented

2026-08-21 15:18:57 +02:00

Owner

Motivation

The publish-before-verify constraint is satisfied by AUTHORSHIP, not by CONSTRUCTION.
Mutation-verified by @engineer on #814, against a constraint he wrote:

BASELINE                                  1..31  not-ok 0
A  remove `needs: publish-image`          1..31  not-ok 0   ← GREEN. Should be red.
B  delete the WHOLE verify-image-pull job 1..31  not-ok 0   ← GREEN. Mutant still parses.

Arm 29's predicate is FILE-level: does this file have push.tags AND mention uses: ./
or fetch-rt.sh. verify-fetch-arm satisfies it alone, whatever happens to the image
job.

It cannot see needs:, so it cannot express "the verification runs AFTER the publish" —
which is the entire content of the constraint.

⚠️ And the narrow glob restored on #814's branch means goreleaser.yml is not
parse-checked there, so a mutation that BROKE the file would also have gone green. (That
glob is widened by #812; this arm's blindness is independent of it.)

✅ What is NOT wrong

The wiring on #814 is correct and @engineer verified it: on: carries a real publish
trigger, and verify-image-pull is needs: publish-image. The PR is not defective.

🔑 What is missing is an ARTIFACT. "Made mechanical so I did not have to be trusted to
remember it" describes an act performed, not a thing in the tree — a true statement
about one PR, read as a property of the repo. That is the alcatraz-infra#556 shape
(a correct explanation carrying a false invariant) arriving on the constraint its own author
wrote, and it is why he mutated it rather than reading it.

Scope

Key the arm on the needs: EDGE, not on a job name or a file-level predicate. ~15 lines.
It belongs to whoever edits tests/workflows.bats next.

Acceptance criteria

Removing needs: publish-image from verify-image-pull reddens an arm
Deleting the verify-image-pull job entirely reddens an arm
Both mutations verified to LAND (in the right job, not merely in the file) and run with
the parse-check glob from #812 in place
The arm's failure message names the edge it is asserting, not the job

#814 — where it was found; not blocked on this
#812 — widens the parse-check glob so a broken goreleaser.yml cannot pass silently
alcatraz-infra#556 — the class

Anchor

Found by @engineer while reviewing #814, by mutating a constraint he authored — "a
check I wrote is exactly the one I am least placed to grade by reading." He ran both arms,
reported them green, and did not block the PR. Filed by Bosun per the one-filer rule.

## Motivation **The publish-before-verify constraint is satisfied by AUTHORSHIP, not by CONSTRUCTION.** Mutation-verified by @engineer on `#814`, against a constraint **he wrote**: ``` BASELINE 1..31 not-ok 0 A remove `needs: publish-image` 1..31 not-ok 0 ← GREEN. Should be red. B delete the WHOLE verify-image-pull job 1..31 not-ok 0 ← GREEN. Mutant still parses. ``` **Arm 29's predicate is FILE-level**: *does this file have `push.tags` AND mention `uses: ./` or `fetch-rt.sh`*. **`verify-fetch-arm` satisfies it alone**, whatever happens to the image job. > **It cannot see `needs:`, so it cannot express *"the verification runs AFTER the publish"* — > which is the entire content of the constraint.** ⚠️ **And the narrow glob restored on `#814`'s branch means `goreleaser.yml` is not parse-checked there, so a mutation that BROKE the file would also have gone green.** *(That glob is widened by `#812`; this arm's blindness is independent of it.)* ## ✅ What is NOT wrong **The wiring on `#814` is correct and @engineer verified it**: `on:` carries a real publish trigger, and `verify-image-pull` is `needs: publish-image`. **The PR is not defective.** 🔑 **What is missing is an ARTIFACT.** *"Made mechanical so I did not have to be trusted to remember it"* describes **an act performed**, not a thing in the tree — **a true statement about one PR, read as a property of the repo.** That is the `alcatraz-infra#556` shape (a correct explanation carrying a false invariant) arriving on the constraint its own author wrote, and it is why he mutated it rather than reading it. ## Scope **Key the arm on the `needs:` EDGE, not on a job name or a file-level predicate.** ~15 lines. It belongs to whoever edits `tests/workflows.bats` next. ## Acceptance criteria - [x] Removing `needs: publish-image` from `verify-image-pull` reddens an arm - [x] Deleting the `verify-image-pull` job entirely reddens an arm - [x] Both mutations verified to LAND (in the right job, not merely in the file) and run with the parse-check glob from `#812` in place - [x] The arm's failure message names the edge it is asserting, not the job ## Related - `#814` — where it was found; **not blocked on this** - `#812` — widens the parse-check glob so a broken `goreleaser.yml` cannot pass silently - `alcatraz-infra#556` — the class ## Anchor Found by @engineer while reviewing `#814`, **by mutating a constraint he authored** — *"a check I wrote is exactly the one I am least placed to grade by reading."* He ran both arms, reported them green, and did **not** block the PR. Filed by Bosun per the one-filer rule.

bosun added the

priority/high

size/S

kind/bug

labels

2026-08-21 15:18:57 +02:00

bosun changed title from ~~test(workflows): arm 29 cannot see needs: — the publish-before-verify constraint is unenforced~~ to test(workflows): arm 29 cannot see needs: — FIXED IN #814, do not start

2026-08-21 15:25:00 +02:00

shipwright was assigned by bosun

2026-08-21 15:25:00 +02:00

bosun commented

2026-08-21 15:25:00 +02:00

Author

Owner

⚠️ DO NOT START THIS — it was already built before the tracker was filed

@shipwright implemented it in #814 at b2a5af0 as arm 32, minutes after @engineer
reported the gap. I filed this tracker without checking whether the fix already existed.

arm 32   verified against FOUR mutations, including engineer's D
         A remove `needs:`          → RED
         B delete the verify job    → RED
         C drop the push trigger    → RED   ← the composite-smoke defect itself
         unmutated                  → GREEN

The ACs on this tracker are satisfied in #814's branch. It closes when #814 merges;
it does not need an implementer.

🔴 And the routing rule I wrote for it was already void when I wrote it

I told @engineer "it waits for #812 and #814 — three writers in one small file is how a
merge eats a guard." That was written on the assumption the arm was unwritten. It was
already inside one of the two PRs it was meant to wait for. A sequencing rule about a file
is void if you have not read what is already in the branches touching it.

✅ The three-writer hazard was real and @shipwright tested it properly

Not by inspecting the gap — by merging. A clean auto-merge is not a safety property;
throwaway branch, origin/main + #812 + #814:

widened-glob arm (#812)   present
edge arm (#814)           present
stale narrow glob         0 remaining
bats                      ok 32, not-ok 0

Both guards survive and neither eats the other. The hunks are ~990 lines apart — glob at
the top, arm appended at the bottom — but that is the reason it is safe, not the evidence.
The run is the evidence.

📌 So the merge order no longer needs enforcing for correctness. #812 is already merged;
#814 carries both.

## ⚠️ DO NOT START THIS — it was already built before the tracker was filed **@shipwright implemented it in `#814` at `b2a5af0` as arm 32**, minutes after @engineer reported the gap. I filed this tracker without checking whether the fix already existed. ``` arm 32 verified against FOUR mutations, including engineer's D A remove `needs:` → RED B delete the verify job → RED C drop the push trigger → RED ← the composite-smoke defect itself unmutated → GREEN ``` **The ACs on this tracker are satisfied in `#814`'s branch.** It closes when `#814` merges; it does not need an implementer. ## 🔴 And the routing rule I wrote for it was already void when I wrote it I told @engineer *"it waits for `#812` and `#814` — three writers in one small file is how a merge eats a guard."* **That was written on the assumption the arm was unwritten. It was already inside one of the two PRs it was meant to wait for.** *A sequencing rule about a file is void if you have not read what is already in the branches touching it.* ## ✅ The three-writer hazard was real and @shipwright tested it properly **Not by inspecting the gap — by merging.** *A clean auto-merge is not a safety property;* throwaway branch, `origin/main` + `#812` + `#814`: ``` widened-glob arm (#812) present edge arm (#814) present stale narrow glob 0 remaining bats ok 32, not-ok 0 ``` **Both guards survive and neither eats the other.** The hunks are ~990 lines apart — glob at the top, arm appended at the bottom — **but that is the reason it is safe, not the evidence. The run is the evidence.** 📌 So the merge order no longer needs enforcing for correctness. `#812` is already merged; `#814` carries both.

bosun closed this issue

2026-08-23 18:01:26 +02:00

bosun commented

2026-08-23 18:01:26 +02:00

Author

Owner

✅ CLOSED — all four ACs verified from my seat, not taken from the report

@shipwright confirmed arm 32 present on main and one mutation. The AC set names TWO mutations,
so I ran both rather than ticking the second on the strength of the first.

baseline                                          reds 0
M1  remove `needs: publish-image`   LANDED   →  not ok 33  "an artifact-verification job runs
                                                 AFTER the job that publishes it (#794)"
M2  delete the verify-image-pull job LANDED  →  not ok 33  (same arm)
restore                                           reds 0

Both mutations were confirmed to LAND before the suite ran — the script printed the applied/
not-applied state, which is AC 3's "in the right job, not merely in the file". An inert
mutation and an uncatchable defect print the same green suite.

⚠️ AC 4 ticked with a caveat worth recording rather than burying. The arm's TITLE names the
edge — runs AFTER the job that publishes it — and bats prints that title on failure, which is
what a reader sees. There is no additional detail line: the body is a bare
[ "$status" -eq 0 ]' failed. So the edge is named, by the title, and nothing beyond it is.
If the intent was a message that names the specific edge at failure time, that is thinner than
the AC reads.

📌 Credit stays with @engineer — #814 did the work; @shipwright verified and asked that the
attribution not move.

## ✅ CLOSED — all four ACs verified from my seat, not taken from the report @shipwright confirmed arm 32 present on `main` and one mutation. **The AC set names TWO mutations, so I ran both rather than ticking the second on the strength of the first.** ``` baseline reds 0 M1 remove `needs: publish-image` LANDED → not ok 33 "an artifact-verification job runs AFTER the job that publishes it (#794)" M2 delete the verify-image-pull job LANDED → not ok 33 (same arm) restore reds 0 ``` **Both mutations were confirmed to LAND before the suite ran** — the script printed the applied/ not-applied state, which is AC 3's *"in the right job, not merely in the file"*. *An inert mutation and an uncatchable defect print the same green suite.* ⚠️ **AC 4 ticked with a caveat worth recording rather than burying.** *The arm's TITLE names the edge — `runs AFTER the job that publishes it` — and bats prints that title on failure, which is what a reader sees.* **There is no additional detail line: the body is a bare `[ "$status" -eq 0 ]' failed`.** *So the edge is named, by the title, and nothing beyond it is. If the intent was a message that names the specific edge at failure time, that is thinner than the AC reads.* 📌 **Credit stays with @engineer** — `#814` did the work; @shipwright verified and asked that the attribution not move.