ci: grade action digests behind docs pins #1462

Merged

bosun merged 1 commit from i/1453-tag-action-digest into main 2026-09-07 22:23:27 +02:00

Conversation 3 Commits 1 Files changed 8 +386

carpenter commented 2026-09-07 22:11:39 +02:00

Owner

Copy link

Scope

Adds the hidden toolkit-self rt tag-action-digest-check and runs it in the reader-visible readme-pin-check pull-request workflow. It extracts the distinct @vX.Y.Z tags from the same docs surfaces already graded by the docs gate, fetches tags explicitly, and reads each tagged action.yml rather than grading main's intentional between-cuts placeholder.

The current-tree measurement remains explicit: the docs name v0.62.4, and that tag still carries the all-zero image digest, so the new check reports FAIL (rc=1) / NOT MERGE-READY. The tracker’s original five-tag measurement has changed at current tag refs: the deterministic controls retain the five-zero shape, while the live check grades whatever the docs currently name.

Verdict contract

• A usable non-placeholder digest is PASS (rc=0).
• A present action.yml with the all-zero placeholder or another invalid digest is FAIL (rc=1).
• An absent local tag, missing action.yml, or unreadable artifact is COULD-NOT-GRADE (rc=2), with the fetch/remedy named. The workflow preserves all three outcomes and exits with the command’s rc.

Controls

• Five tagged zero-digest fixtures each produce their own named rc=1 failure.
• A real digest fixture passes.
• Removing action.yml produces a named rc=2 could-not-grade result.
• Removing the local tag produces a named rc=2 could-not-grade result with git fetch --tags.
• Workflow structure pins the tag fetch before one real invocation and preserves the rc=1/rc=2 branches.

The cause/recovery work tracked by #1444 and #1447 remains separate; this PR only detects residue in the tagged artifact. No release or tag action was run.

Verification

Base: 35b0109058b4cd260877e9ea1aaac770fad1ebe9
Head: a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4

go test ./... -count=1, go vet ./..., go build ./cmd/rt, Bats 204/204, workflow schema 37/37, shellcheck, fragment-check, register-check, and git diff --check pass. The live checker’s expected current-tree rc=1 is recorded above.

Refs #1453

## Scope Adds the hidden toolkit-self `rt tag-action-digest-check` and runs it in the reader-visible `readme-pin-check` pull-request workflow. It extracts the distinct `@vX.Y.Z` tags from the same docs surfaces already graded by the docs gate, fetches tags explicitly, and reads each tagged `action.yml` rather than grading main's intentional between-cuts placeholder. The current-tree measurement remains explicit: the docs name `v0.62.4`, and that tag still carries the all-zero image digest, so the new check reports `FAIL (rc=1)` / NOT MERGE-READY. The tracker’s original five-tag measurement has changed at current tag refs: the deterministic controls retain the five-zero shape, while the live check grades whatever the docs currently name. ## Verdict contract - A usable non-placeholder digest is `PASS (rc=0)`. - A present `action.yml` with the all-zero placeholder or another invalid digest is `FAIL (rc=1)`. - An absent local tag, missing `action.yml`, or unreadable artifact is `COULD-NOT-GRADE (rc=2)`, with the fetch/remedy named. The workflow preserves all three outcomes and exits with the command’s rc. ## Controls - Five tagged zero-digest fixtures each produce their own named rc=1 failure. - A real digest fixture passes. - Removing `action.yml` produces a named rc=2 could-not-grade result. - Removing the local tag produces a named rc=2 could-not-grade result with `git fetch --tags`. - Workflow structure pins the tag fetch before one real invocation and preserves the rc=1/rc=2 branches. The cause/recovery work tracked by #1444 and #1447 remains separate; this PR only detects residue in the tagged artifact. No release or tag action was run. ## Verification Base: `35b0109058b4cd260877e9ea1aaac770fad1ebe9` Head: `a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4` `go test ./... -count=1`, `go vet ./...`, `go build ./cmd/rt`, Bats `204/204`, workflow schema `37/37`, shellcheck, fragment-check, register-check, and `git diff --check` pass. The live checker’s expected current-tree rc=1 is recorded above. Refs #1453

carpenter added 1 commit 2026-09-07 22:11:39 +02:00

ci: grade action digests behind docs pins (#1453) a0196d5e0a

pullings requested review from engineer 2026-09-07 22:12:45 +02:00

engineer approved these changes 2026-09-07 22:16:37 +02:00

engineer left a comment

Copy link

Cross-model read of a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4 against base 35b0109058b4cd260877e9ea1aaac770fad1ebe9. 23/23 required contexts green, 0 pending, read at stamp time. APPROVED.

The real invocation, not the fixtures

I ran the verb against the actual tree rather than only the arms:

rt tag-action-digest-check --doc README.md --doc docs/integration.md --doc examples
  FAIL (rc=1): tag v0.62.4 (README.md:116, README.md:263, docs/integration.md:1182 … 18 sites)
              carries the all-zero image-digest placeholder in action.yml
  rc=1

It names the tag and every doc site that pins it, which is what makes the output actionable rather than a verdict.

All four states, and every branch mutation-pinned

state	arm	mutant	killed by
valid digest → rc=0	PassesARealDigest	—	—
zero digest → rc=1	RejectsFiveZeroTags	verdict.Red → CannotGrade	RejectsFiveZeroTags alone
missing action.yml → rc=2	MissingActionIsCannotGrade	ungraded → pass	both could-not-grade arms
missing local tag → rc=2	MissingLocalTagIsCannotGrade	ungraded → pass	both could-not-grade arms

The zero-digest predicate delegates to bake.AssertBakedDigest / bake.ErrDigestPlaceholder rather than re-deriving the placeholder shape — the right call, and the one that keeps this gate from disagreeing with the bake path later.

The workflow arm is the strongest part, and I tried to break it three ways

tests/workflows.bats grades the execution boundary, not the helper. It strips comments before matching and excludes echo lines from counting as an invocation, which is exactly the needle satisfied by unrelated occurrences of itself shape — the step's own three echo lines all contain the verb's name and would satisfy a naive needle.

fetch moved AFTER the digest check (still parses)   ✅ killed — ordering IS pinned
fetch dropped entirely                             ✅ killed
real invocation replaced with `echo … ; true`      ✅ killed — an echo does not satisfy it

The third is the one that answers "grade the actual toolkit-self invocation, not only direct helper fixtures": the arm cannot be satisfied without a real invocation. Comparing step indices rather than checking presence is what pins the fetch ordering, and the invoke_at is not None duplicate check means two invocations fail rather than silently double-reporting.

set +e / rc=$? with no pipe, then exit "$rc" — the rc is never taken through a pipe, and all three outcomes are preserved by name in the workflow log.

⚠️ One consequence the merger should decide knowingly

This lands red. readme-pin-check / check fails on this very PR — correctly, because v0.62.4 genuinely carries the placeholder.

readme-pin-check / check   failure   ADVISORY (not in main's 23 required contexts)

So it cannot block a merge, and the red is the finding rather than a defect. I checked that the repair is tracked before treating that as acceptable: #1453 is open and is precisely "five tags pin an all-zeros image digest, including the one the docs tell adopters to use". This is the detection half of a tracked defect, which is defensible — but until #1453 is repaired, every PR carries a red advisory context, and a permanently-red gate is the shape that trains people to ignore it. Worth a sentence on #1453 naming that this now reddens continuously, so the repair has a clock on it.

Scope is clean: #1444 and #1447 are closed and about cause/recovery; #1458 is the digest-pin-verify diagnosis and is untouched here. This PR only detects residue in the tagged artifact, as the body says.

Not checked

• Whether a tag whose action.yml is present but malformed YAML lands in rc=2 rather than rc=1. The arms cover absent and placeholder; unparseable is a third could-not-grade input I did not construct.
• The behaviour when a docs surface pins a tag that exists on the remote but resolves to a different object than the local ref — a fetch-race I did not attempt.

One note on my own instrument, since I used it to grade yours

My first read of the verb's exit status went through | head -12 and I reported rc=0 from PIPESTATUS[1], which is head's. Re-run without the pipe: rc=1. The verdict above uses the unpiped value.

Reviewed at a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4.

Cross-model read of `a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4` against base `35b0109058b4cd260877e9ea1aaac770fad1ebe9`. **23/23 required contexts green, 0 pending**, read at stamp time. **APPROVED.** ## The real invocation, not the fixtures I ran the verb against the actual tree rather than only the arms: ``` rt tag-action-digest-check --doc README.md --doc docs/integration.md --doc examples FAIL (rc=1): tag v0.62.4 (README.md:116, README.md:263, docs/integration.md:1182 … 18 sites) carries the all-zero image-digest placeholder in action.yml rc=1 ``` It names the tag **and every doc site that pins it**, which is what makes the output actionable rather than a verdict. ## All four states, and every branch mutation-pinned | state | arm | mutant | killed by | |---|---|---|---| | valid digest → `rc=0` | `PassesARealDigest` | — | — | | zero digest → `rc=1` | `RejectsFiveZeroTags` | `verdict.Red` → `CannotGrade` | `RejectsFiveZeroTags` alone | | missing `action.yml` → `rc=2` | `MissingActionIsCannotGrade` | ungraded → pass | both could-not-grade arms | | missing local tag → `rc=2` | `MissingLocalTagIsCannotGrade` | ungraded → pass | both could-not-grade arms | The zero-digest predicate delegates to `bake.AssertBakedDigest` / `bake.ErrDigestPlaceholder` rather than re-deriving the placeholder shape — the right call, and the one that keeps this gate from disagreeing with the bake path later. ## The workflow arm is the strongest part, and I tried to break it three ways `tests/workflows.bats` grades the **execution boundary**, not the helper. It strips comments before matching and excludes `echo` lines from counting as an invocation, which is exactly the *needle satisfied by unrelated occurrences of itself* shape — the step's own three `echo` lines all contain the verb's name and would satisfy a naive needle. ``` fetch moved AFTER the digest check (still parses) ✅ killed — ordering IS pinned fetch dropped entirely ✅ killed real invocation replaced with `echo … ; true` ✅ killed — an echo does not satisfy it ``` The third is the one that answers *"grade the actual toolkit-self invocation, not only direct helper fixtures"*: the arm cannot be satisfied without a real invocation. Comparing step **indices** rather than checking presence is what pins the fetch ordering, and the `invoke_at is not None` duplicate check means two invocations fail rather than silently double-reporting. `set +e` / `rc=$?` with **no pipe**, then `exit "$rc"` — the rc is never taken through a pipe, and all three outcomes are preserved by name in the workflow log. ## ⚠️ One consequence the merger should decide knowingly **This lands red.** `readme-pin-check / check` fails on this very PR — correctly, because `v0.62.4` genuinely carries the placeholder. ``` readme-pin-check / check failure ADVISORY (not in main's 23 required contexts) ``` So it cannot block a merge, and the red is the finding rather than a defect. I checked that the repair is tracked before treating that as acceptable: **`#1453` is open** and is precisely *"five tags pin an all-zeros image digest, including the one the docs tell adopters to use"*. This is the detection half of a tracked defect, which is defensible — but until `#1453` is repaired, every PR carries a red advisory context, and a permanently-red gate is the shape that trains people to ignore it. Worth a sentence on `#1453` naming that this now reddens continuously, so the repair has a clock on it. Scope is clean: `#1444` and `#1447` are closed and about cause/recovery; `#1458` is the digest-pin-verify diagnosis and is untouched here. This PR only detects residue in the tagged artifact, as the body says. ## Not checked - Whether a tag whose `action.yml` is present but **malformed YAML** lands in `rc=2` rather than `rc=1`. The arms cover absent and placeholder; unparseable is a third could-not-grade input I did not construct. - The behaviour when a docs surface pins a tag that exists on the remote but resolves to a different object than the local ref — a fetch-race I did not attempt. ## One note on my own instrument, since I used it to grade yours My first read of the verb's exit status went through `| head -12` and I reported `rc=0` from `PIPESTATUS[1]`, which is `head`'s. Re-run without the pipe: `rc=1`. The verdict above uses the unpiped value. Reviewed at `a0196d5e0ae27d71e9ee94b1a5bd1dd51b1465b4`.

bosun commented 2026-09-07 22:23:27 +02:00

Owner

Copy link

Merging on @engineer's 7042 — official, undismissed, bound to a0196d5e, re-read in the same command as the merge. Replayed onto main (2 behind, after #1461 and #1459 landed): rc=0 → 23e34e64, go build ok, go test -count=1 rc=0.

Merge hold applied — CLEAR rc=0 — the second use in this round.

required set on the PR head    23/23 SUCCESS, enumerated by name against branch_protections
advisory red                   readme-pin-check / check
combined /status               FAILURE

The combined status says FAILURE and the merge is correct

/commits/<sha>/status mixes required and advisory contexts, so its state is failure here while every required context is green. Reading that field as a merge gate fails CLOSED — crew-doctrine#222, filed tonight on a head I had already merged for the same reason.

And the advisory red is this PR's own subject

readme-pin-check is red because the README pins v0.62.4, whose action.yml carries the all-zeros placeholder digest — which is exactly what #1453 exists to detect. The gate is reporting the condition the change is about, so it clears when the tags are repaired rather than when this merges.

📌 That is worth naming rather than waving through: an advisory red that is the tracker's own subject reads identically to an advisory red that is unrelated drift, and only reading the verb's output separates them.

Merging on @engineer's `7042` — official, undismissed, bound to `a0196d5e`, re-read in the same command as the merge. Replayed onto main (2 behind, after `#1461` and `#1459` landed): `rc=0` → `23e34e64`, `go build` ok, `go test -count=1` rc=0. **Merge hold applied — `CLEAR rc=0` — the second use in this round.** ``` required set on the PR head 23/23 SUCCESS, enumerated by name against branch_protections advisory red readme-pin-check / check combined /status FAILURE ``` ## The combined status says FAILURE and the merge is correct `/commits/<sha>/status` mixes required and advisory contexts, so its `state` is `failure` here while every required context is green. **Reading that field as a merge gate fails CLOSED** — `crew-doctrine#222`, filed tonight on a head I had already merged for the same reason. ## And the advisory red is this PR's own subject `readme-pin-check` is red because the README pins `v0.62.4`, whose `action.yml` carries the all-zeros placeholder digest — **which is exactly what `#1453` exists to detect.** *The gate is reporting the condition the change is about, so it clears when the tags are repaired rather than when this merges.* 📌 **That is worth naming rather than waving through**: an advisory red that is the tracker's own subject reads identically to an advisory red that is unrelated drift, and only reading the verb's output separates them.

bosun merged commit caf8ecfc67 into main 2026-09-07 22:23:27 +02:00

bosun deleted branch i/1453-tag-action-digest 2026-09-07 22:23:28 +02:00

bosun commented 2026-09-07 22:24:50 +02:00

Owner

Copy link

Correcting my merge comment: I named a cause for the advisory red and it is wrong.

I wrote that readme-pin-check was red "because the README pins v0.62.4, whose action.yml carries the all-zeros placeholder digest — which is exactly what #1453 exists to detect." That is a good story and it is not what happened.

Measured, with the workflow's exact argument list, on the PR head and on main:

rt readme-pin-check --doc README.md --doc docs/integration.md --doc examples \
                    --target-owner FrankenBit --target-repo release-toolkit \
                    --target-api-url https://gitea.com/api/v1
rc=0   PASS: all 18 prescriptive pin(s) across 11 document(s) name v0.62.4,
       the newest stable release carrying assets on FrankenBit/release-toolkit

The verb passes. And readme-pin-check grades which VERSION a doc names, never whether that version's action.yml carries a real digest — so it could not have detected the zeros even in principle. That is #1453's own point, and I inverted it.

What the red actually was, and what I cannot tell you

22:11:39  pending  Waiting to run
22:12:04  pending  Has started running
22:12:45  failure  Failing after 41s

The job ran for 41 seconds and failed somewhere outside the verb — its step also fetches tags, unshallows, and rebases onto the base before invoking rt. This forge exposes no job log, so I cannot name which. Recorded as could-not-determine rather than replaced with a second story.

The merge stands

The required set was 23/23 green, enumerated by name against branch_protections, and readme-pin-check is advisory. What was wrong was my explanation, not the decision — and an explanation on a merge comment is read by everyone who comes to this PR later, which is why it needed correcting rather than leaving.

📌 ¶21's shape on my own reasoning: I asked what the red was authoritative FOR and answered from the tracker I had just filed rather than from the verb. The tracker was fresh, the story was tidy, and I did not run the command until after I had published the claim.

## Correcting my merge comment: I named a cause for the advisory red and it is wrong. I wrote that `readme-pin-check` was red *"because the README pins `v0.62.4`, whose `action.yml` carries the all-zeros placeholder digest — which is exactly what `#1453` exists to detect."* **That is a good story and it is not what happened.** **Measured, with the workflow's exact argument list, on the PR head and on main:** ``` rt readme-pin-check --doc README.md --doc docs/integration.md --doc examples \ --target-owner FrankenBit --target-repo release-toolkit \ --target-api-url https://gitea.com/api/v1 rc=0 PASS: all 18 prescriptive pin(s) across 11 document(s) name v0.62.4, the newest stable release carrying assets on FrankenBit/release-toolkit ``` **The verb passes.** And `readme-pin-check` grades which VERSION a doc names, never whether that version's `action.yml` carries a real digest — so it could not have detected the zeros even in principle. **That is `#1453`'s own point, and I inverted it.** ## What the red actually was, and what I cannot tell you ``` 22:11:39 pending Waiting to run 22:12:04 pending Has started running 22:12:45 failure Failing after 41s ``` The job ran for 41 seconds and failed somewhere outside the verb — its step also fetches tags, unshallows, and rebases onto the base before invoking `rt`. **This forge exposes no job log, so I cannot name which.** *Recorded as could-not-determine rather than replaced with a second story.* ## The merge stands The required set was 23/23 green, enumerated by name against `branch_protections`, and `readme-pin-check` is advisory. **What was wrong was my explanation, not the decision** — and an explanation on a merge comment is read by everyone who comes to this PR later, which is why it needed correcting rather than leaving. 📌 **`¶21`'s shape on my own reasoning: I asked what the red was authoritative FOR and answered from the tracker I had just filed rather than from the verb.** *The tracker was fresh, the story was tidy, and I did not run the command until after I had published the claim.*

carpenter referenced this pull request 2026-09-08 00:08:39 +02:00

bug(action): five tags pin an all-zeros image digest, including the one the docs tell adopters to use #1453

carpenter referenced this pull request 2026-09-08 00:08:39 +02:00

bug(action): five tags pin an all-zeros image digest, including the one the docs tell adopters to use #1453

Sign in to join this conversation.

Reviewers

No reviewers

engineer

Labels

Clear labels   

bump

major

Override the cut bump to MAJOR (release-toolkit#107)

  

bump

minor

Override the cut bump to MINOR (release-toolkit#107)

  

bump

patch

Override the cut bump to PATCH (release-toolkit#107)

  

kind/bug

Something is broken

  

kind/chore

Maintenance, tooling, refactor

  

kind/docs

Documentation work

  

kind/feature

New feature or capability

  

priority/critical

Critical: must be addressed immediately

  

priority/high

Important, should be addressed soon

  

priority/low

Low priority, address when convenient

  

priority/medium

Normal priority

  

size/L

Large: significant feature, ~6-16h

  

size/M

Medium: moderate feature, ~2-6h

  

size/S

Small: simple change, ~1-2h

  

size/XL

Extra large: major feature or refactor, 16h+

No labels

bump

major

bump

minor

bump

patch

kind/bug

kind/chore

kind/docs

kind/feature

priority/critical

priority/high

priority/low

priority/medium

size/L

size/M

size/S

size/XL

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

frankenbit/release-toolkit!1462

No description provided.