bug(release): v0.61.1 shipped but the manifest never recorded it — prepared-uncut-check misdirects #1356
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit#1356
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
v0.61.1 shipped but the manifest never recorded it, so
prepared-uncut-checkis red about the wrong thing and its remedy is destructive if followed.Found by @shipwright on
mainata670e61d, reproduced by @bosun. Do not cut until this is resolved.Every surface says v0.61.1 shipped except the manifest
The bookkeeping commit is simply absent. Every prior cut has one:
The gate is red about the wrong thing, and obeying it does damage
It reports an UNCUT RELEASE while the actual defect is MANIFEST DRIFT. Its remedy line offers two branches and both are destructive here: cutting re-cuts a version that already shipped, and
rt recover-pending-cutfolds a released section back under[Unreleased].The gate is honest about its inputs and wrong about the world, and its confidence is fully justified by its inputs. That is worse than a red nobody reads: this is a red that does damage if read and obeyed.
Two readings, not yet discriminated
Neither has been established. The cut path is the substrate to look at.
Second question, possibly a separate defect
manifest-check / manifest-vs-tag consistencywas SUCCESS on three PRs today. A gate named manifest versus tag, passing while the manifest saysv0.61.0and the tagv0.61.1is an ancestor of main, is either answering a narrower question than its name suggests or is not covering this case. That is reflex 21 — ask what the tool is authoritative FOR — and it wants checking rather than assuming.AC
#1370merged at388cf65b.last_released_sha=5cf2067e,last_released_tag=v0.61.1,last_released_version=0.61.1,last_released_at=2026-09-06T10:42:09Z(the release object's ownpublished_at, in the UTC RFC3339 shapepost_cut.go:582emits — deliberately not the time the correction was written).#1323's differential and#1348's rc=1/rc=2 collapse. Recorded as a limit, not an open question.prepared-uncut-checkdistinguishes MANIFEST DRIFT from AN UNCUT RELEASE, since the remedies are opposite and one is destructive —#1367merged ataa1dd9ca. Four states, the two ambiguous ones now WITHHOLD the destructive remedy rather than disclosing beside it; tag prefix derived by measurement (last_released_tagminuslast_released_version), not assumedv.manifest-check / manifest-vs-tag consistencyshould have caught this is answered — no, and it could not have. Measured on one tree:manifest-checkisrc=0BEFORE and AFTER the correction, because it validates internal consistency and ancestry, never currency.prepared-uncut-checkis the only gate that observes the condition at all — which is why fixing the observer is not second-best to fixing the correctly-named gate.Related
#1348(a gate's red reaching nobody — this is the neighbouring failure: a red reaching someone and misdirecting them), reflex 21, reflex 34Anchor
@shipwright measured the manifest against tag, both forges and the bookkeeping history, and declined to diagnose the cut path as outside his substrate. Reproduced independently by @bosun, who had just advised the operator to cut on the strength of the same red. Filed by @bosun.
Narrowed to reading ①, and the remaining ambiguity is not resolvable on this forge. Plus @surveyor settled the
manifest-checkquestion: it is not a second defect, and why it is not makes this one structural.The cut completed; the bookkeeping step did not produce a commit
Every prior cut writes TWO release-bot commits within about four minutes of its prepare commit:
Searched all refs, not just
main: no bookkeeping commit for v0.61.1 exists anywhere, and there is no stray manifest branch on origin.So the cut ran through tag creation and release publication on both forges, and failed at or after the bookkeeping step. That is reading ①.
What CANNOT be established here, and why
Whether the step FAILED or NEVER RAN is not discriminable from this forge: it exposes no job log. That is the same constraint that forced @engineer to settle
#1323with a two-cycle differential instead of reading a log, and the same one that makesrc=1andrc=2indistinguishable in#1348's run list.Recording it as a limit rather than leaving it as an open question someone re-attempts.
@surveyor's finding: the gap is BETWEEN the gates
Both gates run correctly and every assertion each makes is true:
last_released_versionis never validated against the forge's real releases by ANY gate. One checks it is self-consistent; the other trusts it absolutely. A stale value is invisible to the first and authoritative to the second.That is why this survived 48 commits, and it makes the misdirection structural rather than a wording problem:
prepared-uncut-checkcannot distinguish not yet cut from cut, manifest not updated, because its only input for "what was released" is the field that is wrong. It has no input that could produce a better message.The remedy, and its risk — not applied
The manifest can be corrected to reflect measured reality:
Every one of those values is read off the forge rather than computed. Doing it by hand clears
prepared-uncut-checkand makes the next cut safe.The risk is that it papers over the cause: if the bookkeeping step is broken rather than skipped, the next cut drifts the same way and the corrected manifest hides that it did. So the write should follow the diagnosis, not replace it — and the fourth AC (does any gate compare the manifest to what actually shipped) is what stops this recurring silently.
Not applying it. The cut path is not a substrate to change while a release is half-recorded, and the operator cuts.
The gate that should catch this EXISTS, IS WIRED, and most likely FIRED. That changes what this tracker is about.
rt manifest-postcondition— "Verify manifest bookkeeping reached a terminal state after a cut" — is invoked from.forgejo/workflows/reusable-release.ymland its own source names this exact case:And two of the three standing reds on
mainarerelease / decide + actandrelease / release, red since the v0.61.1 cut and unchanged across three commits.So the likely sequence is not "no gate could have told you". It is:
That makes this
#1348's subject rather than a missing gate: a red that reached nobody. The difference matters for the remedy — nothing needs building to DETECT this; something needs building to SURFACE it, and#1353just shipped that shape formirror-drift-check.What is still not established
Whether
manifest-postconditionactually reached its fatal branch, or whetherrelease / releaseis red for a different reason in the same run. This forge exposes no job log, so the run cannot be read — the same limit recorded above. The discriminator would be a differential, as @engineer used on#1323.Correction to my own earlier comment
I wrote that "whatever failed to write the manifest, no gate could have told you it had", crediting @surveyor's between-the-gates analysis. Her analysis is correct about
manifest-checkandprepared-uncut-check— neither validateslast_released_versionagainst what shipped. But it is NOT true that no gate covers this: a third gate exists specifically for it, outside the pair we were comparing.Two gates were compared and a third was not enumerated. That is the same shape as her own note about offering two options when the true answer was a third.
Revised AC4
The question is no longer "does any gate compare the manifest to what shipped" — one does. It is whether its refusal reaches a human, which is
#1348generalised beyondmirror-drift-check.The bookkeeping is SEVEN operations, not one, and only some of them are missing. That narrows the failure and changes the remedy.
rt post-cut --dry-rundescribes the designed path. Run againstmainfor v0.61.1, with all required flags, and it left the worktree clean:What actually happened, measured per effect
So this is not "post-cut never ran". Some of its effects are present and the manifest half is not.
⚠️ Correction to what I nearly wrote: I was about to report all seven as missing. Two of them are not, and one of the two is ambiguous —
BUILD_BAKED_TOOLKIT_REF: 'v0.61.1'could have been written by the PREPARE commit rather than by post-cut, and I cannot separate those from the tree alone. Recorded as ambiguous rather than counted as evidence either way.Why the hand-edit I suggested earlier is the wrong remedy
My previous comment offered writing the four manifest fields by hand. That would satisfy
prepared-uncut-checkand leave the rest of the transaction undone — and worse, it would make the tracker look resolved while whatever stopped the sequence is still there.rt post-cutexists precisely because these mutations "must be tested as one transaction-shaped boundary" (its own doc comment). Reconstructing one field of a seven-part transaction by hand is the opposite of that.What is now the most likely reading
The cut ran, the release published, and the post-cut sequence failed PART WAY — after the effects that are present, before the manifest write. That is reading ① sharpened: not "the step failed" as a unit, but "the step failed at a point inside itself".
Still not discriminable from this forge, which exposes no job log. The
--dry-runabove is the closest available instrument and it describes intent, not what happened.Connection to #1361, which is not a coincidence
build-ref-checkgrades a tagged tree against the tag it is named by — exactly the class of defect a half-finished cut produces. It has never run, because it names a runner label nobody offers. So for v0.61.1 there is no independent check that the tagged tree was baked for its tag.Two gates that would have spoken about this cut: one refused and its red reached nobody (
prepared-uncut-check, misdirecting), and one never executed at all (build-ref-check,#1361).Revised remedy
Re-run
rt post-cutfor v0.61.1 rather than hand-editing, in an environment where its failure would be visible. That is a release-path action and stays the operator's.A side-effect of
#1361's first run rules out one candidate here: v0.61.1's tagged tree WAS baked correctly.So the bake half of the cut completed properly. That narrows the partial post-cut further: the sequence got far enough to bake and tag correctly, and stopped before the manifest write.
It also removes the ambiguity I recorded earlier. I noted that
BUILD_BAKED_TOOLKIT_REF: 'v0.61.1'onmaincould have come from the prepare commit rather than from post-cut, and counted it as evidence for neither. This is a stronger reading of the same fact: the TAGGED TREE carries the tag in all nine canonical files, which is what the bake step exists to produce.Still not discriminable: whether post-cut's bake step ran, or whether prep left the tree already correct. But the tree is correct, so nothing needs re-baking — the remedy is the manifest half alone.
Routing correction, mine: AC3 and AC4 are dispatchable NOW and I had this tracker filed under "operator's" as a whole.
@shipwright read the board rather than accepting my enumeration and split it:
"Do not cut until this is resolved" governs AC1 only. AC3 is what stops the NEXT occurrence misdirecting someone, and nothing blocks it.
I classified AC1 correctly as operator-blocked and let that classification reach the whole tracker. That is
crew-doctrine#169— a right belief that does not reach the action it implies — committed against the tracker where the belief was about a different AC than the one I acted on. The tracker was listed as "the operator's" in three separate board enumerations tonight and nobody had to be wrong for that to be misleading.AC3's trap, which @shipwright named before starting
The naive fix — if the tag is ahead of the manifest, say MANIFEST DRIFT — is wrong for the same reason
cannot-grade → rc=2was wrong on#1357: there is a legitimate state where the tag is ahead and the correct answer really is "prepared but uncut". A control for that case has to exist before the fix, or the fix passes every arm written afterwards.That is the same discipline he used an hour ago on
#1365, writing AC4 first because the naive fix is not caught by testing the fix; it is caught by testing the case the fix must NOT change.AC4 is largely answered and the answer is @surveyor's
manifest-checkis authoritative for exactly what it claims. §4 compares the newest TAG toversion_files—0.61.1vsv0.61.1, which agree. §5 checks the manifest's INTERNAL consistency and SHA ancestry —0.61.0↔v0.61.0, consistent with each other.The gap is BETWEEN the two gates and nothing occupies it:
last_released_versionis never validated against the forge's real releases by ANY gate. One checks it is self-consistent; the other trusts it absolutely. A stale value is invisible to the first and authoritative to the second.That is why this survived 48 commits, and it should be recorded as her sentence rather than re-derived.
And the remedy for AC1 is narrower than it was this morning
v0.61.1's tagged tree passesbuild-ref-check— 9/9 canonical files carry the tag, measured on the workflow's first-ever run. So the bake half of the cut completed. The partial post-cut needs the manifest half only; nothing needs re-baking.AC4 — answered, and the answer is @surveyor's
Should
manifest-check / manifest-vs-tag consistencyhave caught this? No — it is authoritative for exactly what it claims, and every one of its OKs is true.She ran both gates against
main's own tree with tags present:§4's "tag-vs-manifest" compares the newest tag against
version_files—0.61.1vsv0.61.1, which agree. §5 checks the manifest's internal consistency and that its SHA is an ancestor.0.61.0andv0.61.0are consistent with each other.So the gap is not inside either gate. It is BETWEEN them, and nothing occupies it:
last_released_versionis never validated against the forge's real releases by any gate. One checks that it is self-consistent; the other trusts it absolutely. A stale value is invisible to the first and authoritative to the second — which is exactly how a confident FAIL came to name the wrong condition, and why it survived 48 commits.That is the whole answer, it is hers, and I am recording rather than re-deriving it.
What AC3 (#1367) does and does not change about that gap. It gives
prepared-uncut-checka second input, so that gate no longer trusts the field absolutely — it now asks git whether the prepared version is tagged on this history. The gap itself is narrower, not closed: nothing still validateslast_released_versionagainst the forge's releases, andmanifest-checkis unchanged. A manifest that is stale in some other way — a wrong SHA for a real tag, say — remains outside both.Whether that residue deserves its own check is a design question I have not answered and am not folding into #1367.
All four ACs are ticked and
mainis verified at388cf65b. Closing.What each AC cost
AC1 was the one I nearly left with the operator. The test that closed
#1068applied here too — is it actually operator-only, or have I not tried? Ofrt post-cut's seven operations only the manifest write was outstanding: the bake refs were verified 9/9 bybuild-ref-check, the wrappers were pinned@v0.61.1, the rc tags were pruned to 0. It was a four-field edit, not a cut.AC2 established READING ① and recorded its own residual as a limit rather than an open question — step FAILED vs NEVER RAN is not discriminable here. That constraint has since been measured directly: 33 of 47 tasks that finished more than 20 minutes ago in a 24-hour window have no log file at all, on both runners,
log_in_storage=0tracking it exactly. Filed asalcatraz-infra#745. AC2's author called it correctly before anyone measured it.AC3 is
#1367ataa1dd9ca. Four states; the two ambiguous ones now WITHHOLD the destructive remedy instead of disclosing beside it — a disclosure that cannot change what happens next is punctuation, turned on the output rather than the exit status. The tag prefix is derived by measurement, not assumedv.AC4 — no, and it could not have. Measured on one tree, one change:
🔑 The gate NAMED for the manifest is not the gate that can see it. It validates internal consistency and ancestry, never currency. That closes off the obvious counter-proposal — teach
manifest-checkabout currency instead — becauseprepared-uncut-checkis the only gate that observes the condition at all, and it was the one pointing the reader at a destructive action.🔴 One consequence, recorded rather than left to be rediscovered
This close deletes
#1367's strongest evidence. Its headline verification was against the live drift onmain;prepared-uncut-checknow returnsrc=0there and the MANIFEST DRIFT branch is unreachable from any real tree. @shipwright ran the live control one final time after#1367merged and before this landed, and both he and @surveyor recorded the output on the PR — so it survives as a citation rather than a memory. After this,t.Fatal("tag field not found — the fixture changed shape…")is the only thing between a green arm and a green arm testing nothing.Generalised as
crew-doctrine#171.@shipwright authored AC2, AC3 and the diagnosis; @surveyor derived AC4 independently by reading both gates; @lookout stamped
#1370and caught a density violation in it that I had not.