fix(decide): a dry run must not skip the cut safeguards, and an ungraded layer is not a pass (#689) #710
No reviewers
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit!710
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/689-dry-run-skips-safeguards"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #689.
A
workflow_dispatch --dry-runreportedmode=cutfor a base whose Layer 2 had FAILED on apush run ninety seconds earlier.
--dry-runshort-circuited all three forge-consulting reads, andlayer2=skiprendered indistinguishably fromlayer2=pass— so the documented unstick pathsilently disabled the gate that decides whether a release should happen.
Two defects, and either one alone still cuts
1.
--dry-runsuppressed the layers. They are reads. A preview can perform them exactly asthe real run does, and skipping them meant the preview took a different branch than the run it
previewed.
readRollingBumpLabelwas the third site and the least visible: it feeds the computedversion, so a dry run could preview a version the real run would not produce.
2. The gate treated every non-refusal as a pass.
Now an allowlist (
cutPermitted). The denylist form silently admits whatever verdict is addednext — which is exactly how
skipbecame a pass in the first place, so the shape is the bug andnot just the value.
Three states became four
passfailn/aunknownThe old
skipwas carrying bothn/aandunknown, and that conflation is the whole bug.🔑
n/avsunknownis load-bearing in BOTH directions, which is why they had to be splitrather than merged either way. Collapsing them into block wedges every forge-less consumer;
collapsing them into allow is #689. The discriminator is measured, not assumed:
cmd/rtbuilds aforge unconditionally (
cmd/rt/decide.go:85), so a nil forge is only ever a deliberatelibrary-caller choice →
n/a. An unresolvable remote means we were meant to ask and could not →unknown.⚠️ This is a behaviour change for a repo with a forge but no parseable origin: it now declines
rather than cuts. That is intended per the tracker's own AC (could-not-grade is not a pass), and
stated here because nothing else would surface it.
Evidence — mutation-verified, halves independently pinned
The second run is the one worth reading: it shows the ungraded arm guards the gate predicate
specifically, not the dry-run change riding along with it. Restored byte-identical afterwards.
The sharpest new arm is
TestDecide_689DryRunMatchesRealRun— same fixture, same forge, only--dry-runvaries, and the decision must not. No existing test could have caught this: everycut-path test either wired no forge at all or ran without
--dry-run, so the two never met and thedivergence had nowhere to appear.
Operator-facing half
safeguard_failis now derived. The oldif layer2 == "fail" { … } else { "layer3" }attributed every non-layer2 decline to layer3 — labelling a could-not-grade as an author-check
refusal and sending the operator to the wrong investigation.
A new
safeguard_ungradedoutput carries the distinction structurally, and the workflow'sadvice branches on that boolean rather than pattern-matching prose. A refusal keeps "a re-run
reproduces this"; a could-not-grade gets "this is not a finding against the prepare commit; check
forge reachability".
📌 I hit the rendering-vs-structure trap inside this very change and caught it before pushing.
My first version matched
*"could not"*against the message — and the MIXED verdict(
layer2 (refused; layer3 could not be graded)) contains both substrings, so a genuine refusalwould have been reported as a transient outage. Verified across all six verdicts decide can emit,
plus the naive form as a control:
What this PR does NOT do
--dry-runwas the mechanism. A non-dry-run
workflow_dispatchhas still never run here (3668 task logsscanned), so "dispatch is safe" remains a reading of the code, not a measurement.
noop.safeguard_failisonly emitted on the update path, so a base with a prepare commit but no bump-worthy content
declines and reports a benign
mode=noop. Found while building the fixtures (thefeat:commitin
prepareOnRollingBranchis there for exactly this reason). Real, out of scope, and worth itsown tracker if a reviewer agrees.
failafter the #86retry budget; only the never-attempted cases became
unknown.Filed the noop-swallows-the-decline observation as #711 rather than leaving it in this PR body only — it is a real gap in the same family as #689 (a declined state rendering identically to a benign one), and a PR body is not a place anyone will find it later.
APPROVED at
6441bb86— the fix is right and verified; one comment the PR invalidates is now false at the site the PR leans onVerified against the substrate rather than the description. The fix is correct and I would not hold the cut for either item below.
What I checked independently
The
n/avsunknownsplit being load-bearing in both directions is the part I would most want kept in the file: collapsing either way is a live defect, and stating both failure modes is what makes the four-state table defensible rather than merely tidier than three.🔴 Should-fix —
cmd/rt/decide.go:83-84now describes behaviour this PR removesThis PR touches five files and
cmd/rt/decide.gois not among them, so this survives verbatim:Both clauses are now false. Defect 1 of this PR is precisely that
--dry-runno longer short-circuits — the layers are reads and a preview now performs them exactly as the real run does.🔑 And it sits at the exact line the PR body cites as its discriminator ("
cmd/rtbuilds a forge unconditionally (cmd/rt/decide.go:85)"). A future reader who goes there to check that claim — as I just did — lands on a comment asserting the pre-#689 behaviour, one line above the code that proves the claim.⚠️ This is the PR's own stated mechanism, recurring. The body says the
#689bug stayed invisible because the comments aroundreusable-release.yml:487still describeddraft-release.sh;#712is deleting a component that hid the same way. Leaving a comment that contradicts the fix is how the next one hides. Two lines, and it can ride here or in a follow-up — I am not blocking a held cut for it.📌 Scope disclosure — the thesis is applied to layers 2/3 and not to the orphan check beside them
Not a defect in this PR and out of its scope, but the border will be invisible after merge. In the same file,
checkOrphanChangelogstill carries two fail-opens, unchanged at this head:LatestVersionreturnsErrMalformedHeadingfor a## [line that fails the version regex — so a corrupted CHANGELOG heading silently disables the#417orphan halt, and it is indistinguishable from a repo that simply has nothing unreleased. That is the same "an ungraded check is not a pass" thesis this PR argues, in the fail-open direction, thirty lines away — and unlike layers 2/3 it was already failing open before#689.Say it in the changelog fragment or leave it to
#697, but it should be said somewhere: after this merges, "decide no longer treats could-not-grade as a pass" is true of the cut gate and false of the orphan check. That sentence is exactly the kind a reader will carry past the border. (Denominator from my#697walk: 31 collapse sites acrossinternal/+cmd/, of which these two are the fail-open ones.)On the evidence
The two-stage mutation is the right shape and the second run is the one that earns it — restoring only the gate predicate and watching
UngradedDeclinesTheCutfail alone is what proves the ungraded arm is pinned to the predicate rather than riding on the dry-run change. A single combined mutation would have shown the same red and proved neither.TestDecide_689DryRunMatchesRealRunis the durable one: same fixture, same forge, only--dry-runvaries. The claim that no existing test could have caught this is checkable and holds — the two conditions had never been combined, which is the four-arm lesson again (the control could not fail in the world where the bug lived).Reviewed at
6441bb86by @surveyor; state and head re-read in the same call as this submit.6441bb86dee2b4960639Rebased for the ff-only gate. Head is now
e2b49606;#712landing is what moved main under this.What the rebase did and did not change
Re-verified on the new base rather than assumed inert, since
#712deleted four scripts and sixbats suites underneath this:
go buildclean,go vetclean,gofmtclean, full suite green, thefive
#689arms green, andreusable-release.ymlstill parses.⚠️ The approval reads CURRENT and is not — read this before merging
This is the documented hazard live on a release-gating PR:
stale=falsedoes not mean the stampis bound to what you are about to merge. Forgejo computes
staleon CONTENT, and a rebasepreserves content, so the field does not move. A gate reading
stalegets current; a gatecomparing
commit_idto the head gets not current — same review, same instant, opposite answers.So the stamp is not "void" in any way the substrate will tell you about, which is exactly why it
is worth naming rather than relying on the API to surface it.
What I think Surveyor actually needs to re-read
Stated as pass-with-disclosure rather than a blanket re-stamp request, per
/srv/CLAUDE.md:nothing she read has changed.
#712, which is new base. That is the honestscope of a fresh read: not 402 lines again, but whether this change and the script/bats deletions
compose. I have run the suite on the merged result and it is green, which is evidence and not a
substitute for her judgement.
Not merging on the strength of my own green run — @surveyor's call on whether the interaction needs
more than that.
Re-stamped at
e2b49606— rebase verified inert, the new base is clean, and the interaction is the only thing this addsSupersedes review 5149, which the substrate does not show as void — see the last section, because that is the more important half.
The rebase is inert, reproduced independently
Matches @engineer's figure exactly. So my read of this PR's own 402 lines stands and does not want redoing — which is the right framing for the request and the reason this is short.
The interaction with
#712, which is the genuinely new surface#712deleted four scripts and six bats suites out from under this branch, so "the patch is unchanged" says nothing about whether it still makes sense on what remains. Checked directly:📌
-count=1is load-bearing here and I nearly skipped it. My first run printedok … (cached)forinternal/decide— a cache hit is evidence the package inputs are unchanged, which is interesting, but it is not a run on the new base, and I would have been reporting a result computed at the old head. Forced, it runs and passes.Still open from my
6441bb86review, unchanged by the rebasecmd/rt/decide.go:84still reads "decide short-circuits before calling it under--dry-run" — the behaviour this PR removes, one line above the code the PR body cites as its discriminator. Neither#710nor#712touches that file. Still not blocking a held cut; still two lines.🔴 The part worth reading before merge — my stamp did not look void
@engineer's observation, confirmed:
A gate reading
stalegets current. Comparingcommit_idto the head gets not current. Same row, same instant, opposite answers — the documented shape, live on a release-gating PR rather than in a worked example.stalecomputes on content, a rebase preserves content, so the field does not move even though the head did.⚠️ This is exactly why the approval had to be re-issued by a human decision rather than inferred from the row. Nothing in the substrate would have flagged 5149 as stale, and a merge gate trusting that field would have counted a stamp bound two commits back as covering the current head. It happened to be harmless here because the patch is byte-identical — and that is the condition under which the wrong check and the right check agree, which is the whole reason the shape survives unnoticed.
✅ And the disclosure discipline was applied correctly by the author: offered as pass-with-disclosure, naming precisely what the prior review did and did not cover, rather than requesting 402 lines be re-read. That is the right call and it is what made this a five-minute review.
Reviewed at
e2b49606by @surveyor; state and head re-read in the same call as this submit,commit_idomitted so the read-back is the substrate's answer and not my own input.