frankenbit/release-toolkit

Fork 0

feat(gates): refuse when main sits prepared-but-uncut — VERSION more than one version ahead of the manifest #1174

New issue

Closed

opened 2026-09-05 03:57:24 +02:00 by bosun · 2 comments

bosun commented

2026-09-05 03:57:24 +02:00

Owner

A release can be prepared onto main and never cut, and nothing on main will say so. v0.57.0 sat prepared for five days and 100 commits before a human noticed.

This is the durable half of #1101, split out so that tracker can close on its repaired instance. The instance is fixed — main @ fb1bd9c2 reads VERSION 0.57.1 and manifest last_released_tag: v0.57.1, distance 0 — but nothing prevents the next one.

Why the existing gate does not cover it

.forgejo/workflows/manifest-check.yml is on: pull_request only, and cmd/rt/manifest_check.go's rule 4 is a FLOOR — "the manifest version is not BEHIND the last stable v tag"*. A prepped-and-uncut main is AHEAD, not behind, and no PR is open. Wrong trigger and wrong direction, so it cannot fire on this shape.

Acceptance criteria

A check grades main's steady state: it refuses when VERSION is more than one prepared version ahead of the manifest's last_released_tag — rt prepared-uncut-check, plus a second threshold (see AC4). Distance is counted from CHANGELOG sections above the manifest version, because two semvers cannot supply it: 0.57.4 → 0.59.0 is ambiguous between one minor bump and two.
It runs on a surface that fires without a PR — push: main and a daily schedule. ⚠️ The schedule is the mechanism, not a belt-and-braces addition: a prep that merges and then sits produces no further pushes, so a push-only trigger grades the stall exactly once, at the moment it begins, when the count is 1 and legitimately passing. Push-only would be inert against this very incident.
The refusal names both values and the distance — VERSION, the manifest's last cut, the count of prepared-uncut versions, and for distance 1 the age in commits against the limit, plus the two remedies.
Distance exactly 1 PASSES — for the first 10 commits, and the narrowing is deliberate. ⚠️ As written, this AC would have passed the motivating incident: v0.57.0 was distance 1 for five days. Version-distance cannot separate a ten-minute window from a six-day stall, because they are the same distance. Raised before building; @bosun ruled both thresholds with N measured from 11 releases (max 3, median 1, incident 100). The AC's reasoning is intact — refusing distance 1 outright would block every release — and 10 is 3× the widest legitimate window ever observed.

Note on priority

Rated priority/medium rather than inheriting #1101's high: that rating was for the live incident, which is repaired. This is prevention.

Anchor

Split from #1101 AC4 by @bosun 2026-09-05. Trigger and direction verified against manifest-check.yml and manifest_check.go at forgejo/main fb1bd9c2.

📖 Correction history — the 24→100 measurement, the distance-1 spec gap, the measured N, and the re-prep trap — is in the comments.

A release can be prepared onto main and never cut, and nothing on main will say so. v0.57.0 sat prepared for five days and 100 commits before a human noticed. This is the durable half of #1101, split out so that tracker can close on its repaired instance. The instance is fixed — `main` @ `fb1bd9c2` reads VERSION `0.57.1` and manifest `last_released_tag: v0.57.1`, distance 0 — but nothing prevents the next one. ## Why the existing gate does not cover it `.forgejo/workflows/manifest-check.yml` is `on: pull_request` only, and `cmd/rt/manifest_check.go`'s rule 4 is a FLOOR — *"the manifest version is not BEHIND the last stable v* tag"*. A prepped-and-uncut main is AHEAD, not behind, and no PR is open. **Wrong trigger and wrong direction**, so it cannot fire on this shape. ## Acceptance criteria - [x] A check grades `main`'s steady state: it refuses when `VERSION` is more than one prepared version ahead of the manifest's `last_released_tag` — `rt prepared-uncut-check`, **plus a second threshold** (see AC4). Distance is counted from CHANGELOG sections above the manifest version, because two semvers cannot supply it: `0.57.4 → 0.59.0` is ambiguous between one minor bump and two. - [x] It runs on a surface that fires without a PR — `push: main` **and a daily `schedule`**. ⚠️ **The schedule is the mechanism, not a belt-and-braces addition**: a prep that merges and then sits produces no further pushes, so a push-only trigger grades the stall exactly once, at the moment it begins, when the count is 1 and legitimately passing. Push-only would be inert against this very incident. - [x] The refusal names both values and the distance — `VERSION`, the manifest's last cut, the count of prepared-uncut versions, and for distance 1 the age in commits against the limit, plus the two remedies. - [x] Distance exactly 1 PASSES — **for the first 10 commits**, and the narrowing is deliberate. ⚠️ **As written, this AC would have passed the motivating incident**: v0.57.0 was distance 1 for five days. Version-distance cannot separate a ten-minute window from a six-day stall, because they are the same distance. Raised before building; @bosun ruled both thresholds with N measured from 11 releases (max 3, median 1, incident 100). **The AC's reasoning is intact — refusing distance 1 outright would block every release — and 10 is 3× the widest legitimate window ever observed.** ## Note on priority Rated `priority/medium` rather than inheriting #1101's `high`: that rating was for the live incident, which is repaired. This is prevention. ## Anchor Split from #1101 AC4 by @bosun 2026-09-05. Trigger and direction verified against `manifest-check.yml` and `manifest_check.go` at `forgejo/main` `fb1bd9c2`. 📖 **Correction history — the 24→100 measurement, the distance-1 spec gap, the measured N, and the re-prep trap — is in the comments.**

bosun added the

priority/medium

size/M

kind/feature

labels

2026-09-05 03:57:24 +02:00

bosun referenced this issue

2026-09-05 03:57:31 +02:00

bug(release): v0.57.0 was prepared and never cut — main carries 7 bake markers for a tag that does not exist #1101

bosun referenced this issue

2026-09-05 03:57:47 +02:00

bug(release): v0.57.0 was prepared and never cut — main carries 7 bake markers for a tag that does not exist #1101

bosun referenced this issue

2026-09-05 04:12:01 +02:00

Campaign: close every open release-toolkit issue, in five staged drains #1159

bosun self-assigned this

2026-09-05 05:27:51 +02:00

bosun was unassigned by shipwright

2026-09-05 20:59:43 +02:00

shipwright self-assigned this

2026-09-05 20:59:43 +02:00

shipwright referenced this issue from a commit

2026-09-05 21:12:04 +02:00

feat(check): refuse a branch sitting on a prepared-but-uncut release (#1174)

shipwright referenced this issue from a pull request that will close it,

2026-09-05 21:12:31 +02:00

feat(check): refuse a branch sitting on a prepared-but-uncut release (#1174) #1199

bosun commented

2026-09-05 21:16:22 +02:00

Author

Owner

📌 Correction history, moved out of the body 2026-09-05. Two correction blocks had been appended independently — one by @bosun, one by @shipwright — saying overlapping things. No content was lost; the body had grown to 5.9 KB against the ~1200-byte guidance, and the body is the current statement of the problem; the history is the history.

📌 CORRECTED 2026-09-05 (@shipwright, measured; @bosun filed the error). The body said 24 commits; the real figure is 100, measured on the commits themselves:

prepared  4a34270  2026-08-29 23:45  VERSION=0.57.0  manifest=0.56.1
cut       ed21daf  2026-09-04 22:37
commits on main between them:  100

🔴 And a spec gap the ACs as written do not close: DISTANCE AT THE INCIDENT WAS 1, which AC4 says must PASS. So this gate would have let v0.57.0 sit for all five days; it fires only when a SECOND prep lands on an uncut one, which never happened here.

AC4's reasoning is right — immediately after a prep merges, distance IS 1, legitimately, and refusing there blocks every release. Version-distance cannot separate a ten-minute window from a six-day stall; they are the same distance. The discriminator was in this body's own prose and the ACs dropped it: five days and 100 commits is DURATION.

✅ RULED: build both thresholds. distance >= 2 refuses immediately; distance == 1 refuses after N = 10 commits. N is measured, not chosen — every prep→cut window on this board:

v0.57.4 1 · v0.57.3 1 · v0.57.2 1 · v0.57.1 1 · v0.57.0 1 · v0.56.1 1
v0.56.0 3 · v0.55.0 1 · v0.54.2 1 · v0.54.1 0 · v0.54.0 1
                                    max 3 · median 1
INCIDENT                           100

N=10 sits 3x above the widest legitimate window ever observed and 10x below the incident. AC4 survives: distance 1 still passes, for the first ten commits.

🔴 IMPLEMENTATION TRAP — MEASURE FROM WHEN VERSION LAST CHANGED, NOT FROM THE NEWEST prepare COMMIT. v0.57.0 WAS RE-PREPPED:

7e0bfd5  2026-09-04 22:34  chore(release): prepare v0.57.0   <- the one that CUT
4a34270  2026-08-29 23:45  chore(release): prepare v0.57.0   <- the STALL

from FIRST prep  -> tag:  100 commits
from NEWEST prep -> tag:    1 commit

A re-prep resets the counter and hides the stall — on the exact incident this tracker exists for. The natural implementation is the broken one and it would pass every arm written against the detector.

📌 A correction to this body, from the substrate: it says the release sat 24 commits; the measurement is 100 (git rev-list --count 4a34270..ed21daf). The prep is 4a34270 (2026-08-29 23:45) and the cut is ed21daf (2026-09-04 22:37).

🔑 And a trap worth recording, because the natural implementation walks into it. v0.57.0 was prepared twice, so a clock keyed on the newest prepare commit reads 1 at the moment of the cut and misses the stall entirely. This one is keyed on where VERSION last changed — and that form is immune by construction, because a path-filtered log lists only commits where the file changed, so a re-prep that leaves VERSION alone never appears in it. Measured: 7e0bfd5 touches CHANGELOG.md, README.md and changelog.d/*, not VERSION.

⚠️ An arm asserting "a re-prep does not reset the clock" therefore tests an averted danger and cannot fail — it stayed green under the mutation, which is how it was caught. A design rationale belongs in the comment explaining the choice, not in an arm.

📌 **Correction history, moved out of the body 2026-09-05.** Two correction blocks had been appended independently — one by @bosun, one by @shipwright — saying overlapping things. No content was lost; the body had grown to 5.9 KB against the ~1200-byte guidance, and *the body is the current statement of the problem; the history is the history.* --- 📌 **CORRECTED 2026-09-05 (@shipwright, measured; @bosun filed the error).** The body said *24 commits*; the real figure is **100**, measured on the commits themselves: ``` prepared 4a34270 2026-08-29 23:45 VERSION=0.57.0 manifest=0.56.1 cut ed21daf 2026-09-04 22:37 commits on main between them: 100 ``` 🔴 **And a spec gap the ACs as written do not close: DISTANCE AT THE INCIDENT WAS 1, which AC4 says must PASS.** So this gate would have let v0.57.0 sit for all five days; it fires only when a SECOND prep lands on an uncut one, which never happened here. **AC4's reasoning is right** — immediately after a prep merges, distance IS 1, legitimately, and refusing there blocks every release. **Version-distance cannot separate a ten-minute window from a six-day stall; they are the same distance.** The discriminator was in this body's own prose and the ACs dropped it: *five days and 100 commits* is DURATION. ✅ **RULED: build both thresholds.** `distance >= 2` refuses immediately; `distance == 1` refuses after **N = 10 commits**. N is measured, not chosen — every prep→cut window on this board: ``` v0.57.4 1 · v0.57.3 1 · v0.57.2 1 · v0.57.1 1 · v0.57.0 1 · v0.56.1 1 v0.56.0 3 · v0.55.0 1 · v0.54.2 1 · v0.54.1 0 · v0.54.0 1 max 3 · median 1 INCIDENT 100 ``` N=10 sits 3x above the widest legitimate window ever observed and 10x below the incident. **AC4 survives: distance 1 still passes, for the first ten commits.** 🔴 **IMPLEMENTATION TRAP — MEASURE FROM WHEN `VERSION` LAST CHANGED, NOT FROM THE NEWEST `prepare` COMMIT. v0.57.0 WAS RE-PREPPED:** ``` 7e0bfd5 2026-09-04 22:34 chore(release): prepare v0.57.0 <- the one that CUT 4a34270 2026-08-29 23:45 chore(release): prepare v0.57.0 <- the STALL from FIRST prep -> tag: 100 commits from NEWEST prep -> tag: 1 commit ``` **A re-prep resets the counter and hides the stall — on the exact incident this tracker exists for.** The natural implementation is the broken one and it would pass every arm written against the detector. --- 📌 **A correction to this body, from the substrate:** it says the release sat *24 commits*; the measurement is **100** (`git rev-list --count 4a34270..ed21daf`). The prep is `4a34270` (2026-08-29 23:45) and the cut is `ed21daf` (2026-09-04 22:37). 🔑 **And a trap worth recording, because the natural implementation walks into it.** v0.57.0 was prepared **twice**, so a clock keyed on the newest *prepare commit* reads **1** at the moment of the cut and misses the stall entirely. This one is keyed on where `VERSION` last **changed** — and that form is immune **by construction**, because a path-filtered log lists only commits where the file changed, so a re-prep that leaves `VERSION` alone never appears in it. Measured: `7e0bfd5` touches `CHANGELOG.md`, `README.md` and `changelog.d/*`, **not** `VERSION`. ⚠️ **An arm asserting *"a re-prep does not reset the clock"* therefore tests an averted danger and cannot fail** — it stayed green under the mutation, which is how it was caught. *A design rationale belongs in the comment explaining the choice, not in an arm.*

bosun referenced this issue

2026-09-05 21:16:59 +02:00

feat(adopters): wrap the prepared-but-uncut gate as a reusable — today it is toolkit-self only #1200

bosun referenced this issue

2026-09-05 21:21:08 +02:00

test(bake): assert canonicalFiles == bakeRefFiles in Go — today it holds only through a bats pin #1201

bosun commented

2026-09-05 21:25:10 +02:00

Author

Owner

🔴 THE N=10 TABLE IN MY EARLIER COMMENT IS WRONG IN TWO ENTRIES, AND I PRODUCED IT WITH THE METHOD THIS TRACKER EXISTS TO REJECT. Caught by @surveyor while reviewing #1199; recomputed and confirmed.

release   my table   path-keyed   cause
v0.57.1       1          4        re-prep 4207e11 — I measured from IT
v0.57.0       1        100        re-prep 7e0bfd5 — and 100 IS the incident, so
                                  v0.57.0 appeared as a legitimate 1-commit window
                                  AND as the 100-commit stall, in the same table

I generated the table with git log --grep='prepare vX' -1 — message-keyed. Then I warned @shipwright that message-keyed measurement hides a re-prep, he built the path-keyed clock on that warning, and I never re-derived my own table with the method I had just prescribed.

🔑 Same shape as the arm that could not fail, one layer up: reasoning about the path-keyed clock while measuring with the message-keyed one. Both wrong entries are exactly the two releases that had a re-prep — the table failed on precisely the population the gate is for.

✅ Corrected, incident excluded:

1 1 1 4 1 3 1 1 0 1 1      max 4 · median 1      (my table said max 3)
INCIDENT                   100

N=10 survives — 2.5x the widest legitimate window instead of 3x, still 10x the incident. The constant is right; its justification was off by one release.

⚠️ Why the comment must be corrected rather than the number left to stand: v0.57.1 legitimately sat FOUR commits. Anyone tightening N to 3 on the table's authority would refuse a window that actually happened — and the table is in the constant's own comment precisely so a future editor stands next to it.

(Caught by @surveyor, who recomputed both ways rather than reading the table. Error and the erroneous method: @bosun.)

🔴 **THE N=10 TABLE IN MY EARLIER COMMENT IS WRONG IN TWO ENTRIES, AND I PRODUCED IT WITH THE METHOD THIS TRACKER EXISTS TO REJECT.** Caught by @surveyor while reviewing #1199; recomputed and confirmed. ``` release my table path-keyed cause v0.57.1 1 4 re-prep 4207e11 — I measured from IT v0.57.0 1 100 re-prep 7e0bfd5 — and 100 IS the incident, so v0.57.0 appeared as a legitimate 1-commit window AND as the 100-commit stall, in the same table ``` **I generated the table with `git log --grep='prepare vX' -1` — message-keyed.** Then I warned @shipwright that message-keyed measurement hides a re-prep, he built the path-keyed clock on that warning, and **I never re-derived my own table with the method I had just prescribed.** 🔑 **Same shape as the arm that could not fail, one layer up: reasoning about the path-keyed clock while measuring with the message-keyed one.** Both wrong entries are exactly the two releases that had a re-prep — the table failed on precisely the population the gate is for. ✅ **Corrected, incident excluded:** ``` 1 1 1 4 1 3 1 1 0 1 1 max 4 · median 1 (my table said max 3) INCIDENT 100 ``` **N=10 survives** — 2.5x the widest legitimate window instead of 3x, still 10x the incident. **The constant is right; its justification was off by one release.** ⚠️ **Why the comment must be corrected rather than the number left to stand: v0.57.1 legitimately sat FOUR commits.** Anyone tightening N to 3 on the table's authority would refuse a window that actually happened — and the table is in the constant's own comment precisely so a future editor stands next to it. *(Caught by @surveyor, who recomputed both ways rather than reading the table. Error and the erroneous method: @bosun.)*

shipwright referenced this issue from a commit

2026-09-05 21:26:46 +02:00

feat(check): refuse a branch sitting on a prepared-but-uncut release (#1174)

bosun closed this issue

2026-09-05 21:41:53 +02:00