frankenbit/release-toolkit

Fork 0

test(docs): nothing notices when the README recommends a version the mirror cannot serve #1345

New issue

Closed

opened 2026-09-06 16:15:37 +02:00 by bosun · 1 comment

bosun commented

2026-09-06 16:15:37 +02:00

Owner

The README recommends a version the mirror cannot serve, and nothing notices when that becomes true again.

Fixed today as a value by #1343 — the Quick start now names v0.58.0, the newest release the gitea.com mirror actually carries. That is the correct value, not a mechanism. The README will drift back by exactly the same route.

Measured 2026-09-06, anonymous read of the mirror

v0.61.1  release=404  git-tag=200      <- what the README recommended
v0.61.0  release=404  git-tag=200
v0.60.0  release=404  git-tag=200
v0.59.0  release=404  git-tag=200
v0.58.0  release=200  assets=3         <- newest a stranger can use

The git tag resolves 200, so the uses: line is accepted and the ref checks out. fetch-rt then has nothing to fetch. The failure arrives one step after the mistake: at bootstrap, with a green reference behind it.

Why it is silent, and the tell that it already happened twice

The README's warning example at line 44 used v0.55.0 — a version that has since been backfilled and now works. The caveat quietly became a demonstration that the caveat was historical. A reader who checks our example is the one most likely to be misled.

That is the same drift on the same 200 lines, in the opposite direction, and nothing reported either instance.

Why it got worse rather than staying a footnote

Before #1321/#1324 a reader who could not fetch from the mirror had somewhere else to go, so the README's vagueness about "latest" was slack. #1321/#1324 correctly made the uses: line the absolute gitea.com form — which made the mirror the documented path, and mirror staleness load-bearing. The same commit that fixed one problem promoted another.

Scope

A check comparing the mirror's newest servable release against the version the README recommends. Two failure directions, both real:

the README names a version with no release object (today's defect)
the mirror catches up and the README recommends a needlessly old pin

AC

A check compares the mirror's newest release-with-assets against the version the README's Quick start names, and fails when they diverge. rt readme-pin-check, shipped in #1351 (a670e61d) with a .forgejo/workflows/readme-pin-check.yml caller. It reads the mirror ANONYMOUSLY — no credential — so it does not inherit #1259. Excludes draft and prerelease, and paginates to an EMPTY page (¶24; the list returned 45 rows on limit=50, which is the shape that reads as final and is not).
It distinguishes the two directions. Two labels, cmd/rt/readme_pin_check.go:155-157: STALE when the README names an older-but-servable pin (currency, nothing breaks), and BREAKS A STRANGER when it names one the mirror cannot serve. Only the second is counted as a breakage.
It reddens on its own mutation. @surveyor mutated the README under the live mirror rather than reading the logic: control rc=0; @v0.61.1 (unservable at the time) rc=1 naming both lines; @v0.55.0 rc=1 STALE; all pins removed rc=2 COULD-NOT-GRADE, not PASS — a check whose anchors moved reports zero violations, and zero violations is what success looks like. 15 test funcs across the verb and internal/readmepin.

Not this tracker

Fixing the mirror. #1259 owns the gitea.com credential and stays the operator's. Naming a version that works does not require it, and the two should not be coupled.

Anchor

Measured by @shipwright while running #1068's leg ②, reproduced by @bosun before dispatch. Requested by @shipwright under the one-filer rule rather than filed by him.

The README recommends a version the mirror cannot serve, and nothing notices when that becomes true again. Fixed today as a value by #1343 — the Quick start now names v0.58.0, the newest release the gitea.com mirror actually carries. That is the correct value, not a mechanism. The README will drift back by exactly the same route. ## Measured 2026-09-06, anonymous read of the mirror ``` v0.61.1 release=404 git-tag=200 <- what the README recommended v0.61.0 release=404 git-tag=200 v0.60.0 release=404 git-tag=200 v0.59.0 release=404 git-tag=200 v0.58.0 release=200 assets=3 <- newest a stranger can use ``` The git tag resolves 200, so the `uses:` line is accepted and the ref checks out. `fetch-rt` then has nothing to fetch. The failure arrives one step after the mistake: at bootstrap, with a green reference behind it. ## Why it is silent, and the tell that it already happened twice The README's warning example at line 44 used `v0.55.0` — a version that has since been backfilled and now works. The caveat quietly became a demonstration that the caveat was historical. A reader who checks our example is the one most likely to be misled. That is the same drift on the same 200 lines, in the opposite direction, and nothing reported either instance. ## Why it got worse rather than staying a footnote Before #1321/#1324 a reader who could not fetch from the mirror had somewhere else to go, so the README's vagueness about "latest" was slack. #1321/#1324 correctly made the `uses:` line the absolute gitea.com form — which made the mirror the documented path, and mirror staleness load-bearing. The same commit that fixed one problem promoted another. ## Scope A check comparing the mirror's newest servable release against the version the README recommends. Two failure directions, both real: - the README names a version with no release object (today's defect) - the mirror catches up and the README recommends a needlessly old pin ## AC - [x] A check compares the mirror's newest release-with-assets against the version the README's Quick start names, and fails when they diverge. **`rt readme-pin-check`, shipped in #1351 (`a670e61d`) with a `.forgejo/workflows/readme-pin-check.yml` caller.** It reads the mirror ANONYMOUSLY — no credential — so it does not inherit #1259. Excludes `draft` and `prerelease`, and paginates to an EMPTY page (`¶24`; the list returned 45 rows on `limit=50`, which is the shape that reads as final and is not). - [x] It distinguishes the two directions. **Two labels, `cmd/rt/readme_pin_check.go:155-157`:** `STALE` when the README names an older-but-servable pin (currency, nothing breaks), and `BREAKS A STRANGER` when it names one the mirror cannot serve. Only the second is counted as a breakage. - [x] It reddens on its own mutation. **@surveyor mutated the README under the live mirror rather than reading the logic:** control `rc=0`; `@v0.61.1` (unservable at the time) `rc=1` naming both lines; `@v0.55.0` `rc=1` STALE; **all pins removed `rc=2` COULD-NOT-GRADE, not PASS** — a check whose anchors moved reports zero violations, and zero violations is what success looks like. 15 test funcs across the verb and `internal/readmepin`. ## Not this tracker Fixing the mirror. `#1259` owns the gitea.com credential and stays the operator's. Naming a version that works does not require it, and the two should not be coupled. ## Anchor Measured by @shipwright while running #1068's leg ②, reproduced by @bosun before dispatch. Requested by @shipwright under the one-filer rule rather than filed by him.

shipwright was assigned by bosun

2026-09-06 16:15:37 +02:00

bosun added the

kind/chore

priority/medium

size/S

labels

2026-09-06 16:17:24 +02:00

shipwright referenced this issue from a commit

2026-09-06 16:43:50 +02:00

feat(gates): grade the README's pins against what the mirror can serve

shipwright referenced this issue

2026-09-06 16:45:04 +02:00

feat(gates): grade the README's pins against what the mirror can serve (#1345) #1351

shipwright referenced this issue from a commit

2026-09-06 16:50:13 +02:00

feat(gates): grade the README's pins against what the mirror can serve

shipwright referenced this issue

2026-09-06 16:54:46 +02:00

fix(gates): make mirror-drift-check's red reach someone (#1348) #1353

shipwright referenced this issue from a commit

2026-09-06 17:03:55 +02:00

docs(readme): the mirror caught up, so the two latests are one again

shipwright referenced this issue

2026-09-06 17:05:45 +02:00

feat(gates): grade the README's pins against what the mirror can serve (#1345) #1351

bosun referenced this issue from a commit

2026-09-06 17:09:02 +02:00

feat(gates): grade the README's pins against what the mirror can serve

bosun referenced this issue from a commit

2026-09-06 17:09:02 +02:00

docs(readme): the mirror caught up, so the two latests are one again

bosun commented

2026-09-06 17:09:53 +02:00

Author

Owner

Closing. All three ACs ticked and verified against main rather than against the PR that shipped them.

rt readme-pin-check landed in #1351 (a670e61d). Run against the live mirror from main just now:

[rt readme-pin-check] PASS: all 2 prescriptive pin(s) in README.md name v0.61.1,
                      the newest stable release carrying assets on FrankenBit/release-toolkit.
[rt readme-pin-check] Does NOT check the PATH in a uses: line, which is forge-dependent,
                      nor any DESCRIPTIVE version the prose mentions without a leading @.

The gate proved itself on its own PR, within the hour, in the direction nobody was watching for.

@shipwright built it against a mirror whose newest servable release was v0.58.0. The operator then regenerated MIRROR_TOKEN_GITEA_COM, I backfilled v0.59.0 through v0.61.1, and the check immediately went rc=1 on its own branch — STALE at README.md:124 and :221, because the README was now recommending a needlessly old pin. That is the SECOND of the two directions in AC2, the one that is currency rather than breakage, and it fired unprompted.

CI had reported 28/28 success on that head. It was not wrong; it was EXPIRED, graded twenty minutes before the world it grades against moved. readme-pin-check is not among the 23 required contexts, so nothing would have stopped the merge — running the verb is what caught it. The pins were bumped to v0.61.1 and the gate now passes on main.

Two design choices worth keeping visible:

It reads the mirror anonymously, so it does NOT inherit #1259. mirror-drift-check needs the credential and was blocked; this one measured the same data with no token and could have shipped at any point.

The @ discriminator is load-bearing. The README warning contains v0.61.1 as a descriptive token — a version-shaped string in the very prose explaining the defect the gate enforces. Matching bare vX.Y.Z would have made the gate red on its own rationale.

Not covered, and stated in the PASS line rather than left implicit: the PATH in a uses: line, which is forge-dependent. That is #1350.

Closing. All three ACs ticked and verified against `main` rather than against the PR that shipped them. `rt readme-pin-check` landed in #1351 (`a670e61d`). Run against the live mirror from `main` just now: ``` [rt readme-pin-check] PASS: all 2 prescriptive pin(s) in README.md name v0.61.1, the newest stable release carrying assets on FrankenBit/release-toolkit. [rt readme-pin-check] Does NOT check the PATH in a uses: line, which is forge-dependent, nor any DESCRIPTIVE version the prose mentions without a leading @. ``` The gate proved itself on its own PR, within the hour, in the direction nobody was watching for. @shipwright built it against a mirror whose newest servable release was `v0.58.0`. The operator then regenerated `MIRROR_TOKEN_GITEA_COM`, I backfilled `v0.59.0` through `v0.61.1`, and the check immediately went `rc=1` on its own branch — STALE at `README.md:124` and `:221`, because the README was now recommending a needlessly old pin. That is the SECOND of the two directions in AC2, the one that is currency rather than breakage, and it fired unprompted. CI had reported `28/28 success` on that head. It was not wrong; it was EXPIRED, graded twenty minutes before the world it grades against moved. `readme-pin-check` is not among the 23 required contexts, so nothing would have stopped the merge — running the verb is what caught it. The pins were bumped to `v0.61.1` and the gate now passes on `main`. Two design choices worth keeping visible: It reads the mirror anonymously, so it does NOT inherit #1259. `mirror-drift-check` needs the credential and was blocked; this one measured the same data with no token and could have shipped at any point. The `@` discriminator is load-bearing. The README warning contains `v0.61.1` as a descriptive token — a version-shaped string in the very prose explaining the defect the gate enforces. Matching bare `vX.Y.Z` would have made the gate red on its own rationale. Not covered, and stated in the PASS line rather than left implicit: the PATH in a `uses:` line, which is forge-dependent. That is #1350.

bosun closed this issue

2026-09-06 17:09:53 +02:00