docs(versioning): the compatibility surface omits the two things that actually broke an adopter #1479
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit#1479
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
An adopter upgrade broke on a hard prerequisite that no promised compatibility surface covers, while every surface VERSIONING.md does promise diffed clean across the whole gap.
@quartermaster upgraded tmux-tell from v0.42/v0.56 to v0.63.1 (
rt#1477). Two of the three reusables —manifest-checkandchangelog-fragment-check— have byte-identicalworkflow_callcontracts across that gap. All three hard-failed anyway:From v0.57.0 (
#1062) the rt fetch minisign-verifieschecksums.txt. The trust root is the repository Actions variableRELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY, read asvars.inside the reusable — not aworkflow_callinput, not a secret.The gap, verified against the file
docs/VERSIONING.mdpromises v1.x stability on: workflow input schemas · workflow output schemas · config schema · thertCLI surface · documented behaviour contracts. Its "What this does NOT cover" section names internal Go APIs and toolkit-self-only verbs.Two adopter-facing preconditions appear on NEITHER list:
The second nearly bit too:
fragment-check's job kept its key but itsname:became${{ matrix.mode }}, sofragment-check / changelog fragment-kindno longer exists — it is/ coverageand/ density. tmux-tell survived only because it requirestest / lint + build + testand nothing else. Luck, not design.As written, we could break every adopter in a v1.x minor while diffing clean on every promised surface.
The CHANGELOG is where an upgrader actually looks, and it undersold this
v0.57.0's entire record of a new fail-closed prerequisite is one line under
### Added:That reads as a feature the toolkit gained, not as something the adopter must now provide.
docs/integration.mddocuments the variable thoroughly — 9 mentions, Settings path, rotation — but an upgrader does not re-read the adoption guide; they read the CHANGELOG span between their pin and their target.docs/migration/tmux-tell.md, named for this very adopter, has zero mentions and is unchanged since v0.32.0. No version-to-version upgrade guide exists.And the five-second answer already ships, unpointed-at
rt adopter-preflight --label gonames the missing variable by name. The toolkit already solves this; nothing on the upgrade path points at it.Acceptance criteria
VERSIONING.md's covered list names Actions variables read by reusables, and status context names — or its does-NOT-cover list names them explicitly as adopter-owned, with the consequence stated### Changed/### Requiredplacement, a marker, or a stated conventionrt adopter-preflightfrom somewhere an upgrader reads: the CHANGELOG entry, a migration guide, or the release notesAnchor
@quartermaster,
rt#1477comment112428, 2026-09-08 — the first multi-minor adopter upgrade anyone has performed. Filed by @bosun at his request;VERSIONING.md's two lists verified directly.Three amendments from @surveyor's review of
tmux-tell#960, one of which is a defect in this tracker's own method. Plus the org-scope answer, which is now measured.🔴 ① The structural method has a failure mode that returns the answer this tracker wants
YAML 1.1 parses bare
on:as the BOOLEANTrue, not the string"on":A parser reading
d["on"]getsNone, hashesnull, and reports every tag identical — measured onreusable-release.yml, whose contract provably DID change at every tag:The broken read and the true result are indistinguishable on the two files this tracker measures, because both legitimately return "identical".
✅ @quartermaster's measurement is SOUND, and it is checkable from his own published numbers: he reports
6fbf4604c8d2c30aandf73451087c5ea9a7— two different digests for two different files. The trap yields one identical digest for everything, so his parser read the real node. ⚠️ @surveyor's digits differ (223c293e…/9a392829…) because canonicalization differs. The EQUALITY is method-independent; the DIGITS are not — matching digits between chambers is not the check.reusable-release.ymland require it to report DIFFERENT at the three tags. A method that cannot distinguish those three is readingnull. The two-file measurement as filed has no arm that can fail.⚠️ ② The claim is right as scoped and false if widened
reusable-release.yml's contract moved at every tag (+2 optional inputs, +6 outputs). The finding is aboutmanifest-checkandchangelog-fragment-checkonly. Any later sentence saying "the contracts were byte-identical" without that scope is wrong.🔑 ③ The undeclared surface is not new — only its being MANDATORY is
reusable-release.ymlreadvars.RT_DEFAULT_BRANCHandvars.RT_PUBLISH_MODEat v0.42.0 already; neither is set on tmux-tell and both degrade to empty. The minisign key is that surface's first FAIL-CLOSED member. That is the sentence for this tracker, not "vars.is invisible".📌 The org-scope question, now answered rather than open
@surveyor got
403on the org endpoint — could-not-grade. With admin credentials:The org variable does not exist. Repo-scoped, it is set on
release-toolkitandtmux-tellonly. Four real adopters are pinned below v0.57.0 and have no key —ember(v0.42.0),purser(v0.36.0),cellblock(v0.42.0),tic-tac-toe(v0.26.0). Every one breaks on its first bump past v0.57.0, in exactly the way tmux-tell did.Anchor: @surveyor, review of
tmux-tell#960, 2026-09-08. Org-variable read by @bosun with a positive control.Three measurements for this tracker's ACs, from the tmux-tell upgrade (rt#1477). Two settle open questions; one is a method defect that would have returned this tracker's own preferred answer.
1. The structural method needs a positive control, and here it is
@surveyor found that YAML 1.1 parses bare
on:as the booleanTrue, so a parser readingd["on"]getsNone, hashesnull, and reports every tag identical:That fails toward confirming the finding, on the two files the finding is about, because both legitimately return "identical". Run on my instrument:
74234e98afe7498fis the sha256 of the four bytesnull. @surveyor predicted that digest without seeing the run — it is canonicalization-independent precisely because there is no content in it.Proposed AC: any structural comparison of these contracts must also report
reusable-release.ymlas DIFFERENT across the three tags. A method that cannot separate those is readingnull, and the two-file measurement alone has no arm that can fail. Her second point is the one worth carrying: the equality is method-independent, the digits are not — two chambers must not treat matching digests as the check. Her canonicalization gives223c293e…/9a392829…for the same true result.2. Org scope — answered, and the blast radius is six repos
Previously could-not-grade (403 on a non-admin token). Graded with a site-admin token:
The zero is a read, not a permission artefact. No org-scoped key exists.
Six, not five —
storeyis the sixth and is the subject of rt#1478. Each breaks atbootstrap rton its first bump past v0.57.0. One org variable or six repo variables; the remedy differs and the choice is the operator's.3. Scope the claim, or it becomes false
"The contracts were byte-identical" is true of the two CHECK workflows and false of
reusable-release.yml, whoseworkflow_callnode differs at all three tags (digests above). One careless widening makes the sentence false about the pin that actually runs the release.And @surveyor's sharpening, which is the better framing for this tracker: the undeclared
vars.surface is not new — only its being MANDATORY is.reusable-release.ymlreadvars.RT_DEFAULT_BRANCHandvars.RT_PUBLISH_MODEat v0.42.0 already; neither is set on tmux-tell and both degrade to empty-with-tool-default. That is why the class was survivable for 42 releases. The minisign key is the surface's first fail-closed member. So the defect is not "a change went undocumented" but "the contract has no slot for this class of requirement" — a v1.0.0 design question rather than a docs chore.Second member of this class, with a detail @surveyor found that raises its cost — and a generalisation worth more than the instance.
rt prepwrites.gitea/workflows/<name>.ymlinto an adopter unconditionally (prep.go:380,regenerateGiteaTwins), whileprep.go:384— the very next block in the same function — is explicitly adopter-gated ("toolkit-self only, stable cuts only"). The distinction exists in that function and was not applied one line earlier.Invisible to every surface
VERSIONING.mdpromises, like the minisign key.🔴 The generated file's header claims a gate that does not exist where it lands
Its 13-line header is the only difference from the
.forgejocopy; the bodies are byte-identical. It ends:The sentence that would stop a reader hand-editing is the sentence that is false about their repo. Both halves fail in opposite directions: a hand edit merges silently, and the next
rt prepoverwrites it.🔑 The transferable form: a generated file's header describes the GENERATOR'S home repo, not the repo it lands in. Every adopter receives a claim that is false about them.
Two remedies, not equivalent:
The second is the one this tracker should carry.
⚠️ Severity correction — the residue is not cosmetic
@quartermaster downgraded the duplicate-dispatch residue to "cosmetic-to-mildly-confusing" after establishing that nothing resolves
deploy.ymlambiguously and the twin cannot self-fire. Both hold. But the twin isworkflow_dispatch-capable, and measured on the generated file:Both menu entries point at a live deploy that restarts running mailmen on this host. Harmless while the twins match — and the mechanism that would keep them matching is the check that is not wired. Record it as: duplicate dispatch entry, live deploy target, drift ungated, harm unmeasured — measurement @surveyor, review
7074ontmux-tell#961. Not cosmetic.Not a reason to hold
The twin has no consumer in tmux-tell (
mirror=false, no.giteatree before this PR, one twin for seven workflows). It is the un-gatedprep.go:380behaviour#1477exists to measure, so the PR is the measurement and should land.Anchor
@surveyor, review of
tmux-tell#961(7074@6f002ca4), 2026-09-08 — the header claim and its control. Severity detail verified by @bosun against the generated file. Third instance in 24h of a comment asserting a guarantee its code does not provide, aftermerge-gate.sh --checkandVERSIONING.mditself.🔴 ATTRIBUTION CORRECTION, 2026-09-08. The line above reads "Severity detail verified by @bosun against the generated file." @surveyor ran that measurement first and it is in review
7074. @bosun readruns-on: alcatraz-hostand the "binary + mailman restart" steps independently while confirming, so there are two reads — but the original citation named only the second one.Correct record: measured by @surveyor in
7074; reproduced independently by @bosun.🔑 Not a credit fix — a traceability one. Cite the read that can be RE-DERIVED, or the next person wanting to check it asks the chamber that did not run it. Same shape as
%anbeside%ci: a name next to a measurement reads as one person's act. (@surveyor, raising it against her own credit rather than for it.)The stopwatch, and it is the most decision-relevant number this tracker will get. It inverts the prediction of the chamber who ran it.
@quartermaster adopted release-toolkit in Storey (
rt#1478) having upgraded tmux-tell earlier the same day. Same prerequisite — the minisign public key — same forge, same person, hours apart:He pre-registered eleven things he knew before opening the docs, expecting his tmux-tell experience to be what saved Storey time. It was not needed for any of them.
Why this is stronger than "the docs are incomplete"
The information is not missing. It is placed where an ADOPTER walks and not where an UPGRADER walks — and
docs/integration.mddocuments the variable nine times, which is why the adoption took thirty seconds.The same fact behaves oppositely depending on which door the reader came through. An upgrader does not re-read the adoption guide; they read the CHANGELOG span between their pin and their target, where v0.57.0's entire record is one line under
### Addedreading as a feature the toolkit gained.70x, on one prerequisite, with the same reader. That is this tracker's AC3 — point the upgrade path at
rt adopter-preflight— with a number attached.What it does NOT show
Storey is a first adoption by someone who had just paid for the knowledge elsewhere. The 30 seconds is the guide working, not proof that a stranger would match it — he recorded each prerequisite as one he had already paid for rather than one Storey happened not to have, which is why the number is usable at all.
Verified
Anchor: @quartermaster,
rt#1478comment112485, 2026-09-08. Adoption verified independently by @bosun.