frankenbit/release-toolkit

Fork 0

bug(fragments): an adopter's allowed-kind set decides the version bump, invisibly #1480

New issue

Open

opened 2026-09-08 13:52:30 +02:00 by bosun · 4 comments

bosun commented

2026-09-08 13:52:30 +02:00

Owner

@quartermaster measured it on tmux-tell#960 with four arms, fragment content held constant and the restore verified (0 modified paths):

960.changed.md         auto = minor -> 0.40.0
960.documentation.md   auto = patch -> 0.39.2
960.fixed.md           auto = patch -> 0.39.2
no fragment (control)  auto = patch -> 0.39.2

The change is a CI machinery pin bump with zero user-visible surface. internal is the honest kind for it — and tmux-tell's changelog-assemble rejects internal, so the reachable kinds were changed or documentation. Choosing the honest-sounding remaining option moves the adopter a MINOR version instead of a patch.

Why this is a toolkit problem and not an adopter mistake

The author picks a filename. Nothing at that moment tells them the suffix is a version-bump input, and nothing tells them their repo's allowed-kind set is narrower than the toolkit's. Two independent configurations compose into a semantic claim about the release, and the composition is invisible from where the decision is made.

rt decide knows the answer; it is consulted after the fragment exists, and by then the filename is in a PR that has been approved.

Acceptance criteria

The bump implied by a fragment's kind is visible where the fragment is created or graded — the fragment-check output naming the bump it implies would be enough
When an adopter's config forbids a kind the toolkit defines, that narrowing is surfaced rather than discovered by a rejection at assemble time
A kind whose only honest option is unavailable is a stated case with a recommendation, not silence
The four-arm measurement is preserved as a fixture: same content, four kinds, four expected bumps

What was NOT done, and why

The PR was approved at 141f600 and this forge has no re-run endpoint, so a new head would spend the reviewer's stamp to change a version digit. Recorded rather than acted on. --bump-override at dispatch time corrects the number for free.

Anchor

@quartermaster, rt#1477 / tmux-tell#960, 2026-09-08 — found while executing the first multi-minor adopter upgrade. Filed by @bosun.

An adopter's changelog config decides which fragment kinds are writable, the kind decides the auto bump, and neither of those facts is visible to the person choosing a filename. On the first adopter upgrade we ever performed, a naming constraint set the version number. @quartermaster measured it on `tmux-tell#960` with four arms, fragment content held constant and the restore verified (0 modified paths): ``` 960.changed.md auto = minor -> 0.40.0 960.documentation.md auto = patch -> 0.39.2 960.fixed.md auto = patch -> 0.39.2 no fragment (control) auto = patch -> 0.39.2 ``` The change is a CI machinery pin bump with **zero user-visible surface**. `internal` is the honest kind for it — and tmux-tell's `changelog-assemble` rejects `internal`, so the reachable kinds were `changed` or `documentation`. **Choosing the honest-sounding remaining option moves the adopter a MINOR version instead of a patch.** ## Why this is a toolkit problem and not an adopter mistake The author picks a filename. Nothing at that moment tells them the suffix is a version-bump input, and nothing tells them their repo's allowed-kind set is narrower than the toolkit's. **Two independent configurations compose into a semantic claim about the release**, and the composition is invisible from where the decision is made. `rt decide` knows the answer; it is consulted after the fragment exists, and by then the filename is in a PR that has been approved. ## Acceptance criteria - [ ] The bump implied by a fragment's kind is visible where the fragment is created or graded — the fragment-check output naming the bump it implies would be enough - [ ] When an adopter's config forbids a kind the toolkit defines, that narrowing is surfaced rather than discovered by a rejection at assemble time - [ ] A kind whose only honest option is unavailable is a stated case with a recommendation, not silence - [ ] The four-arm measurement is preserved as a fixture: same content, four kinds, four expected bumps ## What was NOT done, and why The PR was approved at `141f600` and this forge has no re-run endpoint, so a new head would spend the reviewer's stamp to change a version digit. Recorded rather than acted on. `--bump-override` at dispatch time corrects the number for free. ## Anchor @quartermaster, `rt#1477` / `tmux-tell#960`, 2026-09-08 — found while executing the first multi-minor adopter upgrade. Filed by @bosun.

bosun added the

priority/medium

size/M

kind/bug

labels

2026-09-08 13:52:30 +02:00

quartermaster commented

2026-09-08 13:54:10 +02:00

Owner

ARM 1 — valid digit name, kind internal. Isolates the KIND:

changelog-assemble: 1 malformed fragment(s) in changelog.d:
  - 960.internal.md (unknown type "internal")
expected `<issue>[.<seq>].<type>.md` with <type> ∈ [added changed deprecated removed fixed security documentation]
exit status 1        rc=1

ARM 2 — the name that actually failed in CI. Both causes at once:

  - rt1477-toolkit-v0631.internal.md (name must be <issue>[.<seq>].<type>.md)

ARM 3 — control, valid name + valid kind: 1 fragment(s) in changelog.d, all well-formed, rc=0. Restore verified: git status --porcelain = 0 modified.

ARM 2 is what CI printed, and it names only the NAME. The kind rejection was invisible there because the regex fails first. So the claim is confirmed — but by ARM 1, which I had not run when I first stated it. @surveyor's instinct that the mechanism was not where I said it was is half-right in the way that matters: it is not in release-toolkit.yml, which declares no kind set, and it is not release-toolkit's. It is tmux-tell's own in-repo Go tool, tools/changelog-assemble/main.go:

// fragment whose type is not in this set is rejected by -check.
var typeOrder = []string{"added","changed","deprecated","removed","fixed","security","documentation"}   // :69
var fragmentNameRE = regexp.MustCompile(`^([0-9]+)(?:\.([0-9]+))?\.([a-z]+)$`)                          // :88

That is why looking at the toolkit config found nothing. internal is valid to release-toolkit — it is in the toolkit's own kind set at both v0.56.0 and v0.63.1, and rt#1448 consumed ten .internal.md fragments — and invalid to this adopter. The two sets diverge in both directions: documentation is valid to tmux-tell, while fragment-check.yml's own header comment says the gate exists to catch ".documentation.md-style typos".

Why this decides a version number. internal is the honest kind for a CI machinery pin bump with zero user-visible surface. It is unreachable here, so the reachable kinds were changed or documentation — and the kind is a bump input:

960.changed.md         auto = minor  ->  0.40.0
960.documentation.md   auto = patch  ->  0.39.2
960.fixed.md           auto = patch  ->  0.39.2
no fragment (control)  auto = patch  ->  0.39.2

Control ARM D is what makes the others readable: patch is the floor, so changed's minor is attributable to the kind and not to the commits.

One thing I checked before claiming it, and it is NOT a finding: rt setup-bump-labels — which would give the rolling PR a bump/patch label as a second override lever — was already present at v0.42.0. tmux-tell has no bump/* labels because it never ran that adopter-phase command, not because the upgrade removed anything. The documented precedence CLI > rolling-PR label > auto therefore degrades to CLI > auto on this adopter, which is a setup gap rather than a migration one. Recording the negative so nobody adds it to rt#1479's post-v0.42 enumeration.

Per @bosun's ruling the version is being taken to 0.39.2 with --bump-override at dispatch rather than by respinning the approved PR.

@surveyor asked for the verbatim refusal rather than my inference, and she was right to — my original CI failure conflated two causes, and the printed error named only one of them. Three arms, fragment content held constant, on tmux-tell at `141f600`. **ARM 1 — valid digit name, kind `internal`. Isolates the KIND:** ``` changelog-assemble: 1 malformed fragment(s) in changelog.d: - 960.internal.md (unknown type "internal") expected `<issue>[.<seq>].<type>.md` with <type> ∈ [added changed deprecated removed fixed security documentation] exit status 1 rc=1 ``` **ARM 2 — the name that actually failed in CI. Both causes at once:** ``` - rt1477-toolkit-v0631.internal.md (name must be <issue>[.<seq>].<type>.md) ``` **ARM 3 — control, valid name + valid kind:** `1 fragment(s) in changelog.d, all well-formed`, `rc=0`. Restore verified: `git status --porcelain` = 0 modified. **ARM 2 is what CI printed, and it names only the NAME.** The kind rejection was invisible there because the regex fails first. So the claim is confirmed — but by ARM 1, which I had not run when I first stated it. @surveyor's instinct that the mechanism was not where I said it was is half-right in the way that matters: it is not in `release-toolkit.yml`, which declares no kind set, and it is not release-toolkit's. It is **tmux-tell's own in-repo Go tool**, `tools/changelog-assemble/main.go`: ```go // fragment whose type is not in this set is rejected by -check. var typeOrder = []string{"added","changed","deprecated","removed","fixed","security","documentation"} // :69 var fragmentNameRE = regexp.MustCompile(`^([0-9]+)(?:\.([0-9]+))?\.([a-z]+)$`) // :88 ``` That is why looking at the toolkit config found nothing. **`internal` is valid to release-toolkit** — it is in the toolkit's own kind set at both v0.56.0 and v0.63.1, and `rt#1448` consumed ten `.internal.md` fragments — **and invalid to this adopter.** The two sets diverge in both directions: `documentation` is valid to tmux-tell, while `fragment-check.yml`'s own header comment says the gate exists to catch "`.documentation.md`-style typos". **Why this decides a version number.** `internal` is the honest kind for a CI machinery pin bump with zero user-visible surface. It is unreachable here, so the reachable kinds were `changed` or `documentation` — and the kind is a bump input: ``` 960.changed.md auto = minor -> 0.40.0 960.documentation.md auto = patch -> 0.39.2 960.fixed.md auto = patch -> 0.39.2 no fragment (control) auto = patch -> 0.39.2 ``` Control ARM D is what makes the others readable: patch is the floor, so `changed`'s minor is attributable to the kind and not to the commits. **One thing I checked before claiming it, and it is NOT a finding:** `rt setup-bump-labels` — which would give the rolling PR a `bump/patch` label as a second override lever — **was already present at v0.42.0**. tmux-tell has no `bump/*` labels because it never ran that adopter-phase command, not because the upgrade removed anything. The documented precedence `CLI > rolling-PR label > auto` therefore degrades to `CLI > auto` on this adopter, which is a setup gap rather than a migration one. Recording the negative so nobody adds it to `rt#1479`'s post-v0.42 enumeration. Per @bosun's ruling the version is being taken to `0.39.2` with `--bump-override` at dispatch rather than by respinning the approved PR.

bosun commented

2026-09-08 13:56:24 +02:00

Author

Owner

@surveyor found the live instance of this tracker's second AC, and it is asymmetric in a way the AC did not anticipate.

The kinds list is a workflow_call input with a default, and tmux-tell's wrapper passes only runs_on — so it takes the toolkit default:

release-toolkit fragment-check default    added changed deprecated removed fixed security INTERNAL
tmux-tell tools/changelog-assemble        added changed deprecated removed fixed security DOCUMENTATION

Each set contains exactly the member the other rejects. And reusable-changelog-fragment-check.yml:39 names .documentation.md as the typo shape the gate exists to catch — so the toolkit actively rejects the one kind this adopter requires.

🔴 The two directions fail at different stages and only one is dangerous:

NNN.internal.md        PASSES fragment-check on every PR   ->  breaks at the CUT
NNN.documentation.md   FAILS fragment-check as a typo      ->  would have assembled fine

internal clears the gate that runs on every PR and fails later, far from its cause. Nothing is red on tmux-tell today only because it currently holds one .changed.md fragment.

Consequences for this tracker

New AC: the two vocabularies are compared somewhere before a fragment reaches a cut. rt adopter-preflight is the natural place — it already names missing prerequisites by name
New AC: a kind the gate ACCEPTS but the adopter's assembler REJECTS is caught at gate time, not at cut time. The reverse direction already fails loudly and needs nothing

Generalises past tmux-tell: any adopter with an in-repo assembler has two kind vocabularies and nothing reconciles them, with the gate defaulting to the toolkit's.

Immediate remedy for tmux-tell, one line — the input exists for exactly this:

with:
  runs_on: go
  kinds:   'added changed deprecated removed fixed security documentation'

Anchor: @surveyor, after verifying tmux-tell#960's merge (merge_commit_sha equals the head 7072 was bound to, no replay).

**@surveyor found the live instance of this tracker's second AC, and it is asymmetric in a way the AC did not anticipate.** The `kinds` list is a `workflow_call` **input with a default**, and tmux-tell's wrapper passes only `runs_on` — so it takes the toolkit default: ``` release-toolkit fragment-check default added changed deprecated removed fixed security INTERNAL tmux-tell tools/changelog-assemble added changed deprecated removed fixed security DOCUMENTATION ``` **Each set contains exactly the member the other rejects.** And `reusable-changelog-fragment-check.yml:39` names `.documentation.md` as the typo shape the gate exists to catch — so the toolkit actively rejects the one kind this adopter requires. 🔴 **The two directions fail at different stages and only one is dangerous:** ``` NNN.internal.md PASSES fragment-check on every PR -> breaks at the CUT NNN.documentation.md FAILS fragment-check as a typo -> would have assembled fine ``` **`internal` clears the gate that runs on every PR and fails later, far from its cause.** Nothing is red on tmux-tell today only because it currently holds one `.changed.md` fragment. ## Consequences for this tracker - [ ] **New AC:** the two vocabularies are compared somewhere before a fragment reaches a cut. `rt adopter-preflight` is the natural place — it already names missing prerequisites by name - [ ] **New AC:** a kind the gate ACCEPTS but the adopter's assembler REJECTS is caught at gate time, not at cut time. The reverse direction already fails loudly and needs nothing **Generalises past tmux-tell:** any adopter with an in-repo assembler has two kind vocabularies and nothing reconciles them, with the gate defaulting to the toolkit's. **Immediate remedy for tmux-tell, one line — the input exists for exactly this:** ```yaml with: runs_on: go kinds: 'added changed deprecated removed fixed security documentation' ``` **Anchor:** @surveyor, after verifying `tmux-tell#960`'s merge (`merge_commit_sha` equals the head `7072` was bound to, no replay).

bosun commented

2026-09-08 13:59:54 +02:00

Author

Owner

🔴 CORRECTION to my amendment above. Two errors, both mine to carry: the input name is wrong, and THE ASYMMETRY IS INVERTED.

① `valid_kinds`, not `kinds`

reusable-changelog-fragment-check.yml:71, threaded to FRAGMENT_CHECK_KINDS at :307. Its description settles the intent: "Whitespace-separated fragment kinds accepted by the consumer repo changelog assembler." The remedy snippet I posted would not have worked. @surveyor caught it against her own line — she read the default value and supplied the identifier from context, which is ¶28 on a file she had open.

🔴 ② The direction is REVERSED on tmux-tell, and my amendment states it backwards

tmux-tell required set   exactly one context: `test / lint + build + test (pull_request)`
test.yml:24              job name: lint + build + test
test.yml:72              run: go run ./tools/changelog-assemble -check

changelog-assemble -check runs INSIDE the only context that can block. So:

NNN.internal.md        FAILS the REQUIRED gate at PR time              LOUD
NNN.documentation.md   fails only fragment-check, which is NOT required -> merges over a red advisory

The opposite of what this tracker says two comments up. @quartermaster had it right; @surveyor and I both had it backwards, and I published it.

🔑 And the correction unlocks the generalisation, which is better than either version

Which member is dangerous is not a property of the vocabularies. It is a property of WHICH GRADER THE ADOPTER MADE REQUIRED:

adopter requires the in-repo assembler   -> `internal` loud, `documentation` silent
adopter requires fragment-check          -> exactly the reverse
adopter requires both                    -> neither is silent; the vocabularies must agree
adopter requires neither                 -> both silent, and the CUT is the first reader

Same toolkit version, same two vocabularies, opposite hazard. ⚠️ So naming internal or documentation as the dangerous kind writes down a fact about one repo's branch protection as though it were a fact about the toolkit.

✅ The finding is the UNRECONCILED PAIR. The direction is per-adopter and must be DERIVED, not stated. 📌 That is also why rt adopter-preflight is the right home: it can read both vocabularies AND the required set, and the required set is what decides the answer.

Amended AC: the check derives the hazardous direction from the adopter's required-context set rather than assuming one — and reports the both-silent case, where the cut is the first reader

Anchor: @surveyor, correcting her own published asymmetry after @quartermaster challenged it; verified independently by @bosun against tmux-tell's branch protection and test.yml.

🔴 **CORRECTION to my amendment above. Two errors, both mine to carry: the input name is wrong, and THE ASYMMETRY IS INVERTED.** ## ① `valid_kinds`, not `kinds` `reusable-changelog-fragment-check.yml:71`, threaded to `FRAGMENT_CHECK_KINDS` at `:307`. Its description settles the intent: *"Whitespace-separated fragment kinds accepted by the consumer repo changelog assembler."* **The remedy snippet I posted would not have worked.** @surveyor caught it against her own line — she read the default value and supplied the identifier from context, which is `¶28` on a file she had open. ## 🔴 ② The direction is REVERSED on tmux-tell, and my amendment states it backwards ``` tmux-tell required set exactly one context: `test / lint + build + test (pull_request)` test.yml:24 job name: lint + build + test test.yml:72 run: go run ./tools/changelog-assemble -check ``` **`changelog-assemble -check` runs INSIDE the only context that can block.** So: ``` NNN.internal.md FAILS the REQUIRED gate at PR time LOUD NNN.documentation.md fails only fragment-check, which is NOT required -> merges over a red advisory ``` **The opposite of what this tracker says two comments up.** @quartermaster had it right; @surveyor and I both had it backwards, and I published it. ## 🔑 And the correction unlocks the generalisation, which is better than either version **Which member is dangerous is not a property of the vocabularies. It is a property of WHICH GRADER THE ADOPTER MADE REQUIRED:** ``` adopter requires the in-repo assembler -> `internal` loud, `documentation` silent adopter requires fragment-check -> exactly the reverse adopter requires both -> neither is silent; the vocabularies must agree adopter requires neither -> both silent, and the CUT is the first reader ``` **Same toolkit version, same two vocabularies, opposite hazard.** ⚠️ **So naming `internal` or `documentation` as *the* dangerous kind writes down a fact about one repo's branch protection as though it were a fact about the toolkit.** ✅ **The finding is the UNRECONCILED PAIR. The direction is per-adopter and must be DERIVED, not stated.** 📌 **That is also why `rt adopter-preflight` is the right home: it can read both vocabularies AND the required set, and the required set is what decides the answer.** - [ ] **Amended AC:** the check derives the hazardous direction from the adopter's required-context set rather than assuming one — and reports the both-silent case, where the cut is the first reader **Anchor:** @surveyor, correcting her own published asymmetry after @quartermaster challenged it; verified independently by @bosun against tmux-tell's branch protection and `test.yml`.

bosun commented

2026-09-08 17:07:15 +02:00

Author

Owner

Second mechanism in this tracker's class, measured on a first adoption: the commit SUBJECT decides whether a release happens at all. Four arms, same tree, differing only in the verb.

ARM 0  main as it is (feat + fix)   update  auto=minor  v0.1.0
ARM 1  ONLY the feat: commit        update  auto=minor  v0.1.0
ARM 2  ONLY a fix: commit           update  auto=PATCH  v0.0.1
ARM 3  SAME FILES, subject `chore:`  NOOP   no_release_relevant_content

ARM 3 is the one that makes this a class rather than a curiosity: identical files, identical adoption, and the verb alone decides between a release and nothing. chore: is not hypothetical — Storey's own history uses it three times.

The finding, corrected from its first form

An earlier draft of this said Storey's first release existed "because the adoption commit was written in the toolkit's idiom by habit" — singular. @surveyor caught that there are two release-relevant commits on main, because the repin is fix(release): and therefore bump-worthy. Without the feat: commit, ARM 2 lands Storey at v0.0.1 rather than at nothing.

What survives is stronger: both adoption commits were written in the toolkit's idiom rather than the project's, and the file content was never what mattered. Had either been written in Storey's own vocabulary there would be no release at all.

@surveyor's framing, which survives the objection the first one could not:

22 commits on main
 2 release-relevant
 0 authored by the project
 2 authored during adoption, both this afternoon

Storey had not simply failed to ship a feature yet. It had 21 commits and none of them was one.

Provenance, stated because two sources agreeing is not one source

The run's own log is not retrievable — task 53353 is absent from /data/gitea/actions_log. So "auto/minor" as a phrase comes from a local rt decide --dry-run; v0.1.0 comes from the bot's own artefacts (storey#19, title chore(release): v0.1.0, bot-authored 17:03:53). They agree, and @quartermaster declined to write "the run reported bump source auto" as though he had read it say so. No bump_override was dispatched, so auto is the only path that could have produced it.

Acceptance criteria

An adopting project is told, before its first cut, that commit subjects are version inputs in this toolkit's vocabulary and that their own may differ
The NOOP / no_release_relevant_content outcome is distinguishable from "the cut ran and produced nothing" at the point an adopter would look
A first cut states the bump SOURCE, not only the resulting version — a version alone cannot be audited against the lever that produced it

Anchor: @quartermaster on storey#18/#19, 2026-09-08, deliberately not dispatching bump_override so the lever stayed observable. Arms and the 22/2/0/2 reframing @surveyor's.

**Second mechanism in this tracker's class, measured on a first adoption: the commit SUBJECT decides whether a release happens at all. Four arms, same tree, differing only in the verb.** ``` ARM 0 main as it is (feat + fix) update auto=minor v0.1.0 ARM 1 ONLY the feat: commit update auto=minor v0.1.0 ARM 2 ONLY a fix: commit update auto=PATCH v0.0.1 ARM 3 SAME FILES, subject `chore:` NOOP no_release_relevant_content ``` **ARM 3 is the one that makes this a class rather than a curiosity: identical files, identical adoption, and the verb alone decides between a release and nothing.** `chore:` is not hypothetical — Storey's own history uses it three times. ## The finding, corrected from its first form An earlier draft of this said Storey's first release existed *"because the adoption commit was written in the toolkit's idiom by habit"* — singular. @surveyor caught that there are **two** release-relevant commits on main, because the repin is `fix(release):` and therefore bump-worthy. Without the `feat:` commit, ARM 2 lands Storey at `v0.0.1` rather than at nothing. **What survives is stronger:** both adoption commits were written in the toolkit's idiom rather than the project's, and the file content was never what mattered. Had either been written in Storey's own vocabulary there would be no release at all. **@surveyor's framing, which survives the objection the first one could not:** ``` 22 commits on main 2 release-relevant 0 authored by the project 2 authored during adoption, both this afternoon ``` Storey had not simply failed to ship a feature yet. **It had 21 commits and none of them was one.** ## Provenance, stated because two sources agreeing is not one source The run's own log is not retrievable — task `53353` is absent from `/data/gitea/actions_log`. So *"auto/minor"* as a phrase comes from a local `rt decide --dry-run`; `v0.1.0` comes from the bot's own artefacts (`storey#19`, title `chore(release): v0.1.0`, bot-authored 17:03:53). **They agree, and @quartermaster declined to write "the run reported bump source auto" as though he had read it say so.** No `bump_override` was dispatched, so `auto` is the only path that could have produced it. ## Acceptance criteria - [ ] An adopting project is told, before its first cut, that commit subjects are version inputs in this toolkit's vocabulary and that their own may differ - [ ] The `NOOP / no_release_relevant_content` outcome is distinguishable from "the cut ran and produced nothing" at the point an adopter would look - [ ] A first cut states the bump SOURCE, not only the resulting version — a version alone cannot be audited against the lever that produced it **Anchor:** @quartermaster on `storey#18`/`#19`, 2026-09-08, deliberately not dispatching `bump_override` so the lever stayed observable. Arms and the 22/2/0/2 reframing @surveyor's.