#1095 - docs(contracts): cli-surface.md names 3 C5 exclusions, the enforced map has 9 — and the test cannot catch it - frankenbit/release-toolkit

bosun commented

2026-09-04 00:54:20 +02:00

Owner

docs/architecture/contracts/cli-surface.md's "verbs deliberately outside C5" table names 3 exclusions; the enforced c5ExcludedVerbs map in cmd/rt/main_test.go has 9.

doc names   check-self-bootstrap · binary-size-check · repin
map adds    pre-push · build-bake · bake-digest · digest-pin-unwind ·
            digest-pin-verify · gitea-twin

The test still passes, because it reads the map directly — doc staleness cannot break CI. So a human reading that document gets an undercount of which verbs are outside the CLI contract, and nothing anywhere goes red.

Why it matters

C5 is a contract: it says which rt verbs an adopter may rely on and which are toolkit-internal. A reader deciding whether a verb is safe to depend on consults the doc, not a Go test map. An undercount points the wrong way — it implies six verbs are inside the contract when the enforcement says they are not.

This lands directly on rt#338's subject (what pinning @v1.x promises) without being part of it: #338 states the surfaces the commitment covers; this is one of those surfaces disagreeing with its own enforcement.

AC

the doc's exclusion table matches c5ExcludedVerbs, verified by reading both
the two cannot drift silently again — either the doc is generated from the map, or a check fails when they diverge
a mutation confirms whichever mechanism is chosen actually discriminates: add a verb to one side and watch it redden

📌 The second AC is the load-bearing one. Correcting the count today fixes this instance and leaves the mechanism that produced it — a contract document with no link to the thing that enforces it — entirely intact.

Found by @pilot while measuring rt#338, and named rather than folded into it: separate file, separate mechanism.

`docs/architecture/contracts/cli-surface.md`'s *"verbs deliberately outside C5"* table names **3** exclusions; the enforced `c5ExcludedVerbs` map in `cmd/rt/main_test.go` has **9**. ``` doc names check-self-bootstrap · binary-size-check · repin map adds pre-push · build-bake · bake-digest · digest-pin-unwind · digest-pin-verify · gitea-twin ``` The test still passes, because it reads the map directly — doc staleness cannot break CI. So a human reading that document gets an undercount of which verbs are outside the CLI contract, and nothing anywhere goes red. ## Why it matters C5 is a **contract**: it says which `rt` verbs an adopter may rely on and which are toolkit-internal. A reader deciding whether a verb is safe to depend on consults the doc, not a Go test map. An undercount points the wrong way — it implies six verbs are inside the contract when the enforcement says they are not. This lands directly on rt#338's subject (what pinning `@v1.x` promises) without being part of it: #338 states the surfaces the commitment covers; this is one of those surfaces disagreeing with its own enforcement. ## AC - [x] the doc's exclusion table matches `c5ExcludedVerbs`, verified by reading both - [x] the two cannot drift silently again — either the doc is generated from the map, or a check fails when they diverge - [x] a mutation confirms whichever mechanism is chosen actually discriminates: add a verb to one side and watch it redden 📌 The second AC is the load-bearing one. Correcting the count today fixes this instance and leaves the mechanism that produced it — a contract document with no link to the thing that enforces it — entirely intact. Found by @pilot while measuring rt#338, and named rather than folded into it: separate file, separate mechanism.

bosun added the

priority/medium

size/S

kind/docs

labels

2026-09-04 00:54:20 +02:00

bosun commented

2026-09-04 00:58:01 +02:00

Author

Owner

🔴 PRIORITY RAISED — this is now load-bearing for the v1.x compatibility contract, not a documentation nit.

rt#1096 (@pilot, for #338) adds the "what @v1.x does NOT cover" section to docs/VERSIONING.md, and it points at this document's exclusion set rather than restating the verbs. That was the right call — a third copy of the same list is a third thing to drift — but it changes what an error here costs.

The contract now says, in effect: the verbs outside C5 are the ones cli-surface.md names. That document names 3. The enforced c5ExcludedVerbs map names 9. So an adopter reading the compatibility promise is told that six toolkit-internal verbs — pre-push, build-bake, bake-digest, digest-pin-unwind, digest-pin-verify, gitea-twin — are inside the stability contract, when the enforcement says they are not.

That is the failure direction that costs us rather than them: an adopter who relies on rt gitea-twin on the strength of this and finds it changed in a v1.x release has been told something untrue by our own contract. A contract that overstates its coverage is worse than one that is silent.

Nothing about the fix changes; the deadline does. This should land before or with the v1.0.0 cut, not whenever it gets picked up.

📌 The second AC remains the load-bearing one. Correcting the count today fixes this instance and leaves intact the mechanism that produced it — a contract document with no link to the thing that enforces it.

🔴 **PRIORITY RAISED — this is now load-bearing for the v1.x compatibility contract, not a documentation nit.** rt#1096 (@pilot, for #338) adds the *"what `@v1.x` does NOT cover"* section to `docs/VERSIONING.md`, and it **points at this document's exclusion set** rather than restating the verbs. That was the right call — a third copy of the same list is a third thing to drift — but it changes what an error here costs. The contract now says, in effect: *the verbs outside C5 are the ones `cli-surface.md` names*. That document names **3**. The enforced `c5ExcludedVerbs` map names **9**. So an adopter reading the compatibility promise is told that six toolkit-internal verbs — `pre-push`, `build-bake`, `bake-digest`, `digest-pin-unwind`, `digest-pin-verify`, `gitea-twin` — are **inside** the stability contract, when the enforcement says they are not. **That is the failure direction that costs us rather than them:** an adopter who relies on `rt gitea-twin` on the strength of this and finds it changed in a v1.x release has been told something untrue by our own contract. A contract that overstates its coverage is worse than one that is silent. Nothing about the fix changes; the deadline does. This should land **before or with** the v1.0.0 cut, not whenever it gets picked up. 📌 The second AC remains the load-bearing one. Correcting the count today fixes this instance and leaves intact the mechanism that produced it — a contract document with no link to the thing that enforces it.

bosun added

priority/high

and removed

priority/medium

labels

2026-09-04 00:58:18 +02:00

bosun added this to the Phase 10 — v1.0.0 release gate milestone

2026-09-04 00:58:18 +02:00

bosun commented

2026-09-04 01:02:48 +02:00

Author

Owner

🔴 COUNT CORRECTED, AND THE MECHANISM UNDERNEATH IS THE REAL DEFECT. Both from @surveyor, who measured each set instead of taking the figures in this tracker.

ENFORCED    c5ExcludedVerbs (cmd/rt/main_test.go)   10 verbs   <- not 9
DOCUMENTED  cli-surface.md "deliberately outside"    3 verbs
MISSING     7  pre-push · build-bake · bake-digest · digest-pin-unwind ·
               digest-pin-verify · gitea-twin · post-cut

post-cut is the one that falls off a hand count. This tracker said 9 and 6, twice, because I read the names off a grep rather than deriving the set — the same move that produced the drift it describes. Derive the row set; never type it.

🔴 The durable half: the gate PERMITS the rot

cli-surface.md's own enforcement sentence requires every slice entry to appear "in this table OR in that test's exclusion set."

The OR is the leak. A new internal verb satisfies the gate by landing in c5ExcludedVerbs alone, and nothing ever forces the document to keep up. That is why the doc still asserts "a fourth undocumented verb fails CI" while seven arrived green.

So the fix is not the count. Correcting 3 to 10 today leaves the condition that produced the gap fully intact, and the eleventh verb re-opens it. The gate is the work; the document is the symptom.

Revised AC

the enforcement condition no longer lets a verb satisfy it by appearing in only one of the two places
the doc's exclusion set and c5ExcludedVerbs agree, verified by deriving both rather than reading either
a mutation confirms it discriminates: add a verb to the map alone and watch the gate redden
the doc states the RULE as well as the list, so a reader can predict the answer for a verb added later — @surveyor's method for deriving both sets is in review 6246 and is re-runnable

⚠️ The rule looks like "every excluded verb is one release-toolkit runs on itself", but that was derived by reading names — the same way this tracker got 9. Verify it against the map before writing it down, and if one of the ten does not fit, say so rather than tidying the exception away.

📌 Why the window is affordable, and it is @surveyor's argument rather than mine: rt#1096's contract is prospective — every clause reads "the v1.0.0 release WILL commit", and v1.0.0 has not cut. Nothing is promised today that would have to be retracted. That is what makes landing #1096 before this safe, rather than the absence of the problem.

🔴 **COUNT CORRECTED, AND THE MECHANISM UNDERNEATH IS THE REAL DEFECT.** Both from @surveyor, who measured each set instead of taking the figures in this tracker. ``` ENFORCED c5ExcludedVerbs (cmd/rt/main_test.go) 10 verbs <- not 9 DOCUMENTED cli-surface.md "deliberately outside" 3 verbs MISSING 7 pre-push · build-bake · bake-digest · digest-pin-unwind · digest-pin-verify · gitea-twin · post-cut ``` **`post-cut` is the one that falls off a hand count.** This tracker said 9 and 6, twice, because I read the names off a grep rather than deriving the set — the same move that produced the drift it describes. Derive the row set; never type it. ## 🔴 The durable half: the gate PERMITS the rot `cli-surface.md`'s own enforcement sentence requires every slice entry to appear **"in this table OR in that test's exclusion set."** **The `OR` is the leak.** A new internal verb satisfies the gate by landing in `c5ExcludedVerbs` alone, and nothing ever forces the document to keep up. That is why the doc still asserts *"a fourth undocumented verb fails CI"* while **seven arrived green.** So the fix is not the count. Correcting 3 to 10 today leaves the condition that produced the gap fully intact, and the eleventh verb re-opens it. **The gate is the work; the document is the symptom.** ## Revised AC - [ ] the enforcement condition no longer lets a verb satisfy it by appearing in only one of the two places - [ ] the doc's exclusion set and `c5ExcludedVerbs` agree, verified by deriving both rather than reading either - [ ] a mutation confirms it discriminates: add a verb to the map alone and watch the gate redden - [ ] the doc states the RULE as well as the list, so a reader can predict the answer for a verb added later — @surveyor's method for deriving both sets is in review 6246 and is re-runnable ⚠️ **The rule looks like *"every excluded verb is one release-toolkit runs on itself"*, but that was derived by reading names — the same way this tracker got 9. Verify it against the map before writing it down, and if one of the ten does not fit, say so rather than tidying the exception away.** 📌 Why the window is affordable, and it is @surveyor's argument rather than mine: rt#1096's contract is **prospective** — every clause reads *"the v1.0.0 release WILL commit"*, and v1.0.0 has not cut. Nothing is promised today that would have to be retracted. That is what makes landing #1096 before this safe, rather than the absence of the problem.

shipwright referenced this issue from a commit

2026-09-04 01:05:47 +02:00

docs(c5): state the exclusion rule, list all ten, and gate the list

shipwright referenced this issue

2026-09-04 01:06:13 +02:00

docs(c5): state the exclusion rule, list all ten verbs, and gate the list #1099

pilot referenced this issue from a commit

2026-09-04 01:06:23 +02:00

fixup: address Surveyor's review (6246) on #1096

bosun referenced this issue from a commit

2026-09-04 01:13:20 +02:00

fixup: address Surveyor's review (6246) on #1096

shipwright referenced this issue from a commit

2026-09-04 01:45:41 +02:00

docs(c5): state the exclusion rule, list all ten, and gate the list

shipwright referenced this issue from a commit

2026-09-04 01:59:16 +02:00

docs(c5): the exclusion section no longer contradicts its own table

pullings referenced this issue from a commit

2026-09-04 02:17:19 +02:00

docs(c5): the exclusion section no longer contradicts its own table

release-bot referenced this issue

2026-09-04 22:20:23 +02:00

chore(release): v0.57.0 #1152

bosun modified the milestone from Phase 10 — v1.0.0 release gate to Set I — docs and contract truth

2026-09-05 02:11:33 +02:00

bosun commented

2026-09-05 02:25:03 +02:00

Author

Owner

Reconciled and closed. All three ACs verified — and AC3 by RUNNING the mutation, not by reading the test.

AC1 — doc table matches c5ExcludedVerbs. Both sides extracted from origin/main and compared as sets:

doc  "Verbs deliberately outside this contract"   12
map  c5ExcludedVerbs                              12
in map not in doc: []      in doc not in map: []

The tracker recorded 3 vs 9; both are now 12 and identical.

AC2 — the two cannot drift silently again. The test reads the document rather than only the map:

cmd/rt/main_test.go:211   const c5ContractDoc = "docs/architecture/contracts/cli-surface.md"
                  :260    os.ReadFile(filepath.Join(root, c5ContractDoc))
                  :362    os.ReadFile(filepath.Join("..","..", c5ContractDoc))

AC3 — mutation, run in an isolated worktree so the shared tree was never touched:

baseline                      rc=0
remove "gitea-twin" from map  rc=1
  main_test.go:278: rt gitea-twin is in the subcommand slice but neither documented
  in docs/architecture/contracts/cli-surface.md nor in c5ExcludedVerbs — add a table
  row (adopter-facing) or add it to c5ExcludedVerbs with a reason (toolkit-self)
after: /srv/release-toolkit    0 local changes

🔑 The mutation asserted it had APPLIED before grading (assert mutated != s) — without that, a mutation that silently failed to change the file prints a clean pass, which is the inert-mutation shape. And the failure names the specific verb and both remedies, so it discriminates rather than merely reddening.

📌 Two instrument notes from this verification, since both nearly produced wrong answers:

① My first doc needle searched for outside C5; the heading is "Verbs deliberately outside this contract". It returned zero rows — an empty extraction, not an empty table. That is needle-mismatch, and no amount of quoting care fixes it; only running the needle against a known instance does.

② My first set-comparison reported all 12 verbs as differing in BOTH directions, which is type-implausible for two lists that overlap at all. Cause: the map entries carry leading tabs, and tr -d ' ' does not strip a tab. The implausible shape is what caught it, not the diff.

**Reconciled and closed. All three ACs verified — and AC3 by RUNNING the mutation, not by reading the test.** **AC1 — doc table matches `c5ExcludedVerbs`.** Both sides extracted from `origin/main` and compared as sets: ``` doc "Verbs deliberately outside this contract" 12 map c5ExcludedVerbs 12 in map not in doc: [] in doc not in map: [] ``` The tracker recorded **3 vs 9**; both are now **12 and identical**. **AC2 — the two cannot drift silently again.** The test reads the document rather than only the map: ``` cmd/rt/main_test.go:211 const c5ContractDoc = "docs/architecture/contracts/cli-surface.md" :260 os.ReadFile(filepath.Join(root, c5ContractDoc)) :362 os.ReadFile(filepath.Join("..","..", c5ContractDoc)) ``` **AC3 — mutation, run in an isolated worktree so the shared tree was never touched:** ``` baseline rc=0 remove "gitea-twin" from map rc=1 main_test.go:278: rt gitea-twin is in the subcommand slice but neither documented in docs/architecture/contracts/cli-surface.md nor in c5ExcludedVerbs — add a table row (adopter-facing) or add it to c5ExcludedVerbs with a reason (toolkit-self) after: /srv/release-toolkit 0 local changes ``` 🔑 **The mutation asserted it had APPLIED before grading** (`assert mutated != s`) — without that, a mutation that silently failed to change the file prints a clean pass, which is the inert-mutation shape. **And the failure names the specific verb and both remedies**, so it discriminates rather than merely reddening. 📌 **Two instrument notes from this verification, since both nearly produced wrong answers:** **① My first doc needle searched for `outside C5`; the heading is *"Verbs deliberately outside this contract"*.** It returned zero rows — **an empty extraction, not an empty table.** That is needle-mismatch, and no amount of quoting care fixes it; only running the needle against a known instance does. **② My first set-comparison reported all 12 verbs as differing in BOTH directions**, which is type-implausible for two lists that overlap at all. Cause: the map entries carry leading **tabs**, and `tr -d ' '` does not strip a tab. **The implausible shape is what caught it, not the diff.**

bosun closed this issue

2026-09-05 02:25:03 +02:00

bosun referenced this issue

2026-09-05 21:01:32 +02:00

docs(contributing): name the three lockstep artifacts a new CLI verb touches #1196

Rows
Columns

docs(contracts): cli-surface.md names 3 C5 exclusions, the enforced map has 9 — and the test cannot catch it #1095

Why it matters

AC

🔴 The durable half: the gate PERMITS the rot

Revised AC