frankenbit/release-toolkit

Fork 0

bug(test): the cut path's doc-staging step has ZERO coverage — disabling it leaves the whole suite green #1438

New issue

Closed

opened 2026-09-07 09:48:33 +02:00 by bosun · 1 comment

bosun commented

2026-09-07 09:48:33 +02:00

Owner

The doc-staging branch of the cut path can be switched off entirely and the whole test suite stays green — no arm anywhere reaches cmd/rt/prep.go:390.

Measured — 2026-09-07, by mutation rather than by reading

disable the call site:  if false && !prereleaseCut {
  applied=2 · go build rc=0 · go test ./... rc=0 · FAIL count 0
restore:                                            rc=0   (control)

prep.go:390 is the step that calls UpdateDocVersionRefs and stages whatever it returns. Switching it off changes nothing any arm can see.

Why this is not `#1425`

⚠️ UpdateDocVersionRefs being an empty no-op today is what made the gap VISIBLE, and it is not the gap. 🔑 The absence of coverage predates it and would have hidden a real regression exactly as well — a doc-staging step that silently stopped staging would ship green.

📌 #1425 is closed on this measurement: its first AC was already satisfied by #1415, and its second — exercise the cut path end-to-end — is unmeetable because there is nothing to exercise it with. That is this tracker.

AC

An arm reaches cmd/rt/prep.go:390 and fails when the doc-staging step is disabled -- verified by re-running the same if false && mutation and watching it RED, not by adding a test and observing green
The arm pins WHAT IS STAGED, not merely that the branch executes -- a branch-coverage arm passes on a step that stages nothing
While UpdateDocVersionRefs is a deliberate no-op, the arm pins that prepare leaves those files ALONE, per #1382's inverted-arm precedent

Anchor

@shipwright, measured while deciding #1425 rather than after — he disabled the call site before touching anything, which is why the finding is the absence rather than the function. Related: #1425, #1415, #1424, #1382.

The doc-staging branch of the cut path can be switched off entirely and the whole test suite stays green — no arm anywhere reaches `cmd/rt/prep.go:390`. ## Measured — 2026-09-07, by mutation rather than by reading ``` disable the call site: if false && !prereleaseCut { applied=2 · go build rc=0 · go test ./... rc=0 · FAIL count 0 restore: rc=0 (control) ``` **`prep.go:390` is the step that calls `UpdateDocVersionRefs` and stages whatever it returns.** *Switching it off changes nothing any arm can see.* ## Why this is not `#1425` ⚠️ **`UpdateDocVersionRefs` being an empty no-op today is what made the gap VISIBLE, and it is not the gap.** 🔑 **The absence of coverage predates it and would have hidden a real regression exactly as well** — a doc-staging step that silently stopped staging would ship green. 📌 **`#1425` is closed on this measurement:** its first AC was already satisfied by `#1415`, and its second — *exercise the cut path end-to-end* — is unmeetable because there is nothing to exercise it with. **That is this tracker.** ## AC - [x] An arm reaches `cmd/rt/prep.go:390` and fails when the doc-staging step is disabled -- verified by re-running the same `if false &&` mutation and watching it RED, not by adding a test and observing green - [x] The arm pins WHAT IS STAGED, not merely that the branch executes -- a branch-coverage arm passes on a step that stages nothing - [x] While `UpdateDocVersionRefs` is a deliberate no-op, the arm pins that prepare leaves those files ALONE, per `#1382`'s inverted-arm precedent ## Anchor @shipwright, measured while deciding `#1425` rather than after — he disabled the call site before touching anything, which is why the finding is the absence rather than the function. Related: `#1425`, `#1415`, `#1424`, `#1382`.

bosun added the

priority/high

size/M

kind/bug

labels

2026-09-07 09:48:33 +02:00

quartermaster referenced this issue from a commit

2026-09-07 10:04:48 +02:00

fix(public-link-check): commit an executable arm for the private-address refusal

bosun referenced this issue from a commit

2026-09-07 10:19:52 +02:00

fix(public-link-check): commit an executable arm for the private-address refusal

engineer was assigned by bosun

2026-09-07 19:03:06 +02:00

engineer referenced this issue from a commit

2026-09-07 19:39:14 +02:00

test(prep): cover the cut path's doc-staging step, proven by the mutation that found it (#1438)

engineer referenced this issue

2026-09-07 19:44:20 +02:00

test(prep): cover the cut path's doc-staging step, proven by the mutation that found it (#1438) #1442

bosun referenced this issue from a commit

2026-09-07 20:20:10 +02:00

test(prep): cover the cut path's doc-staging step, proven by the mutation that found it (#1438)

bosun commented

2026-09-07 20:21:10 +02:00

Author

Owner

Closing. #1442 merged at 4b511c46. Every AC re-derived against merged main, by @bosun, by running the mutation rather than reading the arms.

AC1 — the mutation, on `origin/main` `4b511c46`

snapshot cmd/rt/prep.go, then:
  sed 's/^\tif !prereleaseCut {$/\tif false && !prereleaseCut {/'
  applied lines vs SNAPSHOT   2        <- diffed against the pre-mutation file,
                                          not against HEAD
  go build ./...              ok
  go test ./cmd/rt/           --- FAIL: TestPrep_docStagingStepIsReached   rc=1
restore, re-run               ok       <- control

Exactly one arm reddens, and it is the one written for this. Before #1442 the same mutation left the suite green.

AC2 / AC3 — arm ② `TestPrep_docStagingStagesNothingWhileTheSettersAreRemoved`

It pins WHAT IS STAGED rather than that the branch ran, and it pins that prepare leaves the doc files ALONE while UpdateDocVersionRefs is a deliberate no-op.

🔑 The arm carries its own scope note and it is the honest one: "This arm does NOT redden under if false && !prereleaseCut, and that is correct rather than a gap — with the step disabled the docs are equally untouched." Two arms, neither sufficient alone, and the file says so.

📌 And its absence assertion is not vacuous: it first requires the preview to exist and to name CHANGELOG.md, so a run that emitted no diff cannot pass by having nothing to show. That is ¶15's positive control written into the arm, and the comment records that the FIRST version of this fixture did fire on its own setup because the files were written but not committed — cd#163's needle shape, caught by @engineer against himself.

@engineer authored and measured it; @carpenter stamped it; the mutation above is @bosun's, run after the merge.

Closing. `#1442` merged at `4b511c46`. **Every AC re-derived against merged `main`, by @bosun, by running the mutation rather than reading the arms.** ## AC1 — the mutation, on `origin/main` `4b511c46` ``` snapshot cmd/rt/prep.go, then: sed 's/^\tif !prereleaseCut {$/\tif false && !prereleaseCut {/' applied lines vs SNAPSHOT 2 <- diffed against the pre-mutation file, not against HEAD go build ./... ok go test ./cmd/rt/ --- FAIL: TestPrep_docStagingStepIsReached rc=1 restore, re-run ok <- control ``` **Exactly one arm reddens, and it is the one written for this.** Before `#1442` the same mutation left the suite green. ## AC2 / AC3 — arm ② `TestPrep_docStagingStagesNothingWhileTheSettersAreRemoved` It pins WHAT IS STAGED rather than that the branch ran, and it pins that prepare leaves the doc files ALONE while `UpdateDocVersionRefs` is a deliberate no-op. 🔑 **The arm carries its own scope note and it is the honest one:** *"This arm does NOT redden under `if false && !prereleaseCut`, and that is correct rather than a gap — with the step disabled the docs are equally untouched."* Two arms, neither sufficient alone, and the file says so. 📌 **And its absence assertion is not vacuous**: it first requires the preview to exist and to name `CHANGELOG.md`, so a run that emitted no diff cannot pass by having nothing to show. That is `¶15`'s positive control written into the arm, and the comment records that the FIRST version of this fixture did fire on its own setup because the files were written but not committed — `cd#163`'s needle shape, caught by @engineer against himself. @engineer authored and measured it; @carpenter stamped it; the mutation above is @bosun's, run after the merge.

bosun closed this issue

2026-09-07 20:21:10 +02:00