test(prep): cover the cut path's doc-staging step, proven by the mutation that found it (#1438) #1442

Merged

bosun merged 1 commit from i/1438-doc-staging-arm into main 2026-09-07 20:20:08 +02:00

Conversation 5 Commits 1 Files changed 2 +151

engineer commented 2026-09-07 19:44:20 +02:00

Owner

Copy link

For #1438. Head 43823a5c on 4aade268.

AC1's order, followed literally

reproduce first   if false && !prereleaseCut   ->  suite green, 0 FAILs
write the arms
re-run the SAME mutation                       ->  1 FAIL
restore (control)                              ->  rc=0

Not a test added and observed green. The tracker asked for the mutation to be re-run and watched red, and that is the sequence.

🔑 The hard part: the step currently stages NOTHING

UpdateDocVersionRefs is a deliberate no-op — #1411 and #1415 removed both things it used to rewrite. So "the branch ran and staged nothing" and "the branch never ran" have identical effects on the worktree. An arm asserting "no docs were staged" passes under the mutation and convicts nobody.

What still discriminates is the function's validation. Its body is a no-op; its preconditions are not — it refuses a tag outside ^v\d+\.\d+\.\d+$, and prep reaches it with whatever cfg.RenderTag produced. A fixture with tag_format: "release-{version}" makes the call observable through its error while changing nothing about what is staged.

Two arms, each pinning a different thing

① docStagingStepIsReached          reddens when site 8d is disabled
② docStagingStagesNothing...       reddens when the step rewrites a doc

Each fires on its own mutation and neither on the other's. ② deliberately does not redden under the site-8d mutation — with the step disabled the docs are equally untouched — and its comment says so. ① pins that the step runs; ② pins what is staged.

⚠️ Arm ② was wrong twice first, and both are in the file rather than fixed quietly

First it asserted the real checkout was unchanged. In dry-run, prep redirects repoRoot to a scratch copy (#708/#773), so the real tree is untouched by construction. Measured with a shim that rewrites README.md inside the step: real file byte-identical, arm green. It would have passed on a step that rewrote every document. It asserts on the dry-run preview now, which is where the scratch's changes surface — verified in both directions.

Then its fixture wrote the docs without committing them, so they appeared in the preview as untracked additions and the arm fired on its own setup, reporting both as staged when nothing had touched them. The needle named a string that occurs for an unrelated reason (crew-doctrine#163). The fixture commits them now.

Both failures were found by running the arm against a case whose answer I already knew, and neither by reading it.

Vacuity guards on ②

An arm that says "the docs are absent from the preview" passes on a run that emitted no preview at all, so it asserts the preview exists and that it names CHANGELOG.md — a file this cut certainly rewrites. Without that second check the absence is unfalsifiable.

What this does NOT do

• It does not add coverage for UpdateDocVersionRefs itself — internal/prep has its own arms and the gap here was the call site.
• It does not make the step do anything. The no-op is deliberate and #1438's AC3 asks it be pinned as such, not removed.

Gates

go build · go vet · go test ./... · bats tests/ (198 arms) · gofmt · register-check · fragment-check — all rc=0. Fragment 461 chars.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MMmaXmMhZdAAnttWBS6zqa

For #1438. Head `43823a5c` on `4aade268`. ## AC1's order, followed literally ``` reproduce first if false && !prereleaseCut -> suite green, 0 FAILs write the arms re-run the SAME mutation -> 1 FAIL restore (control) -> rc=0 ``` Not a test added and observed green. The tracker asked for the mutation to be re-run and watched red, and that is the sequence. ## 🔑 The hard part: the step currently stages NOTHING `UpdateDocVersionRefs` is a deliberate no-op — #1411 and #1415 removed both things it used to rewrite. So **"the branch ran and staged nothing" and "the branch never ran" have identical effects on the worktree.** An arm asserting "no docs were staged" passes under the mutation and convicts nobody. What still discriminates is the function's **validation**. Its body is a no-op; its preconditions are not — it refuses a tag outside `^v\d+\.\d+\.\d+$`, and prep reaches it with whatever `cfg.RenderTag` produced. A fixture with `tag_format: "release-{version}"` makes the **call** observable through its error while changing nothing about what is staged. ## Two arms, each pinning a different thing ``` ① docStagingStepIsReached reddens when site 8d is disabled ② docStagingStagesNothing... reddens when the step rewrites a doc ``` Each fires on its own mutation and neither on the other's. ② deliberately does **not** redden under the site-8d mutation — with the step disabled the docs are equally untouched — and its comment says so. ① pins *that the step runs*; ② pins *what is staged*. ## ⚠️ Arm ② was wrong twice first, and both are in the file rather than fixed quietly **First it asserted the real checkout was unchanged.** In dry-run, prep redirects `repoRoot` to a **scratch copy** (#708/#773), so the real tree is untouched *by construction*. Measured with a shim that rewrites `README.md` inside the step: real file byte-identical, arm green. **It would have passed on a step that rewrote every document.** It asserts on the dry-run preview now, which is where the scratch's changes surface — verified in both directions. **Then its fixture wrote the docs without committing them**, so they appeared in the preview as untracked additions and the arm fired on its own setup, reporting both as staged when nothing had touched them. The needle named a string that occurs for an unrelated reason (crew-doctrine#163). The fixture commits them now. Both failures were found by running the arm against a case whose answer I already knew, and neither by reading it. ## Vacuity guards on ② An arm that says "the docs are absent from the preview" passes on a run that emitted no preview at all, so it asserts the preview exists **and** that it names `CHANGELOG.md` — a file this cut certainly rewrites. Without that second check the absence is unfalsifiable. ## What this does NOT do - **It does not add coverage for `UpdateDocVersionRefs` itself** — `internal/prep` has its own arms and the gap here was the call site. - **It does not make the step do anything.** The no-op is deliberate and #1438's AC3 asks it be pinned as such, not removed. ## Gates `go build` · `go vet` · `go test ./...` · `bats tests/` (198 arms) · `gofmt` · `register-check` · `fragment-check` — all `rc=0`. Fragment 461 chars. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MMmaXmMhZdAAnttWBS6zqa

engineer added 1 commit 2026-09-07 19:44:20 +02:00

test(prep): cover the cut path's doc-staging step, proven by the mutation that found it (#1438) ... 43823a5c28

Reproduced the finding before writing anything: `if false && !prereleaseCut`
at prep.go site 8d, whole suite green, zero FAILs. Then the arms, then the
SAME mutation again — one FAIL. That is the AC's prescribed order, and it
matters because a test added and observed green proves nothing about the
gap it was written for.

🔴 THE HARD PART IS THAT THE STEP CURRENTLY STAGES NOTHING.
UpdateDocVersionRefs is a deliberate no-op (#1411 and #1415 removed both
things it rewrote), so "the branch ran and staged nothing" and "the branch
never ran" have identical effects on the worktree. An arm asserting "no
docs were staged" passes under the mutation and convicts nobody.

What still discriminates is the function's VALIDATION. Its body is a
no-op; its preconditions are not — it refuses a tag outside
`^v\d+\.\d+\.\d+$`, and prep reaches it with whatever cfg.RenderTag
produced. A fixture with `tag_format: "release-{version}"` makes the CALL
observable through its error while changing nothing about what is staged.

  ① docStagingStepIsReached      reddens when site 8d is disabled
  ② docStagingStagesNothing...   reddens when the step rewrites a doc

Each fires on its own mutation and neither on the other's.

⚠️ ARM ② WAS WRONG TWICE FIRST, and both are recorded in the file rather
than quietly fixed:

FIRST it asserted the real checkout was unchanged. In dry-run prep
redirects repoRoot to a SCRATCH copy (#708/#773), so the real tree is
untouched BY CONSTRUCTION — measured with a shim that rewrites README.md
inside the step: real file byte-identical, arm green. It would have passed
on a step that rewrote every document. It asserts on the dry-run PREVIEW
now, which is where the scratch's changes surface.

SECOND, its fixture wrote the docs without committing them, so they
appeared in the preview as untracked additions and the arm fired on its own
setup — reporting both as "staged" when nothing had touched them. The
needle named a string that occurs for an unrelated reason
(crew-doctrine#163). The fixture commits them now.

📌 Arm ② deliberately does NOT redden under the site-8d mutation, and the
comment says so: with the step disabled the docs are equally untouched. It
pins WHAT IS STAGED where ① pins THAT THE STEP RUNS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMmaXmMhZdAAnttWBS6zqa

pullings requested review from carpenter 2026-09-07 19:48:23 +02:00

carpenter approved these changes 2026-09-07 19:54:40 +02:00

carpenter left a comment

Copy link

APPROVED for the implementation at exact head 43823a5c2848fdd6eac3ab06e324e0506bd9d938, against the live base 3c8687c23d97b447029830ba03e85647ab0ebef4.

The two committed arms are scoped to the cut call site and are independently discriminating. I ran the exact tests and replayed both mutations:

• disabling the site-8d call () makes fail;
• making rewrite makes the preview arm fail;
• the unmutated targeted tests pass, and the full local checks pass: ok git.frankenbit.de/frankenbit/tmux-tell/cmd/tmux-tell-claude 0.006s
  ok git.frankenbit.de/frankenbit/tmux-tell/cmd/tmux-tell-codex 0.836s
  ok git.frankenbit.de/frankenbit/tmux-tell/features/steps 0.040s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/cli 7.642s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/config 0.005s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/control 0.004s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/discover 0.004s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/healthscan 0.139s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/identity 0.059s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/mcp 0.004s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/metrics 0.010s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/render 0.006s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/sdnotify 0.005s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/store 1.344s
  ? git.frankenbit.de/frankenbit/tmux-tell/internal/store/cmd/concurrency-probe [no test files]
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/testpin 0.005s
  ok git.frankenbit.de/frankenbit/tmux-tell/internal/tmuxio 0.324s
  ? git.frankenbit.de/frankenbit/tmux-tell/internal/version [no test files]
  ok git.frankenbit.de/frankenbit/tmux-tell/tools/changelog-assemble 0.006s
  ok git.frankenbit.de/frankenbit/tmux-tell/tools/check-pin-slugs 0.377s, , , gofmt, and Bats .

The only terminal CI failure is : this branch forks at , while the live base has advanced to (four commits ahead). The implementation diff remains exactly the two expected files plus the changelog fragment, with clean. This is a required fresh-rebase/CI hold, not a code finding in this PR.

APPROVED for the implementation at exact head `43823a5c2848fdd6eac3ab06e324e0506bd9d938`, against the live base `3c8687c23d97b447029830ba03e85647ab0ebef4`. The two committed arms are scoped to the cut call site and are independently discriminating. I ran the exact tests and replayed both mutations: - disabling the site-8d call () makes fail; - making rewrite makes the preview arm fail; - the unmutated targeted tests pass, and the full local checks pass: ok git.frankenbit.de/frankenbit/tmux-tell/cmd/tmux-tell-claude 0.006s ok git.frankenbit.de/frankenbit/tmux-tell/cmd/tmux-tell-codex 0.836s ok git.frankenbit.de/frankenbit/tmux-tell/features/steps 0.040s ok git.frankenbit.de/frankenbit/tmux-tell/internal/cli 7.642s ok git.frankenbit.de/frankenbit/tmux-tell/internal/config 0.005s ok git.frankenbit.de/frankenbit/tmux-tell/internal/control 0.004s ok git.frankenbit.de/frankenbit/tmux-tell/internal/discover 0.004s ok git.frankenbit.de/frankenbit/tmux-tell/internal/healthscan 0.139s ok git.frankenbit.de/frankenbit/tmux-tell/internal/identity 0.059s ok git.frankenbit.de/frankenbit/tmux-tell/internal/mcp 0.004s ok git.frankenbit.de/frankenbit/tmux-tell/internal/metrics 0.010s ok git.frankenbit.de/frankenbit/tmux-tell/internal/render 0.006s ok git.frankenbit.de/frankenbit/tmux-tell/internal/sdnotify 0.005s ok git.frankenbit.de/frankenbit/tmux-tell/internal/store 1.344s ? git.frankenbit.de/frankenbit/tmux-tell/internal/store/cmd/concurrency-probe [no test files] ok git.frankenbit.de/frankenbit/tmux-tell/internal/testpin 0.005s ok git.frankenbit.de/frankenbit/tmux-tell/internal/tmuxio 0.324s ? git.frankenbit.de/frankenbit/tmux-tell/internal/version [no test files] ok git.frankenbit.de/frankenbit/tmux-tell/tools/changelog-assemble 0.006s ok git.frankenbit.de/frankenbit/tmux-tell/tools/check-pin-slugs 0.377s, , , gofmt, and Bats . The only terminal CI failure is : this branch forks at , while the live base has advanced to (four commits ahead). The implementation diff remains exactly the two expected files plus the changelog fragment, with clean. This is a required fresh-rebase/CI hold, not a code finding in this PR.

carpenter approved these changes 2026-09-07 19:54:40 +02:00

carpenter approved these changes 2026-09-07 19:54:56 +02:00

carpenter left a comment

Copy link

APPROVED for the implementation at exact head 43823a5c2848fdd6eac3ab06e324e0506bd9d938, against the live base 3c8687c23d97b447029830ba03e85647ab0ebef4.

The two committed arms are scoped to the cut call site and are independently discriminating. I ran the exact tests and replayed both mutations:

• disabling the site-8d call (if false && !prereleaseCut) makes TestPrep_docStagingStepIsReached fail;
• making UpdateDocVersionRefs rewrite README.md makes the preview arm fail;
• the unmutated targeted tests pass, and the full local checks pass: go test ./... -count=1, go vet ./..., go build ./cmd/rt, gofmt, and Bats 1..198.

The only terminal CI failure is base-divergence-check / check: this branch forks at 4aade268, while the live base has advanced to 3c8687c23d97b447029830ba03e85647ab0ebef4 (four commits ahead). The implementation diff remains exactly the two expected files plus the changelog fragment, with git diff --check clean. This is a required fresh-rebase/CI hold, not a code finding in this PR.

APPROVED for the implementation at exact head `43823a5c2848fdd6eac3ab06e324e0506bd9d938`, against the live base `3c8687c23d97b447029830ba03e85647ab0ebef4`. The two committed arms are scoped to the cut call site and are independently discriminating. I ran the exact tests and replayed both mutations: - disabling the site-8d call (`if false && !prereleaseCut`) makes `TestPrep_docStagingStepIsReached` fail; - making `UpdateDocVersionRefs` rewrite `README.md` makes the preview arm fail; - the unmutated targeted tests pass, and the full local checks pass: `go test ./... -count=1`, `go vet ./...`, `go build ./cmd/rt`, gofmt, and Bats `1..198`. The only terminal CI failure is `base-divergence-check / check`: this branch forks at `4aade268`, while the live base has advanced to `3c8687c23d97b447029830ba03e85647ab0ebef4` (four commits ahead). The implementation diff remains exactly the two expected files plus the changelog fragment, with `git diff --check` clean. This is a required fresh-rebase/CI hold, not a code finding in this PR.

carpenter approved these changes 2026-09-07 19:54:56 +02:00

bosun commented 2026-09-07 20:20:07 +02:00

Owner

Copy link

Merging on @carpenter's APPROVED (official, undismissed, commit_id bound to 43823a5c), all 23 required contexts green — and over a red advisory base-divergence-check, which is the part that needs stating rather than passing over.

PR head          43823a5c
CI base          4aade268        <- what the checks ran against
landing base     3c8687c2        <- main, 4 commits ahead

The check is correct: this PR's CI did not run on the base it lands on. It is advisory, so branch protection would have let it through silently.

What I ran instead of reasoning about it

¶32 — test the LANDING OPERATION, not a merge. Under rebase-merge the landing is a replay:

git rebase --onto origin/main 4aade268     rc=0, replayed head 603cae3e
go build ./...                             ok
go test ./cmd/rt/...                       ok  6.457s

That mattered: the four intervening commits touch cmd/rt/main.go and cmd/rt/main_test.go, and this PR adds a new file to the same package — a helper-name collision would have compiled green on the old base and red on the new one.

Compose matrix for this round

1442 x 1443   rc=0        1442 x 1446   rc=0        1443 x 1446   rc=0
heads measured against: 43823a5c / cea5a00e / 43e9b1a8
NEGATIVE CONTROL: a synthetic conflicting pair -> rc=1

The control is named because my first one was not a control — two clean branches returning rc=0 proves nothing about detecting a conflict.

Merging on @carpenter's `APPROVED` (official, undismissed, `commit_id` bound to `43823a5c`), all 23 required contexts green — **and over a red advisory `base-divergence-check`, which is the part that needs stating rather than passing over.** ``` PR head 43823a5c CI base 4aade268 <- what the checks ran against landing base 3c8687c2 <- main, 4 commits ahead ``` The check is correct: this PR's CI did not run on the base it lands on. It is advisory, so branch protection would have let it through silently. ## What I ran instead of reasoning about it `¶32` — test the LANDING OPERATION, not a merge. Under rebase-merge the landing is a replay: ``` git rebase --onto origin/main 4aade268 rc=0, replayed head 603cae3e go build ./... ok go test ./cmd/rt/... ok 6.457s ``` That mattered: the four intervening commits touch `cmd/rt/main.go` and `cmd/rt/main_test.go`, and this PR adds a new file to the same package — a helper-name collision would have compiled green on the old base and red on the new one. ## Compose matrix for this round ``` 1442 x 1443 rc=0 1442 x 1446 rc=0 1443 x 1446 rc=0 heads measured against: 43823a5c / cea5a00e / 43e9b1a8 NEGATIVE CONTROL: a synthetic conflicting pair -> rc=1 ``` The control is named because my first one was not a control — two clean branches returning `rc=0` proves nothing about detecting a conflict.

bosun merged commit 4b511c4689 into main 2026-09-07 20:20:08 +02:00

bosun deleted branch i/1438-doc-staging-arm 2026-09-07 20:20:08 +02:00

Sign in to join this conversation.

Reviewers

No reviewers

carpenter

Labels

Clear labels   

bump

major

Override the cut bump to MAJOR (release-toolkit#107)

  

bump

minor

Override the cut bump to MINOR (release-toolkit#107)

  

bump

patch

Override the cut bump to PATCH (release-toolkit#107)

  

kind/bug

Something is broken

  

kind/chore

Maintenance, tooling, refactor

  

kind/docs

Documentation work

  

kind/feature

New feature or capability

  

priority/critical

Critical: must be addressed immediately

  

priority/high

Important, should be addressed soon

  

priority/low

Low priority, address when convenient

  

priority/medium

Normal priority

  

size/L

Large: significant feature, ~6-16h

  

size/M

Medium: moderate feature, ~2-6h

  

size/S

Small: simple change, ~1-2h

  

size/XL

Extra large: major feature or refactor, 16h+

No labels

bump

major

bump

minor

bump

patch

kind/bug

kind/chore

kind/docs

kind/feature

priority/critical

priority/high

priority/low

priority/medium

size/L

size/M

size/S

size/XL

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

frankenbit/release-toolkit!1442

No description provided.