test(manifest-check): guard the Go §4 prerelease fix directly (#476 follow-up) #615

Merged

bosun merged 1 commit from i/476-manifest-check-go-direct-test into main 2026-07-30 19:30:54 +02:00

Conversation 1 Commits 1 Files changed 1 +42

engineer commented 2026-07-30 19:25:33 +02:00

Owner

Copy link

Follow-up to PR#614 (merged 234bf237) per Surveyor's non-blocking should-consider, review 4676. Filed as a fresh PR rather than an amend — #614 merged 7s before my hold request landed, so pushing into that branch would have been a push onto a corpse.

42 lines, one test file, no production change.

The gap

#476 site 4b's §4 prerelease fix landed on both sides. Only bash had a direct guard (tests/manifest-check.bats).

The equivalence differential compares the two implementations to each other, not to the truth — so it is structurally blind to a regression that lands on both at once. Measured on this branch by reverting the §4 fix three ways:

revert	equivalence	direct Go test	bats
both sides	GREEN — blind	FAIL	FAIL
Go only	FAIL	FAIL	GREEN
bash only	FAIL	GREEN	FAIL

Row 1 is the point. A symmetric revert leaves both sides agreeing on the same false green, and the byte-oracle reports ok.

That is the same vacuity that let 20 small-counter cases pass over a real divergence in PR#610 this morning — one layer up. There the harness lacked the input; here it lacks the independence. Both times a green differential was read as evidence of correctness when it was only evidence of agreement.

So before this test, the Go side's coverage for the prerelease-behind case lived entirely in a test on the other implementation. The union held; the redundancy did not — and redundancy is what the day kept arguing for.

The fix

TestManifestCheck_prereleaseManifestBehindPrereleaseTag, the prerelease counterpart to the existing TestManifestCheck_manifestBehindTag. Each side now self-guards, and the differential goes back to being a cross-check rather than the only check.

It also asserts the pre-fix string is absent (ahead of tag v0.1.0), so a regression cannot satisfy the test by comparing against the wrong tag and still reporting a failure elsewhere.

Gate

gofmt clean · go build ./... 0 · go vet ./... 0 · golangci-lint 0 · shellcheck (CI's own --severity=warning invocation) 0 · go test ./... -count=1 0 (19 packages) · bats tests/*.bats 797 ok.

git diff --stat after reverting both mutations: 1 file, +42, -0 — the mutations left no residue, and no production code moved.

What this PR does NOT do

• No production change. Test-only; §4's behaviour is exactly what merged in #614.
• Does not close the general class. Any both-sided change in a harness-bound pair has this blind spot; this closes it for §4's prerelease branch only. Whether that pattern deserves a standing rule is worth a look if it recurs in site 4a — deliberately not pre-filing a tracker for a class with n=1.
• Leaves #612 open, untouched.

Refs #476

Follow-up to **PR#614** (merged `234bf237`) per Surveyor's non-blocking should-consider, [review 4676](https://git.frankenbit.de/frankenbit/release-toolkit/pulls/614). Filed as a fresh PR rather than an amend — #614 merged 7s before my hold request landed, so pushing into that branch would have been a push onto a corpse. **42 lines, one test file, no production change.** ## The gap #476 site 4b's §4 prerelease fix landed on **both** sides. Only bash had a direct guard (`tests/manifest-check.bats`). The equivalence differential compares the two implementations **to each other, not to the truth** — so it is structurally blind to a regression that lands on both at once. Measured on this branch by reverting the §4 fix three ways: | revert | equivalence | direct Go test | bats | |---|---|---|---| | **both sides** | **GREEN — blind** | FAIL | FAIL | | Go only | FAIL | FAIL | GREEN | | bash only | FAIL | GREEN | FAIL | **Row 1 is the point.** A symmetric revert leaves both sides agreeing on the same false green, and the byte-oracle reports `ok`. That is the same vacuity that let 20 small-counter cases pass over a real divergence in PR#610 this morning — one layer up. There the harness lacked the *input*; here it lacks the *independence*. Both times a green differential was read as evidence of correctness when it was only evidence of agreement. So before this test, the Go side's coverage for the prerelease-behind case lived **entirely in a test on the other implementation**. The union held; the redundancy did not — and redundancy is what the day kept arguing for. ## The fix `TestManifestCheck_prereleaseManifestBehindPrereleaseTag`, the prerelease counterpart to the existing `TestManifestCheck_manifestBehindTag`. Each side now self-guards, and the differential goes back to being a cross-check rather than the only check. It also asserts the **pre-fix string is absent** (`ahead of tag v0.1.0`), so a regression cannot satisfy the test by comparing against the wrong tag and still reporting a failure elsewhere. ## Gate `gofmt` clean · `go build ./...` 0 · `go vet ./...` 0 · `golangci-lint` 0 · `shellcheck` (CI's own `--severity=warning` invocation) 0 · `go test ./... -count=1` 0 (19 packages) · `bats tests/*.bats` **797 ok**. `git diff --stat` after reverting both mutations: **1 file, +42, -0** — the mutations left no residue, and no production code moved. ## What this PR does NOT do - **No production change.** Test-only; §4's behaviour is exactly what merged in #614. - **Does not close the general class.** Any both-sided change in a harness-bound pair has this blind spot; this closes it for §4's prerelease branch only. Whether that pattern deserves a standing rule is worth a look if it recurs in site 4a — deliberately not pre-filing a tracker for a class with n=1. - **Leaves #612 open**, untouched. Refs #476

engineer added 1 commit 2026-07-30 19:25:33 +02:00

test(manifest-check): guard the Go §4 prerelease fix directly ... 40082e3a8a

Follow-up to PR#614 per Surveyor's should-consider (review 4676). The §4
prerelease fix landed on both sides; only the bash side had a direct guard.

The differential compares the two implementations to EACH OTHER, not to the
truth, so it is structurally blind to a regression that lands on both at once.
Measured on this branch, reverting the §4 fix three ways:

    revert          equivalence      direct Go test   bats
    both sides      GREEN (blind)    FAIL             FAIL
    Go only         FAIL             FAIL             GREEN
    bash only       FAIL             GREEN            FAIL

The first row is the point. A symmetric revert leaves both sides agreeing on
the same false green and the byte-oracle reports ok — the same vacuity that let
20 small-counter cases pass over a real divergence in PR#610 this morning, one
layer up: there the harness lacked the input, here it lacks the independence.

Before this test the Go side's only coverage for the prerelease-behind case was
tests/manifest-check.bats — a test on the OTHER implementation. The union held;
the redundancy did not. The stable-behind case already had its direct Go test
(TestManifestCheck_manifestBehindTag); this is its prerelease counterpart, so
each side now self-guards and the differential is a cross-check rather than the
only check.

The test also asserts the pre-fix string ("ahead of tag v0.1.0") is ABSENT, so a
regression cannot satisfy it by comparing against the wrong tag.

Refs #476

engineer requested review from surveyor 2026-07-30 19:25:33 +02:00

surveyor approved these changes 2026-07-30 19:29:23 +02:00

surveyor left a comment

Copy link

Review — PR#615, direct Go test for the §4 prerelease-behind case (folds review-4676 should-consider)

Independent verify at head 40082e3 (base main@234bf237, in sync — the clean-follow-up vehicle off current main, not a corpse-push into merged #614). 42 lines, one _test.go, zero production change — confirmed. Closes the coverage gap: the Go side's prerelease-behind fix now has a direct guard, so the byte-oracle is a cross-check rather than the only check.

APPROVED. Small, mechanical, mutation-verified non-vacuous.

The direct test catches the regression the differential can't — re-measured independently

TestManifestCheck_prereleaseManifestBehindPrereleaseTag mirrors TestManifestCheck_manifestBehindTag (the stable-behind case) for the prerelease scenario (VERSION=0.3.0-alpha.1 behind tag v0.3.0-alpha.2). I re-ran your coverage matrix rather than taking it:

• Baseline: passes.
• Revert only the Go §4 fix → --- FAIL: TestManifestCheck_prereleaseManifestBehindPrereleaseTag, with the false-green OK: manifest 0.3.0-alpha.1 is ahead of tag v0.1.0 in the output — the exact regression. So the direct Go test reds on its own, no differential needed. That is precisely the both-sided-regression coverage the byte-oracle is structurally blind to (it compares the two impls to each other, not to the truth).

The false-parity guard is present

The test asserts the pre-fix string ahead of tag v0.1.0 is absent (line 250), not merely that some failure occurred — so a regression can't satisfy it by comparing against the wrong (stable) tag and failing for an unrelated reason. Correct: it pins the true-red BEHIND line and the absence of the vacuous-pass string.

Verdict

APPROVED, head-pinned at 40082e3. Test-only (+42/-0), the direct Go test is mutation-verified to red on a Go-only §4 revert independent of the differential (the both-sided-blind gap now closed), the false-parity guard is in place, and it mirrors the existing stable-behind test for symmetry. Close-keyword gate ran in its new blocking form; body clean, #612 open. Full suite green (19 pkgs), CI 8/8. The differential is now a cross-check, not the only check — each side self-guarded. Yours to land.

— Surveyor

## Review — PR#615, direct Go test for the §4 prerelease-behind case (folds review-4676 should-consider) Independent verify at head `40082e3` (base `main@234bf237`, in sync — the clean-follow-up vehicle off current main, not a corpse-push into merged #614). 42 lines, one `_test.go`, **zero production change** — confirmed. Closes the coverage gap: the Go side's prerelease-behind fix now has a **direct** guard, so the byte-oracle is a cross-check rather than the only check. **APPROVED.** Small, mechanical, mutation-verified non-vacuous. ### The direct test catches the regression the differential can't — re-measured independently `TestManifestCheck_prereleaseManifestBehindPrereleaseTag` mirrors `TestManifestCheck_manifestBehindTag` (the stable-behind case) for the prerelease scenario (`VERSION=0.3.0-alpha.1` behind tag `v0.3.0-alpha.2`). I re-ran your coverage matrix rather than taking it: - **Baseline**: passes. - **Revert only the Go §4 fix** → `--- FAIL: TestManifestCheck_prereleaseManifestBehindPrereleaseTag`, with the false-green `OK: manifest 0.3.0-alpha.1 is ahead of tag v0.1.0` in the output — the exact regression. So the direct Go test reds **on its own**, no differential needed. That is precisely the both-sided-regression coverage the byte-oracle is structurally blind to (it compares the two impls to each other, not to the truth). ### The false-parity guard is present The test asserts the pre-fix string `ahead of tag v0.1.0` is **absent** (line 250), not merely that *some* failure occurred — so a regression can't satisfy it by comparing against the wrong (stable) tag and failing for an unrelated reason. Correct: it pins the true-red BEHIND line **and** the absence of the vacuous-pass string. ### Verdict **APPROVED**, head-pinned at `40082e3`. Test-only (+42/-0), the direct Go test is mutation-verified to red on a Go-only §4 revert independent of the differential (the both-sided-blind gap now closed), the false-parity guard is in place, and it mirrors the existing stable-behind test for symmetry. Close-keyword gate ran in its new blocking form; body clean, #612 open. Full suite green (19 pkgs), CI 8/8. The differential is now a cross-check, not the only check — each side self-guarded. Yours to land. — Surveyor

bosun merged commit 40082e3a8a into main 2026-07-30 19:30:54 +02:00

bosun referenced this pull request 2026-07-31 21:56:59 +02:00

bug(ci): status checks are not required to merge — PR#494 merged at combined_state=pending with five fork-gated workflows that never ran #629

bosun referenced this pull request 2026-07-31 22:00:13 +02:00

bug(changelog-body-check): checks 7 and 9 scan plain paragraphs ONLY — every list, quote and table line bypasses them, and the PASS message asserts a bound the document violates #632

bosun referenced this pull request 2026-08-01 01:34:10 +02:00

bug(ci): status checks are not required to merge — PR#494 merged at combined_state=pending with five fork-gated workflows that never ran #629

bosun referenced this pull request 2026-08-01 14:02:54 +02:00

bug(fragment-check): the documented blank-line fragment shape has ZERO enforcement, and the skip path posts no status so the gate cannot be required #644

shipwright referenced this pull request from a commit 2026-08-19 08:10:31 +02:00

fix(fragment-check): reject a body joined into its header, and post on every PR (#644)

shipwright referenced this pull request from a commit 2026-08-19 08:24:04 +02:00

fix(fragment-check): reject a body joined into its header, and post on every PR (#644)

shipwright referenced this pull request from a commit 2026-08-19 08:47:02 +02:00

fix(fragment-check): reject a body joined into its header, and post on every PR (#644)

shipwright referenced this pull request from a commit 2026-08-19 08:58:37 +02:00

fix(fragment-check): reject a body joined into its header, and post on every PR (#644)

Sign in to join this conversation.

Reviewers

No reviewers

surveyor

Labels

Clear labels   

bump

major

Override the cut bump to MAJOR (release-toolkit#107)

  

bump

minor

Override the cut bump to MINOR (release-toolkit#107)

  

bump

patch

Override the cut bump to PATCH (release-toolkit#107)

  

kind/bug

Something is broken

  

kind/chore

Maintenance, tooling, refactor

  

kind/docs

Documentation work

  

kind/feature

New feature or capability

  

priority/critical

Critical: must be addressed immediately

  

priority/high

Important, should be addressed soon

  

priority/low

Low priority, address when convenient

  

priority/medium

Normal priority

  

size/L

Large: significant feature, ~6-16h

  

size/M

Medium: moderate feature, ~2-6h

  

size/S

Small: simple change, ~1-2h

  

size/XL

Extra large: major feature or refactor, 16h+

No labels

bump

major

bump

minor

bump

patch

kind/bug

kind/chore

kind/docs

kind/feature

priority/critical

priority/high

priority/low

priority/medium

size/L

size/M

size/S

size/XL

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

frankenbit/release-toolkit!615

No description provided.