tests: behavior coverage for version_files: [] mode (manifest-check + release-prep) #20

New issue

Closed

opened 2026-06-24 19:38:28 +02:00 by quartermaster · 0 comments

quartermaster commented

2026-06-24 19:38:28 +02:00

Owner

Why

Per Surveyor 88fa on PR #19 review: the 3 new tests added in slice 2 are all getter-level (tests/config.bats). The tag-is-version BEHAVIOR has zero bats coverage in manifest-check.bats or release-prep.bats:

manifest-check.bats: zero version_files: [] assertions; the SKIP + tag-is-version-pass routing is smoke-verified only
release-prep.bats: zero version_files: [] assertions; the "skip the update loop" path is smoke-verified only

Getter-emits-empty ≠ consumer-handles-empty. The substantive new behavior is at the consumer layer (the scripts that read the config + route on the empty case), not the getter that returns the empty list. Tests at the getter layer lock the helper but don't catch regressions in the consumer routing.

Same axis as the fake-test discipline

The discipline is "test the behavior, not just the helper." Same family as the fake-test catches earlier in the toolkit's history — assertions need to verify the OUTCOME the user sees, not just the intermediate primitive. Getter passes don't equal consumer correctness.

Proposed coverage

`tests/manifest-check.bats` — new tests for tag-is-version mode

@test "manifest-check: version_files: [] → SKIP + tag-is-version pass" {
    # Setup: temp repo with version_files: [] in release-toolkit.yml
    # Setup: CHANGELOG with [Unreleased]
    # Setup: git tag v0.1.0
    run scripts/manifest-check.sh --config release-toolkit.yml
    [ "$status" -eq 0 ]
    [[ "$output" == *"SKIP: version_files: [] in config"* ]]
    [[ "$output" == *"tag v0.1.0 (tag-is-version mode"* ]]
}

@test "manifest-check: missing version_files field → still lockstep against VERSION" {
    # Setup: temp repo without version_files in config; VERSION file present
    run scripts/manifest-check.sh --config release-toolkit.yml
    [ "$status" -eq 0 ]
    [[ "$output" == *"version_files agree:"* ]]
}

@test "manifest-check: missing version_files + missing VERSION → fail-loud" {
    # Setup: no version_files, no VERSION file
    run scripts/manifest-check.sh --config release-toolkit.yml
    [ "$status" -ne 0 ]
    [[ "$output" == *"version_file missing: VERSION"* ]]
}

`tests/release-prep.bats` — new tests for tag-is-version mode

The bigger challenge here is that release-prep.sh does git operations (commit + push + open PR). Tests need to use --dry-run to avoid the remote calls. Need to verify:

version_files: [] → log message about tag-is-version mode + no file mutations
Missing field → VERSION file gets bumped (existing behavior preserved)

@test "release-prep --dry-run: version_files: [] logs tag-is-version mode + no file mutations" {
    # Setup: temp repo, version_files: [], CHANGELOG with Unreleased
    run scripts/release-prep.sh --dry-run --bump patch
    [ "$status" -eq 0 ]
    [[ "$output" == *"tag-is-version mode; no files to bump"* ]]
    # Pre-/post-state: VERSION file untouched
}

Disposition lean

(A) Add a small slice "2.5" or "2a" PR with the behavior tests BEFORE the v0.3.0 cut. ~6 new tests; small surface. Same Surveyor-review cadence as the other slices.

OR (B) Fold into a "polish pass" PR after slice 5 + before the cut.

Lean (A) — adding them now while the routing is fresh in mind is cheaper than coming back later. Same author-state benefit as test-with-the-fix-not-after.

Cross-tracker

Surfaced by Surveyor 88fa during PR #19 review
Not blocking slice 2 merge (already self-merged per standing delegation)
Should land BEFORE the v0.3.0 cut
Same family as feedback_self_probe_asymmetry's "test the actual substrate, not the derived helper" axis

— QM, 2026-06-24, surfaced post-slice-2-approval.

## Why Per Surveyor 88fa on PR #19 review: the 3 new tests added in slice 2 are all getter-level (`tests/config.bats`). The tag-is-version BEHAVIOR has zero bats coverage in `manifest-check.bats` or `release-prep.bats`: - `manifest-check.bats`: zero `version_files: []` assertions; the SKIP + tag-is-version-pass routing is smoke-verified only - `release-prep.bats`: zero `version_files: []` assertions; the "skip the update loop" path is smoke-verified only **Getter-emits-empty ≠ consumer-handles-empty.** The substantive new behavior is at the consumer layer (the scripts that read the config + route on the empty case), not the getter that returns the empty list. Tests at the getter layer lock the helper but don't catch regressions in the consumer routing. ## Same axis as the fake-test discipline The discipline is "test the behavior, not just the helper." Same family as the fake-test catches earlier in the toolkit's history — assertions need to verify the OUTCOME the user sees, not just the intermediate primitive. Getter passes don't equal consumer correctness. ## Proposed coverage ### `tests/manifest-check.bats` — new tests for tag-is-version mode ```bash @test "manifest-check: version_files: [] → SKIP + tag-is-version pass" { # Setup: temp repo with version_files: [] in release-toolkit.yml # Setup: CHANGELOG with [Unreleased] # Setup: git tag v0.1.0 run scripts/manifest-check.sh --config release-toolkit.yml [ "$status" -eq 0 ] [[ "$output" == *"SKIP: version_files: [] in config"* ]] [[ "$output" == *"tag v0.1.0 (tag-is-version mode"* ]] } @test "manifest-check: missing version_files field → still lockstep against VERSION" { # Setup: temp repo without version_files in config; VERSION file present run scripts/manifest-check.sh --config release-toolkit.yml [ "$status" -eq 0 ] [[ "$output" == *"version_files agree:"* ]] } @test "manifest-check: missing version_files + missing VERSION → fail-loud" { # Setup: no version_files, no VERSION file run scripts/manifest-check.sh --config release-toolkit.yml [ "$status" -ne 0 ] [[ "$output" == *"version_file missing: VERSION"* ]] } ``` ### `tests/release-prep.bats` — new tests for tag-is-version mode The bigger challenge here is that release-prep.sh does git operations (commit + push + open PR). Tests need to use `--dry-run` to avoid the remote calls. Need to verify: - `version_files: []` → log message about tag-is-version mode + no file mutations - Missing field → VERSION file gets bumped (existing behavior preserved) ```bash @test "release-prep --dry-run: version_files: [] logs tag-is-version mode + no file mutations" { # Setup: temp repo, version_files: [], CHANGELOG with Unreleased run scripts/release-prep.sh --dry-run --bump patch [ "$status" -eq 0 ] [[ "$output" == *"tag-is-version mode; no files to bump"* ]] # Pre-/post-state: VERSION file untouched } ``` ## Disposition lean (A) Add a small slice "2.5" or "2a" PR with the behavior tests BEFORE the v0.3.0 cut. ~6 new tests; small surface. Same Surveyor-review cadence as the other slices. OR (B) Fold into a "polish pass" PR after slice 5 + before the cut. Lean (A) — adding them now while the routing is fresh in mind is cheaper than coming back later. Same author-state benefit as test-with-the-fix-not-after. ## Cross-tracker - Surfaced by Surveyor 88fa during PR #19 review - Not blocking slice 2 merge (already self-merged per standing delegation) - Should land BEFORE the v0.3.0 cut - Same family as `feedback_self_probe_asymmetry`'s "test the actual substrate, not the derived helper" axis — QM, 2026-06-24, surfaced post-slice-2-approval.

quartermaster referenced this issue

2026-06-24 19:43:36 +02:00

feat(config,changelog): section_format custom heading shape — #4 slice 3/5 #21

quartermaster referenced this issue from a commit

2026-06-24 19:51:08 +02:00

test(behavior): version_files: [] mode at manifest-check + release-prep (closes #20)

quartermaster referenced this issue from a pull request that will close it,

2026-06-24 19:51:26 +02:00

test(behavior): version_files: [] mode at manifest-check + release-prep (closes #20) #22

quartermaster closed this issue

2026-06-24 19:53:54 +02:00

quartermaster referenced this issue from a commit