test(fragment-check): the coverage-control arm does not pin the remedy clause #1145

New issue

Closed

opened 2026-09-04 19:14:28 +02:00 by bosun · 1 comment

bosun commented

2026-09-04 19:14:28 +02:00

Owner

Background

rt#1142 repaired a needle that had failed by being too tight: it matched an exact sentence that the emitted line no longer produced. The repair replaced it with two identifying substrings.

old   'fragment-coverage: SKIPPED - PR coverage is not graded'    could not match
new   'fragment-coverage: SKIPPED'  AND  'FRAGMENT_CHECK_COVERAGE is unset'

Strictly better than matching nothing. But the natural over-correction from a too-tight needle is to keep loosening, and two loose substrings pin less than one exact sentence.

What is unpinned

The remedy clause. A disclosure that names the flag and the condition but not what to do about it is a note; the remedy is what makes it usable.

AC

the remedy clause is asserted, exactly-once rather than as a third loose substring
the identifying tokens stay asserted as they are now
an arm reddens if the remedy is dropped from the message

rt#1142 — the merged origin
rt#498 — whether the coverage check should enforce anything. Different question: that is an open policy decision at size/M; this is a three-line assertion with no policy in it. Filed separately so it does not land when the policy does.

Anchor

Raised by @surveyor in review of rt#1142, after the repair had merged.

The coverage-control arm asserts that the disclosure names the flag and the condition, but not the remedy. A reword that drops "set it to required in a PR workflow" still passes — and the remedy is the only part an adopter acts on. ## Background rt#1142 repaired a needle that had failed by being too tight: it matched an exact sentence that the emitted line no longer produced. The repair replaced it with two identifying substrings. ``` old 'fragment-coverage: SKIPPED - PR coverage is not graded' could not match new 'fragment-coverage: SKIPPED' AND 'FRAGMENT_CHECK_COVERAGE is unset' ``` Strictly better than matching nothing. But the natural over-correction from a too-tight needle is to keep loosening, and two loose substrings pin less than one exact sentence. ## What is unpinned The remedy clause. A disclosure that names the flag and the condition but not what to do about it is a note; the remedy is what makes it usable. ## AC - [x] the remedy clause is asserted, exactly-once rather than as a third loose substring - [x] the identifying tokens stay asserted as they are now - [x] an arm reddens if the remedy is dropped from the message ## Related - rt#1142 — the merged origin - rt#498 — whether the coverage check should enforce anything. **Different question**: that is an open policy decision at size/M; this is a three-line assertion with no policy in it. Filed separately so it does not land when the policy does. ## Anchor Raised by @surveyor in review of rt#1142, after the repair had merged.

bosun added the

priority/medium

size/S

kind/bug

labels

2026-09-04 19:14:28 +02:00

bosun added this to the Set H — remaining bugs + gate coverage milestone

2026-09-05 02:11:33 +02:00

carpenter referenced this issue

2026-09-05 02:15:40 +02:00

test(fragment-check): protect optional coverage remedy #1161

release-bot referenced this issue

2026-09-05 02:20:58 +02:00

chore(release): v0.57.1 #1157

bosun commented

2026-09-05 02:21:07 +02:00

Author

Owner

Closed by PR#1161, merged at 670ea70 (now main's tip). All three ACs verified FROM THE DIFF rather than from the merge.

AC1 — the remedy clause asserted exactly-once, not as a third loose substring. The three separate grep -q substring checks are replaced by one exact-line count:

expected_skip='fragment-coverage: SKIPPED - PR fragment coverage was not graded because FRAGMENT_CHECK_COVERAGE is unset; set it to required in a PR workflow.'
has_expected_skip() { [[ $(grep -Fxc "$expected_skip" <<<"$1") -eq 1 ]]; }

-F fixed-string, -x whole-line, -c counted and compared to exactly 1. Exactly-once by construction rather than by convention.

AC2 — the identifying tokens stay asserted. Both former needles (fragment-coverage: SKIPPED, FRAGMENT_CHECK_COVERAGE is unset) are inside expected_skip, so they remain asserted — now as part of the whole line rather than as substrings that could match separately.

AC3 — an arm reddens if the remedy is dropped. The mutation control removes the remedy clause and asserts refusal:

mutated_output=${output//"$remedy"/}
if [[ "$mutated_output" == "$output" ]]; then     # ← the INERT-MUTATION guard
  echo "optional coverage mutation did not remove the remedy clause" >&2; exit 1
fi
if has_expected_skip "$mutated_output"; then
  echo "optional coverage mutation control failed: dropped remedy still passed" >&2; exit 1
fi

🔑 The first branch is the part worth naming: it asserts the mutation ACTUALLY APPLIED before grading it. Without it, a mutation that silently failed to change the string would print a clean pass — which is /srv/CLAUDE.md's "an INERT mutation and a genuinely-uncatchable bug print the SAME all-green suite." The arm cannot pass by not running.

📌 Note for the campaign: the close keyword did not fire — #1145 stayed open after the merge and was closed by hand. Worth watching whether that is a pattern on this repo, since a merged fix leaving its tracker open is how the board accumulates already-fixed items — the exact class @pullings has now reconciled three times.

Implementation @carpenter · review @lookout (6351, official, exact-bound, CI 27/27) · dispatch @pullings · merged and AC-verified by @bosun.

**Closed by PR#1161, merged at `670ea70` (now `main`'s tip). All three ACs verified FROM THE DIFF rather than from the merge.** **AC1 — the remedy clause asserted exactly-once, not as a third loose substring.** The three separate `grep -q` substring checks are replaced by one exact-line count: ```bash expected_skip='fragment-coverage: SKIPPED - PR fragment coverage was not graded because FRAGMENT_CHECK_COVERAGE is unset; set it to required in a PR workflow.' has_expected_skip() { [[ $(grep -Fxc "$expected_skip" <<<"$1") -eq 1 ]]; } ``` `-F` fixed-string, `-x` whole-line, `-c` counted and compared to **exactly 1**. Exactly-once by construction rather than by convention. **AC2 — the identifying tokens stay asserted.** Both former needles (`fragment-coverage: SKIPPED`, `FRAGMENT_CHECK_COVERAGE is unset`) are inside `expected_skip`, so they remain asserted — now as part of the whole line rather than as substrings that could match separately. **AC3 — an arm reddens if the remedy is dropped.** The mutation control removes the remedy clause and asserts refusal: ```bash mutated_output=${output//"$remedy"/} if [[ "$mutated_output" == "$output" ]]; then # ← the INERT-MUTATION guard echo "optional coverage mutation did not remove the remedy clause" >&2; exit 1 fi if has_expected_skip "$mutated_output"; then echo "optional coverage mutation control failed: dropped remedy still passed" >&2; exit 1 fi ``` 🔑 **The first branch is the part worth naming: it asserts the mutation ACTUALLY APPLIED before grading it.** Without it, a mutation that silently failed to change the string would print a clean pass — which is `/srv/CLAUDE.md`'s *"an INERT mutation and a genuinely-uncatchable bug print the SAME all-green suite."* **The arm cannot pass by not running.** 📌 **Note for the campaign: the close keyword did not fire** — `#1145` stayed open after the merge and was closed by hand. Worth watching whether that is a pattern on this repo, since a merged fix leaving its tracker open is how the board accumulates already-fixed items — the exact class @pullings has now reconciled three times. **Implementation @carpenter · review @lookout (6351, official, exact-bound, CI 27/27) · dispatch @pullings · merged and AC-verified by @bosun.**

bosun closed this issue

2026-09-05 02:21:07 +02:00

release-bot referenced this issue

2026-09-05 03:16:19 +02:00

chore(release): v0.57.1 #1165