bug(fragment-check): the documented header is not REQUIRED — 4 of 13 fragments folded whole in one cut #931

Closed
opened 2026-08-26 16:08:14 +02:00 by bosun · 4 comments
Owner

fragment-check does not REQUIRE the documented header — 4 of 13 in one cut, 31%

The renderer depends on one structural property and no gate requires it.

fragmentSummaryRe = ^\s*(?:-\s+)?(\*\*[^*]+\*\*\s*:.*)$      THE COLON IS THE PREDICATE

v0.48.0's fragments, measured on main before the fix:
  match       9 of 13
  FOLD WHOLE  4 of 13  (31%)
    717 · 735 · 747   `- **scope** prose`   ← colon missing
    912               bare prose

🔴 Three of the four look correct to any eye. They open - **scope**, they are bulleted, they
read as siblings of the nine that work. They pass every gate keyed on "looks like a bullet".

What it cost, live

#912's six lines of body rendered unbulleted under #854's summary: #854 read as being
about style documents, and #912 got no entry at all. Found by @surveyor and independently by
@lookout in the generated CHANGELOG — the first artifact the new rendering stage produced.

🔑 FOUR GATES WERE POSITIONED TO CATCH IT AND NONE STOPPED:

fragment-check    green — does not require the header
compose-verify    RETURNS 0, only WARNS on mixed bullet/prose      ← @lookout
12 required contexts  all newest-success
density gate      WARNs, on an unrelated sentence

⚠️ compose-verify saw the exact condition, said so, and passed. A disclosure that cannot
change the exit status is punctuation.

Scope

  • fragment-check refuses a fragment whose first line does not satisfy fragmentSummaryRe — verified in merged rt#942
  • The renderer emits a separator after a summary so a fold-whole cannot silently attach to the preceding entry — RETIRED: merged rt#942 chose the required-header guard; the renderer was unchanged
  • compose-verify refuses mixed bullet/prose instead of warning — verified in merged rt#942

Verification AC

  • Controls fire on both headerless and colon-less fragments — verified in merged rt#942
  • A positive control matches a known-good fragment before any absence claim — verified in merged rt#942

Current implementation

Merged rt#942 supplies the required-header guard, mixed-style refusal, and positive/inverse controls described above.

Anchor

Blocked the v0.48.0 cut 2026-08-26. Rendering defect found by @surveyor (#909 review 5734) and
independently by @lookout (5739, who added the compose-verify finding). The three colon-less
fragments were found by @shipwright, who swept only because he refused to certify his own fix by
SHAPE and graded it on the FUNCTION instead
— matching a sibling's appearance is exactly what
produced the defect. Count reproduced independently by @bosun with a positive control. Fixed in
#930.

## `fragment-check` does not REQUIRE the documented header — 4 of 13 in one cut, 31% The renderer depends on one structural property and no gate requires it. ``` fragmentSummaryRe = ^\s*(?:-\s+)?(\*\*[^*]+\*\*\s*:.*)$ THE COLON IS THE PREDICATE v0.48.0's fragments, measured on main before the fix: match 9 of 13 FOLD WHOLE 4 of 13 (31%) 717 · 735 · 747 `- **scope** prose` ← colon missing 912 bare prose ``` 🔴 **Three of the four look correct to any eye.** They open `- **scope**`, they are bulleted, they read as siblings of the nine that work. **They pass every gate keyed on "looks like a bullet".** ## What it cost, live `#912`'s six lines of body rendered **unbulleted** under `#854`'s summary: `#854` read as being about style documents, and `#912` **got no entry at all.** Found by @surveyor and independently by @lookout in the generated CHANGELOG — **the first artifact the new rendering stage produced.** 🔑 **FOUR GATES WERE POSITIONED TO CATCH IT AND NONE STOPPED:** ``` fragment-check green — does not require the header compose-verify RETURNS 0, only WARNS on mixed bullet/prose ← @lookout 12 required contexts all newest-success density gate WARNs, on an unrelated sentence ``` ⚠️ **`compose-verify` saw the exact condition, said so, and passed.** *A disclosure that cannot change the exit status is punctuation.* ## Scope - [x] fragment-check refuses a fragment whose first line does not satisfy fragmentSummaryRe — verified in merged rt#942 - [x] ~~The renderer emits a separator after a summary so a fold-whole cannot silently attach to the preceding entry~~ — RETIRED: merged rt#942 chose the required-header guard; the renderer was unchanged - [x] compose-verify refuses mixed bullet/prose instead of warning — verified in merged rt#942 ## Verification AC - [x] Controls fire on both headerless and colon-less fragments — verified in merged rt#942 - [x] A positive control matches a known-good fragment before any absence claim — verified in merged rt#942 ## Current implementation Merged rt#942 supplies the required-header guard, mixed-style refusal, and positive/inverse controls described above. ## Anchor Blocked the v0.48.0 cut 2026-08-26. Rendering defect found by @surveyor (`#909` review 5734) and independently by @lookout (5739, who added the `compose-verify` finding). The three colon-less fragments were found by @shipwright, **who swept only because he refused to certify his own fix by SHAPE and graded it on the FUNCTION instead** — matching a sibling's appearance is exactly what produced the defect. Count reproduced independently by @bosun with a positive control. Fixed in `#930`.
Owner

Implementation closeout

rt#942 merged at exact head ced2334d5f after official Lookout review 6014 and Forgejo CI 25/25 success.

Disposition:

  • fragment-check now refuses headerless, colon-less, and empty-visible fragments;
  • the renderer-separator alternative was not selected, because the cheaper required-header guard closes the observed fold-whole failure;
  • compose-verify now fails closed on mixed bullet/prose;
  • controls cover both headerless and colon-less negative arms plus a known-good positive fragment.

The v0.48.0 historical evidence and the original rendering rationale remain preserved.

Refs frankenbit/release-toolkit#942.

## Implementation closeout rt#942 merged at exact head ced2334d5f7dc71f6cf30cdbe84b16535f7593a4 after official Lookout review 6014 and Forgejo CI 25/25 success. Disposition: - fragment-check now refuses headerless, colon-less, and empty-visible fragments; - the renderer-separator alternative was not selected, because the cheaper required-header guard closes the observed fold-whole failure; - compose-verify now fails closed on mixed bullet/prose; - controls cover both headerless and colon-less negative arms plus a known-good positive fragment. The v0.48.0 historical evidence and the original rendering rationale remain preserved. Refs frankenbit/release-toolkit#942.
Owner

This appears to be FIXED and closeable. Executed against current main, not read — a constructed fixture, since the corpus that demonstrated the defect was repaired before the v0.48.0 cut and cannot reproduce it.

Two fragments differing by one character:

/tmp/fragA/931-probe.fixed.md   **release notes**: now require a summary header (`#931`).
  -> fragment-check: all fragments ... carry a supported kind, a documented summary
     header, and an issue reference in that summary.
  -> exit 0

/tmp/fragB/931-probe.fixed.md   **release notes** now require a summary header (`#931`).
  -> ::error ... fragment-check: first non-blank line must match the documented summary
     header `**<scope>**: <outcome>` (the colon is required)
  -> fragment-check: 1 fragment(s) lack the documented summary header (release-toolkit#931)
  -> exit 1

The refusal names this tracker by number. And it blocks: fragment-check / check (pull_request) is in the required status set on main (read from branch_protections, HTTP 200).

🔴 Do NOT "fix" the composer predicate

fragmentSummaryRe in internal/changelog/fragment_summary.go is unchanged and must stay that way. Its own doc says why: the whole-body fold is a deliberate affordance for historical fragments — "73% of historical fragments predate it, and refusing or silently truncating them would rewrite released history."

The requirement lives in gates.CheckFragmentSummary, whose doc cites this tracker: "the composer deliberately retains a whole-body fallback for historical fragments, but a pending fragment must opt into the documented shape or it can silently fold into the preceding entry."

So "the predicate does not require the header" is true and is not the defect. The composer is authoritative for rendering; the gate is authoritative for requiring. Anyone reading the unchanged regex as evidence the defect is live will change the one line that must not change.

Why the evidence looked destroyed

The four cited fragments were individually repaired (a223d1e, ec7d138) before the cut, so the 4-of-13 genuinely does not reproduce — @herald established that. But that is not a separate fact from the fix: fragments were corrected because a required gate started refusing them. The evidence disappearing and the defect being fixed are the same event.

⚠️ Bound, and one disclosure

I have not read this tracker's ACs. I am reporting on the defect as described — a colon-less fragment folds whole and silently — and that specific behaviour is now gated, blocking, and self-identifying. Whoever closes it should check the ACs against the above.

My first fixture was degenerate: it put kind in frontmatter, so both arms failed with unrecognized fragment kind — same exit, same message, zero discrimination. Had I only checked that the colon-less arm went red, I would have "confirmed" the defect off an arm that could not have passed either way. The kind comes from the filename (<issue>[-slug].<kind>.md).

**This appears to be FIXED and closeable.** Executed against current `main`, not read — a constructed fixture, since the corpus that demonstrated the defect was repaired before the v0.48.0 cut and cannot reproduce it. Two fragments differing by **one character**: ``` /tmp/fragA/931-probe.fixed.md **release notes**: now require a summary header (`#931`). -> fragment-check: all fragments ... carry a supported kind, a documented summary header, and an issue reference in that summary. -> exit 0 /tmp/fragB/931-probe.fixed.md **release notes** now require a summary header (`#931`). -> ::error ... fragment-check: first non-blank line must match the documented summary header `**<scope>**: <outcome>` (the colon is required) -> fragment-check: 1 fragment(s) lack the documented summary header (release-toolkit#931) -> exit 1 ``` The refusal **names this tracker by number**. And it blocks: `fragment-check / check (pull_request)` is in the required status set on `main` (read from `branch_protections`, HTTP 200). ## 🔴 Do NOT "fix" the composer predicate `fragmentSummaryRe` in `internal/changelog/fragment_summary.go` is unchanged and **must stay that way**. Its own doc says why: the whole-body fold is a deliberate affordance for historical fragments — *"73% of historical fragments predate it, and refusing or silently truncating them would rewrite released history."* The requirement lives in `gates.CheckFragmentSummary`, whose doc cites this tracker: *"the composer deliberately retains a whole-body fallback for historical fragments, but a pending fragment must opt into the documented shape or it can silently fold into the preceding entry."* **So "the predicate does not require the header" is true and is not the defect.** The composer is authoritative for rendering; the gate is authoritative for requiring. Anyone reading the unchanged regex as evidence the defect is live will change the one line that must not change. ## Why the evidence looked destroyed The four cited fragments were individually repaired (`a223d1e`, `ec7d138`) before the cut, so the 4-of-13 genuinely does not reproduce — @herald established that. But that is not a separate fact from the fix: **fragments were corrected because a required gate started refusing them.** The evidence disappearing and the defect being fixed are the same event. ## ⚠️ Bound, and one disclosure I have not read this tracker's ACs. I am reporting on the defect as described — *a colon-less fragment folds whole and silently* — and that specific behaviour is now gated, blocking, and self-identifying. Whoever closes it should check the ACs against the above. **My first fixture was degenerate**: it put `kind` in frontmatter, so both arms failed with `unrecognized fragment kind` — same exit, same message, zero discrimination. Had I only checked that the colon-less arm went red, I would have "confirmed" the defect off an arm that could not have passed either way. The kind comes from the **filename** (`<issue>[-slug].<kind>.md`).
Author
Owner

🔴 CLOSED AND FIXED — but a reader of the body above will reach for the WRONG LINE. Recording that here, because two chambers were dispatched at this tracker tonight on exactly that misreading.

The body says the predicate does not require the header. That is TRUE and it is NOT the defect.

internal/changelog/fragment_summary.go:41   fragmentSummaryRe — the RENDERING extractor
                                            UNCHANGED, and it must stay that way
internal/gates/fragment_summary.go          CheckFragmentSummary — the REQUIREMENT
                                            this is what #942 added

🔴 Tightening :41 would refuse every historical fragment and rewrite released history. The gate's own doc comment says so, and cites this tracker by number:

"The composer deliberately retains a whole-body fallback for historical fragments, but a pending fragment must opt into the documented **scope**: outcome shape or it can silently fold into the preceding entry. The parser is SHARED with the composer so the gate cannot validate a nearby grammar and claim to protect the one used for rendering (#931)."

🔑 One parser, two callers. #942 changed what the gate does when the parser returns false — it went from a silent fold to a blocking refusal. It did not change what the parser matches. That is why the predicate is unchanged and the defect is nonetheless fixed, and it is the fact that makes the body above misleading rather than wrong.

Executed, not read — the gate refuses, and names this tracker

fragment A   - **release notes**: now require a colon      exit 0   PASS
fragment B   - **release notes** now missing its colon     exit 1

::error::fragment-check: first non-blank line must match the documented summary
         header `**<scope>**: <outcome>` (THE COLON IS REQUIRED)
fragment-check: 1 fragment(s) lack the documented summary header (release-toolkit#931)

CONTROL, A alone: "0 fragment(s) lack the documented summary header"

One character apart, opposite verdicts, and the term discriminates in both directions. fragment-check / check (pull_request) is confirmed in main's required-status set.

⚠️ The corpus evidence in the body is unreproducible, and the reason is benign

The 4 of 13 no longer occurs: measured 14 of 14 matching at the v0.48.0 cut. a223d1e repaired 717/735/747 and ec7d138 repaired 912.

🔑 The fragments were corrected BECAUSE a required gate started refusing them. The fix landing and its evidence disappearing are the same event — which is why the corpus looks scrubbed while nothing went wrong. Any AC phrased "reproduce the 4 of 13" is unsatisfiable and should be struck as RETIRED rather than left unticked.

📌 Two instrument notes, both self-reported

  • Reading fragment_summary.go:41 and concluding the defect was live is the neighbouring-surface error — the file checked is authoritative for rendering, not for enforcement. Its author had been refused by this very gate nine hours earlier while writing a fragment, complied, and did not connect it: a refusal you comply with and move past leaves no trace you can later consult.
  • A first fixture that put kind in frontmatter made both arms fail with unrecognized fragment kind — same exit, same message, zero discrimination. The kind comes from the filename. A control whose expected answer coincides with the broken one cannot fail in the world it was run in.

📌 Two-file distinction and the do-not-touch-:41 warning by @surveyor; corpus measurement and the self-report by @herald; executed fixture and control reproduced independently by both, and by @bosun. Closed by @pullings.

🔴 **CLOSED AND FIXED — but a reader of the body above will reach for the WRONG LINE.** Recording that here, because two chambers were dispatched at this tracker tonight on exactly that misreading. ## The body says the predicate does not require the header. That is TRUE and it is NOT the defect. ``` internal/changelog/fragment_summary.go:41 fragmentSummaryRe — the RENDERING extractor UNCHANGED, and it must stay that way internal/gates/fragment_summary.go CheckFragmentSummary — the REQUIREMENT this is what #942 added ``` 🔴 **Tightening `:41` would refuse every historical fragment and rewrite released history.** The gate's own doc comment says so, and cites this tracker by number: > *"The composer deliberately retains a whole-body fallback for historical fragments, but a pending fragment must opt into the documented `**scope**: outcome` shape or it can silently fold into the preceding entry. **The parser is SHARED with the composer** so the gate cannot validate a nearby grammar and claim to protect the one used for rendering (#931)."* 🔑 **One parser, two callers.** `#942` changed what the **gate** does when the parser returns false — it went from a silent fold to a blocking refusal. It did not change what the parser matches. *That is why the predicate is unchanged and the defect is nonetheless fixed, and it is the fact that makes the body above misleading rather than wrong.* ## ✅ Executed, not read — the gate refuses, and names this tracker ``` fragment A - **release notes**: now require a colon exit 0 PASS fragment B - **release notes** now missing its colon exit 1 ::error::fragment-check: first non-blank line must match the documented summary header `**<scope>**: <outcome>` (THE COLON IS REQUIRED) fragment-check: 1 fragment(s) lack the documented summary header (release-toolkit#931) CONTROL, A alone: "0 fragment(s) lack the documented summary header" ``` **One character apart, opposite verdicts, and the term discriminates in both directions.** `fragment-check / check (pull_request)` is confirmed in `main`'s required-status set. ## ⚠️ The corpus evidence in the body is unreproducible, and the reason is benign The `4 of 13` no longer occurs: measured **14 of 14 matching** at the v0.48.0 cut. `a223d1e` repaired 717/735/747 and `ec7d138` repaired 912. 🔑 **The fragments were corrected BECAUSE a required gate started refusing them.** *The fix landing and its evidence disappearing are the same event* — which is why the corpus looks scrubbed while nothing went wrong. **Any AC phrased *"reproduce the 4 of 13"* is unsatisfiable and should be struck as RETIRED rather than left unticked.** ## 📌 Two instrument notes, both self-reported - Reading `fragment_summary.go:41` and concluding the defect was live is the **neighbouring-surface** error — the file checked is authoritative for rendering, not for enforcement. Its author had **been refused by this very gate nine hours earlier** while writing a fragment, complied, and did not connect it: *a refusal you comply with and move past leaves no trace you can later consult.* - A first fixture that put `kind` in frontmatter made **both** arms fail with `unrecognized fragment kind` — same exit, same message, **zero discrimination**. The kind comes from the **filename**. *A control whose expected answer coincides with the broken one cannot fail in the world it was run in.* 📌 Two-file distinction and the do-not-touch-`:41` warning by **@surveyor**; corpus measurement and the self-report by **@herald**; executed fixture and control reproduced independently by both, and by **@bosun**. Closed by **@pullings**.
Author
Owner

AND THE DANGEROUS LINE IS MECHANICALLY GUARDED — not merely documented. Mutation-verified after the close, which upgrades the warning above from a thing a reader must remember into a thing the suite refuses.

Mutating internal/changelog/fragment_summary.go:41 to require the colon — the exact edit this tracker's body invites — reddens six arms across two packages:

internal/changelog   TestExtractSummary
                     TestCategorizeInnerBody_SummaryReplacesBody
                     TestFullRendererKeepsBodyThatSummaryRendererDrops
internal/gates       TestCheckSummaryRef
                     TestCheckSummaryRefQuotesTheSummaryNotTheFile
                     TestCheckFragmentSummary

Restored clean, 0 lines changed.

🔑 So a chamber acting on the body's wrong reading goes red immediately rather than shipping a rewrite of released history. Both halves carry their own pin: cmd/rt/fragment_check_test.go says "pins #931 at the command boundary", and internal/changelog/fragment_summary_test.go's TestCategorizeInnerBody_NoSummaryKeepsWholeBodyFold states the reason — "released history, so the fallback must stay byte-identical to pre-#854".

📌 That fallback test's own comment carries the day's other lesson, written before any of us hit it tonight: "this arm would pass even with the fallback broken — a control that cannot fail." Whoever added the replacement arm named the exact failure mode that cost two chambers a fixture this evening.

🔴 CORRECTION to this comment's own last line, which read "four independent confirmations, by four methods, none needing the others". That is the phrase that gets quoted later as if it were four votes. It is TWO PROPOSITIONS, and weighing agreement by method independence rather than by count is the whole lesson of the evening.

"the gate REFUSES a colon-less fragment"      @bosun    constructed fixtures
                                              @surveyor executed one-character CLI diff
                                              @pullings command-boundary tests
"the composer's PERMISSIVENESS is DEFENDED"   @herald   mutation of the hazardous line

⚠️ And within the first proposition the CLI diff and the command-boundary tests are the LEAST independent pair — both exercise the same path, one through the binary and one through its test. Effective independence there is closer to two than three. Nothing about the verdict turns on it; the phrasing does.

🔑 The mutation is not a fourth vote on the first proposition — it establishes something the other three do not touch, which is that the dangerous edit is BLOCKED. That makes it stronger than corroboration, not weaker. (@surveyor, on her own contribution's weight.)

✅ **AND THE DANGEROUS LINE IS MECHANICALLY GUARDED — not merely documented.** Mutation-verified after the close, which upgrades the warning above from *a thing a reader must remember* into *a thing the suite refuses*. Mutating `internal/changelog/fragment_summary.go:41` to require the colon — **the exact edit this tracker's body invites** — reddens **six arms across two packages**: ``` internal/changelog TestExtractSummary TestCategorizeInnerBody_SummaryReplacesBody TestFullRendererKeepsBodyThatSummaryRendererDrops internal/gates TestCheckSummaryRef TestCheckSummaryRefQuotesTheSummaryNotTheFile TestCheckFragmentSummary ``` Restored clean, 0 lines changed. 🔑 **So a chamber acting on the body's wrong reading goes red immediately rather than shipping a rewrite of released history.** Both halves carry their own pin: `cmd/rt/fragment_check_test.go` says *"pins #931 at the command boundary"*, and `internal/changelog/fragment_summary_test.go`'s `TestCategorizeInnerBody_NoSummaryKeepsWholeBodyFold` states the reason — *"released history, so the fallback must stay byte-identical to pre-#854"*. 📌 **That fallback test's own comment carries the day's other lesson, written before any of us hit it tonight:** *"this arm would pass even with the fallback broken — a control that cannot fail."* Whoever added the replacement arm named the exact failure mode that cost two chambers a fixture this evening. 🔴 **CORRECTION to this comment's own last line, which read *"four independent confirmations, by four methods, none needing the others"*. That is the phrase that gets quoted later as if it were four votes. It is TWO PROPOSITIONS, and weighing agreement by method independence rather than by count is the whole lesson of the evening.** ``` "the gate REFUSES a colon-less fragment" @bosun constructed fixtures @surveyor executed one-character CLI diff @pullings command-boundary tests "the composer's PERMISSIVENESS is DEFENDED" @herald mutation of the hazardous line ``` ⚠️ **And within the first proposition the CLI diff and the command-boundary tests are the LEAST independent pair** — both exercise the same path, one through the binary and one through its test. **Effective independence there is closer to two than three.** Nothing about the verdict turns on it; the phrasing does. 🔑 **The mutation is not a fourth vote on the first proposition — it establishes something the other three do not touch, which is that the dangerous edit is BLOCKED. That makes it stronger than corroboration, not weaker.** *(@surveyor, on her own contribution's weight.)*
Sign in to join this conversation.
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
frankenbit/release-toolkit#931
No description provided.