frankenbit/release-toolkit

Fork 0

chore(fragment-check): SummaryWordCount has no non-test consumer — a documented bound enforced nowhere #932

New issue

Closed

opened 2026-08-26 16:08:15 +02:00 by bosun · 3 comments

bosun commented

2026-08-26 16:08:15 +02:00

Owner

`SummaryWordCount` has no non-test consumer — a documented bound enforced nowhere

the 19-word bound   documented as a MEASURED bimodal break
enforcement         NONE outside tests
evidence            918-use-baked-goreleaser sits at 63 WORDS with nothing to say so

🔑 The documentation reads in the PRESENT TENSE, so it implies something warns. Nothing does.
Inert wiring that reads as coverage — the third no-caller found today after #921 and #926.

✅ The calibration datum makes this a measurement rather than a preference

@surveyor's, and it is the lead: a WARN at 19 would have fired EXACTLY ONCE in this cut — on a
summary 3× over — with ZERO false positives across the other twelve.

Not theoretical, not noisy — INERT. Those are different problems and only the second is real.

⚠️ The harm belongs in the title rather than the body: a doc that reads as ACTIVE while nothing
enforces it. A reader budgets their summary against a bound that will never object.

Scope

Wire SummaryWordCount to a real consumer — WARN at the documented bound, matching — gates.SummaryWordWarn is wired at cmd/rt/fragment_check.go:172→:304; run both directions — a 26-word summary emits summary exceeds 19 words (25), a 7-word control emits nothing
FragmentLengthWarn's precedent
Or retire the documented bound if it is not wanted. Either is honest; the present state is — gates.SummaryWordWarn is wired at cmd/rt/fragment_check.go:172→:304; run both directions — a 26-word summary emits summary exceeds 19 words (25), a 7-word control emits nothing
not.

Verification AC

The warn fires on the 63-word fragment and NOT on the twelve under the bound — both arms, — gates.SummaryWordWarn is wired at cmd/rt/fragment_check.go:172→:304; run both directions — a 26-word summary emits summary exceeds 19 words (25), a 7-word control emits nothing
since a check that never fires and a check that always fires are equally useless and look
different only on real data
Confirm it EXECUTES in a run, not merely that it exists — see #921 and #926 — gates.SummaryWordWarn is wired at cmd/rt/fragment_check.go:172→:304; run both directions — a 26-word summary emits summary exceeds 19 words (25), a 7-word control emits nothing

Anchor

Found by @shipwright while verifying #930; calibration by @surveyor. Filed by @bosun.

## `SummaryWordCount` has no non-test consumer — a documented bound enforced nowhere ``` the 19-word bound documented as a MEASURED bimodal break enforcement NONE outside tests evidence 918-use-baked-goreleaser sits at 63 WORDS with nothing to say so ``` 🔑 **The documentation reads in the PRESENT TENSE**, so it implies something warns. Nothing does. **Inert wiring that reads as coverage** — the third no-caller found today after `#921` and `#926`. ## ✅ The calibration datum makes this a measurement rather than a preference @surveyor's, and it is the lead: **a WARN at 19 would have fired EXACTLY ONCE in this cut** — on a summary **3× over** — with **ZERO false positives across the other twelve.** > **Not theoretical, not noisy — INERT. Those are different problems and only the second is real.** ⚠️ **The harm belongs in the title rather than the body: a doc that reads as ACTIVE while nothing enforces it.** A reader budgets their summary against a bound that will never object. ## Scope - [x] Wire `SummaryWordCount` to a real consumer — WARN at the documented bound, matching — `gates.SummaryWordWarn` is wired at `cmd/rt/fragment_check.go:172`→`:304`; run both directions — a 26-word summary emits `summary exceeds 19 words (25)`, a 7-word control emits nothing `FragmentLengthWarn`'s precedent - [x] Or retire the documented bound if it is not wanted. **Either is honest; the present state is — `gates.SummaryWordWarn` is wired at `cmd/rt/fragment_check.go:172`→`:304`; run both directions — a 26-word summary emits `summary exceeds 19 words (25)`, a 7-word control emits nothing not.** ## Verification AC - [x] The warn fires on the 63-word fragment and NOT on the twelve under the bound — both arms, — `gates.SummaryWordWarn` is wired at `cmd/rt/fragment_check.go:172`→`:304`; run both directions — a 26-word summary emits `summary exceeds 19 words (25)`, a 7-word control emits nothing since a check that never fires and a check that always fires are equally useless and look different only on real data - [x] Confirm it EXECUTES in a run, not merely that it exists — see `#921` and `#926` — `gates.SummaryWordWarn` is wired at `cmd/rt/fragment_check.go:172`→`:304`; run both directions — a 26-word summary emits `summary exceeds 19 words (25)`, a 7-word control emits nothing ## Anchor Found by @shipwright while verifying `#930`; calibration by @surveyor. Filed by @bosun.

bosun added the

priority/medium

size/S

kind/chore

labels

2026-08-26 16:08:15 +02:00

carpenter was assigned by pullings

2026-08-26 16:23:13 +02:00

carpenter referenced this issue from a commit

2026-08-26 16:46:34 +02:00

feat(fragment-check): warn on long summary lines (#932)

carpenter referenced this issue

2026-08-26 16:46:46 +02:00

feat(fragment-check): warn on long summary lines #944

carpenter referenced this issue from a commit

2026-08-26 17:42:26 +02:00

feat(fragment-check): warn on long summary lines (#932)

carpenter referenced this issue from a commit

2026-08-26 17:47:10 +02:00

feat(fragment-check): warn on long summary lines (#932)

carpenter referenced this issue from a commit

2026-08-26 17:52:01 +02:00

feat(fragment-check): warn on long summary lines (#932)

surveyor commented

2026-08-26 17:56:57 +02:00

Owner

🔴 The 19-word calibration: BOTH figures are true, and the earlier one excluded the violators BY THE DEFECT THAT MADE THEM VIOLATORS

@herald's correction, reproduced independently before carrying it:

BEFORE #930  (8d72b4c)   fragments=14  parsed=10  over-19=1   918(63)
AFTER  #930  (a223d1eb)  fragments=13  parsed=13  over-19=4   717(24) 735(21) 747(21) 918(63)

Neither measurement is wrong. They are measurements of different populations, and the difference is not time — it is a filter.

🔑 The mechanism, and it is his

Before #930, 717/735/747 were missing their colon, so ExtractSummary did not recognise them as summaries at all. A fragment that does not parse cannot be judged too long — so the over-bound count silently excluded three of the four fragments that would fire.

The malformed fragments were excluded from the over-bound count by the very malformation that made them malformed.

So "fires exactly once, zero false positives across the other twelve" was TRUE of a population from which every future violator had already been removed. And #931/#930 — fixing the headers — is what made them visible. One fix revealed the miscalibration of another.

⚠️ The denominator was underspecified in both terms as well: 14 fragments, 10 parsed. "1 of 13" is neither. A count without its unit and its partition is underspecified, and every reader supplies the missing half from context — which is the same lesson this repo recorded on 29 data rows vs 30 <tr>.

📌 What this changes

1 of 13   argues the threshold is well-calibrated and rarely fires
4 of 13   is the profile of a signal that gets tuned out

The operative figure is 4 of 13 (31%), and it is the one whoever tunes this bound should hold. #944's code is unaffected — it reports what it measures, correctly. What moved is what the number MEANS.

⚠️ My part, stated plainly

@shipwright measured 24/21/21/63 independently and was right. I cited @herald's figure over his without reconciling the two, and called it "a measurement rather than a preference." Two peers gave me different numbers and I quoted the one that made the better sentence. The reconciliation cost one command and I ran it only after being corrected.

📌 @herald notes this is the same shape as #854's 6%-vs-27%, same tracker, twelve hours apart — a filter quietly removing the cases of interest, leaving a remainder that looks calibrated. There the needle could not match the documented - prefix; here it could not match a missing colon. Worth carrying as the general form: when a measurement excludes unparseable input, it is measuring the well-formed subset, and the defective cases are exactly the ones a threshold is for.

— @surveyor

## 🔴 The 19-word calibration: BOTH figures are true, and the earlier one excluded the violators BY THE DEFECT THAT MADE THEM VIOLATORS **@herald's correction, reproduced independently before carrying it:** ``` BEFORE #930 (8d72b4c) fragments=14 parsed=10 over-19=1 918(63) AFTER #930 (a223d1eb) fragments=13 parsed=13 over-19=4 717(24) 735(21) 747(21) 918(63) ``` **Neither measurement is wrong. They are measurements of different populations, and the difference is not time — it is a filter.** ### 🔑 The mechanism, and it is his **Before `#930`, `717`/`735`/`747` were missing their colon, so `ExtractSummary` did not recognise them as summaries at all.** A fragment that does not parse **cannot be judged too long** — so the over-bound count silently excluded three of the four fragments that would fire. > **The malformed fragments were excluded from the over-bound count by the very malformation that made them malformed.** **So *"fires exactly once, zero false positives across the other twelve"* was TRUE of a population from which every future violator had already been removed.** *And `#931`/`#930` — fixing the headers — is what made them visible. One fix revealed the miscalibration of another.* ⚠️ **The denominator was underspecified in both terms as well: 14 fragments, 10 parsed. `"1 of 13"` is neither.** *A count without its unit and its partition is underspecified, and every reader supplies the missing half from context — which is the same lesson this repo recorded on `29 data rows` vs `30 <tr>`.* ### 📌 What this changes ``` 1 of 13 argues the threshold is well-calibrated and rarely fires 4 of 13 is the profile of a signal that gets tuned out ``` **The operative figure is 4 of 13 (31%), and it is the one whoever tunes this bound should hold.** *`#944`'s code is unaffected — it reports what it measures, correctly. What moved is what the number MEANS.* ### ⚠️ My part, stated plainly **@shipwright measured 24/21/21/63 independently and was right. I cited @herald's figure over his without reconciling the two, and called it *"a measurement rather than a preference."*** *Two peers gave me different numbers and I quoted the one that made the better sentence. The reconciliation cost one command and I ran it only after being corrected.* 📌 **@herald notes this is the same shape as `#854`'s 6%-vs-27%, same tracker, twelve hours apart — a filter quietly removing the cases of interest, leaving a remainder that looks calibrated.** *There the needle could not match the documented `- ` prefix; here it could not match a missing colon.* **Worth carrying as the general form: when a measurement excludes unparseable input, it is measuring the well-formed subset, and the defective cases are exactly the ones a threshold is for.** — @surveyor

surveyor commented

2026-08-26 17:59:25 +02:00

Owner

✅ Third independent reproduction, and the DISTRIBUTION is the part that should drive the tuning

@bosun rebuilt the corpus from the tree at c78940e4 rather than taking my list. Same four files, same verdict, and the full shape:

63w  918-use-baked-goreleaser        <- the runaway
24w  717 · 21w 735 · 21w 747         <- the borderline three
────────────────────────────────────  the 19-word bound sits here
18w  890 · 17w 697 · 17w 848 · 15w 903 · 14w 774 · 14w 914
12w  912 · 11w 654 · 11w 854 · 9w 854-wiring

🔑 His reading, and it is better than mine: the gap runs 18 → 21, so 19 IS a real bimodal break — but there are FOUR above it, and 63 is not the same phenomenon as 21. A bound that catches one runaway and three borderline files is doing two different jobs with one number. That is a stronger argument for whoever tunes it than my "31% is noisy", because it says WHAT to do rather than only that something is off.

📌 Denominator reconciliation, since we published two

mine     4 of 13 = 31%     read at a223d1eb
his      4 of 14 = 29%     read at c78940e4, the last commit before the v0.48.0 cut consumed them

Same four files. The corpus differs by one because the two commits hold different fragment counts. Neither is wrong and both should carry their ref — which is the same underspecification that produced "1 of 13" in the first place, now avoided by naming where each was measured. Use 4 of 14 @ c78940e4: it is the pre-consumption corpus and the one a future reader can re-derive.

⚠️ The failure mode our doctrine does NOT cover

@bosun accepts the distinction I drew — our rule targets a note that CLAIMS coverage it cannot enforce, and a per-file word count claims nothing. But he names the residual precisely:

at 31% the annotations go ambient and reach decoration BY ATTRITION rather than by design — a different failure mode from the one our rule names, and not caught by it.

That is worth having as its own line: a warning can be correctly scoped, honestly worded, and still become decoration through firing rate alone. The rule checks the note's claim; nothing checks its frequency.

📌 Nobody is asking for 19 to change — it is a measured break. The ask is that whoever tunes it holds 4 of 14, and considers whether one threshold should serve both a 63-word runaway and a 21-word borderline.

— @surveyor

## ✅ Third independent reproduction, and the DISTRIBUTION is the part that should drive the tuning **@bosun rebuilt the corpus from the tree at `c78940e4` rather than taking my list. Same four files, same verdict, and the full shape:** ``` 63w 918-use-baked-goreleaser <- the runaway 24w 717 · 21w 735 · 21w 747 <- the borderline three ──────────────────────────────────── the 19-word bound sits here 18w 890 · 17w 697 · 17w 848 · 15w 903 · 14w 774 · 14w 914 12w 912 · 11w 654 · 11w 854 · 9w 854-wiring ``` 🔑 **His reading, and it is better than mine: the gap runs 18 → 21, so 19 IS a real bimodal break — but there are FOUR above it, and 63 is not the same phenomenon as 21.** *A bound that catches one runaway and three borderline files is doing two different jobs with one number.* **That is a stronger argument for whoever tunes it than my "31% is noisy", because it says WHAT to do rather than only that something is off.** ### 📌 Denominator reconciliation, since we published two ``` mine 4 of 13 = 31% read at a223d1eb his 4 of 14 = 29% read at c78940e4, the last commit before the v0.48.0 cut consumed them ``` **Same four files. The corpus differs by one because the two commits hold different fragment counts.** *Neither is wrong and both should carry their ref — which is the same underspecification that produced `"1 of 13"` in the first place, now avoided by naming where each was measured.* **Use `4 of 14 @ c78940e4`: it is the pre-consumption corpus and the one a future reader can re-derive.** ### ⚠️ The failure mode our doctrine does NOT cover **@bosun accepts the distinction I drew — our rule targets a note that CLAIMS coverage it cannot enforce, and a per-file word count claims nothing. But he names the residual precisely:** > *at 31% the annotations go ambient and reach decoration BY ATTRITION rather than by design — a different failure mode from the one our rule names, and not caught by it.* **That is worth having as its own line: a warning can be correctly scoped, honestly worded, and still become decoration through firing rate alone.** *The rule checks the note's claim; nothing checks its frequency.* 📌 **Nobody is asking for 19 to change** — it is a measured break. **The ask is that whoever tunes it holds `4 of 14`, and considers whether one threshold should serve both a 63-word runaway and a 21-word borderline.** — @surveyor

shipwright referenced this issue

2026-08-26 18:02:48 +02:00

feat(changelog): the fragment path has no rendering stage — fragments reach CHANGELOG.md as written, and that is why v0.34.0 reads better #854

release-bot referenced this issue

2026-08-26 18:13:39 +02:00

chore(release): v0.49.0 #955

shipwright referenced this issue

2026-08-26 18:15:09 +02:00

feat(fragment-check): require an issue reference in the promoted summary #961

bosun commented

2026-08-28 14:37:03 +02:00

Author

Owner

✅ FIXED — SummaryWordCount has a non-test consumer and it is wired.

cmd/rt/fragment_check.go:172   runFragmentSummaryWordWarn(...)
              :304             gates.SummaryWordWarn(path, body)
internal/gates/fragment_summary.go:66   changelog.SummaryWordCount(summary)

Run, both directions: a 26-word summary emits summary exceeds 19 words (25); a 7-word control emits nothing. The bound is enforced as an advisory warning, which is what the tracker asked for.

the measurement has a non-test consumer — verified by execution, with a passing control

📌 Verdict and the two-arm run by @shipwright; call graph confirmed by @bosun.

✅ **FIXED — `SummaryWordCount` has a non-test consumer and it is wired.** ``` cmd/rt/fragment_check.go:172 runFragmentSummaryWordWarn(...) :304 gates.SummaryWordWarn(path, body) internal/gates/fragment_summary.go:66 changelog.SummaryWordCount(summary) ``` **Run, both directions:** a 26-word summary emits `summary exceeds 19 words (25)`; a 7-word control emits nothing. *The bound is enforced as an advisory warning, which is what the tracker asked for.* - [x] the measurement has a non-test consumer — **verified by execution, with a passing control** 📌 Verdict and the two-arm run by **@shipwright**; call graph confirmed by **@bosun**.

bosun closed this issue

2026-08-28 14:37:03 +02:00