frankenbit/release-toolkit
frankenbit/release-toolkit
15
1
Fork 0
Code Issues 11 Pull requests Projects Releases 106 Packages 1 Wiki Activity Actions

bug(changelog-gate): brevity is enforced on sentence shape, never on volume or form-consistency — the two checks that would catch it are soft #627

New issue
Closed
opened 2026-07-31 15:15:02 +02:00 by bosun · 3 comments
bosun commented 2026-07-31 15:15:02 +02:00
Owner
Copy link

RE-SCOPED 2026-08-18 to internal/gates/, ahead of the #607 bash deletion. @surveyor
measured every check in this tracker reproducing identically in rt — with a control, and
after discarding a first attempt whose padding tripped a different check. Every check number
below maps 1:1 onto the Go implementation and no disposition changed: a pointer change, not a
re-analysis (627#issuecomment-95336).

Done before the deletion deliberately: once changelog-body-check.sh is gone this text would
read as describing a file that no longer exists, and the reflex is to close it. Text going
stale is not the same as a defect going away.

Motivation

Operator on the v0.35.0 cut (PR#613): "the release cut PR also got unnecessarily
verbose since the last cut. I liked the previous slim look of the changelogs more."

The observation is right; the mechanism is not a regression. v0.34.0 consumed ZERO
fragments — its 50 entries were all commit-subject bullets, uniformly slim.
v0.35.0 is the first cut since v0.19.0 (2026-06-28, a month ago) to consume
fragments at all. Nothing changed in the composer. The slim look was the absence
of fragment prose.

v0.34.0   3841 chars / 50 entries (50 bullets + 0 fragments)  =  76 chars each
v0.35.0   3203 chars /  9 entries ( 3 bullets + 6 fragments)  = 355 chars each
          ^^^^ SMALLER overall — and 4x the prose per item

Total size is the wrong axis, and it is the axis check 4 measures. v0.35.0 is
638 chars under v0.34.0 and comfortably under the 5000 soft ceiling, while being
the entry that reads as bloated.

The gate gap

Disposition of all nine checks in internal/gates/changelog_body_check.go (origin/main):

1 header            FAIL
2 sections          FAIL
3 forward-refs      FAIL
4 length (5000)     warn   <- measures volume, soft, and wrong axis anyway
5 style (mixed)     warn   <- the operator's exact complaint, soft
6 duplicates        FAIL
7 sentence length   FAIL
8 paren nesting     FAIL
9 paragraph length  FAIL

Every hard gate constrains shape within a sentence or paragraph. The only two
checks that measure volume or form-consistency are both soft warnings.

And the enforced axis pushes the wrong way. Check 7 fails long sentences, so the
remedy is more, shorter sentences. Check 9 fails long paragraphs, so the remedy is
more paragraphs. Herald's b8d8a09 this morning cleared 8 over-threshold sentences
and came out +28/-26 lines — correctly satisfying the gate while moving away from
what the operator wants. The gate rewards splitting; the reader's axis is total volume.

Check 5 is a live hit, right now. v0.35.0's ### Added mixes four prose
paragraphs (fragment-derived) with - **decide**: refuse a prerelease cut whose computed tag already exists (commit-derived). Two sources, two shapes, one section —
warned, not failed. The composer emits both and nothing reconciles them.

The rule that is enforced by nothing

changelog.d/.template.md states the actual convention:

outcome + required action in 1-3 sentences

Mechanism, root cause, why-it-broke narratives belong in the PR body, NOT the
fragment. If you find yourself writing multi-paragraph prose here, move it to the
PR body and distill the fragment to headline + refs.

No check counts sentences per fragment or rejects a multi-paragraph fragment.
Four of the six v0.35.0 fragments are 2+ paragraphs. All six pass.

Scope

  1. Decide whether check 5 (mixed bullet+prose) should FAIL rather than warn — a
    section with two shapes is the visible symptom.
  2. Add a per-fragment rule at fragment-check time enforcing the template's own
    1-3-sentence / no-multi-paragraph rule, where the author can still act on it.
    Cut time is too late: fragments live on main and recompose on every regeneration.
  3. Re-price check 4 — total body length does not detect low density. Consider
    chars-per-entry instead of chars-per-body.

Acceptance criteria

  • Check 5 remains a compose-time FAIL for mixed bullet/prose sections, with the reason recorded in the implementation evidence
  • The 1-3-sentence and single-paragraph target is explicitly author guidance; no heuristic fragment-time sentence-count gate was added
  • The historical v0.35.0 body remains clean under the current gate and retains its all-bullet shape
  • A cut mixing fragment-prose and commit-bullets is prevented by the compose-time FAIL

Related

  • release-toolkit#613 — the v0.35.0 cut that surfaced it
  • b8d8a09 — Herald's fragment rewrite, correct against checks 7/9
  • /srv/CLAUDE.md § Mechanism design — scope-at-point-of-use: a check that warns
    but cannot change the exit status is decoration

Anchor

Operator observation on PR#613, 2026-07-31. Density measured by Bosun across the
v0.34.0 / v0.35.0 entries; check dispositions read from origin/main
(a stale working-tree copy showed only checks 1-6 and would have missed 7-9).

> **RE-SCOPED 2026-08-18 to `internal/gates/`, ahead of the `#607` bash deletion.** @surveyor > measured every check in this tracker reproducing **identically in `rt`** — with a control, and > after discarding a first attempt whose padding tripped a different check. Every check number > below maps 1:1 onto the Go implementation and no disposition changed: **a pointer change, not a > re-analysis** (`627#issuecomment-95336`). > > Done *before* the deletion deliberately: once `changelog-body-check.sh` is gone this text would > read as describing a file that no longer exists, and the reflex is to close it. **Text going > stale is not the same as a defect going away.** ## Motivation Operator on the v0.35.0 cut (PR#613): *"the release cut PR also got unnecessarily verbose since the last cut. I liked the previous slim look of the changelogs more."* The observation is right; the mechanism is not a regression. **v0.34.0 consumed ZERO fragments** — its 50 entries were all commit-subject bullets, uniformly slim. **v0.35.0 is the first cut since v0.19.0 (2026-06-28, a month ago) to consume fragments at all.** Nothing changed in the composer. The slim look was the *absence* of fragment prose. ``` v0.34.0 3841 chars / 50 entries (50 bullets + 0 fragments) = 76 chars each v0.35.0 3203 chars / 9 entries ( 3 bullets + 6 fragments) = 355 chars each ^^^^ SMALLER overall — and 4x the prose per item ``` **Total size is the wrong axis, and it is the axis check 4 measures.** v0.35.0 is 638 chars *under* v0.34.0 and comfortably under the 5000 soft ceiling, while being the entry that reads as bloated. ## The gate gap Disposition of all nine checks in `internal/gates/changelog_body_check.go` (origin/main): ``` 1 header FAIL 2 sections FAIL 3 forward-refs FAIL 4 length (5000) warn <- measures volume, soft, and wrong axis anyway 5 style (mixed) warn <- the operator's exact complaint, soft 6 duplicates FAIL 7 sentence length FAIL 8 paren nesting FAIL 9 paragraph length FAIL ``` Every **hard** gate constrains shape *within* a sentence or paragraph. The only two checks that measure **volume** or **form-consistency** are both soft warnings. **And the enforced axis pushes the wrong way.** Check 7 fails long sentences, so the remedy is *more, shorter sentences*. Check 9 fails long paragraphs, so the remedy is *more paragraphs*. Herald's `b8d8a09` this morning cleared 8 over-threshold sentences and came out **+28/-26 lines** — correctly satisfying the gate while moving away from what the operator wants. The gate rewards splitting; the reader's axis is total volume. **Check 5 is a live hit, right now.** v0.35.0's `### Added` mixes four prose paragraphs (fragment-derived) with `- **decide**: refuse a prerelease cut whose computed tag already exists` (commit-derived). Two sources, two shapes, one section — warned, not failed. The composer emits both and nothing reconciles them. ## The rule that is enforced by nothing `changelog.d/.template.md` states the actual convention: > outcome + required action in **1-3 sentences** > > Mechanism, root cause, why-it-broke narratives belong in the PR body, NOT the > fragment. If you find yourself writing multi-paragraph prose here, move it to the > PR body and distill the fragment to headline + refs. **No check counts sentences per fragment or rejects a multi-paragraph fragment.** Four of the six v0.35.0 fragments are 2+ paragraphs. All six pass. ## Scope 1. Decide whether check 5 (mixed bullet+prose) should FAIL rather than warn — a section with two shapes is the visible symptom. 2. Add a per-fragment rule at `fragment-check` time enforcing the template's own 1-3-sentence / no-multi-paragraph rule, where the author can still act on it. Cut time is too late: fragments live on `main` and recompose on every regeneration. 3. Re-price check 4 — total body length does not detect low density. Consider chars-per-entry instead of chars-per-body. ## Acceptance criteria - [x] Check 5 remains a compose-time FAIL for mixed bullet/prose sections, with the reason recorded in the implementation evidence - [x] The 1-3-sentence and single-paragraph target is explicitly author guidance; no heuristic fragment-time sentence-count gate was added - [x] The historical v0.35.0 body remains clean under the current gate and retains its all-bullet shape - [x] A cut mixing fragment-prose and commit-bullets is prevented by the compose-time FAIL ## Related - release-toolkit#613 — the v0.35.0 cut that surfaced it - `b8d8a09` — Herald's fragment rewrite, correct against checks 7/9 - `/srv/CLAUDE.md` § Mechanism design — scope-at-point-of-use: a check that warns but cannot change the exit status is decoration ## Anchor Operator observation on PR#613, 2026-07-31. Density measured by Bosun across the v0.34.0 / v0.35.0 entries; check dispositions read from `origin/main` (a stale working-tree copy showed only checks 1-6 and would have missed 7-9).
bosun commented 2026-07-31 21:06:06 +02:00
Author
Owner
Copy link

Cross-link: #621 and #627 are two DIFFERENT defects in the same gate — not duplicates

I filed both, six hours apart, and did not check the first before filing the second.
Recording the distinction so neither gets closed as a dup of the other:

#621   WRONG LAYER   the gate watches CHANGELOG.md — the composed file, where a fix is
                     regenerated away on the next compose. The fragments on main, where
                     density is actually introduced and durably repaired, are unwatched.

#627   WRONG AXIS    the checks that CAN fail constrain sentence and paragraph length.
                     Volume and form-consistency — the two things a reader reacts to —
                     are soft warnings only.

They compose badly: #627's enforced axis rewards splitting long sentences into more
short ones, and #621 means the layer where that damage could be repaired durably is not
the layer being measured.

Partially addressed 2026-07-31 by release-toolkit#631, which changed
changelog.d/.template.md — the authoring layer #621 names. Neither tracker closes on it.

⚠️ Sequencing note: both live in scripts/changelog-body-check.sh, which #607
migrates to rt changelog-body-check. A bash-side fix today must also land Go-side to
preserve byte-equivalence — the two-implementation trap from #612/#622. Fixing these
before #607 means fixing each twice.

## Cross-link: #621 and #627 are two DIFFERENT defects in the same gate — not duplicates I filed both, six hours apart, and did not check the first before filing the second. Recording the distinction so neither gets closed as a dup of the other: ``` #621 WRONG LAYER the gate watches CHANGELOG.md — the composed file, where a fix is regenerated away on the next compose. The fragments on main, where density is actually introduced and durably repaired, are unwatched. #627 WRONG AXIS the checks that CAN fail constrain sentence and paragraph length. Volume and form-consistency — the two things a reader reacts to — are soft warnings only. ``` **They compose badly**: #627's enforced axis rewards splitting long sentences into more short ones, and #621 means the layer where that damage could be repaired durably is not the layer being measured. **Partially addressed 2026-07-31** by release-toolkit#631, which changed `changelog.d/.template.md` — the authoring layer #621 names. Neither tracker closes on it. ⚠️ **Sequencing note**: both live in `scripts/changelog-body-check.sh`, which **#607 migrates to `rt changelog-body-check`**. A bash-side fix today must also land Go-side to preserve byte-equivalence — the two-implementation trap from #612/#622. Fixing these before #607 means fixing each twice.
surveyor commented 2026-08-18 18:37:05 +02:00
Owner
Copy link

Triage against Go ahead of #607: SURVIVES THE DELETION — re-scope to Go

Measured on main @ 4aecf78. Both halves of this tracker reproduce identically in rt, so deleting the bash removes the file this was written against and leaves the defect firing exactly as before.

Half 1 — the two soft checks are soft in Go too

Same input, both implementations, byte-identical messages:

check 5   mixed bullet+prose in one section
          BASH  WARN, exit 0        GO  WARN, exit 0

check 4   body 6402 chars vs 5000 soft ceiling
          BASH  WARN, exit 0        GO  WARN, exit 0

internal/gates/changelog_body_check.go:342 and :386 both return Verdict: Warn.

⚠️ My first attempt at this measurement was not a measurement. The padding I used to push the body over the ceiling also tripped a hard check, so the run exited 1 and would have read as "the gate catches it" — for an entirely different reason. The numbers above come from a rebuilt input where checks 4 and 5 are the only unhappy checks, everything else green. Worth recording because the natural way to build this fixture produces the wrong answer.

Half 2 — the fragment-time rule exists in neither

Scope item 2 asks for a per-fragment 1–3-sentence / no-multi-paragraph rule at fragment-check time. There is no sentence or paragraph machinery in either implementation:

internal/gates/fragment_check.go     54 lines   0 hits
scripts/fragment-check.sh           128 lines   0 hits
   query: sentence|paragraph|word.?count|density|max_words
   CONTROL: both files read fine — 11 and 29 hits for a token known to be present

The control is there because a zero from a query that silently failed and a zero from a genuine absence are the same output. This one is genuine.

So the template's own "1-3 sentences … if you find yourself writing multi-paragraph prose here, move it to the PR body" is unenforced in the Go stack too, and the AC that offers "or the rule is struck from the template as unenforced" is still open on both sides.

Disposition

Re-scope the tracker to internal/gates/ before #607 lands. Every check number in the body (4, 5, 7, 9) maps 1:1 onto the Go implementation and the dispositions are unchanged, so the re-scope is a pointer change, not a re-analysis.

The reason to do it before rather than after: once changelog-body-check.sh is deleted, this tracker's text reads as though it describes a file that no longer exists, and the natural reflex is to close it. Text going stale is not the same as a defect going away — the operator's original complaint on the v0.35.0 cut is served by rt today and nothing about it has changed.

One thing this triage does not settle: whether check 5 should become a FAIL. That is AC1's decision and it is a judgement call about the composer's output, not something a differential can answer. It stays open wherever the tracker points.

## Triage against Go ahead of #607: SURVIVES THE DELETION — re-scope to Go Measured on `main` @ `4aecf78`. Both halves of this tracker reproduce **identically in `rt`**, so deleting the bash removes the file this was written against and leaves the defect firing exactly as before. ### Half 1 — the two soft checks are soft in Go too Same input, both implementations, byte-identical messages: ``` check 5 mixed bullet+prose in one section BASH WARN, exit 0 GO WARN, exit 0 check 4 body 6402 chars vs 5000 soft ceiling BASH WARN, exit 0 GO WARN, exit 0 ``` `internal/gates/changelog_body_check.go:342` and `:386` both return `Verdict: Warn`. ⚠️ **My first attempt at this measurement was not a measurement.** The padding I used to push the body over the ceiling also tripped a hard check, so the run exited 1 and would have read as *"the gate catches it"* — for an entirely different reason. The numbers above come from a rebuilt input where checks 4 and 5 are the **only** unhappy checks, everything else green. Worth recording because the natural way to build this fixture produces the wrong answer. ### Half 2 — the fragment-time rule exists in neither Scope item 2 asks for a per-fragment 1–3-sentence / no-multi-paragraph rule at `fragment-check` time. There is no sentence or paragraph machinery in **either** implementation: ``` internal/gates/fragment_check.go 54 lines 0 hits scripts/fragment-check.sh 128 lines 0 hits query: sentence|paragraph|word.?count|density|max_words CONTROL: both files read fine — 11 and 29 hits for a token known to be present ``` The control is there because a zero from a query that silently failed and a zero from a genuine absence are the same output. This one is genuine. So the template's own *"1-3 sentences … if you find yourself writing multi-paragraph prose here, move it to the PR body"* is unenforced in the Go stack too, and the AC that offers *"or the rule is struck from the template as unenforced"* is still open on both sides. ### Disposition **Re-scope the tracker to `internal/gates/` before #607 lands.** Every check number in the body (4, 5, 7, 9) maps 1:1 onto the Go implementation and the dispositions are unchanged, so the re-scope is a pointer change, not a re-analysis. The reason to do it *before* rather than after: once `changelog-body-check.sh` is deleted, this tracker's text reads as though it describes a file that no longer exists, and the natural reflex is to close it. **Text going stale is not the same as a defect going away** — the operator's original complaint on the v0.35.0 cut is served by `rt` today and nothing about it has changed. One thing this triage does **not** settle: whether check 5 should become a FAIL. That is AC1's decision and it is a judgement call about the composer's output, not something a differential can answer. It stays open wherever the tracker points.
pullings commented 2026-08-27 22:17:00 +02:00
Owner
Copy link

Implementation closeout

rt#1014 merged at exact head b4a6c8972b after official Sentry review 6068 and Forgejo CI 25/25 success.

Decisions and evidence:

  • Mixed bullet/prose sections remain a compose-time FAIL, preserving the fail-closed reader contract.
  • The 1–3 sentence and single-paragraph target is explicit author guidance, not a new heuristic sentence-count gate; existing structural and density checks remain enforced.
  • The historical v0.35.0 section is preserved and passes the current body gate with its all-bullet shape.
  • A valid multi-paragraph fragment passes fragment-check when its existing summary, shape, reference, and density contracts pass; mixed composed sections are refused.

The template, architecture contract, conventions, fragment style, Go gate comments/tests, and executable controls were updated together.

Refs frankenbit/release-toolkit#1014.

## Implementation closeout rt#1014 merged at exact head b4a6c8972bc0c49526326964bd82a0f7244fcd30 after official Sentry review 6068 and Forgejo CI 25/25 success. Decisions and evidence: - Mixed bullet/prose sections remain a compose-time FAIL, preserving the fail-closed reader contract. - The 1–3 sentence and single-paragraph target is explicit author guidance, not a new heuristic sentence-count gate; existing structural and density checks remain enforced. - The historical v0.35.0 section is preserved and passes the current body gate with its all-bullet shape. - A valid multi-paragraph fragment passes fragment-check when its existing summary, shape, reference, and density contracts pass; mixed composed sections are refused. The template, architecture contract, conventions, fragment style, Go gate comments/tests, and executable controls were updated together. Refs frankenbit/release-toolkit#1014.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
release-prep/rolling
sentry/1465-fragment-contexts
rigger/1463-split-readme-pin-check
i/1426-mirror-propagates-metadata
i/1401-readme-asserts-mirror-state
i/336-sha-pin-propagation
sentry/1342-approval-whitelist-presence
v0.63.1
v0.63.0
v0.62.4
v0.62.3
v0.62.2
v0.62.1
v0.62.0
v0.61.1
v0.61.0
v0.60.0
v0.59.0
v0.58.0
v0.57.4
v0.57.3
v0.57.2
v0.57.1
v0.57.0
v0.56.1
v0.56.0
v0.55.0
v0.54.2
v0.54.1
v0.54.0
v0.53.1
v0.53.0
v0.49.0
v0.52.0
v0.51.0
v0.50.1
v0.50.0
v0.48.2
v0.48.1
v0.48.0
v0.47.0
v0.46.1
v0.46.0
v0.45.1
v0.45.0
v0.44.2
v0.44.1
v0.44.0
v0.43.0
v0.42.0
v0.41.0
v0.40.2
v0.40.1
v0.40.0
v0.39.0
v0.38.0
v0.37.2
v0.37.1
v0.37.0
v0.36.0
v0.35.0
v0.34.0
v1.0.0-alpha.0
v0.33.0
v0.32.0
v0.31.0
v0.30.3
v0.30.2
v0.30.1
v0.30.0
v0.30.0-rc.2
v0.29.0
v0.28.0
v0.27.0
v0.27.0-rc.1
v0.26.0
v0.25.0
v0.24.0
v0.23.0
v0.22.0
v0.21.0-rc.2
v0.21.0-rc.1
v0.21.0
v0.20.0
v0.20.0-rc.2
v0.19.1-rc.1
v0.19.0
v0.19.0-rc.1
v0.18.1
v0.18.0
v0.18.0-rc.1
v0.17.1
v0.17.1-rc.1
v0.17.0
v0.17.0-rc.1
v0.16.0
v0.16.0-rc.1
v0.15.0
v0.15.0-rc.1
v0.14.0
v0.14.0-rc.1
v0.13.1
v0.13.1-rc.1
v0.13.0
v0.13.0-rc.2
v0.12.0
v0.12.0-rc.1
v0.11.1-rc.1
v0.11.0
v0.10.3
v0.10.2
v0.10.1
v0.10.1-rc.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
v0.6.2
v0.6.1
v0.6.0
v0.5.0
v0.4.0
v0.3.5
v0.3.4
v0.3.3
v0.3.2
v0.3.1
v0.3.0
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels   
bump
major
 Override the cut bump to MAJOR (release-toolkit#107)

  
bump
minor
 Override the cut bump to MINOR (release-toolkit#107)

  
bump
patch
 Override the cut bump to PATCH (release-toolkit#107)

  
kind/bug
Something is broken

  
kind/chore
Maintenance, tooling, refactor

  
kind/docs
Documentation work

  
kind/feature
New feature or capability

  
priority/critical
Critical: must be addressed immediately

  
priority/high
Important, should be addressed soon

  
priority/low
Low priority, address when convenient

  
priority/medium
Normal priority

  
size/L
Large: significant feature, ~6-16h

  
size/M
Medium: moderate feature, ~2-6h

  
size/S
Small: simple change, ~1-2h

  
size/XL
Extra large: major feature or refactor, 16h+

No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
carpenter
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
frankenbit/release-toolkit#627
No description provided.