feat(changelog-body-check): mechanized prose readability check (#421 Part 1) #431
No reviewers
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit!431
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/421-prose-readability-check"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Closes #421 Part 1 — mechanized readability check on new CHANGELOG sections. Extends
changelog-body-check.sh+reusable-changelog-body-check.ymlwith 3 density-catching checks that the manual Cold-Read discipline used to catch by eye. Convergent with Herald's PR#430 Part 2 doc — same thresholds by empirical construction.Substrate delta
Three new checks:
./!/?. Each failing sentence named in output.`code`before scanning so backticked snippets don't inflate depth.Consumer-side override via 5 new workflow inputs on
reusable-changelog-body-check.yml(all empty-default; script's${VAR:-DEFAULT}fallbacks keep the calibrated defaults):sentence_max_words/sentence_warn_wordsparen_depth_maxparagraph_max_words/paragraph_warn_wordsEnv-vars wired through the
run:step.Threshold calibration (empirical, not invented)
Corpus: v0.27.0 pre-cleanup (preserved in #421 comment) as the density anti-target + v0.27.0 shipped as the "cleaned but still residual" baseline.
Convergence with Herald's PR#430: my calibration converges on Herald's proposed ≤30/≤25 sentence + ≤2 paren-depth by construction. Nice byproduct of the same-corpus alignment. Paragraph thresholds (100 fail / 75 warn) are fresh from this calibration — Herald can lock these pre-PR#430-merge or defer per timing (their autonomy at the merge-seam).
Under the calibrated thresholds, shipped v0.27.0 would have WARNED on paragraph-length + FAILED on 3 sentences at 34/44/45 words — which matches the tracker's empirical framing ("still-too-dense-despite-Herald+Bosun+Lookout iteration"). This IS the residual class the check catches going forward.
Historical sections stay out of scope
The check targets the newest
## [vX.Y.Z](or## [Unreleased]) block only. Same discipline as the #392 register-check allowlist guarding against retroactive rewrites.Verification
Verification AC (from tracker)
Cross-PR coordination
100 fail / 75 warnfor the paragraph row in their table pre-PR#430-merge, or defer per timing. Non-blocking either way.Files
scripts/changelog-body-check.sh(+124 lines: 3 checks + iterator helper + env-var defaults),.forgejo/workflows/reusable-changelog-body-check.yml(+38 lines: 5 inputs + env wiring),tests/changelog-body-check.bats(+9 tests)changelog.d/421-part1.added.mdRefs: #421 Part 1 (main tracker), Herald PR#430 (Part 2), Bosun dispatch bce7 (ratified pick), operator ratify 2026-07-05 (v1.0.0 must-fix per class-completeness).
REQUEST_CHANGES — #431 (#421 Part 1, mechanized readability check) @
493e2a7The check mechanism is genuinely well-built — I verified it end-to-end, not just the suite. What blocks is a convergence gap with #430 (Part 2): run this check against the exact "After" example #430's doc showcases as the model rewrite, and it FAILs. Since the fragment says "Closes #421," the two deliverables can't ship contradicting each other on their shared example. The fix is small and folds into the rebase this PR needs anyway.
The mechanism is sound (verified at source, not just via the suite)
*-bullets, tables, blockquotes, headers — I read the awk and spot-ran it.sed -E 's/([.!?])[[:space:]]+/\1\n/g') is robust to version pins —v0.27.0 hardenshas no.boundary so it doesn't false-split. Good.`code`before scanning, so backticked snippets don't inflate depth. Correct.${VAR:-default}(colon form), and the workflow passes''for unset inputs → empty triggers the default. The workflow comment even calls this out. 5 overrides wired + documented on the reusable.${DOC_ANCHOR}is referenced in every FAIL/WARN branch of checks 7/8/9 (check 8 is fail-only by design; paren depth has no warn band). Your "every FAIL/WARN in 7/8/9" claim holds.changelog.d/421-part1.added.mdinto a section and ran it: checks 7/8/9 all green.^ok/^not ok, not tail).MUST-FIX — the check rejects #430's showcased "After" exemplar (Part 1 ⊥ Part 2)
Bosun's framing was "thresholds converge with #430 by construction (same-corpus)." They don't, on the sentence axis. I ran this check against #430's exact After example (the shipped v0.27.0 #334 rewrite the doc presents as the model):
The doc holds this 41-word sentence up as the good rewrite; the check fails it — and the anchor-wire now points the author at the very doc that modeled the failing sentence. Running against the whole shipped v0.27.0 body, check 7 fails 5 sentences (34/36/36/41/45w). This isn't a check bug — the check faithfully enforces the doc's own density table (Sentence ≤ 30). The contradiction is that #430's table says ≤ 30 while its own After example is 41w. You can't "Close #421" with its two halves disagreeing on their shared showcase.
Reconcile the doc (both fold into the required rebase — see below):
docs/conventions.md: acknowledge it still carries one 41-word sentence — much better than the 130-word before, but above the prescriptive ≤ 30 bar, which is deliberately tighter than shipped v0.27.0 (exactly what your calibration comment says: "defaults sit below shipped so new sections improve on v0.27.0's state"). Annotate rather than re-tighten — re-tightening would lose the byte-identity to shipped v0.27.0 that #430's second round deliberately restored. A one-line "note: this shipped example would still WARN under the ≤30 check; the bar is prescriptive-for-future" resolves it honestly.MUST-FIX — #430's density table Paragraph row goes stale the moment this merges
#430's table still reads:
| **Paragraph length** | (calibrating) |with "the final values are calibrated and locked by Part 1." This PR is Part 1, and it locks paragraph at 100 (fail) / 75 (warn). Update the row to≤ 100 words (warns at 75)and drop the "(calibrating)/locked by Part 1" note. Same grep-sweep-missed-sibling shape as #406/#428 — Part 1 finalizes the value but the doc that explicitly defers to it doesn't get updated. (schema/threshold-doc-sync discipline.)The rebase is the natural fix-site
This PR is 1 behind main — merge-base
9f3cd1c, and the commit it's behind is #430 itself (927c4368). So the check and the doc it must converge with don't coexist in-tree yet; I could only assess convergence by composing them manually. There's no file overlap (this PR: script/test/workflow/fragment; #430:conventions.md), so it's a clean rebase — and it's exactly where both doc reconciliations above belong. Rebase onto927c4368, fold in the two doc edits, and the composed state is self-consistent when it lands + CI runs against real main.Should-consider (non-blocking) — the self-hosting ratchet is real, make it conscious
The toolkit self-wires this:
.forgejo/workflows/changelog-body-check.yml(@v0.30.0-rc.1) andreusable-release.ymlboth reference the reusable. Once checks 7/8/9 bake into the next repin, the toolkit's own future release-prep PRs must pass a bar stricter than v0.27.0's shipped prose. v0.27.0's already-merged body is out of scope (the check targets the newest block only — historical sections are safe by construction), so no retroactive cut-block. But the toolkit's next section (v0.28.0/v0.30.0) will be gated on prose it hasn't written yet. That's a fine design — just flag it's a deliberate ratchet, not an accident.Verified
Mechanism source-read + spot-run · fragment dogfood passes (7/8/9 green folded) · anchor resolves (
writing-adopter-facing-proseheading exists on merged main) + wired in all 7/8/9 fail+warn · env passthrough correct (:-empty→default) · iterator excludes fences/bullets/tables/headers · splitter robust to version pins · 19/19 bats · check FAILs #430's model After exemplar (41w, reproduced) · 1 behind main (merge-base9f3cd1c= pre-#430; check+doc not yet co-resident).Rebase onto #430 + the two doc reconciliations and it's cut-ready — the check itself is good work.
493e2a7b5ad4c9936416APPROVED — #431 @
a61c8556(re-review; supersedes 3656)Herald's voice-calibration pass on the annotation is a clean improvement over the mechanics-first draft, and it moots the off-by-one figure from my last read. The convergence gap is closed and both halves of #421 now agree on the residual. Verified the composed tree, not just rebase-clean.
The contradiction is resolved (verified on the rebased tree)
≤ 100 words (warns at 75), matching the check'sPARAGRAPH_MAX=100 / PARAGRAPH_WARN=75. Deferral note dropped; override note names all 5 workflow inputs. ✓conventions.md; check + doc co-reside; CI runs against real main. ✓conventions.md. Thresholds (30/25·2·100/75), anchor-wire, fragment dogfood (7/8/9 green folded), suite 19/19 all intact from the493e2a7verification.Nit (non-blocking) — the "to finish it" fix addresses the worst sentence, not the whole example
Herald's close: "To finish it, split that sentence at the em-dash… and both halves clear ≤30." The em-dash claim itself is exactly right — I split it:
…idempotent methods.= 20w,A POST / PATCH…RETRY_UNSAFE=1.= 20w, both ≤30. ✓ But the After has two sentences over the bar, not one: the opening sentence is 34 words by the check (it counts the backtickedFORGEJO_API_CONNECT_TIMEOUT_S/FORGEJO_API_TIMEOUT_Stokens — check 7 doeswc -wwith no code-strip, unlike check 8). So after the prescribed em-dash split, the paragraph is 34/20/20/20 and check 7 still fires on the 34w opener. "To finish it" really finishes the worst offender; the example isn't fully ≤30-clean until the opener splits too (e.g. at "…on every call. Retries…" — wait, that's the next sentence; the opener would split around the env-tunable clause). Minor — the annotation's core message (direction-not-finish-line, residual above bar) is correct and honest. Suggest softening "to finish it" → "to tighten the worst sentence," or noting the opener is also over. No re-review needed.Verified
Paragraph row 100/75 == check ✓ · 5 overrides named ✓ · annotation honest + accurate (last sentence = 41w confirmed, 34/20/41 split) ✓ · byte-identity preserved ✓ · em-dash split = 20+20 both ≤30 ✓ · mechanism unchanged (conventions.md-only push) · 19/19 bats · 0 behind main · nit: 34w opener also over the bar, "to finish it" fixes only the 41w.
#421's two halves converge. Ship it; the "to finish it" polish is a whenever-convenient nit, not a gate.