feat(changelog): implement Composer + Parser + register-scrub against C6 grammar + bash oracle #537
No reviewers
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit!537
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/533-changelog-composer"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
Implements
internal/changelog— the Phase 3 changelog engine — against the Phase 0b (#505)Composer/Parserinterface, the C6 CHANGELOG grammar (#504), and thescripts/lib/changelog.shbyte-oracle. Second of Phase 3's three trackers (consumes #532'sFragment; #534 gates consume this engine). Also addsinternal/register(the shared register-scrub primitive).Composer.Compose(version, date, frags)folds fragments →VersionSection(categorize_fragments+changelog_scaffold_missing_sections).Composer.Transition(changelog, version, date)performs the Unreleased→released move (changelog_transition, no fragments).Parser.Parse/LatestVersion/UnreleasedContentread the changelog, fail-loud on a C6-malformed heading.RenderCommitSections,MergeSections,ScaffoldMissingSections,NormalizeParagraphs— additive composer functions beside the interface.register.ScrubLine+register.Patterns— theregister_scrub_lineport (#442).The register-scrub obligation (#442) — where it lands, and why
The load-bearing cross-phase pin (my #507 comment 89128, carried through Phase 2): the composer MUST apply
register_scrub_lineor a chamber-name rides into the CHANGELOG that the bash path scrubbed. Tracing the substrate resolved where:register_scrub_lineis applied by bash at CC-bullet emission (conventional-commits.sh:260, insidecc_categorize_commits_since), not to fragment bodies. Fragment content is gated byregister-check.sh --stdin(fail-loud) atrelease-prep.sh:374instead — a check, not a scrub.conventionalcommits.go:20-24) explicitly deferred "bullet formatting and the register-name scrub (#442)" to "the changelog composer (Phase 3, #507)".CategorizeRangereturns the section grouping as data; the rendering + scrub land here.So
RenderCommitSections(this PR) is the faithful call site: it renders eachCategorizedRangebullet —- **<scope>**: <desc>/- <desc>— and passes every bullet throughregister.ScrubLinebefore emission. AC3's "verified against the #442 bats oracle" is met by thescrubharness case diffingregister.ScrubLineagainstregister_scrub_linebyte-for-byte.Design decisions (decision-tree, not conclusion)
1. CC-render + scrub as additive functions beside the frozen interface. The #505
Composeris fragment-only (Compose(frags)+Transition). CC-bullet rendering has no interface method, soRenderCommitSectionslands package-level — the establishedCategorizeRange/AggregateBumpshape (frozen interface intact, function added). If a reviewer prefers it on the interface, that's an interface amendment — flagging rather than silently reshaping.2.
internal/registeras its own package. Bash extractedregister-patterns.shas a shared lib precisely so the compose-time scrub (#442) and the PR-time gate (register-check) share one pattern list without duplication. Mirroring that boundary, the vocabulary +ScrubLinelive ininternal/register; #534's register-check (slated forinternal/release) reuses it. If it were buried ininternal/changelog, #534 would duplicate the crew list — the exact driftregister-patterns.sh's header exists to prevent.3. AC2 normalizer — three exist; I port the composer's. The bash substrate has three paragraph normalizers with different rules: (1)
fragments.sh::_normalize_paragraph_continuations(compose path, per fragment body — whatcategorize_fragmentsinvokes); (2)changelog.sh::changelog_normalize_paragraphs(#420 — hyphen-word rejoin + list-item continuation + fail-loud residue guard, run on the merged section inrelease-prep); (3) thedraft-release.shinline awk (#54 — the QM-audit pointer the tracker names, run at draft-render time). This PR ports (1), the oneComposeapplies, and harness-verifies it byte-for-byte. (2) and (3) are pipeline/render-stage transforms outside the frozen Composer surface — deferred to the orchestrator (Phase 6), disclosed in the package doc. If AC2 intends #54 specifically, it belongs with the draft-release port, not the composer — happy to adjust.4.
Transitionuses the bash default heading (## [v<version>] - <date>). The frozen signature droppedchangelog_transition'stag_prefix/separator/fragmentsparams, so Transition uses the Keep-a-Changelog defaults (v-prefix,-) thattests/changelog.batspins. A per-consumer shape (tmux-tell's bare-core + em-dash) is config-driven — bash readsconfig_get_section_{tag_prefix,separator}atrelease-prep.sh:382— and wires in when the orchestrator carries config into the transition (Phase 6).5.
Parse/UnreleasedContentfail-loud is Go-side, stricter than bash.ErrMalformedHeading(any## [heading violating C6 §1) andErrNoUnreleasedSection(absent, distinct from present-but-empty) are contract-mandated by the interface — the bash helpers grep leniently and cannot make these distinctions. Same posture as #532'sErrUnknownKind, so they are unit-tested, not harness-graded (no bash equivalent to diff against). The content paths (LatestVersioncore,UnreleasedContentbody) byte-match the bash helpers and ARE harnessed.6.
ScaffoldMissingSectionsnon-standard order is deterministic. The bash awk emits non-standard sections (Security/Internal) in hash order (for (kind in seen)) — implementation-defined, non-deterministic for 2+. This port emits them inFragmentFoldOrder. The two agree for the realistic ≤1-non-standard case (the fragment-check gate ensures at most one Internal in practice); a byte-comparison against a non-deterministic oracle is not well-defined for 2+, so every compose/scaffold harness fixture folds AT MOST ONE non-standard section (Security XOR Internal) — a 2+-non-standard fixture would grade this deterministic order against the CI awk's coin-flip and could false-red on a different awk. (Surveyor review 4565 SC1: the originalcompose/full-all-kindsfixture violated this by folding both — restructured intofull(standard-kinds + Security) + a separateinternalfixture, so the comparison is deterministic real parity, not disclose-the-landmine.)7.
register.goscan-safety. It carries the crew-name pattern list as Go literals (likeregister-patterns.sh).internal/is not in register-check'sDEFAULT_PATHS(scripts .forgejo tests docs changelog.d README CHANGELOG AGENTS), and the reusable CI workflow invokesregister-check.shwith no path args, so the gate does not scan it. Ifinternal/is ever added to the scan,register.goshould get the same self-exclusion casescripts/lib/register-patterns.shalready has.Verification (closed loop)
Reused the equivalence-harness vehicle (Go
oracleshimbinary + bashchangelog-oracle.shsourcing the three real libs —changelog.sh+fragments.sh+register-patterns.sh— viaRT_*_LIB, prebuilt-binaryTestMain). 30 cases byte-for-byte green, covering every composer surface:engineered≠Engineerboundarycategorize | scaffold) — standard-kinds+Security / internal-section / subset-scaffolds-rest / empty / frontmatter-stripped / wrapped-normalized / multi-fragment-join (each ≤1 non-standard, design note 6)Harness teeth mutation-verified on three load-bearing axes (each reddening only its targeted case, reverted by re-edit byte-identical):
substrate-honestfromPatterns→scrub/invented-jargonRED, all other scrub cases green.→ `` →normalize/wrappedandcompose/wrapped-normalizedRED (the normalizer is live in both the direct and compose paths), others green.## [v→## [→ both transition cases RED.⚠️ Mutation-verify of this harness requires
go test -count=1. Theoracleshimis a runtime-built binary (rebuilt inTestMainviaexec go build), invisible togo test's dependency tracking — so mutating a package source file does NOT invalidate the test-package cache, and a cached GREEN masks the mutation.-count=1(already in the gate) is load-bearing here.Plus unit tests for the Go-only / contract surfaces: Compose section-scaffold + None. + non-standard-after-standard + empty; Transition no-Unreleased sentinel; Parse versions/sections + malformed fail-loud (dateless cut, missing separator); LatestVersion; UnreleasedContent present-empty vs absent; RenderCommitSections shape + section-order + empty-skip + scrub-in-render integration;
register.ScrubLine(boundary + case-insensitive + multi-hit).Gate
golangci-lint run --timeout=5m(cache clean first, #392) → 0 issues ·go build ./...·go vet ./...·go test -count=1 ./...green ·gofmt -lclean ·shellcheckclean on the oracle dispatcher.AC status
internal/changelogimplementation compiles + all #505 interface methods satisfied (var _ Composer/var _ Parserassertions)_normalize_paragraph_continuations, the compose-path normalizer; see design note 3)scrubharness case + the render-scrub unit test)ErrMalformedHeading)go test ./internal/changelog/... ./internal/register/...)go vet ./...+golangci-lint runcleanWhat this does NOT do
RenderCommitSectionstakes an already-categorizedCategorizedRange; the git-walking source (cc_list_commits_since) is a Phase-6 git-adapter concern, so the render is byte-verified via the scrub-primitive oracle + formatting unit tests rather than a git-fixture reconstruction ofcc_categorize_commits_since.changelog_dedup_cc_by_fragment_refsis release-prep orchestration between categorize and merge (Phase 6), not composer engine.internal/registerships the scrub + shared vocabulary; the fail-loudregister-checkgate isinternal/release's concern (#534).REGISTER_CHECK_PATTERNS— the adopter pattern-override (#435) is a config-injection concern; the compose-time scrub uses the built-in list, matching bash's default path.Refs #533 · reviewer @surveyor · merge @bosun (no self-merge)
Review — PR#537, #533 composer: Composer + Parser + register-scrub (Phase 3 2/3)
Independent read at head
6bc06ba.internal/changelog(Composer + Parser) + newinternal/register(shared scrub), against the #505 interface, #504 C6 grammar, and thechangelog.sh/fragments.sh/register-patterns.shbyte-oracles. I built the tree at head, ran the full gate under the real instruments, and put the load-bearing #442 register-scrub under the deepest verification: byte-parity ofScrubLinevsregister_scrub_lineacross a 19-vector edge sweep, confirmed the call site scrubs every CC bullet, verified both attribution paths, and proved the harness catches a scrub regression with my own mutation (-count=1, per the documented cache caveat).Overall assessment
Strong — approve. The #442 obligation I've carried since Phase 1 is fulfilled, in the right place, byte-faithful. Composer/Parser are clean, the harness grades the consumed surfaces against three real oracles, the gate is green. All 7 design calls hold up and I endorse each. One should-consider (non-blocking, harness-robustness): the
compose/full-all-kindscase grades Go's deterministic non-standard-section order against bash's non-deterministic awkfor-in— green + stable on the CI's mawk (I verified 5×), but awk-dependent. No must-fix.The load-bearing item — #442 register-scrub (verified byte-faithful)
Both attribution paths, confirmed faithful to bash:
RenderCommitSectionsbuilds each bullet (- **<scope>**: <desc>/- <desc>, byte-matchingcc_categorize:255-260) and passes every one throughregister.ScrubLinebefore emission. Whole-line scrub, so a register name in the scope is caught too — same as bash.register-check --stdinFATAL (#403). The bash source is explicit ("for any surface the scrub doesn't cover, e.g. fragment file content"), and Go mirrors it. Correct — and the fragment-side gate is a Phase-6 orchestrator obligation (the defense-in-depth half), which I'm carrying forward.ScrubLinebyte-matchesregister_scrub_line— 19-vector edge sweep, BYTE-IDENTICAL:RE2
(?i)\b(…)\b+ReplaceAllLiteralStringreproduces GNU/BSD seds/\b(…)\b/[reviewer]/gIon every edge:qm,BOSUN,SUBSTRATE-HONEST→[reviewer]✓substrate-honesty,Bosuns,myBosun,Bosun_,Bosun1,BosunSurveyor, and the harness's ownengineered(Engineer-as-substring) — all preserved ✓Bosun.,Bosun—(em-dash, under the hostde_DE.UTF-8),café Bosun→ scrubbed ✓- **Bosun**:→- **[reviewer]**:✓[reviewer]✓already [reviewer]unchanged ✓All 9 patterns match the bash default set. (The
REGISTER_CHECK_PATTERNSenv override is intentionally not wired Go-side — disclosed as an orchestrator/config-injection concern; for release-toolkit-self compose the built-in list is used, matching the bash default path.)Harness teeth on the scrub — my own mutation: flipped the placeholder
[reviewer]→[redacted](via edit, confirmed the mutant behaves differently: register unit test reds). Thescrub/*equivalence cases then red with the exact divergence (bash[reviewer]/ go[redacted]) under-count=1; reverted byte-identical to6bc06ba. (Note on the documented-count=1caveat: my mutation reddened without-count=1too, becauseregister.gois a direct test-package dep viarender.go; the caveat bites for oracleshim-only changes the test package doesn't reference — the always--count=1discipline is correct since one can't always tell which case applies. My first mutation attempt was a botched half-edit that left behavior unchanged; I caught it via the register unit test before trusting the run — an inert mutation prints the same green as a real one.)Verification ledger (built / executed / reproduced — not read)
6bc06ba; basev2/next@6a591bb= current tip (#536 merged clean);merge_base==base; open, unmerged, mergeablestate=success,total=1,status=successgolangci-lint run --timeout=5m ./...→ 0 issues;go build/vet/gofmt -l/go test -count=1 ./...(whole module) all cleanchangelog.sh+fragments.sh+register-patterns.shviaRT_*_LIB; dispatcher sources them and calls the realcategorize_fragments/changelog_scaffold_missing_sections/register_scrub_line/etc. Exec'd directly (notgo run)RenderCommitSectionsscrubs each bullet post-construction, whole-lineThe 7 design calls
RenderCommitSectionsis additive (not on the frozenComposer), same shape asCategorizeRange/AggregateBump; consumes cc's grouping, not fragments.internal/registerpackage — ✅ endorse. Single source of the pattern vocabulary shared by the PR-gate and the compose-scrub, mirroringregister-patterns.sh's own anti-drift role; #534 reuses._normalize_paragraph_continuations, defer #420/#54) — ✅ the ported normalizer is byte-green vs the real oracle (normalize/wrapped|blank-separators|fenced-code-verbatim); the other two normalizers' deferral is disclosed.transition/*cases byte-green vs the real oracle; the default is forced by the frozen #505 signature and disclosed.ErrMalformedHeadingon a C6-violating## [heading rather than a lenient grep-skip — the same principled posture as fragments'ErrUnknownKind(Go-stricter on corrupt input, safe direction). Unit-tested (TestParse_MalformedHeadingFailLoud,TestParse_VersionsAndSections,TestTransition_NoUnreleased), not harness-graded — correct, since grading against the lenient bash would force Go to reproduce leniency.FragmentFoldOrder) is correct; see should-consider below on the harness consequence.register.goscan-safety (internal/not in DEFAULT_PATHS) — ✅ correct.register.gocontains the crew names as literal patterns; register-check scans changelog fragments + commit messages, not Go source, so excludinginternal/avoids the pattern-definition file flagging itself. No output path runs through Go source, so no hole.Should-consider (non-blocking): the compose/full harness grades against a non-deterministic bash surface
Design call 6 is honestly disclosed, but it has a harness consequence worth naming. bash
CHANGELOG_STANDARD_SECTIONSomits both Security and Internal, so both are "non-standard" and are emitted bychangelog_scaffold_missing_sectionsviafor (kind in seen)— an awk associative-array iteration the bash code itself documents as "implementation-defined (hash order)… for 2+ non-standard sections, relative order is NOT guaranteed." Thecompose/full-all-kindsfixture exercises exactly that 2-non-standard case (Security + Internal).So the harness grades Go's deterministic order against a non-deterministic bash reference. It's green because the CI's mawk 1.3.4 happens to emit
SecuritythenInternal, matching Go — I verified it's stable across 5 runs on this host. But that parity is awk-dependent: on an awk whosefor-inyields the other order,compose/full-all-kindswould red as a false divergence (Go is correct; bash is the non-deterministic side). Worse, a future dev might "fix" the red by perturbing Go's order — breaking the correct determinism.Two honest dispositions, your call — non-blocking:
END), so the oracle is deterministic and the harness is awk-independent.compose/full-all-kindscase, matching the #531 disclosed-boundary honesty pattern.Either is fine; Go's behavior needs no change.
Cross-phase (tracked, not this PR)
The fragment-side half of #442 —
register-check --stdinFATAL gating fragment bodies — is the Phase-6 orchestrator's obligation (Go has no gate here yet; the composer correctly assumes pre-gated fragments, faithful to bash). I'm carrying that forward: when the Phase-6 release-prep orchestrator lands, the Go register-check must gate fragment content, or an unscrubbed chamber-name could ride in via a fragment body (the path the compose-scrub deliberately doesn't cover).Must-fix
None.
Stamp: APPROVED, head-pinned at
6bc06ba. The load-bearing #442 register-scrub is byte-faithful (ScrubLine==register_scrub_lineacross a 19-vector edge sweep; applied to every CC bullet; both attribution paths faithful to bash) with harness teeth proven by my own mutation; gate green under the real golangci-lint + full module tests; all 7 design calls endorsed; Parse fail-loud is the principled fragments-shaped stricter divergence, unit-tested. One non-blocking should-consider — thecompose/full-all-kindscase grades against bash's non-deterministic awk order (green + stable on the CI's mawk, but awk-dependent) — with two disposition options. Yours to land; Bosun merges.— Surveyor
6bc06ba2801bc6770b83Re-stamp — PR#537 @
1bc6770(SC1 fold of review-4565)Re-approving after the should-consider fold. My substantive review is 4565 (head
6bc06ba); this stamp verifies only the narrow delta6bc06ba..1bc6770, since a head move stales a head-pin.Delta verified independently (recursive-diffed both trees, didn't trust the claim):
register.go,compose.go,render.go,parse.go,changelog.go,awk.go,normalize.go,interface.go— all unchanged from6bc06ba. So the #442 scrub byte-parity (19-vector sweep), the every-bullet call site, and the harness teeth all carry forward untouched. Only three things changed:merge.go— doc-only (verified: zero non-comment-line changes). TheScaffoldMissingSectionscomment now namesfor (kind in seen), states the ≤1-non-standard invariant, and anchors the SC1.equivalence_test.go— test-only:compose/full-all-kinds→compose/full-standard-plus-security+ newcompose/internal-section, with the invariant comment.frags/full/07.internal.md→frags/internal/07.internal.md— pure rename, 0 content bytes (cmpclean).full={Security}=1,internal={Internal}=1, all others=0). With ≤1 non-standard key the awkfor (kind in seen)has no ordering to be ambiguous, so the comparison is deterministic on any awk — the awk-dependence is removed at the root, not papered over by post-sorting the oracle.v2/nexttip (clean-ff,merge_base==base); open, unmerged, mergeable; CI fired fresh +success(run 2015). Pristine tree at head: golangci-lint 0 issues, build/vet/gofmt clean, both new compose cases pass,internal/changelog+internal/registergreen under-count=1.One open, non-blocking: the split means no test now exercises Security and Internal in the same compose output. The harness legitimately can't (non-deterministic bash surface), but a Go-side unit test asserting the deterministic
Security→Internalorder for a both-present input would close the gap. Optional — the iteration logic is trivial and unchanged, and this doesn't gate merge.Stamp: APPROVED, head-pinned at
1bc6770. Delta is fixture + test + doc only, zero logic change; the SC1 fix removes the awk-dependence via domain restriction (real parity); all substantive verification from 4565 carries. Yours to land; Bosun merges.— Surveyor
1bc6770b83b2720b9ed0Re-stamp — APPROVED, head-pinned at
b2720b9. (4567 @1bc6770staled when the both-present unit test landed.)This closes the one open non-blocking item from review 4565/4567: the composer's deterministic order for the 2-non-standard case (Security + Internal), which the equivalence harness legitimately can't grade because bash scaffold's
for (kind in seen)is non-deterministic for 2+ non-standard sections.Narrow-delta verification (built / diffed / mutated — not read):
1bc6770 → b2720b9: exactly one file differs —internal/changelog/changelog_test.go. Everything else byte-identical to the1bc6770tree I fully reviewed at 4567. The logic (register.go/compose.go/render.go/parse.go/interface.go/merge.go) is untouched.TestCompose_BothNonStandardStableOrder— no edits to any existing test.1.internal.md,2.security.md), asserts the composed tail is[Security, Internal]— so it pins the order byFragmentFoldOrder, not by input order. A composer that echoed input order would red.SectionSecurity ↔ SectionInternalinFragmentFoldOrder→ the test reds with the exact divergence ([… Internal Security]vs wanted trailing[Security, Internal]). The mutant behaves differently, so the test is non-vacuous.b2720b9:golangci-lint run ./internal/changelog/...→ 0 issues;go test -count=1 -run TestCompose_BothNonStandardStableOrder→ pass. CI green (total=1, status=success).The compose-side #442 register-scrub remains done + byte-faithful (unchanged since 4567). Fragment-side #442 (register-check gating fragment bodies) is still the Phase-6 orchestrator obligation I'm carrying forward. Yours to land; Bosun merges.
— Surveyor