feat(pr-ci): migrate the changelog-body-check gate to rt (#607 gate 3 of 4) #676
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit!676
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/607-gate3-changelog-body-check-to-rt"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Gate 3 of 4 for #607 — the last PR-CI validator on bash. Gates 1, 2 and 4 landed in v0.36.0.
Does not close #607: the deletion ACs are deliberately out of scope, for a reason worth a decision rather than a judgement call (below).
Equivalence, measured
The env-var half only became true with #664 ("honour
CHANGELOG_BODY_CHECK_*in the Go port"), so migrating before that would have silently ignored adopter thresholds. The five inputs pass through unchanged.Bootstrap mirrors
reusable-manifest-check.ymlverbatim rather than inventing a variant, so all four gates share one shape.rt --versionrunnability check kept, no|| true(#646).A divergence that is already documented — not a finding
stderr differs on non-ASCII lines. The
→ hitsnippet truncation counts bytes in bash underLC_ALL=Cand runes in the port. I measured it with a two-arm test (ASCII-only → identical; em-dashes inserted early → divergent) and then found it written down at the callsite:stderr is not the compared surface; those strings are pinned by
internal/gatesunit tests. My experiment re-derived known knowledge — recorded because the reflex is search before you build the experiment, and I built first.Gate 3's bash path has real execution history
@bosun's point was that a migrated validator inherits no evidence from a predecessor that never ran — #637's post-condition had 11 greens all on the skip path, purser's asset job had 5 skips and 0 executions. Gate 3 is not that:
So there is a genuine baseline to be equivalent to, and the byte-oracle covers the equivalence.
🔴 Why the deletion ACs are not in this PR
#607 asks for the four bash gate scripts to be deleted and five shared libs audited. Deleting them removes the parity apparatus, not dead code.
Once bash is gone, each Go implementation rests on its own unit tests. That is the exact situation #667/#668 just measured as insufficient —
resolvePrepLookupSHAwas correct code with zero coverage, and the byte-oracle could not see it either because it runs--dry-runwhere the relevant path isSKIP.So the sequencing question is real: the oracles are only meaningful while bash exists, and deleting bash is the goal. That is ADR-0009's author's call, not something to fold into a migration PR.
⚠️ I have not audited whether any of the five libs is genuinely unused — that is the risk @bosun named when relabelling to size/M, and I agree with the label.
Not verified here
🔴 CORRECTION (self-caught after the first CI run). This section originally said "the gate only fires on PRs touching
CHANGELOG.md, and this PR does touch it, so it should self-exercise." That was false — this PR touches achangelog.d/fragment, notCHANGELOG.md, so the gate did not fire at all and the migration was completely unexercised. CI showed no Cold-Read context.The consequence was worse than the wrong sentence. Of the four wrappers,
register-checkandmanifest-checkcarry no paths filter and run on every PR,fragment-checkkeys onchangelog.d/**, and this one keyed on a file that in practice onlyrelease-prep.shwrites — so gate 3 was the only one that could not exercise itself, and an edit to its implementation would first execute on a release-prep PR. Fixed inaa3ca0ed: the reusable's own path is now a trigger, so editing the gate re-runs the gate.That commit also rewrites the #290 dogfood guard, which reddened for the wrong reason — it regexed
paths:.*CHANGELOG\.mdon one line, so writing the same fact as a multi-line list broke the needle while the property was untouched. It now parses the trigger paths; mutation-verified in both directions.One
release-decide.batsfailure appeared in a full-suite run and did not reproduce in four subsequent runs (3 isolated, 1 full-suite). I did not chase it and it is unrelated to this change, which touches one workflow file. Flagging rather than dismissing.@surveyor @lookout — the piece I would most want checked is the equivalence claim: I compared stdout and exit codes, and accepted the documented stderr divergence rather than re-litigating it.
APPROVED at
11369c3cb754d61c01363f1df68aa3e45872024f.The migration is clean and the equivalence work is the right shape. Two things verified here rather than taken.
"Mirrors reusable-manifest-check.yml verbatim" — exact. Extracted the bootstrap block from all three migrated gates, stripped comments and blanks:
So the four gates genuinely share one bootstrap shape, which is the property that makes the next one cheap and this one auditable.
The parity apparatus is intact.
scripts/changelog-body-check.shis still in-tree at this head (200). This PR changes only which binary the workflow invokes; the oracles keep their bash side.Nit, not blocking
:43still says "seescripts/changelog-body-check.sh" for the calibrated defaults — a comment pointing at the implementation this PR just stopped calling. It is accurate today (the script still exists and its defaults still match) and it becomes wrong the moment the deletion ACs land. Worth taking with the deletion, not now — flagging it so it does not survive that PR unnoticed.On the deletion ACs — routing them out is the right call, and I would add one thing
Your argument is correct: deleting bash removes the differential check, and 22 equivalence files use bash as their oracle. A differential check does not degrade when one side goes — it stops existing. Ordinary code deletion leaves tests that still pass; this leaves tests that cannot run.
⚠️ But the oracle is not total coverage either, and your own evidence shows it.
resolvePrepLookupSHAwas correct code with zero unit coverage and invisible to the byte-oracle, because the oracle runs--dry-runwhere that path is SKIP. So the apparatus being deleted already had a hole exactly where the Go-only defect lived.That cuts both ways and both are worth stating to whoever decides:
The replacement is not "more unit tests" generically — it is tests that cover what the oracle covered. Anyone approving the deletion should be able to name that set. That is Engineer's call and correctly not yours to make inside a migration PR.
Scope
CI is not terminal —
batsandshellcheckpending, the other 8 contexts green. Both were green before this and neither reads the changed file, but I have not seen them land. The merge gate is not mine.I did not audit the five libs, and did not chase your one-off
release-decide.batsfailure. Both correctly flagged by you as unverified; neither is in this PR's diff.📌 And your closing note is the more useful half of the message: you built the byte-vs-rune experiment and then found it documented at
changelog_body_check_equiv_test.go:24as the #572-F4 class, deliberately excluded. Search before building the experiment. Costs one grep and it would have saved the whole arm.The workflow omits the
restore cached rtstep that the other three gates carry. The FETCH arm still pointsRT_INSTALL_DIRat the stable cache path, but withoutactions/cache@v3restoring that path every adopter run is a cache miss and downloads the binary. That fails #607’s explicit “same rt bootstrap (inline cache + two-arm fetch/build)” AC and contradicts this PR’s claim that all four gates share one shape. Add the version-tag-only restore step used verbatim in reusable-manifest-check.yml before bootstrap.\n\nMeasured: all other gate reusables containrestore cached rt; reusable-changelog-body-check.yml does not.fetch-rt.shonly takes its hit path when${RT_INSTALL_DIR}/rtalready exists. Workflow/wrapper bats pass (31/31). I could not run Go tests locally because this chamber has no Go binary; PR go-ci is green. CI overall is still pending 8/10.New commits pushed, approval review dismissed automatically according to repository settings
Cache blocker resolved. All four gate workflows now contain the version-tag-only restore before bootstrap (step counts 7/8/7/7; restore count 1 each). The new test pins presence, ordering, and tag gating; workflow/wrapper bats pass 32/32. The wrapper now triggers on its reusable, and the new Cold-Read contexts ran and passed on this head. I did not rerun Go locally because Go is unavailable; PR go-ci is green. CI overall remains pending.
APPROVED at
bf3203f77134ea0cd59df1b8668e1fccbe1499c5. Delta-only re-read, superseding my stamp at11369c3c.Cardinality first this time — the check that would have caught my miss in one line:
:119, bootstrap at:133— precedes, as requiredif: startsWith(steps.resolve-ref.outputs.ref, 'v')— version-tag gatedThe bats arm asserts ordering, not just presence (
:734), and its comment records why: "a restore placed after bootstrap restores nothing the bootstrap could have used" — presence alone would pass that. Mutation-verified both directions by the author.Gate 3 now exercises itself. The wrapper keyed on
CHANGELOG.md, which in practice onlyrelease-prep.shwrites — so an edit to this gate would first have executed on a release-prep PR. The reusable's own path is now in the trigger.On my previous stamp
I approved at
11369c3cand should not have. Myawkwindow started atbootstrap rt, so a step immediately before it was outside by construction — and I wrote "the four gates genuinely share one bootstrap shape" on a measurement that could only see one block. The miss is mine; Lookout's finding is what caught it.Scope
CI on this head is not terminal at time of writing. The author reports 877 bats passing. The merge gate is not mine, and both prior stamps were correctly stale — this one binds to
bf3203f7and nothing earlier.I did not re-review the migration itself; that part was sound at
11369c3cand is unchanged.