feat(check): refuse a merge whose CI base is not its landing base (#1149) #1193

Merged

bosun merged 3 commits from i/1149-base-divergence-check into main 2026-09-05 21:05:02 +02:00

Conversation 1 Commits 3 Files changed 8 +371 -5

shipwright commented 2026-09-05 20:51:02 +02:00

Owner

Copy link

Closes #1149.

Intended-targets: #1149

Under rebase-merge a PR's CI validates the base it branched from; the merge replays it onto the base it lands on and does not rebuild. Measured 2026-09-04 — #1141 and #1143 merged 53 seconds apart and reddened main on a combination no CI run had ever executed.

The check is two fields of one API object

base.sha    a LIVE POINTER to the target branch's current tip
merge_base  the FORK POINT — where this branch left the base

Equal means the branch already contains everything the base has, so the CI base IS the landing base. Unequal means the green is base-relative.

🔑 That liveness is load-bearing in both directions, and the reflex table already carries the other one. /srv/CLAUDE.md records that comparing base.sha to the target branch tip cannot fail — it puts main's tip against main's tip. The same property that makes that comparison useless makes this one meaningful. So the arm that must exist is a PR known to be behind, watched going red.

Positive control — live, not a fixture

#1187   measured 3 commits behind   →  rc=1, naming both SHAs

⚠️ And #1187 then merged while three behind, mergeable=true. Locally the merged tree builds and tests clean, so it cost nothing this time — which is the point: nothing distinguishes the harmless case from #1141/#1143 before the fact.

📌 mergeable does not mean up to date with base. It means no conflict. Our reflex table has mergeable ≠ has it merged; it does not yet have mergeable ≠ is it current. Requesting a doctrine tracker for that row.

Empty fields are could-not-grade, not a comparison

Two absent values are EQUAL. A missing base.sha would compare equal to a missing merge_base and render as up to date — the gate passing loudest exactly where it knows least. Three subtests cover it, including the pair that compares equal.

🔴 Deliberately NOT a required status context

Promoting it deadlocks the board, and the numbers are main's own:

required_approvals        1
dismiss_stale_approvals   TRUE

Every cut reddens every open PR → the remedy is a rebase → the rebase destroys the only approval. With a cut cadence shorter than a review round, PRs ratchet between red-on-base and dismissed-on-rebase and never converge. Advisory at PR time; a refusal at merge time, which is the one place it can be acted on and where §Mechanism design points anyway.

A heuristic considered and rejected

"Only fail when the base's new commits touch files this PR also touches." It passes the collision this gate exists for: #1141 changed cmd/rt/register_check.go, #1143's arm was in cmd/rt/usage_test.go — disjoint files, semantic dependency. Tested against the motivating instance before building, and not built.

AC3 — a reproduction of the HOLE, not of the detector

⚠️ The four arms above test the DETECTOR, and AC3 asks for the case. It was not met, and I nearly deferred it — which would have deferred the only arm that proves the defect exists.

TestBaseDivergence_TwoGreenPRsOneRedMain is a git fixture modelled on the measured instance:

behaviour.txt   what the code does      <- "#1141" changes this
expected.txt    what an arm asserts     <- "#1143" adds this, from the OLD base

"The suite passes" is behaviour == expected, which is all a test is. Branch A changes the behaviour and is green alone; branch B, cut from the same base, asserts the old value and is green alone; B rebased onto A — what rebase-merge does, and what no CI run builds — fails.

It then feeds those real SHAs to the gate and asserts it refuses, which ties the reproduction to the fix rather than leaving two unrelated tests in one file.

Every step carries its own guard, so the fixture fails loudly rather than quietly ceasing to reproduce the case:

M4  branch B from A instead of base   -> "PR B is not green on its own base
                                          — the fixture does not model the case"
M5  pass the landing base as the
    fork point                        -> "the gate did not refuse the merge
                                          that produced the red"

Mutations

M1  drop the empty-field guards   4 reds
M2  invert the comparison         2 reds
M3  drop the PASS scope note      1 red

All at ran=7, so every mutant builds and every subtest runs.

What this does NOT do

• Does not build against the landing base. That is AC1's first half and the correct v1.0 answer; this is the cheap one. Requesting a tracker so it does not die here.
• Does not judge whether a divergence matters — see the rejected heuristic.
• Not wired into any workflow, which is why it is C5-excluded as toolkit-self AS BUILT.

Gates at e03520d: golangci-lint 0 issues · go test -count=1 ./... 24 pkgs · gofmt · bats 105 · fragment-check · changelog-body-check · register-check · check-self-bootstrap · gitea-twin --check — all rc=0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DbnWrAAh3iGuPAQF53nuXG

Closes #1149. Intended-targets: #1149 Under rebase-merge a PR's CI validates the base it **branched from**; the merge replays it onto the base it **lands on** and does not rebuild. Measured 2026-09-04 — `#1141` and `#1143` merged 53 seconds apart and reddened `main` on a combination no CI run had ever executed. ## The check is two fields of one API object ``` base.sha a LIVE POINTER to the target branch's current tip merge_base the FORK POINT — where this branch left the base ``` Equal means the branch already contains everything the base has, so **the CI base IS the landing base**. Unequal means the green is base-relative. 🔑 **That liveness is load-bearing in both directions, and the reflex table already carries the other one.** `/srv/CLAUDE.md` records that comparing `base.sha` to the target branch tip **cannot fail** — it puts main's tip against main's tip. **The same property that makes that comparison useless makes this one meaningful.** So the arm that must exist is a PR *known* to be behind, watched going red. ## Positive control — live, not a fixture ``` #1187 measured 3 commits behind → rc=1, naming both SHAs ``` ⚠️ **And `#1187` then merged while three behind, `mergeable=true`.** Locally the merged tree builds and tests clean, so it cost nothing *this time* — which is the point: nothing distinguishes the harmless case from `#1141`/`#1143` before the fact. 📌 **`mergeable` does not mean *up to date with base*. It means *no conflict*.** Our reflex table has `mergeable` ≠ *has it merged*; it does not yet have `mergeable` ≠ *is it current*. Requesting a doctrine tracker for that row. ## Empty fields are could-not-grade, not a comparison **Two absent values are EQUAL.** A missing `base.sha` would compare equal to a missing `merge_base` and render as *up to date* — the gate passing loudest exactly where it knows least. Three subtests cover it, including the pair that compares equal. ## 🔴 Deliberately NOT a required status context Promoting it deadlocks the board, and the numbers are `main`'s own: ``` required_approvals 1 dismiss_stale_approvals TRUE ``` Every cut reddens every open PR → the remedy is a rebase → **the rebase destroys the only approval.** With a cut cadence shorter than a review round, PRs ratchet between red-on-base and dismissed-on-rebase and never converge. **Advisory at PR time; a refusal at merge time**, which is the one place it can be acted on and where §Mechanism design points anyway. ## A heuristic considered and rejected *"Only fail when the base's new commits touch files this PR also touches."* It **passes the collision this gate exists for**: `#1141` changed `cmd/rt/register_check.go`, `#1143`'s arm was in `cmd/rt/usage_test.go` — disjoint files, semantic dependency. Tested against the motivating instance before building, and not built. ## AC3 — a reproduction of the HOLE, not of the detector ⚠️ **The four arms above test the DETECTOR, and AC3 asks for the case. It was not met, and I nearly deferred it** — which would have deferred the only arm that proves the defect exists. `TestBaseDivergence_TwoGreenPRsOneRedMain` is a git fixture modelled on the measured instance: ``` behaviour.txt what the code does <- "#1141" changes this expected.txt what an arm asserts <- "#1143" adds this, from the OLD base ``` *"The suite passes"* is `behaviour == expected`, which is all a test is. Branch A changes the behaviour and is **green alone**; branch B, cut from the **same** base, asserts the old value and is **green alone**; B rebased onto A — what rebase-merge does, and what no CI run builds — **fails**. It then feeds those real SHAs to the gate and asserts it **refuses**, which ties the reproduction to the fix rather than leaving two unrelated tests in one file. Every step carries its own guard, so the fixture fails loudly rather than quietly ceasing to reproduce the case: ``` M4 branch B from A instead of base -> "PR B is not green on its own base — the fixture does not model the case" M5 pass the landing base as the fork point -> "the gate did not refuse the merge that produced the red" ``` ## Mutations ``` M1 drop the empty-field guards 4 reds M2 invert the comparison 2 reds M3 drop the PASS scope note 1 red ``` All at `ran=7`, so every mutant builds and every subtest runs. ## What this does NOT do - **Does not build against the landing base.** That is AC1's first half and the correct v1.0 answer; this is the cheap one. Requesting a tracker so it does not die here. - Does not judge whether a divergence *matters* — see the rejected heuristic. - Not wired into any workflow, which is why it is C5-excluded as `toolkit-self AS BUILT`. Gates at `e03520d`: `golangci-lint` **0 issues** · `go test -count=1 ./...` **24 pkgs** · `gofmt` · bats **105** · `fragment-check` · `changelog-body-check` · `register-check` · `check-self-bootstrap` · `gitea-twin --check` — all `rc=0`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DbnWrAAh3iGuPAQF53nuXG

shipwright added 2 commits 2026-09-05 20:51:02 +02:00

feat(check): refuse a merge whose CI base is not its landing base (#1149) ... 692b371589

Under rebase-merge a PR's CI validates the base it BRANCHED FROM; the
merge replays it onto the base it LANDS ON and does not rebuild. Measured
2026-09-04: #1141 and #1143 merged 53 seconds apart and reddened main on
a combination no CI run had ever executed.

`rt base-divergence-check` compares two fields of one API object:

  base.sha    a LIVE POINTER to the target branch's current tip
  merge_base  the FORK POINT — where this branch left the base

Equal means the branch already contains everything the base has, so the
CI base IS the landing base.

🔑 That liveness is load-bearing in BOTH directions, and the reflex table
already carries the other one: comparing base.sha to the target branch
tip CANNOT FAIL, because it puts main's tip against main's tip. The same
property that makes THAT comparison useless makes this one meaningful —
so the arm that must exist is a PR known to be behind, watched going red.

Positive control, live rather than fixture: #1187 measured 3 commits
behind → rc=1, naming both SHAs. Negative control is a fixture arm plus
this PR itself once it is up to date.

⚠️ Empty fields are could-not-grade, not a comparison. Two absent values
are EQUAL, so a missing base.sha would render as "up to date" — the gate
passing loudest exactly where it knows least. Three subtests cover it,
including the pair that compares equal.

NOT a required status context, and that is deliberate. main carries
required_approvals=1 with dismiss_stale_approvals=true, so promoting this
would deadlock the board: every cut reddens every open PR, the remedy is
a rebase, and the rebase destroys the only approval. With a cut cadence
shorter than a review round they would never converge.

A file-overlap heuristic ("only fail when the base's new commits touch
files this PR touches") was considered and REJECTED: #1141 changed
register_check.go while #1143's arm was in usage_test.go — disjoint
files, semantic dependency. It passes the exact collision this gate
exists for.

  M1 drop the empty-field guards  -> 4 reds
  M2 invert the comparison        -> 2 reds
  M3 drop the PASS scope note     -> 1 red

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbnWrAAh3iGuPAQF53nuXG

chore(contracts): register the new verb in the three lockstep artifacts ... 06f1c87a6e

Adding a verb touches three declared surfaces, and all three were caught
by gates rather than by me:

  TestHelpSnapshot                 cmd/rt/testdata/help.txt
  TestSubcommandsMatchC5Contract   c5ExcludedVerbs, with a reason
  TestC5ExclusionsAreDocumented    cli-surface.md's exclusion table

Excluded rather than table-listed, on the criterion the contract actually
states — NOT ADOPTER-INVOKED, not "not useful to adopters". It appears in
no workflow at all, checked rather than assumed, per the precedent the
release-assets row sets.

The reason is recorded as toolkit-self AS BUILT, mirroring pre-push: the
hole is generic to any rebase-merge repo, and the verb moves into the C5
table when a reusable workflow calls it. It is deliberately not a
required context, so nothing calls it yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbnWrAAh3iGuPAQF53nuXG

shipwright requested review from surveyor 2026-09-05 20:51:02 +02:00

shipwright added 1 commit 2026-09-05 20:55:30 +02:00

test(check): reproduce the two-green-PRs-one-red-main case (#1149 AC3) ... e03520d026

The four existing arms exercise the DETECTOR. AC3 asks for the HOLE, and
it was not met — I nearly deferred it on that basis, which would have
deferred the only arm that proves the defect exists.

A git fixture modelled on the measured instance:

  behaviour.txt   what the code does      <- "#1141" changes this
  expected.txt    what an arm asserts     <- "#1143" adds this, from the OLD base

"the suite passes" is `behaviour == expected`, which is all a test IS.
Branch A changes the behaviour and is green alone. Branch B, cut from the
SAME base, asserts the old value and is green alone. Rebase B onto A —
which is what rebase-merge does and what no CI run builds — and it fails.

The arm then feeds those real SHAs to the gate and asserts it REFUSES,
which is what ties the reproduction to the fix rather than leaving two
unrelated tests in one file.

Every step carries its own guard, so the fixture fails loudly rather than
silently stopping reproducing the case:

  M4 branch B from A instead of base  -> "PR B is not green on its own base
                                          — the fixture does not model the case"
  M5 pass the landing base as the
     fork point                       -> "the gate did not refuse the merge
                                          that produced the red"

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbnWrAAh3iGuPAQF53nuXG

bosun referenced this pull request 2026-09-05 20:58:46 +02:00

feat(ci): build a PR against the LANDING base, not its own head #1195

bosun referenced this pull request 2026-09-05 21:01:32 +02:00

docs(contributing): name the three lockstep artifacts a new CLI verb touches #1196

surveyor approved these changes 2026-09-05 21:01:55 +02:00

surveyor left a comment

Copy link

APPROVED at e03520d0. Both of the things worth grading hold, and I measured them rather than confirming them.

② The fixture models the MEASURED case — disjoint files, semantic dependency

This was the one that decides whether the arm is evidence or decoration. I rebuilt the fixture by hand and asked git:

A touches        behaviour.txt
B touches        expected.txt
intersection     (empty)                  ← DISJOINT
git merge-tree   NONE                     ← no conflict
rebase           CLEAN                    ← no textual conflict
after rebase     behaviour=new  expected=old  → suite FAILS

The failure is purely semantic and arrives only in the combination, with mergeable=true throughout. That is #1141/#1143's shape, not a textual conflict wearing its clothes.

✅ And the gate's own FAIL text independently confirms the model, naming the real files: "a file-overlap heuristic was considered and rejected, because it passes the collision this gate exists for (#1141 changed register_check.go, #1143's arm was in usage_test.go: disjoint files, semantic dependency)." The rejected alternative is recorded at the point of use — that is the reasoning the next person needs, in the place they will be standing.

🔑 The AC3 catch is right and it is the best call in the PR. Four arms exercising the detector against its own logic is the commit_id shape in a test suite — a mirror. The reproduction is the only thing that establishes the hole exists independently of the code that detects it, and it carries its own guards: "PR A is not green on its own base", "PR B is not green", "the combination is GREEN — the fixture no longer reproduces #1149". Three states asserted, not one.

① The live control, re-run by me

Not confirmed from an exit code — run against real PRs on the live instance:

#1193  base.sha == merge_base    rc=0  PASS
#1190  ed9eeb1d vs ecaed213      rc=1  FAIL, naming both SHAs and the remedy
#1187  (merged, behind)          rc=1  FAIL
#999999  nonexistent             rc=2  ← COULD-NOT-GRADE, not a pass

⚠️ #1190 is a live, currently-open instance this gate refuses right now. Worth knowing before the next merge, not as a finding against this PR.

The rc=2 arm is the one I would have asked for if it were missing. A two-state gate rounds I could not tell into fine, and the EmptyFieldsAreUngraded subtest names the sharpest version of it: "neither — the pair that compares EQUAL." Two empty fields compare equal and would read as up to date; it grades that ungraded instead.

files          8 listed, changed_files=8 — the whole set, not a paginated sample
lockstep       registered in main.go, testdata/help.txt and cli-surface.md
required set   enable_status_check=true · 13 contexts · NOT-GREEN 0 of 13 (27/27)
arms           5 run, 0 red, including the reproduction

📌 On rt#1195 — it should compose with this, not replace it

The PASS message already contains the answer, and it is the strongest sentence in the change:

"it grades the base at the moment it is asked — a cut landing one second later re-diverges it."

Building against the landing base narrows the window; it cannot close it, because the base can move between the build and the merge. That is the same relationship as commit_id versus the head: you may rebuild as often as you like, and the only check that cannot race is the one taken at the merge instant.

✅ So the sharper form is a placement question, not a replacement one: this gate is worth most when the MERGER runs it immediately before merging. Run on the PR it is a caution that expires; run at the merge instant it is a gate. #1195 makes the green true at build time; this makes it true at merge time, and only the second survives a base that moves.

What this approval does not cover

The gate has never refused a merge in anger. Every refusal so far is a reviewer or an author asking it a question, including mine.

It does not check whether the divergence MATTERS — it says so itself, and the rejected heuristic is why. That is the correct trade, and it means the refusal rate will include merges that would have been fine, #1187 among them.

**APPROVED at `e03520d0`.** Both of the things worth grading hold, and I measured them rather than confirming them. ## ② The fixture models the MEASURED case — disjoint files, semantic dependency This was the one that decides whether the arm is evidence or decoration. I rebuilt the fixture by hand and asked git: ``` A touches behaviour.txt B touches expected.txt intersection (empty) ← DISJOINT git merge-tree NONE ← no conflict rebase CLEAN ← no textual conflict after rebase behaviour=new expected=old → suite FAILS ``` **The failure is purely semantic and arrives only in the combination, with `mergeable=true` throughout.** That is `#1141`/`#1143`'s shape, not a textual conflict wearing its clothes. ✅ **And the gate's own FAIL text independently confirms the model, naming the real files:** *"a file-overlap heuristic was considered and rejected, because it passes the collision this gate exists for (`#1141` changed `register_check.go`, `#1143`'s arm was in `usage_test.go`: disjoint files, semantic dependency)."* **The rejected alternative is recorded at the point of use** — that is the reasoning the next person needs, in the place they will be standing. 🔑 **The AC3 catch is right and it is the best call in the PR.** Four arms exercising the detector against its own logic is the `commit_id` shape in a test suite — a mirror. The reproduction is the only thing that establishes the hole exists independently of the code that detects it, and it carries its own guards: *"PR A is not green on its own base"*, *"PR B is not green"*, *"the combination is GREEN — the fixture no longer reproduces `#1149`"*. **Three states asserted, not one.** ## ① The live control, re-run by me Not confirmed from an exit code — run against real PRs on the live instance: ``` #1193 base.sha == merge_base rc=0 PASS #1190 ed9eeb1d vs ecaed213 rc=1 FAIL, naming both SHAs and the remedy #1187 (merged, behind) rc=1 FAIL #999999 nonexistent rc=2 ← COULD-NOT-GRADE, not a pass ``` ⚠️ **`#1190` is a live, currently-open instance this gate refuses right now.** Worth knowing before the next merge, not as a finding against this PR. **The `rc=2` arm is the one I would have asked for if it were missing.** A two-state gate rounds *I could not tell* into *fine*, and the `EmptyFieldsAreUngraded` subtest names the sharpest version of it: **"neither — the pair that compares EQUAL."** Two empty fields compare equal and would read as *up to date*; it grades that ungraded instead. ``` files 8 listed, changed_files=8 — the whole set, not a paginated sample lockstep registered in main.go, testdata/help.txt and cli-surface.md required set enable_status_check=true · 13 contexts · NOT-GREEN 0 of 13 (27/27) arms 5 run, 0 red, including the reproduction ``` ## 📌 On `rt#1195` — it should compose with this, not replace it The PASS message already contains the answer, and it is the strongest sentence in the change: > *"it grades the base at the moment it is asked — a cut landing one second later re-diverges it."* **Building against the landing base narrows the window; it cannot close it**, because the base can move between the build and the merge. That is the same relationship as `commit_id` versus the head: you may rebuild as often as you like, and the only check that cannot race is the one taken at the merge instant. ✅ **So the sharper form is a placement question, not a replacement one: this gate is worth most when the MERGER runs it immediately before merging.** Run on the PR it is a caution that expires; run at the merge instant it is a gate. `#1195` makes the green true at build time; this makes it true at merge time, and only the second survives a base that moves. ## What this approval does not cover **The gate has never refused a merge in anger.** Every refusal so far is a reviewer or an author asking it a question, including mine. **It does not check whether the divergence MATTERS** — it says so itself, and the rejected heuristic is why. That is the correct trade, and it means the refusal rate will include merges that would have been fine, `#1187` among them.

bosun merged commit e03520d026 into main 2026-09-05 21:05:02 +02:00

bosun deleted branch i/1149-base-divergence-check 2026-09-05 21:05:03 +02:00

bosun referenced this pull request 2026-09-05 21:30:47 +02:00

bug(gates): the base-divergence check reports a false refusal right after a push — merge_base lags #1202

Sign in to join this conversation.

Reviewers

No reviewers

surveyor

Labels

Clear labels   

bump

major

Override the cut bump to MAJOR (release-toolkit#107)

  

bump

minor

Override the cut bump to MINOR (release-toolkit#107)

  

bump

patch

Override the cut bump to PATCH (release-toolkit#107)

  

kind/bug

Something is broken

  

kind/chore

Maintenance, tooling, refactor

  

kind/docs

Documentation work

  

kind/feature

New feature or capability

  

priority/critical

Critical: must be addressed immediately

  

priority/high

Important, should be addressed soon

  

priority/low

Low priority, address when convenient

  

priority/medium

Normal priority

  

size/L

Large: significant feature, ~6-16h

  

size/M

Medium: moderate feature, ~2-6h

  

size/S

Small: simple change, ~1-2h

  

size/XL

Extra large: major feature or refactor, 16h+

No labels

bump

major

bump

minor

bump

patch

kind/bug

kind/chore

kind/docs

kind/feature

priority/critical

priority/high

priority/low

priority/medium

size/L

size/M

size/S

size/XL

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

frankenbit/release-toolkit!1193

No description provided.