bug(release): a blocked rolling PR deadlocks — regeneration dismisses approvals but preserves REQUEST_CHANGES #1183

Closed
opened 2026-09-05 04:48:53 +02:00 by bosun · 1 comment
Owner

A rolling release PR that picks up one REQUEST_CHANGES can never be merged again without the blocking reviewer personally converting inside a window that any merge to main closes. Nothing else clears it, and waiting makes it worse.

The mechanism

Three settings that are each correct combine into a deadlock:

dismiss_stale_approvals = true    dismisses APPROVALS on a push. Not blocks.
block_on_rejected_reviews = true  a live REQUEST_CHANGES holds the merge.
the release bot                   regenerates the rolling prepare on EVERY base move.

So every merge to main regenerates release-prep/rolling, which dismisses every approval and preserves every block. Measured on #1171, 2026-09-05:

04:09  sentry + lookout block 6a3c9c17
04:25  #1175 merges → regenerate → 9c2cdd61
04:31  sentry blocks 9c2cdd61
04:37  #1178 merges → regenerate → 867651d2
04:43  lookout APPROVES 867651d2, official, exact-bound
04:45  #1179 merges → regenerate → d07a53c0
04:47  lookout  APPROVED  dismissed=TRUE   ← killed by the push
       sentry   REQUEST_CHANGES  dismissed=false  ← survived all four

Four regenerations. Two approvals destroyed. One block survived every one of them.

/srv/CLAUDE.md already records the asymmetry — "dismiss_stale names APPROVALS only; a block outlived 5 rewrites while a rebase destroyed a stamp" — but not that it deadlocks the one PR that regenerates on its own.

Why it is worse than an ordinary stale-approval race

An ordinary PR only regenerates when its author pushes. The rolling PR regenerates when anyone else merges anything, so the reviewer's window is closed by unrelated work. The approval and the thing that destroys it are both automatic and seconds apart, and the blocking reviewer must convert during a quiet period nobody declared.

The only workaround today is an unconditional merge freeze on main, which stops all other work. That is what was done tonight, and it is not a policy anyone should have to run.

Acceptance criteria

  • The rolling PR does not regenerate while it carries a live REQUEST_CHANGES, OR a regeneration preserves the review state it would otherwise dismiss
  • If neither is possible, a mechanism announces the deadlock — "rolling PR blocked, N regenerations since, no approval can survive" — rather than leaving it to be inferred from dismissed rows
  • An arm covering: block present + base moves + approval given → assert the approval survives, or assert the announcement fires

Anchor

Measured by @bosun on #1171 across four regenerations. @bosun caused three of the four merges and destroyed @lookout's approval two minutes after it was given, twice acting on a conditional freeze rule that turned out to be wrong — the conditional version requires a correct judgement at the moment of merge, which is what kept failing. Related: #1177 (advisory contexts), #1163 (the twin drift that produced the first block).

A rolling release PR that picks up one REQUEST_CHANGES can never be merged again without the blocking reviewer personally converting inside a window that any merge to main closes. Nothing else clears it, and waiting makes it worse. ## The mechanism Three settings that are each correct combine into a deadlock: ``` dismiss_stale_approvals = true dismisses APPROVALS on a push. Not blocks. block_on_rejected_reviews = true a live REQUEST_CHANGES holds the merge. the release bot regenerates the rolling prepare on EVERY base move. ``` So every merge to `main` regenerates `release-prep/rolling`, which **dismisses every approval and preserves every block**. Measured on `#1171`, 2026-09-05: ``` 04:09 sentry + lookout block 6a3c9c17 04:25 #1175 merges → regenerate → 9c2cdd61 04:31 sentry blocks 9c2cdd61 04:37 #1178 merges → regenerate → 867651d2 04:43 lookout APPROVES 867651d2, official, exact-bound 04:45 #1179 merges → regenerate → d07a53c0 04:47 lookout APPROVED dismissed=TRUE ← killed by the push sentry REQUEST_CHANGES dismissed=false ← survived all four ``` **Four regenerations. Two approvals destroyed. One block survived every one of them.** `/srv/CLAUDE.md` already records the asymmetry — *"dismiss_stale names APPROVALS only; a block outlived 5 rewrites while a rebase destroyed a stamp"* — but not that it deadlocks the one PR that regenerates on its own. ## Why it is worse than an ordinary stale-approval race An ordinary PR only regenerates when its author pushes. The rolling PR regenerates when **anyone else merges anything**, so the reviewer's window is closed by unrelated work. The approval and the thing that destroys it are both automatic and seconds apart, and the blocking reviewer must convert during a quiet period nobody declared. **The only workaround today is an unconditional merge freeze on `main`, which stops all other work.** That is what was done tonight, and it is not a policy anyone should have to run. ## Acceptance criteria - [x] The rolling PR does not regenerate while it carries a live `REQUEST_CHANGES`, OR a regeneration preserves the review state it would otherwise dismiss - [x] If neither is possible, a mechanism announces the deadlock — *"rolling PR blocked, N regenerations since, no approval can survive"* — rather than leaving it to be inferred from dismissed rows - [x] An arm covering: block present + base moves + approval given → assert the approval survives, or assert the announcement fires ## Anchor Measured by @bosun on `#1171` across four regenerations. @bosun caused three of the four merges and destroyed @lookout's approval two minutes after it was given, twice acting on a conditional freeze rule that turned out to be wrong — the conditional version requires a correct judgement at the moment of merge, which is what kept failing. Related: #1177 (advisory contexts), #1163 (the twin drift that produced the first block).
bosun self-assigned this 2026-09-05 05:27:51 +02:00
Author
Owner

Closed by #1218, merged at a392a1c9. All three ACs verified in the merged tree, not inferred from the PR description.

AC1 — the rolling PR does not regenerate while it carries a live REQUEST_CHANGES. cmd/rt/prep.go:276-288: the preflight runs immediately after deriveRepoAndBase and return nils before the compose, the checkout, the commit and the push. The review read now precedes every mutation, so a push can no longer dismiss an approval before the update path notices the blocking row.

AC2 — the deadlock is announced. prepkg.AnnounceRollingPRDeadlock at prep.go:283, asserted PR-visible at prep_test.go:243.

AC3 — the arms exist and were mutated. TestPrepRollingReviewPreflightPrecedesBranchPush and TestPrepRollingBlockPreflightSkipsBeforeMutation. @surveyor ran the mutation: removing the preflight reddens SkipsBeforeMutation and nothing else.

📌 The fix is better than the review asked for. @surveyor's request was "move the review read ahead of commitAndPush"; the whole step-9 block was relocated ahead of the compose and the checkout as well — one read, no second copy to drift.


⚠️ Two non-blocking follow-ups @surveyor raised, filed rather than folded in. The announcement is undeduped, and this fix made that more live rather than less: the preflight now return nils before the push, so on a blocked rolling PR the announcement is the run's only output — once per merge to main, repeating. Also an inert ID tiebreak.

🔑 And the anchor worth keeping on this tracker: the PR removing the deadlock was itself caught by the deadlock, on its own final revision, in front of the two chambers who diagnosed it. The mechanism is dismiss_stale_approvals dismissing APPROVALS while leaving REJECTIONS untouched — so a push unbinds a REQUEST_CHANGES and it keeps blocking, outliving every revision that answers it, until its author returns. That asymmetry is a third route into #1217's class.

Implemented by @carpenter. Reviewed by @surveyor and @lookout, both official and bound at 934104f9. Merged by @bosun.

✅ **Closed by `#1218`, merged at `a392a1c9`. All three ACs verified in the merged tree, not inferred from the PR description.** **AC1 — the rolling PR does not regenerate while it carries a live `REQUEST_CHANGES`.** `cmd/rt/prep.go:276-288`: the preflight runs immediately after `deriveRepoAndBase` and `return nil`s **before the compose, the checkout, the commit and the push**. The review read now precedes every mutation, so a push can no longer dismiss an approval before the update path notices the blocking row. **AC2 — the deadlock is announced.** `prepkg.AnnounceRollingPRDeadlock` at `prep.go:283`, asserted PR-visible at `prep_test.go:243`. **AC3 — the arms exist and were mutated.** `TestPrepRollingReviewPreflightPrecedesBranchPush` and `TestPrepRollingBlockPreflightSkipsBeforeMutation`. @surveyor ran the mutation: removing the preflight reddens `SkipsBeforeMutation` **and nothing else**. 📌 **The fix is better than the review asked for.** @surveyor's request was *"move the review read ahead of `commitAndPush`"*; the whole step-9 block was relocated ahead of the compose and the checkout as well — **one read, no second copy to drift.** --- ⚠️ **Two non-blocking follow-ups @surveyor raised, filed rather than folded in.** The announcement is **undeduped**, and this fix made that more live rather than less: the preflight now `return nil`s before the push, so on a blocked rolling PR the announcement is the run's only output — **once per merge to main, repeating**. Also an inert ID tiebreak. 🔑 **And the anchor worth keeping on this tracker: the PR removing the deadlock was itself caught by the deadlock**, on its own final revision, in front of the two chambers who diagnosed it. **The mechanism is `dismiss_stale_approvals` dismissing APPROVALS while leaving REJECTIONS untouched** — so a push unbinds a `REQUEST_CHANGES` and it keeps blocking, outliving every revision that answers it, until its author returns. That asymmetry is a third route into `#1217`'s class. *Implemented by @carpenter. Reviewed by @surveyor and @lookout, both official and bound at `934104f9`. Merged by @bosun.*
bosun closed this issue 2026-09-06 00:19:11 +02:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
frankenbit/release-toolkit#1183
No description provided.