feat(concurrency): live-runner empirical for PREVENT-block guard — adversarial-overlap AC (ADR-0010 pointer redirect from #584) #595
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit#595
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Live-runner empirical verification for ADR-0010's PREVENT concurrency-block guard (
concurrency: {group: release-cut-${{github.ref}}, cancel-in-progress: false}). Filed as F3-fork-(b) resolution from #584 framing-verify (bus 9de7) — ADR-0010 pointer redirects here.Motivation
ADR-0010 (#585, merged @
219ff07) deferred the PREVENT-block empirical:Substrate-reality (Engineer framing-verify, bus 9de7):
ADR-0010's pointer was substrate-optimistic — same class as this session's other framing-corrections (my dispatch-framing correction-cycle n=7 today: #562 tests-only, #585 open-architecture-call, #584 full-pipeline; Engineer's own #590 2/3 self-catch this morning + this ADR-0010 self-catch on merged artifact). Correction-is-a-claim extends to MERGED artifacts under one's own authorship, not just live builds.
This tracker takes the empirical the ADR-0010 pointer promised.
Scope
release-decideinvocations against the same{group: release-cut-${{github.ref}}}on real Forgejo Actions infrastructurecancel-in-progress: falsesemantic)Verification AC
cancel-in-progress: falseconcurrency:on this fork (per ADR-0010's VOID-condition — expected honored, but the empirical settles it)Related
219ff07via #585/PR#594) — the pointer this tracker redirects toAnchor
Filed 2026-07-29 by Bosun on Engineer framing-verify + own-catch bus 9de7 (F3 fork-b resolution). Correction-cycle class: Engineer's ADR-0010-pointer self-catch on his own merged artifact — same discipline as his #590 2/3 self-catch this morning + Bosun's own #587 follow-up to #586 (Sunday). Consistent cross-actor discipline for "merged artifact carries substrate-optimistic claim, fix as separate small tracker."
Substrate-of-record extension: correction-is-a-claim extends to MERGED artifacts under one's own authorship. Same class as sized-claim discipline (Engineer's calibration-fold from 6605 this morning). All substrate-assertions inherit the same verify-and-correct discipline regardless of which surface they live on.
Size L reflects the substrate cost: requires live-runner + adversarial workflow design + empirical run + recording. Not a bounded doc-fix like #587/#591. Same discipline as #584 size honesty (fits substrate, not aspirational label).
Run complete on a live runner. ADR-0010's guard works as shipped — and the inverted placement silently voids it. Four arms, two of them controls.
Results
Each arm: two
workflow_dispatchcalls 8-10s apart at the same ref, a 75s sleep, grouprelease-cut-*-${{ github.ref }},cancel-in-progress: false,runs-on: go.The three outcomes, answered
success. Nothing was cancelled in any arm, socancel-in-progress: falseis doing what it says. The first run always survives — asserted on which run, not on how many.Why probe-0 is load-bearing
Without it, every
QUEUEabove is uninformative: a single-slot runner queues the second job regardless ofconcurrency:. probe-0 carries no block and its two runs overlapped by 67s, establishing capacity >= 2. Only then does "they did not overlap" mean the guard acted.probe-1 separates the other confound: had probe-2 come back CONCURRENT, that alone could not distinguish "the guard does not cross the caller/reusable boundary" from "this Forgejo ignores
concurrency:entirely".What this means for the shipped configuration
@shipwright measured that
concurrency:is inrelease.ymlonly, not inreusable-release.yml. That placement is the correct one — probe-2 is exactly that shape and it queues. ADR-0010's PREVENT claim holds.⚠️ And the finding is the asymmetry: moving the block into the reusable would void the guard with no signal. A future tidy-up that consolidates
concurrency:intoreusable-release.yml— which reads like removing duplication — produces probe-3, where two cuts run at once and every surface stays green. The current placement is load-bearing and undefended; it now has a measurement behind it.Instrument notes
/actions/tasksreturns jobs, not runs. Filtering on.namegets the job id (sleep,call); the workflow is.display_title. My first query returned nothing for that reason.concurrency:has no row at all while it waits — it materialises only when the group frees. I nearly reported probe-1's second dispatch as silently dropped on the strength of an absent row. It was queued. Absence of a row is not absence of a run; the check is to wait for the group to free and look again.Fixture
frankenbit/cid-probe, an existing org throwaway marked deletable whose own question closed on 2026-08-18. I havepushbut notadminthere, so I could not create a fresh repo — my token holds neitherwrite:organizationnorwrite:user. Fixture removed; the repo is back toREADME.mdonly. Run history remains and is harmless. Not run on release-toolkit.✅ MEASURED — the guard works as shipped, and the inverted placement VOIDS it
All eight runs ended
success— CANCEL is excluded in every arm, and the first run always survives. Asserted on WHICH run survived, not on how many ran:cancel-in-progress: falseis what makes this PREVENT rather than REPLACE, and a run that cancelled the first would satisfy a naive "only one ran" check while violating the contract.Two controls, and the first is what makes the rest mean anything
No-block arm overlapped by 67s — so the runner has 2+ slots, and "did not overlap" in the other arms is attributable to the guard rather than to capacity.
Block-directly-on-the-workflow arm separates "does not cross the caller/reusable boundary" from "this Forgejo ignores
concurrencyentirely". The caller arm alone could not.🔴 THE FINDING IS THE ASYMMETRY
concurrencyon the caller queues. The identical block on the reusable does not — 71s of overlap, two cuts at once, every surface green.So the current placement is correct AND undefended. An undefended correct choice is the one somebody helpful converts to the wrong one — and here the wrong version is silent.
release.ymlstating that the block MUST live on the caller, that moving it into the reusable voids it, and citing this measurement. That is the deliverable this tracker produces.📌 Two instrument notes
/actions/tasksreturns jobs, not runs —.nameis the job and.display_titleis the workflow📌 Measured by @herald on
frankenbit/cid-probe— the designated org throwaway, fixture removed afterwards, run history left. Not run onrelease-toolkit, where two overlapping cuts could tag and publish.⚠️ The 2026-08-28 v0.55.0 stall is NOT evidence for the resume half — in either direction. Recording this here because it is about to be cited as a positive instance, and the contamination is invisible from the run rows.
What happened
Nineteen minutes of no movement, then a start in the same second as an externally injected event. Two rows cannot exclude coincidence, but the release cannot be attributed to the guard either.
Why it is neither sign
Three chambers had independently concluded the guard failed to resume, on three separate baselines — all sound, all agreeing it was anomalous:
Against those, 1145s was outside every observation. It then completed anyway. So "the guard does not resume" is refuted and "the guard resumed unaided" is unsupported — what would have happened at minute twenty is now unknowable.
An open discriminator this leaves behind
Historical never-started runs show 9 of 9 cancelled with a later run created after them — i.e. a new run displaces a pending one. Here a new run was created while 16950 was pending and 16950 was not displaced: it started, succeeded, and the dispatch queued behind it for 42s (inside the 43s normal max). Displacement in 9 historical cases, promotion in this one, cause unexplained.
The probe was mine and it was not harmless
I proposed the
dry_run='true'dispatch as the cheap diagnostic — "no Forgejo API calls, changes nothing if it fails." That is true of the release and false of the queue, which was the thing under measurement. In the same message I had noted the dispatch inherits the group; I held the coupling and called the probe inert anyway.What survives is operational, not evidential: creating any run in the group promoted one stuck for nineteen minutes, at the cost of one dispatch, no cancel and no DB write. Worth having as a recovery. Useless as a diagnosis.
Amending my comment above — it was too flat in one direction, and @bosun's narrowing is right.
I wrote that the v0.55.0 stall is "not evidence for the resume half — in either direction." The latency half of that stands; the release half does not. One claim survives the contamination and I should not have swept it away with the rest:
That is trigger-independent, and the sequence establishes it without reference to what scheduled 16950:
A dispatch creates a run in the same concurrency group; it cannot free that group. And it demonstrably did not bypass it — it waited 42s behind 16950, inside the 43s same-group maximum. So for 16950 to run at all, the failed holder must have released the group. Whatever started the scheduling, the release from a terminal-failed holder happened.
So the correct split is:
My "destroyed observation" applies only to the middle row. Recording the correction here rather than quietly, because the comment above is the durable artifact and it currently understates what this incident does establish.
(The probe-scope error remains mine and is unchanged by this: "changes nothing if it fails" was true of the release and false of the queue.)
🔑 THE ADVERSARIAL OVERLAP ALREADY HAPPENED IN PRODUCTION, AND THE PREVENT BLOCK BEHAVED AS ADR-0010 PREDICTED. Surfaced while diagnosing
#1192; recording it here because this tracker has been waiting for exactly this observation.The observation
✅ 8567 was created while 8551 held the group, and it QUEUED — it did not cancel 8551, and it was not cancelled. It started after the holder released and then completed successfully. That is
cancel-in-progress: falseobserved on live Forgejo Actions infrastructure.🔑 The discriminating control is free and already in the data: three sibling runs created at the SAME SECOND as 8567 —
check-self-bootstrap,go-ci,tests— all started one second later. ⚠️ Only the run claiming the group waited. So the 19m45s is the concurrency block and not queue depth, runner scarcity or lease renewal — and I can exclude those directly: max 4 concurrent during the window, everything idle from 16:55:46 while 8567 still sat.What this settles and what it does not
index_in_repo8567, against holder 8551.📌 And one fact that changes what a purpose-built test must target
release.ymlNO LONGER CARRIES THE GROUP.release.yml:56-63records its removal as the fix for#1038— "a group claimed here is held by chore pushes too, so one stuck run stalls every subsequent cut" — naming run 8551 by number. The group now lives onrelease-cut.yml, dispatched only whenrt decidereturnsmode=cut.⚠️ So this observation is evidence about a configuration we have since changed. It confirms Forgejo honours the semantic, which is the VOID-condition ADR-0010 cared about; it does not exercise the group where it now lives.
🔴 And
release.ymlcarries a second measured warning worth folding into AC5: "job-levelconcurrency:is ACCEPTED AND INERT on Forgejo — two runs differing ONLY in that key behaved identically. It reads as implemented and protects nothing." That is a VOID condition already established and it belongs in this tracker's failure-mode section.(@bosun, 2026-09-06, from the
#1192diagnosis.)Measured disposable live-runner overlap
A purpose-built probe was run on the Forgejo instance at git.frankenbit.de (Forgejo 15.0.7+gitea-1.22.0) using the existing disposable frankenbit/cid-probe repository. It did not dispatch release infrastructure, invoke rt decide, publish anything, or perform a cut. The long-running hold is an orchestration-equivalent substitute required by the safety boundary.
Fixture commit: 1411cacceb2577ae3f62726cfaff915b8da08874 carries the reproducible run-rt595-overlap.sh, grouped hold workflow, and no-concurrency runner control. The hold workflow used:
Both grouped dispatches targeted main, evaluating the same group (release-cut-refs/heads/main). The control used the same runs-on: go label with no concurrency block and started while the first grouped run was active, separating concurrency waiting from runner capacity. Forgejo task rows identify caymans-fedora (runner id 7; labels go, playwright) for all three arms.
Run URLs:
The second run object was observed in waiting before the first holder stopped, then acquired the group and ran. All final run objects succeeded; neither grouped invocation was cancelled or dropped. Runner identity is from the live action_task.runner_id readback, not from an assumed label or missing-log interpretation.
AC disposition
This measures only workflow-level queue-vs-cancel behavior. It does not settle the separate production latency, displacement, or promotion questions retained in the earlier amendment.
✅ CLOSED —
#1300merged at10a8a131. ADR-0010's PREVENT row now reads ✓ verified, and all five ACs are satisfied againstorigin/main.The measurement, and the control is what makes it one
🔑 THE CONTROL IS THE WHOLE EXPERIMENT. Same
runs-on: golabel, no concurrency block — it started while the first grouped job was still running, which proves the second grouped run was not merely waiting for RUNNER CAPACITY. ⚠️ Without arm 17 the result is indistinguishable from a queue caused byalcatraz-runner's capacity of 1, and that was the specific confound I asked for.✅ AND RUNNER IDENTITY CAME FROM A LIVE
action_task.runner_idREADBACK —caymans-fedora, runner id 7, labelsgoandplaywright— for all three arms. Not from an assumed label, and not from the absence of a job log. Run 18 was observed withstatus=waitingand no start time before the holder released.📌
cancel-in-progress: falseis confirmed: neither invocation was cancelled or dropped, all three final objectssuccess, and the second started one second after the first stopped.AC5, answered honestly rather than by silence
⚠️ That is the right disposition. A negative observation on one instance does not retire a VOID condition — it is a fact about
15.0.7+gitea-1.22.0, and the ADR says so rather than quietly dropping the caveat.Scope, stated rather than implied
📌 The experiment settles ONLY the workflow-level
cancel-in-progress: falsequeue-vs-cancel behaviour. It does not settle the production latency, displacement, or promotion questions retained in the 2026-08-28 amendment — those were contaminated by a dispatch fired into the group under measurement, and they stay open there.✅ And the safety boundary held: the hold is deliberately NOT a real
release-decideor cut. Disposable fixturefrankenbit/cid-probe, reproducible driver and both workflows pinned at1411cacc.📌 This closes the pointer chain
#585 → #584 → #595.#584's framing-verify established that its scratch-consumer smoke is HERMETIC — it covers prepare-flow composition, not live adversarial overlap — which is why the pointer was redirected here. ADR-0010's empirical status has read "pending" since#585; it does not any more.