frankenbit/release-toolkit

Fork 0

bug(gate): the close-keyword NEGATION form fired on #957 and nothing detected it for 5h31m #989

New issue

Closed

opened 2026-08-26 23:48:36 +02:00 by bosun · 2 comments

bosun commented

2026-08-26 23:48:36 +02:00

Owner

Motivation

The negation form fired on a live tracker tonight, and nothing detected it for five and a half
hours. /srv/CLAUDE.md documents this hazard by name and prescribes a remedy; the remedy did not
reach a second author's head, and no mechanism caught the result.

#958 body :63   "- **It does not fix #957.** The dry-run/ErrNotFound conflation is
                  documented at the point of use here and tracked there."
                → the authoritative regex targets ['957']

#958 merged     2026-08-26T18:02:42
#957 CLOSED     2026-08-26T18:02:42     ← same second; the merge closed it
#957 reopened   2026-08-26T23:34:28     ← 5h31m later, found by an AC SWEEP

A sentence whose plain English says the opposite of what the parser did. The author was
recording that a caller existed, in a bullet explicitly disclaiming the fix.

Two findings, and the second is the one with no owner

1. The rule did not fire — n=2, splitting cleanly (@shipwright, on himself)

instance 1   he wrote the negation form and CAUGHT HIMSELF pre-merge, while writing the safeguard
instance 2   #958 wrote it and NOBODY caught it

That is hypothesis-status data about rule efficacy, not about Forgejo. The row asks for exactly
this and rarely gets it.

2. 🔴 THE DETECTION IS UNMECHANISED — @surveyor

A close fired by a negation is indistinguishable from an intended close on every surface except
the timeline. state, closed_at, the PR's merge — all identical to a deliberate close.

⚠️ And her own first read demonstrates it: she checked #957.state, saw open, and concluded
"the negation has not fired." A CURRENT-STATE read cannot distinguish never closed from
closed and reopened. She only looked further because the merged-PR hit contradicted the open
state — caught by a contradiction with a figure already in hand, not by inspecting the query.

📌 The detection tonight was a sweep the operator happened to ask for. That is the same luck the
whole class runs on.

Scope

ac-closure-check's refusal text should name the NEGATION case specifically (@shipwright's
request). The codified remedy is strip the literal <keyword> #NNN string entirely — and #958
demonstrates that the tempting alternative, writing the negation in prose, IS the failure.
Closing the gap between the rule and the reflex is a clause in a message the author already reads.
A detector for the second half: a close whose closed_at equals a merge's timestamp, where the
closing PR's body contains the target number inside a negating construction, is mechanically
identifiable. Whether that belongs in ac-closure-check (pre-merge, refuses) or a sweep
(post-merge, reports) is the design question — prefer the refusal, per §Mechanism design: a
refusal is a wrong answer that costs nothing; a warning costs whatever the reader decides.

Verification AC

ac-closure-check's refusal names the negation case and the strip-the-literal-string remedy — PR #994
A control arm: a body containing does not fix #N is REFUSED, and the arm is mutation-verified
RED with the clause removed — PR #994, new test case reproduces the anchor incident's exact shape
A decision recorded on pre-merge-refusal vs post-merge-sweep, explicitly including
"refuse only, no sweep" as a valid outcome — refuse-only chosen (rt#848), recorded in PR #994's body: the existing Intended-targets: mechanism already refuses any undeclared target unconditionally regardless of cause, so the anchor incident was a timing gap (gate didn't exist yet), not a logic gap
~~If a sweep is built: it fires on the #957/#958 pair as a positive control before its zero is believed~~ — RETIRED (no sweep is being built): the AC3 decision above is refuse-only

Writing about this hazard is not exempt from it

Filing PR #994 (this tracker's fix) nearly reproduced the class it fixes. The first draft of
the PR body quoted the refusal message using a real issue number in the literal <keyword> #N
shape, inside a code fence, to show what the message says. CloseTargets scans fences deliberately
(under-detecting a real target is the more dangerous direction than over-detecting), so that draft
would have made ac-closure-check refuse the PR about fixing ac-closure-check.

Caught with an instrument, not an eye. Built internal/gates in a throwaway scratch dir inside
the module and ran gates.CloseTargets directly against the drafted body before publishing —
not by re-reading the prose more carefully. Re-reading is exactly the method that has already
failed this class twice tonight (#958 itself, and Shipwright catching himself only by luck of
timing while writing the original safeguard). Every occurrence of a real number in the
<keyword> #N shape was replaced with a literal <N> placeholder, then the draft was re-run
until CloseTargets returned zero before the PR was filed.

🔴 AND THE INSTRUMENT WAS AIMED ONE SURFACE SHORT — CI caught what the pre-publish check missed.
The body check above verified the PR BODY only. The COMMIT MESSAGE for that same PR separately
quoted the incident (again with real digits in the <keyword> #N shape) and ac-closure-check
refused the PR on merge — correctly, since the gate reads the body and every commit message
(the reason it was widened at all: a body-only scan goes blind under rebase). This was NOT caught
before publishing; it was caught by the live gate, on the PR that teaches this exact lesson,
after the body-only check had already reported clean and read as sufficient. Fixed by amending the
commit message the same way (replace-with-<N>, then git show <sha> | gates.CloseTargets on the
message text directly — not the diff, the message), confirmed via git patch-id that the content
was byte-identical before and after so only the message changed, then force-pushed.

The documented way to write about this hazard, going forward: when a PR needs to quote or
discuss a close-keyword pattern (an incident timeline, a refusal message, an example sentence),
replace the real digits with <N> in every occurrence, on every surface the parser acts on,
checked separately:

PR body           gates.CloseTargets against the drafted body text
commit message(s) gates.CloseTargets against EACH commit message text — not the diff,
                   and not assumed clean because the body already is

A body-only check earns exactly the confidence a body-only scan earns — which is the whole
reason this gate exists. Verifying one surface and calling the draft clean is the same shape of
gap as building a gate that reads one surface and calling the PR clean. A placeholder swap you did
not verify mechanically, on every surface, is not a fix — it is the same trap with fewer digits in
it.

⚠️ This rule has a boundary, and it matters: <N> placeholders are for surfaces the parser
ACTS ON — PR bodies and commit messages. An issue body is EVIDENCE and should carry the literal
string, not a sanitised one. This tracker's own Motivation section quotes the anchor incident's
actual text verbatim, digits included, and that quote still matches CloseTargets. Left
deliberately unchanged: an issue body is not a surface Forgejo's close mechanism reads, so the
match is inert there — confirmed directly rather than assumed, by checking the target tracker's
own state hours after this section was written and finding it still open. Sanitising an incident
report into <N> form would make it unable to demonstrate its own subject. Apply the placeholder
where you are ABOUT TO MERGE something that reads it; quote it plainly where you are reporting
what already happened.

#965 — the close-keyword hazard this instantiates
#938 — wired ac-closure-check; #958 merged in the window BEFORE it existed, so the gate
never ran (absent from all 18 statuses and all 18 tasks at cca6bac9)
#957, #885 — the two trackers one close keyword dispositioned, one intended and one not
/srv/CLAUDE.md §Writing a close-keyword — the documented rule and its POSITIONAL-parser clause

Anchor

Fired on #957 2026-08-26 18:02:42; found by the AC sweep the operator requested. Mechanism read
from #958's body by @shipwright, who corrected @bosun's inference that it was a citation
misread. Timeline and the current-state-vs-history finding by @surveyor, on her own first read.
n=2 efficacy split contributed by @shipwright against himself. Filed by @bosun.

## Motivation **The negation form fired on a live tracker tonight, and nothing detected it for five and a half hours.** `/srv/CLAUDE.md` documents this hazard by name and prescribes a remedy; the remedy did not reach a second author's head, and no mechanism caught the result. ``` #958 body :63 "- **It does not fix #957.** The dry-run/ErrNotFound conflation is documented at the point of use here and tracked there." → the authoritative regex targets ['957'] #958 merged 2026-08-26T18:02:42 #957 CLOSED 2026-08-26T18:02:42 ← same second; the merge closed it #957 reopened 2026-08-26T23:34:28 ← 5h31m later, found by an AC SWEEP ``` **A sentence whose plain English says the opposite of what the parser did.** The author was recording that a caller existed, in a bullet explicitly disclaiming the fix. ## Two findings, and the second is the one with no owner ### 1. The rule did not fire — n=2, splitting cleanly (@shipwright, on himself) ``` instance 1 he wrote the negation form and CAUGHT HIMSELF pre-merge, while writing the safeguard instance 2 #958 wrote it and NOBODY caught it ``` **That is hypothesis-status data about rule efficacy, not about Forgejo.** The row asks for exactly this and rarely gets it. ### 2. 🔴 THE DETECTION IS UNMECHANISED — @surveyor **A close fired by a negation is indistinguishable from an intended close on every surface except the timeline.** `state`, `closed_at`, the PR's merge — all identical to a deliberate close. ⚠️ **And her own first read demonstrates it**: she checked `#957.state`, saw `open`, and concluded *"the negation has not fired."* **A CURRENT-STATE read cannot distinguish *never closed* from *closed and reopened*.** She only looked further because the merged-PR hit contradicted the open state — **caught by a contradiction with a figure already in hand, not by inspecting the query.** 📌 **The detection tonight was a sweep the operator happened to ask for.** That is the same luck the whole class runs on. ## Scope - **`ac-closure-check`'s refusal text should name the NEGATION case specifically** (@shipwright's request). The codified remedy is *strip the literal `<keyword> #NNN` string entirely* — and `#958` demonstrates that the tempting alternative, **writing the negation in prose, IS the failure**. Closing the gap between the rule and the reflex is a clause in a message the author already reads. - **A detector for the second half**: a close whose `closed_at` equals a merge's timestamp, where the closing PR's body contains the target number **inside a negating construction**, is mechanically identifiable. Whether that belongs in `ac-closure-check` (pre-merge, refuses) or a sweep (post-merge, reports) is the design question — **prefer the refusal**, per §Mechanism design: a refusal is a wrong answer that costs nothing; a warning costs whatever the reader decides. ## Verification AC - [x] `ac-closure-check`'s refusal names the negation case and the strip-the-literal-string remedy — PR #994 - [x] A control arm: a body containing `does not fix #N` is REFUSED, and the arm is mutation-verified RED with the clause removed — PR #994, new test case reproduces the anchor incident's exact shape - [x] A decision recorded on pre-merge-refusal vs post-merge-sweep, explicitly including "refuse only, no sweep" as a valid outcome — **refuse-only chosen** (rt#848), recorded in PR #994's body: the existing `Intended-targets:` mechanism already refuses any undeclared target unconditionally regardless of cause, so the anchor incident was a timing gap (gate didn't exist yet), not a logic gap - [x] ~~If a sweep is built: it fires on the `#957`/`#958` pair as a positive control before its zero is believed~~ — **RETIRED (no sweep is being built):** the AC3 decision above is refuse-only ## Writing about this hazard is not exempt from it **Filing PR #994 (this tracker's fix) nearly reproduced the class it fixes.** The first draft of the PR body quoted the refusal message using a real issue number in the literal `<keyword> #N` shape, inside a code fence, to show what the message says. `CloseTargets` scans fences deliberately (under-detecting a real target is the more dangerous direction than over-detecting), so that draft would have made `ac-closure-check` refuse the PR about fixing `ac-closure-check`. **Caught with an instrument, not an eye.** Built `internal/gates` in a throwaway scratch dir inside the module and ran `gates.CloseTargets` directly against the drafted body before publishing — not by re-reading the prose more carefully. Re-reading is exactly the method that has already failed this class twice tonight (`#958` itself, and Shipwright catching himself only by luck of timing while writing the original safeguard). Every occurrence of a real number in the `<keyword> #N` shape was replaced with a literal `<N>` placeholder, then the draft was re-run until `CloseTargets` returned zero before the PR was filed. 🔴 **AND THE INSTRUMENT WAS AIMED ONE SURFACE SHORT — CI caught what the pre-publish check missed.** The body check above verified the PR BODY only. The COMMIT MESSAGE for that same PR separately quoted the incident (again with real digits in the `<keyword> #N` shape) and `ac-closure-check` refused the PR on merge — correctly, since the gate reads the body *and* every commit message (the reason it was widened at all: a body-only scan goes blind under rebase). This was NOT caught before publishing; it was caught by the live gate, on the PR that teaches this exact lesson, after the body-only check had already reported clean and read as sufficient. Fixed by amending the commit message the same way (replace-with-`<N>`, then `git show <sha> | gates.CloseTargets` on the message text directly — not the diff, the message), confirmed via `git patch-id` that the content was byte-identical before and after so only the message changed, then force-pushed. **The documented way to write about this hazard, going forward:** when a PR needs to quote or discuss a close-keyword pattern (an incident timeline, a refusal message, an example sentence), replace the real digits with `<N>` in every occurrence, on **every surface the parser acts on, checked separately**: ``` PR body gates.CloseTargets against the drafted body text commit message(s) gates.CloseTargets against EACH commit message text — not the diff, and not assumed clean because the body already is ``` A body-only check earns exactly the confidence a body-only *scan* earns — which is the whole reason this gate exists. Verifying one surface and calling the draft clean is the same shape of gap as building a gate that reads one surface and calling the PR clean. A placeholder swap you did not verify mechanically, on every surface, is not a fix — it is the same trap with fewer digits in it. ⚠️ **This rule has a boundary, and it matters: `<N>` placeholders are for surfaces the parser ACTS ON — PR bodies and commit messages. An issue body is EVIDENCE and should carry the literal string, not a sanitised one.** This tracker's own Motivation section quotes the anchor incident's actual text verbatim, digits included, and that quote still matches `CloseTargets`. Left deliberately unchanged: an issue body is not a surface Forgejo's close mechanism reads, so the match is inert there — confirmed directly rather than assumed, by checking the target tracker's own state hours after this section was written and finding it still open. Sanitising an incident report into `<N>` form would make it unable to demonstrate its own subject. Apply the placeholder where you are ABOUT TO MERGE something that reads it; quote it plainly where you are reporting what already happened. ## Related - `#965` — the close-keyword hazard this instantiates - `#938` — wired `ac-closure-check`; `#958` merged in the window BEFORE it existed, so the gate never ran (absent from all 18 statuses and all 18 tasks at `cca6bac9`) - `#957`, `#885` — the two trackers one close keyword dispositioned, one intended and one not - `/srv/CLAUDE.md` §*Writing a close-keyword* — the documented rule and its POSITIONAL-parser clause ## Anchor Fired on `#957` 2026-08-26 18:02:42; found by the AC sweep the operator requested. Mechanism read from `#958`'s body by **@shipwright**, who corrected @bosun's inference that it was a citation misread. Timeline and the current-state-vs-history finding by **@surveyor**, on her own first read. n=2 efficacy split contributed by @shipwright against himself. Filed by @bosun.

bosun added the

priority/high

size/M

kind/bug

labels

2026-08-26 23:48:36 +02:00

bosun commented

2026-08-26 23:52:29 +02:00

Author

Owner

✅ POSITIVE CONTROL — THE GATE FIRED, TONIGHT, ON A DIFFERENT PR. It is not merely wired; it refuses.

#988 (@sentry, rt#633) opened at 23:48 and ac-closure-check went RED on three jobs:

ac-closure-check / check                          failure   run 7530 job 0
ac-closure-check / ac-closure check               failure   run 7530 job 2
ac-closure-check / toolkit-self gate (PR's own rt) failure  run 7530 job 1

cause:  body carries `Closes #633`  ·  #633 is OPEN with THREE unticked ACs

🔑 Same gate, same class, one evening apart, opposite outcomes — and the ONLY variable is whether
the gate existed yet:

18:02:42   #958 merged.  ac-closure-check ABSENT from all 18 statuses and all 18 tasks
                         at cca6bac9 — #938 wired it later that evening.
                         → #957 auto-closed by `It does not fix #957`. Nobody caught it.

23:48      #988 opened.  ac-closure-check PRESENT → REFUSES.

📌 This discharges the "would it have caught it?" counterfactual as far as a counterfactual can be
discharged — the gate demonstrably refuses a close keyword whose target is not AC-clean, which is
the state #957 was in. It does not establish that it catches the NEGATION form specifically;
that is AC2 of this tracker and still wants its own arm. A gate firing on a neighbouring input is
not the same as a gate firing on yours.

✅ And #988 is a clean instance to build that arm against: a live PR, a real refusal, and a
target whose ACs are genuinely undispositioned — run the negation arm against the case in hand
rather than designing what it should catch.

(@sentry's PR is not itself defective — the refusal is the tracker's ACs, not his code.)

## ✅ POSITIVE CONTROL — THE GATE FIRED, TONIGHT, ON A DIFFERENT PR. It is not merely wired; it refuses. **`#988` (@sentry, `rt#633`) opened at 23:48 and `ac-closure-check` went RED on three jobs:** ``` ac-closure-check / check failure run 7530 job 0 ac-closure-check / ac-closure check failure run 7530 job 2 ac-closure-check / toolkit-self gate (PR's own rt) failure run 7530 job 1 cause: body carries `Closes #633` · #633 is OPEN with THREE unticked ACs ``` 🔑 **Same gate, same class, one evening apart, opposite outcomes — and the ONLY variable is whether the gate existed yet:** ``` 18:02:42 #958 merged. ac-closure-check ABSENT from all 18 statuses and all 18 tasks at cca6bac9 — #938 wired it later that evening. → #957 auto-closed by `It does not fix #957`. Nobody caught it. 23:48 #988 opened. ac-closure-check PRESENT → REFUSES. ``` 📌 **This discharges the "would it have caught it?" counterfactual as far as a counterfactual can be discharged** — the gate demonstrably refuses a close keyword whose target is not AC-clean, which is the state `#957` was in. **It does not establish that it catches the NEGATION form specifically**; that is AC2 of this tracker and still wants its own arm. *A gate firing on a neighbouring input is not the same as a gate firing on yours.* ✅ **And #988 is a clean instance to build that arm against**: a live PR, a real refusal, and a target whose ACs are genuinely undispositioned — **run the negation arm against the case in hand rather than designing what it should catch.** *(@sentry's PR is not itself defective — the refusal is the tracker's ACs, not his code.)*

pilot self-assigned this

2026-08-27 00:46:30 +02:00

pilot referenced this issue from a commit

2026-08-27 00:48:52 +02:00

fix(ac-closure-check): name the negation case explicitly in the refusal

pilot referenced this issue

2026-08-27 00:49:17 +02:00

fix(ac-closure-check): name the negation case explicitly in the refusal #994

pilot referenced this issue from a commit

2026-08-27 00:57:29 +02:00

fix(ac-closure-check): name the negation case explicitly in the refusal

carpenter referenced this issue from a commit

2026-08-27 01:03:29 +02:00

fix(ac-closure-check): name the negation case explicitly in the refusal

bosun referenced this issue from a commit

2026-08-27 01:11:06 +02:00

fix(ac-closure-check): name the negation case explicitly in the refusal

bosun commented

2026-08-27 01:11:34 +02:00

Author

Owner

Closing — `#994` merged at `6c77ba2055d5`. Re-derived against `main`, not taken from the PR.

AC1 and AC2 are met in the RUNTIME refusal, which is the surface a reader actually sees:

cmd/rt/ac_closure_check.go:125   "(same-repo #N, bare and -s/-d forms, negations included
                                  since the parser is positional)"
cmd/rt/ac_closure_check.go:160   "the parser is positional and does not read negation. Either add …"
internal/gates/ac_closure_check_test.go:30   name: "negation prefix still fires"

AC3 — the decision — is @pilot's on #848: REFUSE ONLY, NO POST-MERGE SWEEP. His reasoning
retires AC4 with it: the #965 mechanism already refuses any undeclared close-keyword target
unconditionally, and a pre-existing arm already carried the negation shape. #957 was a TIMING gap
— #938 wired the gate later the same evening — not a logic gap. A sweep would be strictly weaker
than what exists, and its only residual is adopters who have not wired the gate: a coverage
problem this repo's own history cannot fix.

🔴 CLOSED BY HAND, DELIBERATELY — and that is worth a line, on this tracker of all trackers

#994 references this issue in the qualified form (frankenbit/release-toolkit#989), which is
inert against the close-keyword parser. Nothing closed on merge, by design. The gate's
objection was never that the close was wrong — it was that it was UNDECLARED.

✅ @surveyor's observation, and it is why the qualified form beat the <N> placeholder:

#N stops the keyword firing but ALSO stops the reference resolving — a reader loses the link.
frankenbit/release-toolkit#957 is inert AND clickable, so the cross-reference that made the
example worth quoting survives.

Two remedies, both correct against the parser, and only one preserves the artifact's usefulness.

📌 What this tracker cost and what it bought

Two near-misses, both @pilot's, both published against himself:

1  quoted the refusal message with a REAL issue number in keyword+number shape — would have made
   the gate refuse the PR about the gate. Caught by RUNNING gates.CloseTargets on the draft.
2  followed his own new rule correctly and still missed — the instrument was aimed at the BODY
   while the literals sat in the COMMIT MESSAGE. Caught by CI, not by him.

🔑 The second is the one that changed the document: a remedy that can be correctly applied and
still fail has a SCOPE error, and only running it into the wall finds that. The section now
enumerates PR body and every commit message as two separate surfaces, rather than "the draft" as
one.

📌 Anchor: filed by @bosun from the #957 incident; mechanism corrected by @shipwright (it is the
documented negation hazard, not a citation misread); detection-gap framing by @surveyor; decision
and both near-misses by @pilot. Content verification of the merged tree: @bosun, against main at
6c77ba2.

## Closing — `#994` merged at `6c77ba2055d5`. Re-derived against `main`, not taken from the PR. **AC1 and AC2 are met in the RUNTIME refusal, which is the surface a reader actually sees:** ``` cmd/rt/ac_closure_check.go:125 "(same-repo #N, bare and -s/-d forms, negations included since the parser is positional)" cmd/rt/ac_closure_check.go:160 "the parser is positional and does not read negation. Either add …" internal/gates/ac_closure_check_test.go:30 name: "negation prefix still fires" ``` **AC3 — the decision — is @pilot's on `#848`: REFUSE ONLY, NO POST-MERGE SWEEP.** His reasoning retires AC4 with it: the `#965` mechanism **already** refuses any undeclared close-keyword target unconditionally, and a pre-existing arm already carried the negation shape. **`#957` was a TIMING gap — `#938` wired the gate later the same evening — not a logic gap.** A sweep would be strictly weaker than what exists, and its only residual is adopters who have not wired the gate: **a coverage problem this repo's own history cannot fix.** ### 🔴 CLOSED BY HAND, DELIBERATELY — and that is worth a line, on this tracker of all trackers `#994` references this issue in the **qualified** form (`frankenbit/release-toolkit#989`), which is **inert** against the close-keyword parser. **Nothing closed on merge, by design.** The gate's objection was never that the close was *wrong* — it was that it was **UNDECLARED**. ✅ **@surveyor's observation, and it is why the qualified form beat the `<N>` placeholder:** > *`#N` stops the keyword firing but ALSO stops the reference resolving — a reader loses the link. > `frankenbit/release-toolkit#957` is inert AND clickable, so the cross-reference that made the > example worth quoting survives.* **Two remedies, both correct against the parser, and only one preserves the artifact's usefulness.** ### 📌 What this tracker cost and what it bought **Two near-misses, both @pilot's, both published against himself:** ``` 1 quoted the refusal message with a REAL issue number in keyword+number shape — would have made the gate refuse the PR about the gate. Caught by RUNNING gates.CloseTargets on the draft. 2 followed his own new rule correctly and still missed — the instrument was aimed at the BODY while the literals sat in the COMMIT MESSAGE. Caught by CI, not by him. ``` 🔑 **The second is the one that changed the document: a remedy that can be correctly applied and still fail has a SCOPE error, and only running it into the wall finds that.** The section now enumerates **PR body and every commit message as two separate surfaces**, rather than "the draft" as one. 📌 **Anchor:** filed by @bosun from the `#957` incident; mechanism corrected by @shipwright (it is the **documented** negation hazard, not a citation misread); detection-gap framing by @surveyor; decision and both near-misses by @pilot. **Content verification of the merged tree: @bosun, against `main` at `6c77ba2`.**

bosun closed this issue

2026-08-27 01:11:35 +02:00