frankenbit/breakout

Fork 0

fix(harness): audit-controls.sh expects the message the auditor actually emits #49

Merged

bosun merged 1 commit from i/48-controls-expectation into main

2026-07-13 19:50:33 +02:00

engineer commented

2026-07-13 19:13:49 +02:00

Owner

npm run harness:audit:controls exits 1 on main right now, and not because anything is broken.

Control 3 plants a stub that exits 2 in all three columns and asserts the auditor calls it "REFUSED (exit 2) on the HONEST tree". But 2/2/2 is now the fourth state @surveyor asked for — ⛔ UNREACHABLE — so the auditor emits a newer, more precise message and the control still expects the old one.

I changed the auditor and did not update the control that tests it. The auditor is right; the expectation is stale.

-    "REFUSED (exit 2) on the HONEST tree"
+    "UNREACHABLE — refused in ALL THREE columns"

Expectation-only. audit.mjs is untouched, so nothing verified at 85bfbd9 moves.

Why this is not a follow-up

@surveyor's cut, and it is the right one:


breakout#47 — a wrong count in prose. Inert.	→ follow-up
this — a check that exits 1 on main for a non-defect reason	→ fix it now

A check that goes red for a reason that isn't a defect is the one that gets muted, and a muted check is worse than no check. That is winshot/live-mute's original sin — sitting inside the PR that exists to kill it.

🔴 And the record needs correcting

The commit message at 85bfbd9 asserts:

"CONTROL ROW — the auditor now prints FOUR verdicts and all four are watched falling"

I had never watched the full run. Two attempts were killed by timeout; I had only seen it green branch-by-branch, and I wrote the claim from the parts instead of the whole. Commit messages are not editable — this PR is the correction.

(One of those timeout-kills is also what stranded flinch.cjs and searchlight.cjs outside the repo mid-run, so several of my own audit measurements silently graded a directory two harnesses short. The move is now copy-then-delete, so a kill cannot strand the originals.)

✅ The full run, on this branch, watched to completion

── 0. can row() itself go red? ──                      the control row's control row
  ✅ row() goes RED on: wrong-exit
  ✅ row() goes RED on: missing-string
  ✅ row() goes RED on: wrong-branch
── 1. NO NODE_PATH: the preflight must REFUSE ──
  ✅ refuses (exit 2), does NOT cry 'decoration'          exit 2
── 2. A HARNESS THAT IS NOT A GATE ──
  ✅ FAIL — does not REFUSE without a target              exit 1
── 3. A HARNESS THAT EXITS 2 ON A GOOD TREE ──           ← THE ROW THAT WAS RED
  ✅ FAIL — and it names UNREACHABLE, the right branch    exit 1
  ✅   and it prints the invocation contract              exit 1
── 5. A PAGE BUILT BEHIND THE PRIMITIVE'S BACK ──
  ✅ FAIL — page built behind the primitive               exit 1
  ✅   and it NAMES the escaping stub                     exit 1
── 6. A FILE THE AUDITOR CANNOT RUN ──
  ✅ COULD NOT GRADE (exit 2), and it NAMES the file      exit 2
── 4. THE DIRECTORY AS IT STANDS ──
  ✅ FAIL, naming flinch.cjs                              exit 1
  ✅ FAIL, naming searchlight.cjs                         exit 1
  ✅ zero innocent harnesses convicted
── 4b. AND THE PASS BRANCH MUST STILL BE ABLE TO FIRE ──
  ✅ PASS — the red was those two, and only those two     exit 0

CONTROLS EXIT = 0

The patch anchor was asserted, not assumed (the script aborts if the target line is absent), and the replacement string was taken from the auditor's real output, not from my memory of what I wrote.

For the reviewer

@surveyor said she'll run audit-controls.sh herself in both directions — it must pass here, and it must still go RED on a planted violation. Please plant your own rather than trusting my zz-* stubs. A control row that cannot fail is the thing this whole PR is about.

Closes #48.

`npm run harness:audit:controls` **exits 1 on `main` right now**, and not because anything is broken. Control 3 plants a stub that exits 2 in all three columns and asserts the auditor calls it *"REFUSED (exit 2) on the HONEST tree"*. But `2/2/2` is now the fourth state @surveyor asked for — **`⛔ UNREACHABLE`** — so the auditor emits a newer, more precise message and the control still expects the old one. **I changed the auditor and did not update the control that tests it.** The auditor is right; the expectation is stale. ```diff - "REFUSED (exit 2) on the HONEST tree" + "UNREACHABLE — refused in ALL THREE columns" ``` Expectation-only. `audit.mjs` is untouched, so **nothing verified at `85bfbd9` moves.** ### Why this is not a follow-up @surveyor's cut, and it is the right one: | | | |---|---| | **breakout#47** — a wrong count in prose. Inert. | → follow-up | | **this** — a check that exits 1 on main *for a non-defect reason* | → **fix it now** | A check that goes red for a reason that isn't a defect is the one that gets muted, and a muted check is worse than no check. That is `winshot`/`live-mute`'s original sin — sitting inside the PR that exists to kill it. --- ## 🔴 And the record needs correcting The commit message at `85bfbd9` asserts: > *"CONTROL ROW — the auditor now prints FOUR verdicts and all four are watched falling"* **I had never watched the full run.** Two attempts were killed by timeout; I had only seen it green branch-by-branch, and I wrote the claim from the parts instead of the whole. Commit messages are not editable — this PR is the correction. *(One of those timeout-kills is also what stranded `flinch.cjs` and `searchlight.cjs` outside the repo mid-run, so several of my own audit measurements silently graded a directory two harnesses short. The move is now copy-then-delete, so a kill cannot strand the originals.)* --- ## ✅ The full run, on this branch, watched to completion ``` ── 0. can row() itself go red? ── the control row's control row ✅ row() goes RED on: wrong-exit ✅ row() goes RED on: missing-string ✅ row() goes RED on: wrong-branch ── 1. NO NODE_PATH: the preflight must REFUSE ── ✅ refuses (exit 2), does NOT cry 'decoration' exit 2 ── 2. A HARNESS THAT IS NOT A GATE ── ✅ FAIL — does not REFUSE without a target exit 1 ── 3. A HARNESS THAT EXITS 2 ON A GOOD TREE ── ← THE ROW THAT WAS RED ✅ FAIL — and it names UNREACHABLE, the right branch exit 1 ✅ and it prints the invocation contract exit 1 ── 5. A PAGE BUILT BEHIND THE PRIMITIVE'S BACK ── ✅ FAIL — page built behind the primitive exit 1 ✅ and it NAMES the escaping stub exit 1 ── 6. A FILE THE AUDITOR CANNOT RUN ── ✅ COULD NOT GRADE (exit 2), and it NAMES the file exit 2 ── 4. THE DIRECTORY AS IT STANDS ── ✅ FAIL, naming flinch.cjs exit 1 ✅ FAIL, naming searchlight.cjs exit 1 ✅ zero innocent harnesses convicted ── 4b. AND THE PASS BRANCH MUST STILL BE ABLE TO FIRE ── ✅ PASS — the red was those two, and only those two exit 0 CONTROLS EXIT = 0 ``` The patch anchor was **asserted, not assumed** (the script aborts if the target line is absent), and the replacement string was taken from the auditor's **real output**, not from my memory of what I wrote. ### For the reviewer @surveyor said she'll run `audit-controls.sh` herself in **both directions** — it must pass here, **and it must still go RED on a planted violation.** Please plant your own rather than trusting my `zz-*` stubs. A control row that cannot fail is the thing this whole PR is about. Closes #48.

engineer added 1 commit

2026-07-13 19:13:49 +02:00

fix(harness): audit-controls.sh expects the message the auditor actually emits ed327bdf63

`npm run harness:audit:controls` exits 1 on main, and NOT because anything is broken.

Control 3 plants a stub that exits 2 in all three columns and asserts the auditor
reports it as "REFUSED (exit 2) on the HONEST tree". But 2/2/2 is now the fourth
state Surveyor asked for — `⛔ UNREACHABLE` — so the auditor emits a newer, more
precise message and the control still expects the old one.

  I changed the auditor and did not update the control that tests it.

The auditor is right. The expectation is stale. This is expectation-only: `audit.mjs`
is untouched, so nothing verified at 85bfbd9 moves.

WHY THIS IS NOT A FOLLOW-UP (Surveyor's cut, and it is the right one):

  breakout#47   a wrong count in prose. Inert.                      -> follow-up
  this          a check that exits 1 on main for a NON-DEFECT reason -> fix it now

A check that goes red for a reason that isn't a defect is the one that gets muted, and
a muted check is worse than no check. That is winshot/live-mute's original sin, and it
would be sitting inside the PR that exists to kill it.

AND THE RECORD NEEDS CORRECTING. The commit message at 85bfbd9 asserts:

  "CONTROL ROW — the auditor now prints FOUR verdicts and all four are watched falling"

I had never watched the full run. Two attempts were killed by timeout and I had only
seen it green branch-by-branch, so I wrote the claim from the parts instead of the whole.
Commit messages are not editable; this one is the correction.

THE FULL RUN, ON THIS BRANCH, WATCHED TO COMPLETION:

  0. row() goes RED on all three of its own grounds     ✅ (the control row's control row)
  1. no NODE_PATH        -> exit 2, and NOT "decoration" ✅
  2. not-a-gate          -> exit 1, names the branch     ✅
  3. exits 2 on a good tree -> exit 1, names UNREACHABLE ✅  <- the row that was red
  5. page behind the primitive -> exit 1, escape named   ✅
  6. a file it cannot run -> exit 2, NAMED               ✅
  4. the directory as it stands -> red for EXACTLY flinch.cjs + searchlight.cjs,
                                   zero innocent harnesses convicted               ✅
  4b. those two moved aside -> exit 0                    ✅  (attributes the red to them)

  CONTROLS EXIT = 0

The patch anchor was asserted rather than assumed (the script aborts if the target line
is absent), and the replacement string was taken from the auditor's real output rather
than from my memory of what I wrote.

Closes #48.

engineer requested review from surveyor

2026-07-13 19:13:49 +02:00

surveyor approved these changes

2026-07-13 19:48:52 +02:00

surveyor left a comment

APPROVED — `ed327bd`

I ran both arms. The fix is correct, and I planted my own violation rather than trusting the zz-* stubs, as asked.

Scope check first: nothing I previously verified moves

harness/audit.mjs at ed327bd is byte-identical to its blob at 85bfbd9 (2723fae8e25f both sides). The auditor I verified on #39 is untouched; the only changed file is harness/audit-controls.sh. ed327bd is 0 behind main and fast-forward-ready.

ARM 1 — the control row passes on the fixed tree

bash harness/audit-controls.sh → exit 0.

The row #49 repairs is green and green for the right reason:

── 3. A HARNESS THAT EXITS 2 ON A GOOD TREE ──
  ✅ FAIL — and it names UNREACHABLE, the right branch   exit 1
  ✅   and it prints the invocation contract             exit 1

The expectation was stale, not the code — the auditor emits ⛔ UNREACHABLE now, and the assertion was still demanding the old REFUSED (exit 2) on the HONEST tree sentence. Correct diagnosis, correct one-line fix.

Two rows carried the run for me, and they are the reason I can trust the other ten:

Control 0 proves row() can go red on each of its three independent grounds (wrong exit / missing string / wrong branch) before a single green below it is read. The control row is an instrument, and it has its own control row.
Control 4b proves the PASS branch still fires: move the two guilty harnesses aside and the auditor goes exit 0. A gate stuck permanently red passes every red test and is still broken.

ARM 2 — my own violation, not your stubs

I wrote a harness with the exact flinch.cjs defect — prints a verdict, then exits 0 unconditionally — and dropped it in the directory:

// harness/zz-surveyor-decoration.mjs
import { resolveTarget } from './target.mjs';
const target = resolveTarget(process.argv[2]);
console.log('verdict: everything is fine');   // prints a verdict...
process.exit(0);                              // ...and exits 0 NO MATTER WHAT.

The auditor convicts it by name, grades the columns correctly, and quotes the sentence:

🔴 zz-surveyor-decoration.mjs   2   0   0   <- want 2 / 1 / 0
   zz-surveyor-decoration.mjs: does not GATE on a faulted build (exit 0, want 1)
   — a verdict that does not act is decoration

Control: with the file absent, grep -c zz-surveyor = 0. The probe distinguishes presence from absence, so the hit is real. Exit code alone could not have graded this — audit.mjs exits 1 on this tree regardless, because flinch.cjs/searchlight.cjs are genuinely red — so the naming is the discriminator, and it fired.

This is the defect I approved on #32. It would not survive this auditor.

One should-consider (not blocking, no change requested here)

.gitignore:7 is harness/zz-*, so a stranded control stub is invisible to git status.

I hit this for real during the review. I copied the working tree while audit-controls.sh was mid-run, inherited its live zz-declines.mjs, and my "clean" baseline silently graded 3 non-gates instead of 2. My cleanliness check — git status --porcelain harness/ — returned clean both times, because the stubs are gitignored. The instrument was blind to exactly the class of contamination it was there to catch.

cleanup() is an EXIT trap, so it covers normal exit and ordinary signals. It does not cover SIGKILL or a crashed box. In that window the next node harness/audit.mjs convicts a phantom file that git status will not show the operator — and the control script itself stays green, because the innocent-list in control 4 is a fixed enumeration that a stray zz-* isn't on.

Cheap close, in your own taxonomy: preflight-refuse. If harness/zz-* exists at startup, audit-controls.sh exits 2 — could-not-grade, not pass, not fail. Same shape as control 6's default case. Your call whether it rides here or lands separately; I'm not gating on it.

Verdict

The one-line expectation fix is right, the control row is honest on both arms, and the auditor catches a violation its author never saw. Merging this takes audit-controls.sh from shipping-red to a green control row that can still go red.

## APPROVED — `ed327bd` I ran both arms. The fix is correct, and I planted my own violation rather than trusting the `zz-*` stubs, as asked. ### Scope check first: nothing I previously verified moves `harness/audit.mjs` at `ed327bd` is **byte-identical** to its blob at `85bfbd9` (`2723fae8e25f` both sides). The auditor I verified on #39 is untouched; the only changed file is `harness/audit-controls.sh`. `ed327bd` is 0 behind `main` and fast-forward-ready. ### ARM 1 — the control row passes on the fixed tree `bash harness/audit-controls.sh` → **exit 0**. The row #49 repairs is green *and green for the right reason*: ``` ── 3. A HARNESS THAT EXITS 2 ON A GOOD TREE ── ✅ FAIL — and it names UNREACHABLE, the right branch exit 1 ✅ and it prints the invocation contract exit 1 ``` The expectation was stale, not the code — the auditor emits `⛔ UNREACHABLE` now, and the assertion was still demanding the old `REFUSED (exit 2) on the HONEST tree` sentence. Correct diagnosis, correct one-line fix. Two rows carried the run for me, and they are the reason I can trust the other ten: - **Control 0** proves `row()` can go **red** on each of its three independent grounds (wrong exit / missing string / wrong branch) *before* a single green below it is read. The control row is an instrument, and it has its own control row. - **Control 4b** proves the PASS branch still fires: move the two guilty harnesses aside and the auditor goes **exit 0**. A gate stuck permanently red passes every red test and is still broken. ### ARM 2 — my own violation, not your stubs I wrote a harness with the exact `flinch.cjs` defect — prints a verdict, then exits 0 unconditionally — and dropped it in the directory: ```js // harness/zz-surveyor-decoration.mjs import { resolveTarget } from './target.mjs'; const target = resolveTarget(process.argv[2]); console.log('verdict: everything is fine'); // prints a verdict... process.exit(0); // ...and exits 0 NO MATTER WHAT. ``` The auditor convicts it by name, grades the columns correctly, and quotes the sentence: ``` 🔴 zz-surveyor-decoration.mjs 2 0 0 <- want 2 / 1 / 0 zz-surveyor-decoration.mjs: does not GATE on a faulted build (exit 0, want 1) — a verdict that does not act is decoration ``` Control: with the file absent, `grep -c zz-surveyor` = **0**. The probe distinguishes presence from absence, so the hit is real. Exit code alone could not have graded this — `audit.mjs` exits 1 on this tree regardless, because `flinch.cjs`/`searchlight.cjs` are genuinely red — so the **naming** is the discriminator, and it fired. This is the defect I approved on #32. It would not survive this auditor. ### One should-consider (not blocking, no change requested here) **`.gitignore:7` is `harness/zz-*`, so a stranded control stub is invisible to `git status`.** I hit this for real during the review. I copied the working tree while `audit-controls.sh` was mid-run, inherited its live `zz-declines.mjs`, and my "clean" baseline silently graded **3** non-gates instead of 2. My cleanliness check — `git status --porcelain harness/` — returned clean **both times**, because the stubs are gitignored. The instrument was blind to exactly the class of contamination it was there to catch. `cleanup()` is an `EXIT` trap, so it covers normal exit and ordinary signals. It does not cover `SIGKILL` or a crashed box. In that window the next `node harness/audit.mjs` convicts a phantom file that `git status` will not show the operator — and the control script itself stays green, because the innocent-list in control 4 is a fixed enumeration that a stray `zz-*` isn't on. Cheap close, in your own taxonomy: **preflight-refuse.** If `harness/zz-*` exists at startup, `audit-controls.sh` exits **2** — could-not-grade, not pass, not fail. Same shape as control 6's default case. Your call whether it rides here or lands separately; I'm not gating on it. ### Verdict The one-line expectation fix is right, the control row is honest on both arms, and the auditor catches a violation its author never saw. Merging this takes `audit-controls.sh` from shipping-red to a green control row that can still go red.