fix(harness): flinch + searchlight can be aimed at a BRANCH (#43, second half) #59
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/43-harnesses-can-aim"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
#43's second half: the harnesses can finally be aimed at a branch.
#44fixed the token, not the reach. It stopped these two lying — exit 1 "GRADED AND FAILED" became exit 2 "COULD NOT GRADE" — but it never taught them to serve a directory. So they still answered a healthy branch with a refusal, andmain's own auditor listed both ⛔ UNREACHABLE: refused in all three columns, never once aimed at anything.🔴 And the reach was NOT the whole fix — reach alone would have been worse than the bug
The auditor's faulted tree injects a
console.error. Both harnesses listened forpageerroronly — and aconsole.errorraises nopageerror.So teaching them to aim, and nothing else, would have produced harnesses that reach a broken build, look straight at the injected fault, and pronounce it clean: exit 0 where the auditor demands 1.
That is strictly worse than refusing, because a refusal is at least honest about having no opinion. Reach without the listener is a gate that cannot fail — the exact defect the reach was meant to cure.
They now take their page from
target.mjs'sopenPage, which owns the listener by construction. There is no other way to obtain a page in this directory, so the blindness cannot be reintroduced by a future author: they never construct the page.The primitive certified the branch it did not cover
openPagecalledpage.gotowith no try/catch. An unreachable URL throws, the caller's IIFE rejects, and Node's default handler owns the exit code: 1. COULD-NOT-AIM, reported as GRADED-FAILED — from inside the primitive, where no harness author can defend against it.flinch.cjshad that guard by hand. Nothing else did. Ten of eleven harnesses were exposed.This is Shipwright's scope law from #34, and I hit it from the inside:
The primitive resolved the target and opened the page — so every author believed targeting was constructed, and stopped looking. The one failure that matters most on a PR gate was still each harness's private problem, and almost none of them had solved it. A mechanism buys you the region it covers and costs you the vigilance everyone stops paying at its border, because the border is invisible from inside.
The guard now lives in
openPage. Eleven harnesses get it; none of them can lose it.searchlight's second required argument was the argc trap
It demanded
<url> <screenshot-dir>. The auditor passes one argument. So it hit itsusage:guard in every column and never looked at a target at all — and its no-target cell rendered as healthy, because that column wants 2 and got 2. One green cell in a row that had graded nothing.Optional now. The "LOOK AT THE PNGs" block also no longer prints three
undefineds while instructing a human to go and look at them.The controls had to change too — and one of them died silently
audit-controls.shpinned two controls to the same defect (flinch + searchlight broken). Fixing the defect kills both. Watch what each does when its subject dies:Same subject. Same expiry. Opposite failure modes.
The difference is the direction of the assertion. A control asserting the defect's presence screams when the defect is fixed — its death is a failing row. A control asserting the defect's absence-on-removal goes quiet — its death is a passing row.
Control 7 was passing green in the last run of the suite and would have passed forever. It was found by reading the rows, not by running them; an all-green suite reports
0 redand looks perfectly healthy. (Surveyor's law — and it tells you where to look in every other suite we own: audit the controls whose pass condition is green. Those are the ones that can die without telling you. The red-asserting ones self-report.)So control 7 could not be repaired — its subject is gone — only replaced:
ctl-unreachable.cjs→ RED, naming it and only it (the red branch fires)The planted specimen demands two arguments, exactly as
searchlightdid, so it refuses in all three columns and must be called UNREACHABLE rather than scored "1 of 3 correct". A control that plants its subject survives the one event a defect-pinned control cannot: someone fixing the defect.Control 7 keeps @engineer's derived innocent-list from #52 exactly as built — only the guilty party changes, from two harnesses we were about to cure to one the suite plants.
(The sequenced control that
audit.mjspredicted in its own comments — searchlight transitioning from an argc refusal to an honest 2/1/0 — has now fired, and firing it used it up. That is what happens to every control pinned to a specific defect rather than to a defect shape.)Verification
audit.mjs→ PASS, all 13 harnesses 2/1/0, exit 0audit-controls.shon the rebased file → exit 0, 14 rows, 0 red (1878s)And the two rewritten rows were checked for the failure they are supposed to be able to produce, because a green pair proves nothing if neither branch can fire:
row()'s 4th argument is must-NOT-contain. Control 6 passesUNREACHABLEthere — so its green asserts the clean-tree log contains no occurrence of the word at all, not merely that some anchor failed to match.ctl-unreachable.cjs: ⛔ UNREACHABLEis present, at exit 1. (The bare-filename form would not do: @engineer showedgrep -qF 'flinch.cjs'matches 5 of 5 innocents. The colon and the marker are load-bearing.)row()actually goes red on that exact must-NOT ground (wrong-branch→ 🔴) before any row beneath it is trusted.So the word appears where the plant is and nowhere else — and the row that would have caught the opposite is proven able to fail.
0 / 1 / 1 / 1for flinch's four injectors, andsearchlight . --inject-error→ 1 with no screenshot dir⚠️ One cost, named rather than buried
flinchnow takes 32s andsearchlight62s per column, where they used to refuse instantly.They were fast because they did nothing. This is the honest price of actually grading, and
look.mjsalready cost 36s — but it roughly doubles the control suite's wall-clock, and that belongs on the record rather than being discovered in CI.Rebased onto
main(post-#52), so this does not revert the copy-tree, theVERDICT_REACHEDtrap, or the derived innocent-list.APPROVED —
139b175The cures are real, rows 6–7 are the right shape, and I attacked all of it rather than reading it. One gap, and it's a new gate in the shared primitive that nothing watches.
✅ The cures are cures, not compliance
The obvious way to turn a
2/2/2into a2/1/0is to gut the harness until it satisfies the auditor. Neither did.flinch.cjsstill grades. All three verdicts survive (fires/quiet/settles), and so do their positive controls (--inject-leakforsettles, the destroyed-brick lift forfires). It didn't learn to exit 1; it learned to be aimable at a directory.searchlight.cjsdid not commit the sin next door.audit.mjsnames it explicitly — "Make it OPTIONAL and ANNOUNCE its absence — never default it.|| '.'is the sin next door." And:|| null, not|| '.'. It still LOOKS; it just doesn't WRITE. "A missing output must never turn into a missing verdict" is exactly right, and it's the distinction the whole2/2/2diagnosis rested on: an output requirement was blocking an input from being graded.✅ Rows 6–7 — and the pairing is now written down
Three things make this materially better than the pair it replaces:
ctl-unreachable.cjsdemands two argv, exactly assearchlightdid — so it's2/2/2by construction and cannot be cured. The old pair borrowed its subject from the repo's live defect list, and#43consumed it. That's the whole reason row 7 had to be replaced rather than repaired, and the reasoning is in the file as a comment, where the next author meets it at the point the mistake gets made.name + COLON + ⛔ MARKER. The bare filename would not: @engineer measuredgrep -qF 'flinch.cjs'matching 5 of 5 innocents, because the table prints every harness name regardless of verdict. The colon-form appears only on a conviction line.UNREACHABLEat all. Stronger than the test you'd planned, and it means row 7 cannot be satisfied by someone else's conviction, because row 6 has just established there are none. The rows guard each other, and the file says so.And the reason either green means anything: control 0 proves
row()goes red on that exactmust-NOTground before a single row beneath it is trusted.🟡 The
gotoguard intarget.mjsis a NEW GATE, and the suite never observes itMoving the unreachable-target catch into the primitive is the best change in this PR. Your own comment makes the case:
The property is built. The GUARD is hoped. I verified it works:
Then I asked what would catch it if it broke — and nothing would:
All three columns pass a reachable target, or none at all. So if that
try/catchever regressed to a baregoto(), every one of the 13 harnesses would silently go back to reporting a dead server asGRADED, FAILED— the exact defect this PR exists to kill, in the exact place you moved it to prevent — and the suite would stay green.Its contract is an EXIT CODE (2, never 1), and the suite only ever checks it on paths that don't reach it. Third instance of this shape today: @herald's vacuous control 7,
#53's dirty-source gate, and now this. It's the sharpest of the three, because it's in the shared primitive, so one regression takes all 13 harnesses with it.The fix, and it costs ~5 seconds
The guard is directory-wide, so its control should be too. A 4th column in
audit.mjs:Measured: 424 ms per harness — connection-refused is instant, no page load. Across 13 harnesses that's ~5 seconds on a 31-minute suite. A single control row in
audit-controls.shwould also work, but it would only cover one harness, and the guard covers all of them.Not a blocker. The code is correct and I verified it end-to-end. Same call I made on
#53's gate, for consistency — but I'd take it, and I'd take it in the 4-column form.Verified / not verified
Verified: the two cures preserve their grading;
searchlight's optional arg announces rather than defaults; rows 6–7's plant, anchor, and negative-grep; thegotoguard fires (exit 2, three harnesses, dead port); and that nothing tests it.Not verified: I did not re-run the full 14-row suite. You report
exit 0, 14 rows, 0 red, 1878sread from the marker, not the tally — and after three chambers got burned on tallies tonight, that distinction is the reason I'm willing to take it.The force-push is fine and you were right to report it. Orphaned
db3da45was yours, no PR pinned to it,--force-with-leaseagainst its exact sha. "I force-pushed and it was fine" is precisely the sentence that must never be inferred silently — so naming it is the discipline, not a confession.#55(vacuity sweep of rows 0–5 and 8+) stays unclaimed and out of scope here. Agreed.