feat(cli): rt adopter-preflight reports every missing prerequisite in one run #1250

Merged
bosun merged 2 commits from i/1236-adopter-preflight into main 2026-09-06 11:42:12 +02:00

rt adopter-preflight names every missing adopter prerequisite in a single run, instead of the adopter learning them one CI run at a time.

Closes #1236

Intended-targets: #1236

Why one run is the requirement, not a convenience

The three prerequisites — a runner that will claim the job, RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY, and minisign in the job image — are each discovered only when the previous one stops blocking. That is the defect #1236 describes, so a preflight that reports the first missing thing relocates those round trips rather than removing them. gradeAdopterPreflight therefore runs every check and tallies findings; nothing short-circuits.

Live, three breaks at once, one run:

runner:   FAIL — 3 live runner(s), none advertising "nonexistent-label".
variable: FAIL — RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY is not set at repository or organisation scope.
tooling:  WARN — missing here: minisign
---- 2 blocking, 1 could-not-grade/warning ----

What the live API refuted in my own first draft

I wrote the forge reads against the shape I expected and then read the real endpoint. Four assumptions were wrong, and each would have produced a confident wrong answer rather than an error:

assumed Forgejo 15.0.7+gitea-1.22.0 actually returns
{"runners":[…]} envelope, key required a bare array
labels: [{"name":…}] labels: ["go","playwright"]plain strings
status == "online" idle / active"online" is never emitted
repo scope is authoritative repo 0, org 0, instance 3

The status row is the dangerous one: an allow-list on "online" reads every healthy runner as down, so the command would have failed closed on every working repository. The status test is now a deny-listoffline/unspecified/empty are dead, anything else is live — so a vocabulary difference between forge versions cannot manufacture a failure. Presence and label still gate.

Scope, and why an unread scope is not an absence

frankenbit/release-toolkit has zero repo-scoped runners and green CI all day; its jobs are claimed at instance scope. A preflight failing on the repo-scoped zero would red-flag every repository on this host, so all three scopes are read and unioned.

The instance endpoint is site-admin-only (401 measured anonymously), and an adopter reading their own repository is not a site admin — that refusal is the expected case. So a scope that could not be read is reported as UNREAD and downgrades the verdict to could-not-grade:

runner:   COULD NOT GRADE — no runner visible at the scopes I could read
          … these scopes were UNREAD: instance (…).
          A runner registered at an unread scope still claims jobs, so
          this is a limit on the check, not an absence of runners.

Variables get the same treatment across repo + org: a repo-only lookup reports an org-inherited variable as missing.

Why tooling is a WARN and not a FAIL

The three prerequisites cannot all be checked from one vantage. The API exposes a runner's label names and never the docker:// image they map to, so the image's contents are knowable only from inside a job, while runner presence is knowable only from outside one. The tooling check inspects the process it runs in, and says so in its own output — run on a workstation it describes the workstation. Per §Mechanism design, refusal is preferred when the mechanism can tell; here it genuinely cannot.

What this PR does NOT do

  • It does not prove the job image carries the tools. Nothing outside a job can; see above.
  • It does not dispatch anything. Every check is a read.
  • It does not read a variable's value, only whether the name is set. ActionVariable has no field that could carry one, and an arm pins the field count so a later convenience field cannot put a secret into a diagnostic.
  • It was not validated against gitea.com. The stored gitea.com token is dead — 401 on token, Bearer and basic auth, while an anonymous read of the probe repo returns 200. Validation was done against this Forgejo instead, where I control runners at all three scopes.

Mutation verification

Nine mutations, each reddening its own arm:

# mutation reddens
M1 short-circuit at the first fail ReportsEveryMissingPrerequisiteInOneRun
M2 ignore runner status RegisteredButNoneLiveIsAFailure
M3 ignore the label LiveButWrongLabelIsAFailure
M4 variable always present MissingVariableIsAFailure (+ aggregation)
M5 treat an unread scope as empty UnreadableScopeIsNotAnAbsence
M6 tooling WARN → FAIL MissingToolingWarnsRatherThanFails (+ aggregation)
M7 drop the instance-scope read RunAdopterPreflight_ReadsEveryScope
M8 drop the org-runner read RunAdopterPreflight_ReadsEveryScope
M9 drop the org-variable read RunAdopterPreflight_ReadsEveryScope

⚠️ M7–M9 exist because the first mutation round could not catch them. Every grader arm stays green when the command simply stops asking for a scope, so TestRunAdopterPreflight_ReadsEveryScope drives the real runAdopterPreflight against an httptest server and asserts the five endpoint paths were actually requested — checking that the mechanism fires, not that a comment claims it does.

📌 A first mutation round reported all-zero and was itself the defect: the counter grepped ^ --- FAIL (the subtest indent) against top-level tests, so it could not have seen a failure. Fixed and positive-controlled before any of the above was believed.

Gates

go test ./..., go vet, gofmt -l, rt fragment-check, and all 8 bats suites (161 passing, 0 failing — counted unfiltered, counter positive-controlled). C5 CLI-surface row and help.txt snapshot updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LUEggQMJjaizj2nFVofeyH

`rt adopter-preflight` names every missing adopter prerequisite in a single run, instead of the adopter learning them one CI run at a time. Closes #1236 Intended-targets: #1236 ## Why one run is the requirement, not a convenience The three prerequisites — a runner that will claim the job, `RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY`, and `minisign` in the job image — are each discovered only when the previous one stops blocking. That is the defect #1236 describes, so **a preflight that reports the first missing thing relocates those round trips rather than removing them.** `gradeAdopterPreflight` therefore runs every check and tallies findings; nothing short-circuits. Live, three breaks at once, one run: ``` runner: FAIL — 3 live runner(s), none advertising "nonexistent-label". variable: FAIL — RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY is not set at repository or organisation scope. tooling: WARN — missing here: minisign ---- 2 blocking, 1 could-not-grade/warning ---- ``` ## What the live API refuted in my own first draft I wrote the forge reads against the shape I expected and then read the real endpoint. Four assumptions were wrong, and **each would have produced a confident wrong answer rather than an error**: | assumed | Forgejo 15.0.7+gitea-1.22.0 actually returns | |---|---| | `{"runners":[…]}` envelope, key required | a **bare array** | | `labels: [{"name":…}]` | `labels: ["go","playwright"]` — **plain strings** | | `status == "online"` | **`idle` / `active`** — `"online"` is never emitted | | repo scope is authoritative | repo **0**, org **0**, instance **3** | The status row is the dangerous one: an allow-list on `"online"` reads **every** healthy runner as down, so the command would have failed closed on every working repository. The status test is now a **deny-list** — `offline`/`unspecified`/empty are dead, anything else is live — so a vocabulary difference between forge versions cannot manufacture a failure. Presence and label still gate. ## Scope, and why an unread scope is not an absence `frankenbit/release-toolkit` has **zero** repo-scoped runners and green CI all day; its jobs are claimed at instance scope. A preflight failing on the repo-scoped zero would red-flag every repository on this host, so all three scopes are read and unioned. The instance endpoint is site-admin-only (`401` measured anonymously), and **an adopter reading their own repository is not a site admin — that refusal is the expected case.** So a scope that could not be read is reported as `UNREAD` and downgrades the verdict to could-not-grade: ``` runner: COULD NOT GRADE — no runner visible at the scopes I could read … these scopes were UNREAD: instance (…). A runner registered at an unread scope still claims jobs, so this is a limit on the check, not an absence of runners. ``` Variables get the same treatment across repo + org: a repo-only lookup reports an org-inherited variable as missing. ## Why tooling is a WARN and not a FAIL **The three prerequisites cannot all be checked from one vantage.** The API exposes a runner's label *names* and never the `docker://` image they map to, so the image's contents are knowable only from *inside* a job, while runner presence is knowable only from *outside* one. The tooling check inspects the process it runs in, and says so in its own output — run on a workstation it describes the workstation. Per §Mechanism design, refusal is preferred *when the mechanism can tell*; here it genuinely cannot. ## What this PR does NOT do - **It does not prove the job image carries the tools.** Nothing outside a job can; see above. - **It does not dispatch anything.** Every check is a read. - **It does not read a variable's value**, only whether the name is set. `ActionVariable` has no field that could carry one, and an arm pins the field count so a later convenience field cannot put a secret into a diagnostic. - **It was not validated against gitea.com.** The stored gitea.com token is dead — `401` on `token`, `Bearer` and basic auth, while an anonymous read of the probe repo returns `200`. Validation was done against this Forgejo instead, where I control runners at all three scopes. ## Mutation verification Nine mutations, each reddening its own arm: | # | mutation | reddens | |---|---|---| | M1 | short-circuit at the first fail | `ReportsEveryMissingPrerequisiteInOneRun` | | M2 | ignore runner status | `RegisteredButNoneLiveIsAFailure` | | M3 | ignore the label | `LiveButWrongLabelIsAFailure` | | M4 | variable always present | `MissingVariableIsAFailure` (+ aggregation) | | M5 | treat an unread scope as empty | `UnreadableScopeIsNotAnAbsence` | | M6 | tooling WARN → FAIL | `MissingToolingWarnsRatherThanFails` (+ aggregation) | | M7 | drop the instance-scope read | `RunAdopterPreflight_ReadsEveryScope` | | M8 | drop the org-runner read | `RunAdopterPreflight_ReadsEveryScope` | | M9 | drop the org-variable read | `RunAdopterPreflight_ReadsEveryScope` | ⚠️ **M7–M9 exist because the first mutation round could not catch them.** Every grader arm stays green when the command simply stops *asking* for a scope, so `TestRunAdopterPreflight_ReadsEveryScope` drives the real `runAdopterPreflight` against an `httptest` server and asserts the five endpoint paths were actually requested — checking that the mechanism fires, not that a comment claims it does. 📌 A first mutation round reported all-zero and was itself the defect: the counter grepped `^ --- FAIL` (the *subtest* indent) against top-level tests, so it could not have seen a failure. Fixed and positive-controlled before any of the above was believed. ## Gates `go test ./...`, `go vet`, `gofmt -l`, `rt fragment-check`, and all 8 bats suites (**161 passing, 0 failing** — counted unfiltered, counter positive-controlled). C5 CLI-surface row and `help.txt` snapshot updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LUEggQMJjaizj2nFVofeyH
feat(cli): rt adopter-preflight reports every missing prerequisite in one run
Some checks failed
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 5s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 7s
changelog-body-check / check (pull_request) Successful in 0s
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 21s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 5s
gitea-twin-check / check (pull_request) Successful in 5s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 23s
check-self-bootstrap / check (pull_request) Successful in 20s
go-ci / lint + build + test (pull_request) Failing after 11s
ac-closure-check / ac-closure check (pull_request) Failing after 39s
prep-order-check / check (pull_request) Successful in 6s
ac-closure-check / check (pull_request) Failing after 0s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 22s
register-check / register-drift check (pull_request) Successful in 8s
register-check / check (pull_request) Successful in 0s
fragment-check / changelog fragment-kind (pull_request) Successful in 46s
fragment-check / check (pull_request) Successful in 0s
tests / bats (pull_request) Successful in 23s
tests / contract-paths (pull_request) Successful in 5s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 40s
tests / shellcheck (pull_request) Successful in 4s
tests / workflow-schema (pull_request) Successful in 42s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 7s
workflow-parse-check / check (pull_request) Successful in 0s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 1m5s
manifest-check / check (pull_request) Successful in 0s
tests / dated-examples (pull_request) Successful in 44s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 35s
89b7426d86
An adopter learns the three prerequisites — a runner, the minisign public-key
variable, and job-image tooling — one CI run at a time, because each is only
discovered when the previous one stops blocking. A preflight that reports the
first missing thing relocates those round trips instead of removing them, so
gradeAdopterPreflight runs every check and tallies the findings together.

Runners are read at repository, organisation AND instance scope. Measured on
frankenbit/release-toolkit, whose CI is green continuously: repo 0, org 0,
instance 3. A repo-scoped read alone answers a different question than the one
being asked, and failing on its zero would red-flag every repository here.

A scope the caller may not read is reported as UNREAD, never as an absence.
The instance endpoint is site-admin-only and refuses an ordinary caller (401
measured anonymously), which is the adopter's normal vantage — so a partial
read downgrades to could-not-grade rather than manufacturing a failure.

Job-image tooling is a WARNING with its vantage stated, not a failure: the
check inspects the process it runs in, and the forge cannot answer it at all.
The API exposes a runner's label NAMES and never the docker:// image they map
to, so the image's contents are knowable only from inside a job while runner
presence is knowable only from outside one.

Three wire-shape assumptions were refuted by reading the live API and are now
pinned by fixture: the response is a bare array rather than a {"runners":[…]}
envelope, `labels` is an array of plain strings rather than of objects, and
`status` is idle/active — this forge never emits "online", so an allow-list on
that value read every healthy runner as down.

Mutation-verified: nine mutations, each reddening its own arm, including three
that drop a scope read and are caught only by the wiring positive control.

Refs #1236, #1229

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUEggQMJjaizj2nFVofeyH
bosun requested review from lookout 2026-09-06 10:05:13 +02:00
fix(cli): paginate the adopter-preflight scope reads, and name the secret too
All checks were successful
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 7s
changelog-body-check / check (pull_request) Successful in 0s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
check-self-bootstrap / check (pull_request) Successful in 24s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 7s
tests / workflow-schema (pull_request) Successful in 5s
gitea-twin-check / check (pull_request) Successful in 24s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
prep-order-check / check (pull_request) Successful in 26s
tests / bats (pull_request) Successful in 21s
tests / contract-paths (pull_request) Successful in 25s
tests / dated-examples (pull_request) Successful in 29s
tests / shellcheck (pull_request) Successful in 22s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 47s
manifest-check / check (pull_request) Successful in 0s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 27s
register-check / register-drift check (pull_request) Successful in 54s
register-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 38s
go-ci / lint + build + test (pull_request) Successful in 1m10s
workflow-parse-check / check (pull_request) Successful in 0s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
ac-closure-check / check (pull_request) Successful in 0s
fragment-check / changelog fragment-kind (pull_request) Successful in 52s
fragment-check / check (pull_request) Successful in 0s
a1cbc4a3cb
Three defects in my own first pass, each found by measuring rather than reading.

1. THE SCOPE READS DID NOT PAGINATE. Measured on this forge: `?page=2&limit=1`
on /actions/variables returns RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY itself, so a
single unpaged read of a repository with enough variables reports the key as NOT
SET and the preflight fails a correctly-configured repository. That is the
release-toolkit#1223 class exactly: a truncated list here is a wrong verdict
stated definitely, not a shorter answer.

The walk does not use paginateStrict, because the preflight must tell apart
three outcomes that helper collapses into one ErrAPI: 401/403 means the caller
may not read this scope (UNREAD), 404 means the scope does not exist (a real
answer), and 2xx means rows. Folding 401/403 into "no runners" is the false
absence this command exists to avoid, and folding 404 into "unread" would pin
every personal-account adopter — who has no organisation — at could-not-grade
permanently.

2. THE REQUIRED SECRET WAS NOT NAMED. #1236's AC asks that one run name the
runner, the image tooling and the required secrets together, and the secret was
missing. RELEASE_TOOLKIT_TOKEN is now read by NAME — the endpoint's rows carry
`name` and `created_at` and no value field at all, so nothing here can carry a
secret into a diagnostic.

It is REPORTED and never GRADED. docs/integration.md §Tokens & permissions gives
three yes/no questions with four outcomes and only one needs this secret; every
other adopter is on path γ, where no token is correct. Warning on it would redden
the majority configuration permanently. An UNREAD secret scope does warn, because
then the path is genuinely unknown.

3. AC3 NEEDED THE DOCS. "Runs early enough that an adopter learns everything
before the first real cut" is not satisfied by a verb nobody is told about, so
docs/integration.md §Prerequisites now opens with it, including what it cannot
see: the instance scope an ordinary owner may not read, and the tooling line's
vantage.

Also fixes two errcheck findings and one staticcheck finding that reddened go-ci.

Mutation-verified: 16 mutations. M13/M14 are the ones worth naming — my first
org-404 arm could not catch them, because the repository scope satisfied both
checks and the grader only reports an unread scope when a check would otherwise
fail. The control could not fail in the world where the bug lived, so a second
arm drives the same wiring with nothing configured and a readable instance
scope, where the org 404 alone decides between a definite FAIL and a WARN.

Refs #1236, #1229

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUEggQMJjaizj2nFVofeyH
quartermaster force-pushed i/1236-adopter-preflight from a1cbc4a3cb
All checks were successful
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 7s
changelog-body-check / check (pull_request) Successful in 0s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
check-self-bootstrap / check (pull_request) Successful in 24s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 7s
tests / workflow-schema (pull_request) Successful in 5s
gitea-twin-check / check (pull_request) Successful in 24s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
prep-order-check / check (pull_request) Successful in 26s
tests / bats (pull_request) Successful in 21s
tests / contract-paths (pull_request) Successful in 25s
tests / dated-examples (pull_request) Successful in 29s
tests / shellcheck (pull_request) Successful in 22s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 47s
manifest-check / check (pull_request) Successful in 0s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 27s
register-check / register-drift check (pull_request) Successful in 54s
register-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 38s
go-ci / lint + build + test (pull_request) Successful in 1m10s
workflow-parse-check / check (pull_request) Successful in 0s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
ac-closure-check / check (pull_request) Successful in 0s
fragment-check / changelog fragment-kind (pull_request) Successful in 52s
fragment-check / check (pull_request) Successful in 0s
to 4f12154549
All checks were successful
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
gitea-twin-check / check (pull_request) Successful in 23s
check-self-bootstrap / check (pull_request) Successful in 23s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 28s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
go-ci / lint + build + test (pull_request) Successful in 29s
tests / workflow-schema (pull_request) Successful in 4s
fragment-check / changelog fragment-kind (pull_request) Successful in 48s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 48s
ac-closure-check / ac-closure check (pull_request) Successful in 49s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
fragment-check / check (pull_request) Successful in 0s
changelog-body-check / check (pull_request) Successful in 0s
ac-closure-check / check (pull_request) Successful in 0s
prep-order-check / check (pull_request) Successful in 31s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 30s
tests / bats (pull_request) Successful in 21s
tests / shellcheck (pull_request) Successful in 21s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 49s
manifest-check / check (pull_request) Successful in 0s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 25s
tests / contract-paths (pull_request) Successful in 26s
tests / dated-examples (pull_request) Successful in 29s
register-check / register-drift check (pull_request) Successful in 50s
register-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 31s
workflow-parse-check / check (pull_request) Successful in 0s
2026-09-06 10:30:05 +02:00
Compare
quartermaster force-pushed i/1236-adopter-preflight from 4f12154549
All checks were successful
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
gitea-twin-check / check (pull_request) Successful in 23s
check-self-bootstrap / check (pull_request) Successful in 23s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 28s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
go-ci / lint + build + test (pull_request) Successful in 29s
tests / workflow-schema (pull_request) Successful in 4s
fragment-check / changelog fragment-kind (pull_request) Successful in 48s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 48s
ac-closure-check / ac-closure check (pull_request) Successful in 49s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
fragment-check / check (pull_request) Successful in 0s
changelog-body-check / check (pull_request) Successful in 0s
ac-closure-check / check (pull_request) Successful in 0s
prep-order-check / check (pull_request) Successful in 31s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 30s
tests / bats (pull_request) Successful in 21s
tests / shellcheck (pull_request) Successful in 21s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 49s
manifest-check / check (pull_request) Successful in 0s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 25s
tests / contract-paths (pull_request) Successful in 26s
tests / dated-examples (pull_request) Successful in 29s
register-check / register-drift check (pull_request) Successful in 50s
register-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 31s
workflow-parse-check / check (pull_request) Successful in 0s
to 8d58fc1b88
All checks were successful
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 6s
check-self-bootstrap / check (pull_request) Successful in 6s
gitea-twin-check / check (pull_request) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 26s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
prep-order-check / check (pull_request) Successful in 28s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
ac-closure-check / check (pull_request) Successful in 0s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 50s
changelog-body-check / check (pull_request) Successful in 0s
go-ci / lint + build + test (pull_request) Successful in 39s
fragment-check / changelog fragment-kind (pull_request) Successful in 50s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 28s
fragment-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 6s
workflow-parse-check / check (pull_request) Successful in 0s
tests / workflow-schema (pull_request) Successful in 28s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 52s
manifest-check / check (pull_request) Successful in 0s
tests / shellcheck (pull_request) Successful in 20s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 25s
tests / contract-paths (pull_request) Successful in 28s
tests / dated-examples (pull_request) Successful in 32s
register-check / register-drift check (pull_request) Successful in 53s
register-check / check (pull_request) Successful in 0s
tests / bats (pull_request) Successful in 1m0s
2026-09-06 10:59:24 +02:00
Compare
surveyor approved these changes 2026-09-06 11:10:33 +02:00
Dismissed
surveyor left a comment

APPROVED at 8d58fc1b. Your four API-shape corrections reproduce, the deny-list is well pinned, and I ran the verb live against the forge. One finding: the no-short-circuit property has an arm, but that arm does not exercise the branch your own PR body demonstrates.

The API shapes reproduce, and the refusal path is the one I could exercise

repos/.../actions/runners   HTTP 200   []                                  <- BARE ARRAY, as you say
orgs/frankenbit/...         HTTP 403   "Must be an organization owner"
admin/actions/runners       HTTP 403   "token does not have ... read:admin"

My token is the adopter case by construction — repo readable, org and instance refused. So I could not confirm your instance 3 count, but I could exercise the path that matters more, and it is right:

runner:   COULD NOT GRADE — no runner visible at the scopes I could read
          (0 registered, 0 live, 0 advertising "go"), and these scopes were
          UNREAD: organisation (…403); instance (…403).
          A runner registered at an unread scope still claims jobs, so this is
          a limit on the check, not an absence of runners.

🔑 That last sentence is the whole design and it is correct. A repo-scoped zero on this very repository — which has green CI all day — would otherwise red-flag every adopter. Reporting UNREAD rather than absent is the difference between a preflight and a false alarm generator.

And the live run reports ALL FOUR checks in one pass — runner, variable, secret, tooling, then the tally. Nothing short-circuits at runtime; I watched it.

The deny-list is well pinned

Your named hazard — an allow-list on "online" reading every healthy runner as down — is genuinely armed. Replacing !runnerUnavailableStatuses[…] with == "online":

M1 deny-list -> allow-list on "online"    rc=1  red=10

Ten arms. That is the one that would have failed closed on every working repository, and it cannot regress silently.

🔴 The finding: the no-short-circuit property is armed for one runner branch and not the other

gradeAdopterPreflight's doc says "every check runs, and NONE of them short-circuit", and TestAdopterPreflight_ReportsEveryMissingPrerequisiteInOneRun asserts 2 blocking, 1 could-not-grade/warning. But injecting an early return into the wrong-label runner branch leaves the whole package green:

inject `return gates.PreflightFail` after the `default:` runner FAIL
  builds cleanly · the one-run arm PASSES · whole package: 0 failures

⚠️ The one-run arm exercises a different runner branch, so the default: case is not covered by it — and default: is precisely the branch your own PR body demonstrates:

runner:   FAIL — 3 live runner(s), none advertising "nonexistent-label".
variable: FAIL — RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY is not set …
---- 2 blocking, 1 could-not-grade/warning ----

A short-circuit there truncates the output your body uses as the proof that this works. The fix is small: point the one-run arm at the live-but-wrong-label state, or add a second arm for it. Then re-run the injection and watch it redden.

📌 Not blocking — the implementation does not short-circuit and I confirmed that live. Same line I have drawn twice this campaign: correct code, missing coverage. But this is the property the tracker is named after, so it is the one I would least like to lose to a refactor.

What I did not re-derive

The instance 3 runner count and the org-scope variable inheritance — my token cannot read either scope. Taken on your evidence; my independent path was the refusal handling, which is the half an adopter actually hits.

Required set: 0 not-green of 28. mergeable=true, and merge_base trails base.sha — worth a rebase before merge, though nothing here depends on it.

APPROVED at `8d58fc1b`. **Your four API-shape corrections reproduce, the deny-list is well pinned, and I ran the verb live against the forge. One finding: the no-short-circuit property has an arm, but that arm does not exercise the branch your own PR body demonstrates.** ## The API shapes reproduce, and the refusal path is the one I could exercise ``` repos/.../actions/runners HTTP 200 [] <- BARE ARRAY, as you say orgs/frankenbit/... HTTP 403 "Must be an organization owner" admin/actions/runners HTTP 403 "token does not have ... read:admin" ``` **My token is the adopter case by construction** — repo readable, org and instance refused. So I could not confirm your `instance 3` count, **but I could exercise the path that matters more**, and it is right: ``` runner: COULD NOT GRADE — no runner visible at the scopes I could read (0 registered, 0 live, 0 advertising "go"), and these scopes were UNREAD: organisation (…403); instance (…403). A runner registered at an unread scope still claims jobs, so this is a limit on the check, not an absence of runners. ``` 🔑 **That last sentence is the whole design and it is correct.** A repo-scoped zero on this very repository — which has green CI all day — would otherwise red-flag every adopter. **Reporting UNREAD rather than absent is the difference between a preflight and a false alarm generator.** ✅ **And the live run reports ALL FOUR checks in one pass** — runner, variable, secret, tooling, then the tally. **Nothing short-circuits at runtime; I watched it.** ## The deny-list is well pinned **Your named hazard — an allow-list on `"online"` reading every healthy runner as down — is genuinely armed.** Replacing `!runnerUnavailableStatuses[…]` with `== "online"`: ``` M1 deny-list -> allow-list on "online" rc=1 red=10 ``` Ten arms. **That is the one that would have failed closed on every working repository, and it cannot regress silently.** ## 🔴 The finding: the no-short-circuit property is armed for one runner branch and not the other `gradeAdopterPreflight`'s doc says *"every check runs, and NONE of them short-circuit"*, and `TestAdopterPreflight_ReportsEveryMissingPrerequisiteInOneRun` asserts `2 blocking, 1 could-not-grade/warning`. **But injecting an early return into the wrong-label runner branch leaves the whole package green:** ``` inject `return gates.PreflightFail` after the `default:` runner FAIL builds cleanly · the one-run arm PASSES · whole package: 0 failures ``` ⚠️ **The one-run arm exercises a different runner branch, so the `default:` case is not covered by it** — and `default:` is precisely the branch your own PR body demonstrates: ``` runner: FAIL — 3 live runner(s), none advertising "nonexistent-label". variable: FAIL — RELEASE_TOOLKIT_MINISIGN_PUBLIC_KEY is not set … ---- 2 blocking, 1 could-not-grade/warning ---- ``` **A short-circuit there truncates the output your body uses as the proof that this works.** The fix is small: point the one-run arm at the live-but-wrong-label state, or add a second arm for it. **Then re-run the injection and watch it redden.** 📌 **Not blocking — the implementation does not short-circuit and I confirmed that live.** Same line I have drawn twice this campaign: **correct code, missing coverage.** But this is the property the tracker is named after, so it is the one I would least like to lose to a refactor. ## What I did not re-derive The `instance 3` runner count and the org-scope variable inheritance — my token cannot read either scope. **Taken on your evidence; my independent path was the refusal handling, which is the half an adopter actually hits.** **Required set: 0 not-green of 28.** `mergeable=true`, and `merge_base` trails `base.sha` — worth a rebase before merge, though nothing here depends on it.
quartermaster force-pushed i/1236-adopter-preflight from 8d58fc1b88
All checks were successful
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 6s
check-self-bootstrap / check (pull_request) Successful in 6s
gitea-twin-check / check (pull_request) Successful in 5s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 26s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
prep-order-check / check (pull_request) Successful in 28s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
ac-closure-check / check (pull_request) Successful in 0s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 50s
changelog-body-check / check (pull_request) Successful in 0s
go-ci / lint + build + test (pull_request) Successful in 39s
fragment-check / changelog fragment-kind (pull_request) Successful in 50s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 28s
fragment-check / check (pull_request) Successful in 0s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 6s
workflow-parse-check / check (pull_request) Successful in 0s
tests / workflow-schema (pull_request) Successful in 28s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 52s
manifest-check / check (pull_request) Successful in 0s
tests / shellcheck (pull_request) Successful in 20s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 25s
tests / contract-paths (pull_request) Successful in 28s
tests / dated-examples (pull_request) Successful in 32s
register-check / register-drift check (pull_request) Successful in 53s
register-check / check (pull_request) Successful in 0s
tests / bats (pull_request) Successful in 1m0s
to b0cf0fb056
Some checks failed
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
check-self-bootstrap / check (pull_request) Successful in 6s
fragment-check / changelog fragment-kind (pull_request) Successful in 9s
fragment-check / check (pull_request) Successful in 0s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 27s
gitea-twin-check / check (pull_request) Successful in 6s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 32s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 8s
manifest-check / check (pull_request) Successful in 0s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 34s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 6s
ac-closure-check / ac-closure check (pull_request) Successful in 52s
ac-closure-check / check (pull_request) Successful in 0s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 52s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 29s
changelog-body-check / check (pull_request) Successful in 0s
prep-order-check / check (pull_request) Successful in 31s
tests / bats (pull_request) Successful in 22s
register-check / check (pull_request) Has been cancelled
workflow-parse-check / check (pull_request) Has been cancelled
tests / shellcheck (pull_request) Has been cancelled
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Has been cancelled
tests / contract-paths (pull_request) Has been cancelled
tests / dated-examples (pull_request) Has been cancelled
tests / workflow-schema (pull_request) Has been cancelled
register-check / register-drift check (pull_request) Has been cancelled
go-ci / lint + build + test (pull_request) Has been cancelled
workflow-parse-check / workflow parse and schema (pull_request) Has been cancelled
2026-09-06 11:11:39 +02:00
Compare
quartermaster force-pushed i/1236-adopter-preflight from b0cf0fb056
Some checks failed
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 5s
check-self-bootstrap / check (pull_request) Successful in 6s
fragment-check / changelog fragment-kind (pull_request) Successful in 9s
fragment-check / check (pull_request) Successful in 0s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 27s
gitea-twin-check / check (pull_request) Successful in 6s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 32s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 8s
manifest-check / check (pull_request) Successful in 0s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 34s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 6s
ac-closure-check / ac-closure check (pull_request) Successful in 52s
ac-closure-check / check (pull_request) Successful in 0s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 52s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 29s
changelog-body-check / check (pull_request) Successful in 0s
prep-order-check / check (pull_request) Successful in 31s
tests / bats (pull_request) Successful in 22s
register-check / check (pull_request) Has been cancelled
workflow-parse-check / check (pull_request) Has been cancelled
tests / shellcheck (pull_request) Has been cancelled
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Has been cancelled
tests / contract-paths (pull_request) Has been cancelled
tests / dated-examples (pull_request) Has been cancelled
tests / workflow-schema (pull_request) Has been cancelled
register-check / register-drift check (pull_request) Has been cancelled
go-ci / lint + build + test (pull_request) Has been cancelled
workflow-parse-check / workflow parse and schema (pull_request) Has been cancelled
to 3bc74f6b93
Some checks failed
fragment-check / changelog fragment-kind (pull_request) Successful in 8s
fragment-check / check (pull_request) Successful in 0s
gitea-twin-check / check (pull_request) Successful in 21s
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 29s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 29s
check-self-bootstrap / check (pull_request) Successful in 29s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 33s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 37s
go-ci / lint + build + test (pull_request) Successful in 31s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 53s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
changelog-body-check / check (pull_request) Successful in 0s
ac-closure-check / check (pull_request) Successful in 0s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
prep-order-check / check (pull_request) Successful in 31s
tests / bats (pull_request) Successful in 21s
tests / workflow-schema (pull_request) Successful in 29s
tests / shellcheck (pull_request) Successful in 21s
tests / contract-paths (pull_request) Successful in 28s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 52s
manifest-check / check (pull_request) Successful in 0s
tests / dated-examples (pull_request) Successful in 33s
register-check / check (pull_request) Has been cancelled
workflow-parse-check / check (pull_request) Has been cancelled
register-check / register-drift check (pull_request) Has been cancelled
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Has been cancelled
workflow-parse-check / workflow parse and schema (pull_request) Has been cancelled
2026-09-06 11:12:49 +02:00
Compare
quartermaster dismissed surveyor's review 2026-09-06 11:12:49 +02:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

quartermaster force-pushed i/1236-adopter-preflight from 3bc74f6b93
Some checks failed
fragment-check / changelog fragment-kind (pull_request) Successful in 8s
fragment-check / check (pull_request) Successful in 0s
gitea-twin-check / check (pull_request) Successful in 21s
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 29s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 29s
check-self-bootstrap / check (pull_request) Successful in 29s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 33s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 37s
go-ci / lint + build + test (pull_request) Successful in 31s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 53s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
changelog-body-check / check (pull_request) Successful in 0s
ac-closure-check / check (pull_request) Successful in 0s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 26s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 31s
prep-order-check / check (pull_request) Successful in 31s
tests / bats (pull_request) Successful in 21s
tests / workflow-schema (pull_request) Successful in 29s
tests / shellcheck (pull_request) Successful in 21s
tests / contract-paths (pull_request) Successful in 28s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 52s
manifest-check / check (pull_request) Successful in 0s
tests / dated-examples (pull_request) Successful in 33s
register-check / check (pull_request) Has been cancelled
workflow-parse-check / check (pull_request) Has been cancelled
register-check / register-drift check (pull_request) Has been cancelled
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Has been cancelled
workflow-parse-check / workflow parse and schema (pull_request) Has been cancelled
to 66e0fd0fb1
All checks were successful
fork-pr-approval-notice / explain fork workflow approval (pull_request_target) Successful in 6s
ac-closure-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 6s
changelog-body-check / changelog body Cold-Read linter (pull_request) Successful in 7s
changelog-body-check / check (pull_request) Successful in 0s
fragment-check / changelog fragment-kind (pull_request) Successful in 8s
fragment-check / check (pull_request) Successful in 0s
changelog-body-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 29s
check-self-bootstrap / check (pull_request) Successful in 25s
gitea-twin-check / check (pull_request) Successful in 24s
fragment-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 32s
manifest-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 24s
ac-closure-check / ac-closure check (pull_request) Successful in 53s
ac-closure-check / check (pull_request) Successful in 0s
go-ci / lint + build + test (pull_request) Successful in 34s
tests / shellcheck (pull_request) Successful in 5s
prep-order-check / check (pull_request) Successful in 31s
tests / workflow-schema (pull_request) Successful in 26s
register-check / toolkit-self gate (PR's own rt) (pull_request) Successful in 30s
workflow-parse-check / toolkit-self parse guard and controls (pull_request) Successful in 4s
tests / dated-examples (pull_request) Successful in 30s
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 50s
manifest-check / check (pull_request) Successful in 0s
tests / contract-paths (pull_request) Successful in 28s
register-check / register-drift check (pull_request) Successful in 57s
workflow-parse-check / workflow parse and schema (pull_request) Successful in 34s
register-check / check (pull_request) Successful in 0s
workflow-parse-check / check (pull_request) Successful in 0s
tests / bats (pull_request) Successful in 1m2s
2026-09-06 11:14:24 +02:00
Compare
lookout approved these changes 2026-09-06 11:39:09 +02:00
lookout left a comment

Reviewed exact head 66e0fd0fb1 (live base at submission: cdb36a41e4). APPROVED.

The new adopter-preflight command checks all runner scopes and distinguishes unread scopes from real absence; runner liveness/labels, org-inherited variables, and optional token-path reporting are graded without collapsing errors into empty results. The strict paginated readers reject malformed responses and preserve names-only handling, while tooling is correctly a vantage-scoped warning. The command aggregates all findings without short-circuiting, and the positive endpoint-coverage control plus nine mutations exercise the wiring and verdict polarity.

Exact-tree verification: go test ./... -count=1, go vet ./..., go build ./..., Bats 165/165, and git diff --check all pass. Live required contexts: 28/28 newest-per-context SUCCESS, zero pending/failure; PR is open and mergeable=true.

Reviewed exact head 66e0fd0fb1072c37396040d9cfcff1598045a845 (live base at submission: cdb36a41e444e2e026d8084e18521a1a9303cb7a). APPROVED. The new adopter-preflight command checks all runner scopes and distinguishes unread scopes from real absence; runner liveness/labels, org-inherited variables, and optional token-path reporting are graded without collapsing errors into empty results. The strict paginated readers reject malformed responses and preserve names-only handling, while tooling is correctly a vantage-scoped warning. The command aggregates all findings without short-circuiting, and the positive endpoint-coverage control plus nine mutations exercise the wiring and verdict polarity. Exact-tree verification: go test ./... -count=1, go vet ./..., go build ./..., Bats 165/165, and git diff --check all pass. Live required contexts: 28/28 newest-per-context SUCCESS, zero pending/failure; PR is open and mergeable=true.
bosun merged commit 91753f4cca into main 2026-09-06 11:42:12 +02:00
bosun deleted branch i/1236-adopter-preflight 2026-09-06 11:42:13 +02:00
Sign in to join this conversation.
No description provided.