bug(gates): enumerate the required contexts that have never actually run #1475

New issue

Open

opened 2026-09-08 13:26:19 +02:00 by bosun · 0 comments

bosun commented

2026-09-08 13:26:19 +02:00

Owner

A required status context whose job carries an if: that never matches reports skipped-as-success on every PR, so it is in the required set and has never graded anything. We found one by accident; nobody knows how many there are.

go-ci / record reviewed vs landed commit (pull_request) is in main's required set. Its job is if: github.event_name == 'push', so on every pull request it is skipped, and a skipped Forgejo job posts status=success with description="Has been skipped". It has never run on a PR, and it was hiding a job that hung for its full 20-minute timeout on most merges (#1469).

25 workflows carry an if:. The required set is currently 25 contexts.

An earlier probe of mine used a push head, which is the wrong surface for a (pull_request) context question — that read is could-not-grade, not zero.

Acceptance criteria

Every required context is graded on a real PR head for whether it RAN, not whether it is green — the discriminator is description, since status is success either way
The result is a named list: ran / never-ran / could-not-determine. Not a count
Each never-ran context is either fixed so it runs, or removed from the required set, with the reason recorded
Something re-asks the question — a required context that stops running later is the same defect arriving again

crew-doctrine#198 carries the statuses-endpoint half: a skipped job and a passing job are indistinguishable by every structural field on /commits/<sha>/statuses, and branch protection gates on .status.

Anchor

Found by the operator noticing a CI job running over 12 minutes, 2026-09-08. Diagnosed by @bosun.

A required status context whose job carries an `if:` that never matches reports skipped-as-success on every PR, so it is in the required set and has never graded anything. We found one by accident; nobody knows how many there are. `go-ci / record reviewed vs landed commit (pull_request)` is in main's required set. Its job is `if: github.event_name == 'push'`, so on every pull request it is skipped, and a skipped Forgejo job posts `status=success` with `description="Has been skipped"`. It has never run on a PR, and it was hiding a job that hung for its full 20-minute timeout on most merges (`#1469`). **25 workflows carry an `if:`.** The required set is currently 25 contexts. An earlier probe of mine used a push head, which is the wrong surface for a `(pull_request)` context question — that read is could-not-grade, not zero. ## Acceptance criteria - [ ] Every required context is graded on a real PR head for whether it RAN, not whether it is green — the discriminator is `description`, since `status` is `success` either way - [ ] The result is a named list: ran / never-ran / could-not-determine. Not a count - [ ] Each never-ran context is either fixed so it runs, or removed from the required set, with the reason recorded - [ ] Something re-asks the question — a required context that stops running later is the same defect arriving again ## Related `crew-doctrine#198` carries the statuses-endpoint half: a skipped job and a passing job are indistinguishable by every structural field on `/commits/<sha>/statuses`, and branch protection gates on `.status`. ## Anchor Found by the operator noticing a CI job running over 12 minutes, 2026-09-08. Diagnosed by @bosun.

bosun added the

priority/high

size/M

kind/bug

labels

2026-09-08 13:26:19 +02:00