chore(fragment-check): split coverage and density into independently named contexts #1465

New issue

Closed

opened 2026-09-08 00:08:33 +02:00 by bosun · 0 comments

bosun commented

2026-09-08 00:08:33 +02:00

Owner

When fragment-check goes red you cannot tell whether coverage failed or density failed, because one job grades both and posts one status — and this forge exposes no job log to read the difference from.

reusable-changelog-fragment-check.yml has a single job check (name: changelog fragment-kind) running two independent graders:

:315  coverage (#498) — did this PR add a fragment for its visible change?
:306  density  (#621) — sentence/paragraph length inside the fragments

#1449 failed one half and #1455 failed the other an hour apart, and a relayed diagnosis of the first sent its author to add a file he already had. POST /actions/runs/<id>/jobs returns 404 here, so the job output is not retrievable — the context name is the only surface a reader gets.

Scope

Split the job so the two graders post independently named contexts, e.g. fragment-check / coverage and fragment-check / density, and update main's status_check_contexts to require both in place of the current pair.

Acceptance criteria

The two graders post distinct contexts whose names say which predicate failed — DONE: real head e6d69f04b749f3ffdbcbd7cd4ae9d91fed5bcb7f posted fragment-check / coverage (pull_request) and fragment-check / density (pull_request) as separate successful status rows (run 18277).
~~Both are required on main; the old context names are removed from status_check_contexts in the same change, so nothing is required that no longer posts~~ — WRONG (the atomic-swap requirement was unsafe): the live migration removed the old fragment-kind context first (24 -> 23), then observed the new workflow and added coverage+density (23 -> 25) ten minutes later. The final required set is correct; doing both mutations in one change would have required a context before it could post and frozen the repository.
A PR that fails ONLY coverage reddens only the coverage context, verified on a real head — DONE: disposable PR #1470, head 6acb5861e9c74346eb4d69159f3f484921ff83c2, posted coverage failure and density success; it was closed unmerged and its branch deleted after the measurement.
A PR that fails ONLY density reddens only the density context, verified on a real head — DONE: disposable PR #1471, head dee257d2e9d3a1ba8bdd44909f4252db54ce1011, posted coverage success and density failure; it was closed unmerged and its branch deleted after the measurement.
Each PASS names what it did not check — DONE: the shipped coverage and density PASS branches explicitly name the other predicate as ungraded, and tests/workflows.bats executes both mode branches and pins those disclosures. The live #1468 status rows confirm both modes post; no job-log read is claimed because the log endpoint is unavailable.

rt#1463 is the same defect in readme-pin-check.yml with the opposite consequence: there the two graders have different DEPENDENCY profiles (one reads gitea.com), so the bundling prevents the safe half from being required at all. Here both halves are already required and the bundling costs diagnosis instead. Two instances, one shape — one job posting one context for graders that fail for unrelated reasons. Worth catching at design time rather than a third time.

Anchor

Deferred from rt#1456 AC2 by @sentry, 2026-09-08 00:07 CEST, who judged the criterion valid but the work separate. The AC was written loosely by @bosun — it asked #1460 for something that change did not attempt and was not asked to; #1460 delivered the could-not-grade refusal (AC1/AC3) and this is the remainder.

When `fragment-check` goes red you cannot tell whether coverage failed or density failed, because one job grades both and posts one status — and this forge exposes no job log to read the difference from. `reusable-changelog-fragment-check.yml` has a single job `check` (`name: changelog fragment-kind`) running two independent graders: ``` :315 coverage (#498) — did this PR add a fragment for its visible change? :306 density (#621) — sentence/paragraph length inside the fragments ``` `#1449` failed one half and `#1455` failed the other an hour apart, and a relayed diagnosis of the first sent its author to add a file he already had. `POST /actions/runs/<id>/jobs` returns 404 here, so the job output is not retrievable — the context name is the only surface a reader gets. ## Scope Split the job so the two graders post independently named contexts, e.g. `fragment-check / coverage` and `fragment-check / density`, and update `main`'s `status_check_contexts` to require both in place of the current pair. ## Acceptance criteria - [x] The two graders post distinct contexts whose names say which predicate failed — **DONE:** real head `e6d69f04b749f3ffdbcbd7cd4ae9d91fed5bcb7f` posted `fragment-check / coverage (pull_request)` and `fragment-check / density (pull_request)` as separate successful status rows (run `18277`). - [x] ~~Both are required on `main`; the old context names are removed from `status_check_contexts` in the same change, so nothing is required that no longer posts~~ — **WRONG (the atomic-swap requirement was unsafe):** the live migration removed the old fragment-kind context first (`24 -> 23`), then observed the new workflow and added coverage+density (`23 -> 25`) ten minutes later. The final required set is correct; doing both mutations in one change would have required a context before it could post and frozen the repository. - [x] A PR that fails ONLY coverage reddens only the coverage context, verified on a real head — **DONE:** disposable PR `#1470`, head `6acb5861e9c74346eb4d69159f3f484921ff83c2`, posted coverage `failure` and density `success`; it was closed unmerged and its branch deleted after the measurement. - [x] A PR that fails ONLY density reddens only the density context, verified on a real head — **DONE:** disposable PR `#1471`, head `dee257d2e9d3a1ba8bdd44909f4252db54ce1011`, posted coverage `success` and density `failure`; it was closed unmerged and its branch deleted after the measurement. - [x] Each PASS names what it did not check — **DONE:** the shipped coverage and density PASS branches explicitly name the other predicate as ungraded, and `tests/workflows.bats` executes both mode branches and pins those disclosures. The live `#1468` status rows confirm both modes post; no job-log read is claimed because the log endpoint is unavailable. ## Related `rt#1463` is the same defect in `readme-pin-check.yml` with the opposite consequence: there the two graders have different DEPENDENCY profiles (one reads gitea.com), so the bundling prevents the safe half from being required at all. Here both halves are already required and the bundling costs diagnosis instead. **Two instances, one shape — one job posting one context for graders that fail for unrelated reasons.** Worth catching at design time rather than a third time. ## Anchor Deferred from `rt#1456` AC2 by @sentry, 2026-09-08 00:07 CEST, who judged the criterion valid but the work separate. The AC was written loosely by @bosun — it asked `#1460` for something that change did not attempt and was not asked to; `#1460` delivered the could-not-grade refusal (AC1/AC3) and this is the remainder.

bosun added the

priority/medium

size/M

kind/chore

labels

2026-09-08 00:08:33 +02:00

bosun referenced this issue

2026-09-08 00:08:44 +02:00

bug(fragment-check): the local verb passes where the CI context of the same name fails #1456

bosun referenced this issue

2026-09-08 00:08:58 +02:00

bug(fragment-check): the local verb passes where the CI context of the same name fails #1456