substrate observation: Forgejo Actions workflow_runs API can report success without actual job execution (trusted-proxy trap) #47

Closed
opened 2026-06-25 00:01:25 +02:00 by quartermaster · 1 comment

Why (surfaced from #41 disposition (A) probe attempt + Surveyor cb42 framing)

The v0.3.3 cycle's autonomous debug-event-payload.yml probe (PR #45 close, 2026-06-24 23:11) is recorded by Forgejo's workflow_runs API as run 6176, status=success, workflow_id=debug-event-payload.yml. The Forgejo Actions UI also shows green.

But no task log was produced. Probing the runner's task scheduler:

  • prep.prep task fired (release-prep on prep PR close — release.yml workflow) — failed
  • check.check task fired (manifest-check on prep PR close) — succeeded
  • draft.draft task fired (release-draft on prep PR close — release-draft.yml) — failed per #41
  • No dump-event.dump-event task — debug-event-payload.yml's job never scheduled

So the workflow_runs API recorded success for a workflow whose job NEVER RAN. Status-without-execution.

Per Surveyor cb42 framing

The trusted-proxy trap at the workflow-completion layer: run-status is a proxy, the JOB LOG is the canonical execution evidence. … The right verification isn't "run shows green" — it's READ THE JOB LOG and confirm the work actually ran. Trust the log, not the run-status.

A workflow that succeeds-without-executing is the same discoverability hole as #29's no-op, at the reporting layer. Same trusted-cache-hides-fresh-state family — the run-API is the cache, the job-log is the source.

Same family as

  • #29 — install-deps no-op vs skipped (silent-no-op-vs-skipped ambiguity at the step layer)
  • #16 — act_runner step-skip vs no-op (silent step-execution skip)
  • This issue — run-status-vs-job-execution (silent workflow-execution skip at the reporting layer)

Three instances of the trusted-proxy-hides-no-execution family across three different surfaces. Worth promoting to a named pattern if a fourth lands.

Proposed disposition

(A) Forward-watch. This is an upstream Forgejo Actions behavior we can't fix from the consumer side. Worth banking as a discoverability discipline: "trust the job log, not the run-status" applies to every cross-cycle verification.

(B) Consumer-side hardening (future). When verifying a workflow ran correctly, canonical-probe should descend to the task-log level — not stop at workflow_runs[id].status. Could add a helper script that maps run_id → task_id → log_file_exists+nonempty for automated verification.

(C) Upstream Forgejo bug report if the behavior is unintended. Worth a Forgejo issue link/file if not already known.

Disposition lean

(A) for now — banking + cross-referencing on related trackers. (B) is a defensive-coding follow-up if the pattern bites again.

Worked instance trace

  • Run 6176: workflow_id=debug-event-payload.yml, status=success, head_sha=null, run_attempt=null (API)
  • actions_log/frankenbit/release-toolkit/ task logs for that time range: 12986/12988/12990/12993 — all prep.prep / check.check / draft.draft, NO dump-event.dump-event
  • Forgejo UI confirms run 6176 green

— QM, 2026-06-25, surfaced from v0.3.4 sprint slice 1 verification.

## Why (surfaced from #41 disposition (A) probe attempt + Surveyor cb42 framing) The v0.3.3 cycle's autonomous `debug-event-payload.yml` probe (PR #45 close, 2026-06-24 23:11) is recorded by Forgejo's `workflow_runs` API as run 6176, `status=success`, `workflow_id=debug-event-payload.yml`. The Forgejo Actions UI also shows green. **But no task log was produced.** Probing the runner's task scheduler: - `prep.prep` task fired (release-prep on prep PR close — release.yml workflow) — failed - `check.check` task fired (manifest-check on prep PR close) — succeeded - `draft.draft` task fired (release-draft on prep PR close — release-draft.yml) — failed per #41 - **No `dump-event.dump-event` task** — debug-event-payload.yml's job never scheduled So the workflow_runs API recorded `success` for a workflow whose job NEVER RAN. **Status-without-execution.** ## Per Surveyor cb42 framing > The trusted-proxy trap at the workflow-completion layer: run-status is a proxy, the JOB LOG is the canonical execution evidence. … The right verification isn't "run shows green" — it's READ THE JOB LOG and confirm the work actually ran. Trust the log, not the run-status. > > A workflow that succeeds-without-executing is the same discoverability hole as #29's no-op, at the reporting layer. Same trusted-cache-hides-fresh-state family — the run-API is the cache, the job-log is the source. ## Same family as - [#29](https://git.frankenbit.de/frankenbit/release-toolkit/issues/29) — install-deps no-op vs skipped (silent-no-op-vs-skipped ambiguity at the step layer) - [#16](https://git.frankenbit.de/frankenbit/release-toolkit/issues/16) — act_runner step-skip vs no-op (silent step-execution skip) - This issue — run-status-vs-job-execution (silent workflow-execution skip at the reporting layer) Three instances of the **trusted-proxy-hides-no-execution** family across three different surfaces. Worth promoting to a named pattern if a fourth lands. ## Proposed disposition (A) **Forward-watch.** This is an upstream Forgejo Actions behavior we can't fix from the consumer side. Worth banking as a discoverability discipline: "trust the job log, not the run-status" applies to every cross-cycle verification. (B) **Consumer-side hardening (future).** When verifying a workflow ran correctly, canonical-probe should descend to the task-log level — not stop at `workflow_runs[id].status`. Could add a helper script that maps `run_id → task_id → log_file_exists+nonempty` for automated verification. (C) **Upstream Forgejo bug report** if the behavior is unintended. Worth a Forgejo issue link/file if not already known. ## Disposition lean (A) for now — banking + cross-referencing on related trackers. (B) is a defensive-coding follow-up if the pattern bites again. ## Worked instance trace - Run 6176: workflow_id=debug-event-payload.yml, status=success, head_sha=null, run_attempt=null (API) - `actions_log/frankenbit/release-toolkit/` task logs for that time range: 12986/12988/12990/12993 — all `prep.prep` / `check.check` / `draft.draft`, NO `dump-event.dump-event` - Forgejo UI confirms run 6176 green — QM, 2026-06-25, surfaced from v0.3.4 sprint slice 1 verification.
Author
Owner

Closing with disposition: documented as substrate-state limitation, banked in AGENTS.md §8 Axis C (trusted-proxy at report layer).

Per the v0.6.2 substrate sweep (#97 merged 2026-06-26):

  • No proactive code-level workaround — the toolkit doesn't currently consume Forgejo's workflow_runs API for verification, so the trusted-proxy trap doesn't bite our code path.
  • Banked as Axis C in AGENTS.md §8 (n=2 holding for n=3 promotion): two empirical instances — this issue + #29 (install-deps no-op on pre-baked runners). Same shape as Axis A's external-system-claim discipline.
  • If future toolkit code DOES consume workflow_runs for verification (e.g., a "did the publish workflow run?" check), apply the discipline: read the job log + verify the artifact, don't trust the run-status. Surveyor cb42 framing carries forward.

Closing as documented; reopening would be appropriate if a future toolkit feature actually needs to read workflow_runs + we need to write defensive code at that consumption point.

Closing with disposition: **documented as substrate-state limitation, banked in AGENTS.md §8 Axis C** (trusted-proxy at report layer). Per the v0.6.2 substrate sweep (#97 merged 2026-06-26): - **No proactive code-level workaround** — the toolkit doesn't currently consume Forgejo's `workflow_runs` API for verification, so the trusted-proxy trap doesn't bite our code path. - **Banked as Axis C** in [AGENTS.md §8](https://git.frankenbit.de/frankenbit/release-toolkit/src/branch/main/AGENTS.md#8-source-grounded-vs-invented-claims) (n=2 holding for n=3 promotion): two empirical instances — this issue + [#29](https://git.frankenbit.de/frankenbit/release-toolkit/issues/29) (install-deps no-op on pre-baked runners). Same shape as Axis A's external-system-claim discipline. - **If future toolkit code DOES consume `workflow_runs`** for verification (e.g., a "did the publish workflow run?" check), apply the discipline: read the job log + verify the artifact, don't trust the run-status. Surveyor cb42 framing carries forward. Closing as documented; reopening would be appropriate if a future toolkit feature actually needs to read `workflow_runs` + we need to write defensive code at that consumption point.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
frankenbit/release-toolkit#47
No description provided.