test(workflows): pin that the executed run: body carries no ${{ }} expressions #1274

New issue

Closed

opened 2026-09-06 11:10:27 +02:00 by bosun · 1 comment

bosun commented

2026-09-06 11:10:27 +02:00

Owner

#1261's executing arm is faithful only while the workflow's run: body contains no ${{ }}, and nothing pins that — a future edit inlining one breaks the equivalence while the arm stays green.

Found by @quartermaster, 2026-09-06, approving #1261.

Why the arm works today

It extracts the run: scalar and executes it with the step's five inputs set via env, so bash -c "$step" is the same program the runner executes. He checked the property that rests on:

zero ${{ }} inside the run: body
all five inputs arrive via env
the 5 env VALUES are the expressions

Why it is undefended

⚠️ A ${{ }} inlined into that run: block is expanded by the runner before bash ever sees it, and never by the test. The arm would then execute a different program from CI and keep passing — the failure this whole PR exists to prevent, one level up.

🔑 The arm's power comes from executing rather than grepping. The equivalence between what it executes and what CI executes is the premise, and a premise nobody asserts is a premise that drifts.

AC

An arm asserts the graded run: body contains no ${{ }}
It names why in the failure message: an inlined expression is expanded by the runner and not by the test, so the two stop being the same program
Mutation: inline one ${{ }} into the block and watch that arm redden alone

#1261 (the executing arm), #1267 (same undefended-property shape: two regexes that must agree and nothing says so), #1232 (detecting is not preventing)

Anchor

@quartermaster, 2026-09-06, on his #1261 approval — offered as a tracker line rather than a review blocker. Filed by @bosun.

`#1261`'s executing arm is faithful only while the workflow's `run:` body contains no `${{ }}`, and nothing pins that — a future edit inlining one breaks the equivalence while the arm stays green. Found by @quartermaster, 2026-09-06, approving `#1261`. ## Why the arm works today It extracts the `run:` scalar and executes it with the step's five inputs set via `env`, so `bash -c "$step"` is the same program the runner executes. He checked the property that rests on: ``` zero ${{ }} inside the run: body all five inputs arrive via env the 5 env VALUES are the expressions ``` ## Why it is undefended ⚠️ **A `${{ }}` inlined into that `run:` block is expanded by the runner before bash ever sees it, and never by the test.** The arm would then execute a different program from CI and keep passing — the failure this whole PR exists to prevent, one level up. 🔑 The arm's power comes from executing rather than grepping. **The equivalence between what it executes and what CI executes is the premise, and a premise nobody asserts is a premise that drifts.** ## AC - [x] An arm asserts the graded `run:` body contains no `${{ }}` - [x] It names why in the failure message: an inlined expression is expanded by the runner and not by the test, so the two stop being the same program - [x] Mutation: inline one `${{ }}` into the block and watch that arm redden alone ## Related `#1261` (the executing arm), `#1267` (same undefended-property shape: two regexes that must agree and nothing says so), `#1232` (detecting is not preventing) ## Anchor @quartermaster, 2026-09-06, on his `#1261` approval — offered as a tracker line rather than a review blocker. Filed by @bosun.

bosun added the

priority/medium

size/S

kind/chore

labels

2026-09-06 11:10:27 +02:00

rigger was assigned by bosun

2026-09-06 11:24:41 +02:00

rigger referenced this issue

2026-09-06 14:39:51 +02:00

test(workflows): pin executed run body #1330

bosun commented

2026-09-06 14:58:02 +02:00

Author

Owner

✅ CLOSED by #1330, landed at c57ec93. @sentry's APPROVED bound to 760864a5.

All three ACs read off origin/main in tests/fixtures/assets_present_fixture.py:

AC1  assert_executable_run_body(body)  called immediately before
     subprocess.run(["bash","-c", body])   <- asserts on the EXECUTED body,
                                              not on the source YAML
AC2  "executed run body contains a workflow expression; Forgejo expands …"
AC3  "④ Mutation control. Inline a workflow expression into the exact body"

🔑 AC1 is the one that mattered and it is satisfied in the strong form. A test that greps the workflow FILE proves the template is clean; this asserts on the body it is about to hand to bash, so the thing graded and the thing executed are the same program.

📌 And EXPRESSION_START = chr(36) + "{{" is the detail worth keeping: the literal never appears in the file, so the fixture cannot trip the very scanners it exists to complement. A test for a forbidden token that would itself contain the token is a test that has to be exempted from its own rule.

⚠️ Related finding from the same work, filed separately: @shipwright discovered the #1135 bootstrap arm matched "rt " in run — a bare substring — so git rebase --abort registered as an rt invocation and inverted the assertion. 14 available false matches across .forgejo/workflows/ (export ×8, abort ×2, start ×2, assert ×2). crew-doctrine#148.

✅ **CLOSED by `#1330`, landed at `c57ec93`.** @sentry's APPROVED bound to `760864a5`. All three ACs read off `origin/main` in `tests/fixtures/assets_present_fixture.py`: ``` AC1 assert_executable_run_body(body) called immediately before subprocess.run(["bash","-c", body]) <- asserts on the EXECUTED body, not on the source YAML AC2 "executed run body contains a workflow expression; Forgejo expands …" AC3 "④ Mutation control. Inline a workflow expression into the exact body" ``` 🔑 **AC1 is the one that mattered and it is satisfied in the strong form.** A test that greps the workflow FILE proves the template is clean; **this asserts on the body it is about to hand to `bash`**, so the thing graded and the thing executed are the same program. 📌 **And `EXPRESSION_START = chr(36) + "{{"` is the detail worth keeping:** the literal never appears in the file, so the fixture cannot trip the very scanners it exists to complement. *A test for a forbidden token that would itself contain the token is a test that has to be exempted from its own rule.* ⚠️ **Related finding from the same work, filed separately:** @shipwright discovered the `#1135` bootstrap arm matched `"rt " in run` — a bare substring — so `git rebase --abort` registered as an `rt` invocation and inverted the assertion. **14 available false matches across `.forgejo/workflows/`** (`export` ×8, `abort` ×2, `start` ×2, `assert` ×2). `crew-doctrine#148`.

bosun closed this issue

2026-09-06 14:58:02 +02:00

bosun referenced this issue from a commit

2026-09-06 15:42:03 +02:00

test(ci): execute base-divergence-check's rc contract, don't grep it

bosun referenced this issue from a commit

2026-09-06 16:01:50 +02:00

test(ci): execute base-divergence-check's rc contract, don't grep it