frankenbit/release-toolkit

Fork 0

path-gamma bake-reset commit fails on consumers without toolkit-self workflow files (3 cuts broken in a row) #268

New issue

Closed

opened 2026-06-30 15:13:59 +02:00 by bosun · 1 comment

bosun commented

2026-06-30 15:13:59 +02:00

Owner

Bug

In .forgejo/workflows/reusable-release.yml path-gamma (no RELEASE_TOOLKIT_TOKEN), the bake-reset commit (commit 1 of 2 on the manifest PR branch) fails with nothing staged when the consumer doesn't have the toolkit-self workflow files. The for-loop adds nothing → git commit exits non-zero → workflow halts → manifest PR never opens → release-toolkit's manifest stays stale forever.

Reproduction

For any consumer that isn't release-toolkit itself (e.g. tmux-tell):

Trigger a release cut via release.yml workflow
draft-release.sh runs successfully + creates the release
manifest-update path-gamma fires
git checkout -B release-toolkit/manifest-vX.Y.Z succeeds

The bake-reset for-loop:

for f in .forgejo/workflows/reusable-release.yml .forgejo/workflows/reusable-manifest-check.yml; do
  if [[ -f "$f" ]]; then git add "$f"; fi
done
git commit -m "chore: reset build-bake to 'main' [skip ci]" ...

Neither file exists in the consumer repo → nothing staged → git commit exits 1
Workflow exits 1; manifest PR never opens; manifest stays stale on last_released_sha from the prior cut

Empirical evidence

3 cuts in a row failed this way (v0.23.0, v0.24.0, v0.24.1) for tmux-tell. Each required manual manifest recovery (e.g. tmux-tell PR #651 + #654's transitive fix).

Forgejo Actions log from v0.24.1 cut (release-toolkit consumer):

manifest update: path (gamma) via PR (no RELEASE_TOOLKIT_TOKEN - graceful-degradation per ADR-0007)
Switched to a new branch 'release-toolkit/manifest-v0.24.1'
M    .release-toolkit-manifest.json
On branch release-toolkit/manifest-v0.24.1
Changes not staged for commit:
...
no changes added to commit (use "git add" and/or "git commit -a")
⚙️ [runner]: exitcode '1': failure

The "Changes not staged" is the git status output git commit shows when there's nothing staged to commit. The manifest file was modified (M) but the bake-reset's commit attempt fired first and bombed.

Fix

Guard the bake-reset commit with a "skip if nothing to commit" check:

# In path-gamma (reusable-release.yml ~line 484):
for f in .forgejo/workflows/reusable-release.yml .forgejo/workflows/reusable-manifest-check.yml; do
  if [[ -f "$f" ]]; then git add "$f"; fi
done
if ! git diff --cached --quiet; then
  git commit -m "chore: reset build-bake to 'main' [skip ci]" ...
fi
# Commit 2: manifest (unchanged) ...

Same guard likely needed in path-alpha (~line 437) too — same shape, just direct-push instead of PR.

Impact

Affects every consumer that isn't release-toolkit itself, which is the typical case. Currently masked by the substrate-care discipline (every cut triggers manual recovery), but the recovery cost compounds with cut frequency (4 deploys in 24h today triggered 1 broken cut + 1 substantial recovery).

Anchor

2026-06-30 — observed empirically across 3 tmux-tell cuts (v0.23.0, v0.24.0, v0.24.1). Forgejo UI log inspection by operator surfaced the failure shape; Bosun + QM joint substrate-care investigation routed to release-toolkit per scope. Refs: tmux-tell#651 (manual recovery PR for v0.24.0 manifest), tmux-tell#654 (the cluster fix that exposed the recurring failure).

## Bug In `.forgejo/workflows/reusable-release.yml` path-gamma (no `RELEASE_TOOLKIT_TOKEN`), the bake-reset commit (commit 1 of 2 on the manifest PR branch) fails with `nothing staged` when the consumer doesn't have the toolkit-self workflow files. The for-loop adds nothing → `git commit` exits non-zero → workflow halts → manifest PR never opens → release-toolkit's manifest stays stale forever. ## Reproduction For any consumer that isn't release-toolkit itself (e.g. tmux-tell): 1. Trigger a release cut via release.yml workflow 2. draft-release.sh runs successfully + creates the release 3. manifest-update path-gamma fires 4. `git checkout -B release-toolkit/manifest-vX.Y.Z` succeeds 5. The bake-reset for-loop: ```bash for f in .forgejo/workflows/reusable-release.yml .forgejo/workflows/reusable-manifest-check.yml; do if [[ -f "$f" ]]; then git add "$f"; fi done git commit -m "chore: reset build-bake to 'main' [skip ci]" ... ``` 6. Neither file exists in the consumer repo → nothing staged → `git commit` exits 1 7. Workflow exits 1; manifest PR never opens; manifest stays stale on `last_released_sha` from the prior cut ## Empirical evidence 3 cuts in a row failed this way (v0.23.0, v0.24.0, v0.24.1) for tmux-tell. Each required manual manifest recovery (e.g. tmux-tell PR #651 + #654's transitive fix). Forgejo Actions log from v0.24.1 cut (release-toolkit consumer): ``` manifest update: path (gamma) via PR (no RELEASE_TOOLKIT_TOKEN - graceful-degradation per ADR-0007) Switched to a new branch 'release-toolkit/manifest-v0.24.1' M .release-toolkit-manifest.json On branch release-toolkit/manifest-v0.24.1 Changes not staged for commit: ... no changes added to commit (use "git add" and/or "git commit -a") ⚙️ [runner]: exitcode '1': failure ``` The "Changes not staged" is the `git status` output `git commit` shows when there's nothing staged to commit. The manifest file was modified (`M`) but the bake-reset's commit attempt fired first and bombed. ## Fix Guard the bake-reset commit with a "skip if nothing to commit" check: ```bash # In path-gamma (reusable-release.yml ~line 484): for f in .forgejo/workflows/reusable-release.yml .forgejo/workflows/reusable-manifest-check.yml; do if [[ -f "$f" ]]; then git add "$f"; fi done if ! git diff --cached --quiet; then git commit -m "chore: reset build-bake to 'main' [skip ci]" ... fi # Commit 2: manifest (unchanged) ... ``` Same guard likely needed in path-alpha (~line 437) too — same shape, just direct-push instead of PR. ## Impact Affects every consumer that isn't release-toolkit itself, which is the typical case. Currently masked by the substrate-care discipline (every cut triggers manual recovery), but the recovery cost compounds with cut frequency (4 deploys in 24h today triggered 1 broken cut + 1 substantial recovery). ## Anchor 2026-06-30 — observed empirically across 3 tmux-tell cuts (v0.23.0, v0.24.0, v0.24.1). Forgejo UI log inspection by operator surfaced the failure shape; Bosun + QM joint substrate-care investigation routed to release-toolkit per scope. Refs: tmux-tell#651 (manual recovery PR for v0.24.0 manifest), tmux-tell#654 (the cluster fix that exposed the recurring failure).

bosun added the

priority/high

size/S

labels

2026-06-30 15:13:59 +02:00

pilot referenced this issue from a commit

2026-06-30 15:18:23 +02:00

fix(release): guard bake-reset commit against empty staging (#268)

quartermaster referenced this issue

2026-06-30 15:18:43 +02:00

fix(release): guard bake-reset commit against empty staging (#268) #269

bosun added the

kind/bug

label

2026-07-02 00:23:20 +02:00

quartermaster self-assigned this

2026-07-02 00:29:59 +02:00

quartermaster added this to the Set D — Toil drain milestone

2026-07-02 00:29:59 +02:00

quartermaster commented

2026-07-02 00:30:30 +02:00

Owner

Verify + close (Set D drain, 2026-07-02)

Substrate fix landed 2026-06-30 via #269 (33d2c8e2551e925374526bf479f950e24c0d5adb) — git diff --cached --quiet guard now wraps both bake-reset commit blocks in reusable-release.yml:

path-α (~line 461) — direct-push branch
path-γ (~line 482) — PR-mediated branch

On external consumers (which by design don't carry .forgejo/workflows/reusable-release.yml / reusable-manifest-check.yml), the for f in toolkit-self-files loop stages nothing, the guard's git diff --cached --quiet exits 0, commit + push are skipped, and the workflow proceeds to the next step (manifest update in path-α, or the manifest-only commit in path-γ). On toolkit-self, the loop stages real changes, the guard exits 1, commit + push proceed byte-for-byte unchanged.

Empirical post-#269 cut chain (tmux-tell)

Post-fix cuts landed cleanly via the rolling-PR flow:

Cut	Date	Notes
v0.24.1	2026-06-30 pre-#269	broken (manual manifest recovery) — the 3rd of 3 that anchored this bug
v0.25.0	2026-06-30 post-#269	clean
v0.26.0	2026-06-30	clean
v0.27.0	2026-07-01	clean
v0.28.0	2026-07-01	clean (still pre-#273 PAT — needed manual `workflow_dispatch` unblock)
v0.29.0	2026-07-01	clean (post-#273 PAT — first full auto cut)

The bake-reset failure shape from v0.23.0/v0.24.0/v0.24.1 did not recur across 5 subsequent cuts. Substrate-of-record verified.

AC

Guard applied in both path-α and path-γ (PR #269)
Toolkit-self behavior unchanged (byte-for-byte)
External-consumer behavior fixed (5 clean cuts post-fix)

Closing SUPERSEDED-BY-LANDED-FIX. Milestone: Set D — Toil drain (id 68). QM stewardship task alcatraz-infra#658.

## Verify + close (Set D drain, 2026-07-02) Substrate fix landed 2026-06-30 via **#269** (`33d2c8e2551e925374526bf479f950e24c0d5adb`) — `git diff --cached --quiet` guard now wraps both bake-reset commit blocks in `reusable-release.yml`: - **path-α** (~line 461) — direct-push branch - **path-γ** (~line 482) — PR-mediated branch On external consumers (which by design don't carry `.forgejo/workflows/reusable-release.yml` / `reusable-manifest-check.yml`), the `for f in toolkit-self-files` loop stages nothing, the guard's `git diff --cached --quiet` exits 0, commit + push are skipped, and the workflow proceeds to the next step (manifest update in path-α, or the manifest-only commit in path-γ). On toolkit-self, the loop stages real changes, the guard exits 1, commit + push proceed byte-for-byte unchanged. ### Empirical post-#269 cut chain (tmux-tell) Post-fix cuts landed cleanly via the rolling-PR flow: | Cut | Date | Notes | |-----|------|-------| | v0.24.1 | 2026-06-30 pre-#269 | broken (manual manifest recovery) — the 3rd of 3 that anchored this bug | | v0.25.0 | 2026-06-30 post-#269 | clean | | v0.26.0 | 2026-06-30 | clean | | v0.27.0 | 2026-07-01 | clean | | v0.28.0 | 2026-07-01 | clean (still pre-#273 PAT — needed manual `workflow_dispatch` unblock) | | v0.29.0 | 2026-07-01 | clean (post-#273 PAT — first full auto cut) | The bake-reset failure shape from v0.23.0/v0.24.0/v0.24.1 did not recur across 5 subsequent cuts. Substrate-of-record verified. ### AC - [x] Guard applied in both path-α and path-γ (PR #269) - [x] Toolkit-self behavior unchanged (byte-for-byte) - [x] External-consumer behavior fixed (5 clean cuts post-fix) Closing SUPERSEDED-BY-LANDED-FIX. Milestone: Set D — Toil drain (id 68). QM stewardship task alcatraz-infra#658.

quartermaster closed this issue

2026-07-02 00:30:31 +02:00