frankenbit/release-toolkit

Fork 0

fix(check-self-bootstrap): tolerate transient build-bake diff on cut-prep merge (operator-flagged v0.13.0 regression) #184

New issue

Closed

opened 2026-06-27 13:20:54 +02:00 by quartermaster · 1 comment

quartermaster commented

2026-06-27 13:20:54 +02:00

Owner

Why

Operator-flagged 2026-06-27 evening: "v0.13.0 has some issues in the workflows."

Source-probe of the v0.13.0 cut sequence (release-toolkit/actions): check-self-bootstrap.yml reliably red-fails on the cut prep merge commit, every cut, post-#173.

Empirical evidence (v0.13.0 cut, 2026-06-27)

Task	Workflow	sha	ref	Status
13494	release	`caadf757`	main	success
13492	check-self-bootstrap	`caadf757` (rc.2 self-bootstrap)	main	success
13496	decide + act	`5a2efa75` (prep merge)	main	success
13495	check-self-bootstrap	`5a2efa75`	main	🔴 failure
13499	release	`5d4bb67f` (post-cut bake-reset)	main	success
13497	check-self-bootstrap	`5d4bb67f`	main	success

The same false-positive pattern fired on the substrate-PR merges that introduced the build-bake (#173) + post-cut reset (#177):

Task	sha	ref	Notes
13471	`c59176fc`	main	#173 merge — pre-rc.1-bootstrap
13486	`64c4aed0`	main	#177 merge — pre-rc.2-bootstrap
13495	`5a2efa75`	main	v0.13.0 cut prep merge

Source-confirmation

$ git rev-parse 5a2efa75:.forgejo/workflows/_release.yml          # 203544cd
$ git rev-parse v0.13.0-rc.2:.forgejo/workflows/_release.yml      # 736cb2d2
$ git rev-parse 5a2efa75:.forgejo/workflows/_manifest-check.yml   # 9eb94a1c
$ git rev-parse v0.13.0-rc.2:.forgejo/workflows/_manifest-check.yml # 6d0d76a4

Both compose-scripts at the prep-merge HEAD differ from v0.13.0-rc.2. The diff is entirely in the build-bake line:

-BUILD_BAKED_TOOLKIT_REF: 'main'    # release-toolkit-build-ref ...
+BUILD_BAKED_TOOLKIT_REF: 'v0.13.0' # release-toolkit-build-ref ...

check-self-bootstrap.sh compares blob hashes (no awareness of the build-bake marker) → red.

Root cause

The build-bake design (#148) requires the cut prep commit to bake the to-be-cut version into BUILD_BAKED_TOOLKIT_REF, so the cut tag carries the baked ref (construction-enforces consumer-pin per AGENTS.md §2). But the rc.N pin used for the cut's actual execution doesn't have this bake. So between prep-merge and post-cut bake-reset, _release.yml + _manifest-check.yml at HEAD legitimately differ from the pinned rc tag — and check-self-bootstrap.sh sees this as drift requiring re-pin.

The structural backstop was designed pre-#148 when no compose-script line was expected to differ between HEAD and pin. Build-bake introduces a single tolerated-diff line marker-anchored as # release-toolkit-build-ref.

Option tree

(A) Normalize the bake line before hash-compare (recommended)

In check-self-bootstrap.sh, when comparing each compose-script's HEAD vs pinned content, strip or normalize lines containing the release-toolkit-build-ref marker before computing the comparison hash. This treats the build-bake value as a tolerated diff while still detecting:

The marker line being removed entirely
The marker line moving to a new position
Any surrounding YAML structure changing

Implementation sketch:

# Replace blob-hash compare with normalized-content compare
HEAD_CONTENT=$(git show "HEAD:$script" | sed -E "s/^([[:space:]]*[A-Z_]+: ')[^']*('  # release-toolkit-build-ref)/\1__NORMALIZED__\2/")
PINNED_CONTENT=$(git show "${PINNED_REF}:$script" | sed -E "s/^([[:space:]]*[A-Z_]+: ')[^']*('  # release-toolkit-build-ref)/\1__NORMALIZED__\2/")
HEAD_HASH=$(printf '%s' "$HEAD_CONTENT" | sha256sum)
PINNED_HASH=$(printf '%s' "$PINNED_CONTENT" | sha256sum)

Pros: surgical; preserves structural-backstop intent; uses the same marker as the bake mechanism (single source of truth).

Cons: introduces awareness of the build-bake marker into check-self-bootstrap (new coupling, but bounded — both are part of the self-bootstrap mechanism per AGENTS.md §2).

(B) Skip check-self-bootstrap on prep-PR merge commits

Detect the commit subject ^chore(release): prepare v[0-9] and skip. Brittle (depends on commit-subject convention), leaks domain knowledge into the check.

(C) Pin self-bootstrap to v0.13.0 immediately as part of cut prep

Have the cut prep also self-bootstrap the pin to the to-be-cut version (alongside the build-bake), so HEAD and pin stay aligned. Chicken-and-egg: the tag doesn't exist yet when the prep PR is open.

Surveyor calibration

Lean (A). It uses the structural marker that the bake mechanism itself relies on, scopes the tolerated diff narrowly, and keeps the backstop's mechanism-of-touch intent intact for everything else. Bats tests assert that:

Normalization correctly treats a bake-line diff as same
Removing the marker line entirely is detected as diff (red)
Changing surrounding YAML is still detected as diff (red)

Composition

Sibling of #179: both are post-#173/#177 regressions on the cut-path workflow surface. #179 = post-cut manifest-commit bundle violates #139 paths-ignore. This one = cut-prep-merge bake-rewrite trips check-self-bootstrap.
Sibling of #163: alignment-enforcement at re-pin time (the other side of the structural-backstop discipline).
Sibling of #174: resolve-ref extraction (the build-bake's consumer-side counterpart).
Does NOT block #172 (Phase 2 drops the toolkit_ref input): even with Phase 2, the bake mechanism still rewrites the marker line, so check-self-bootstrap still needs the normalization.

What this PR will NOT do

Will NOT change the build-bake mechanism itself
Will NOT change #179's paths-ignore handling (separate surface)
Will NOT broaden the structural-backstop's tolerance to other diffs — only the marker-anchored build-bake line

Refs

v0.13.0 cut empirical (2026-06-27): task 13495 = check-self-bootstrap red on 5a2efa75
#148 (the build-bake feature that introduces the tolerated-diff line)
#173/#177 (the PRs that implemented + post-cut-reset the build-bake)
#124 / Surveyor 116b (the structural-backstop's mechanism-of-touch design)
AGENTS.md §2 (build-bake + structural-backstop joint surface)

Labels

priority/2-soon (recurs every cut; cosmetic-red but noisy), size/S, kind/fix

## Why Operator-flagged 2026-06-27 evening: "v0.13.0 has some issues in the workflows." Source-probe of the v0.13.0 cut sequence (release-toolkit/actions): **`check-self-bootstrap.yml` reliably red-fails on the cut prep merge commit**, every cut, post-#173. ### Empirical evidence (v0.13.0 cut, 2026-06-27) | Task | Workflow | sha | ref | Status | |------|----------|-----|-----|--------| | 13494 | release | caadf757 | main | success | | 13492 | **check-self-bootstrap** | caadf757 (rc.2 self-bootstrap) | main | **success** | | 13496 | decide + act | 5a2efa75 (**prep merge**) | main | success | | **13495** | **check-self-bootstrap** | **5a2efa75** | **main** | **🔴 failure** | | 13499 | release | 5d4bb67f (post-cut bake-reset) | main | success | | 13497 | check-self-bootstrap | 5d4bb67f | main | success | The same false-positive pattern fired on the substrate-PR merges that introduced the build-bake (#173) + post-cut reset (#177): | Task | sha | ref | Notes | |------|-----|-----|-------| | 13471 | c59176fc | main | #173 merge — pre-rc.1-bootstrap | | 13486 | 64c4aed0 | main | #177 merge — pre-rc.2-bootstrap | | 13495 | 5a2efa75 | main | **v0.13.0 cut prep merge** | ### Source-confirmation ``` $ git rev-parse 5a2efa75:.forgejo/workflows/_release.yml # 203544cd $ git rev-parse v0.13.0-rc.2:.forgejo/workflows/_release.yml # 736cb2d2 $ git rev-parse 5a2efa75:.forgejo/workflows/_manifest-check.yml # 9eb94a1c $ git rev-parse v0.13.0-rc.2:.forgejo/workflows/_manifest-check.yml # 6d0d76a4 ``` Both compose-scripts at the prep-merge HEAD differ from `v0.13.0-rc.2`. The diff is entirely in the build-bake line: ```diff -BUILD_BAKED_TOOLKIT_REF: 'main' # release-toolkit-build-ref ... +BUILD_BAKED_TOOLKIT_REF: 'v0.13.0' # release-toolkit-build-ref ... ``` `check-self-bootstrap.sh` compares blob hashes (no awareness of the build-bake marker) → red. ## Root cause The build-bake design (#148) **requires** the cut prep commit to bake the to-be-cut version into `BUILD_BAKED_TOOLKIT_REF`, so the cut tag carries the baked ref (construction-enforces consumer-pin per AGENTS.md §2). But the rc.N pin used for the cut's actual execution doesn't have this bake. So between prep-merge and post-cut bake-reset, `_release.yml` + `_manifest-check.yml` at HEAD legitimately differ from the pinned rc tag — and `check-self-bootstrap.sh` sees this as drift requiring re-pin. The structural backstop was designed pre-#148 when no compose-script line was expected to differ between HEAD and pin. Build-bake introduces a **single tolerated-diff line** marker-anchored as `# release-toolkit-build-ref`. ## Option tree ### (A) Normalize the bake line before hash-compare (recommended) In `check-self-bootstrap.sh`, when comparing each compose-script's HEAD vs pinned content, strip or normalize lines containing the `release-toolkit-build-ref` marker before computing the comparison hash. This treats the build-bake value as a tolerated diff while still detecting: - The marker line being removed entirely - The marker line moving to a new position - Any surrounding YAML structure changing **Implementation sketch:** ```bash # Replace blob-hash compare with normalized-content compare HEAD_CONTENT=$(git show "HEAD:$script" | sed -E "s/^([[:space:]]*[A-Z_]+: ')[^']*(' # release-toolkit-build-ref)/\1__NORMALIZED__\2/") PINNED_CONTENT=$(git show "${PINNED_REF}:$script" | sed -E "s/^([[:space:]]*[A-Z_]+: ')[^']*(' # release-toolkit-build-ref)/\1__NORMALIZED__\2/") HEAD_HASH=$(printf '%s' "$HEAD_CONTENT" | sha256sum) PINNED_HASH=$(printf '%s' "$PINNED_CONTENT" | sha256sum) ``` **Pros**: surgical; preserves structural-backstop intent; uses the same marker as the bake mechanism (single source of truth). **Cons**: introduces awareness of the build-bake marker into check-self-bootstrap (new coupling, but bounded — both are part of the self-bootstrap mechanism per AGENTS.md §2). ### (B) Skip check-self-bootstrap on prep-PR merge commits Detect the commit subject `^chore(release): prepare v[0-9]` and skip. Brittle (depends on commit-subject convention), leaks domain knowledge into the check. ### (C) Pin self-bootstrap to v0.13.0 immediately as part of cut prep Have the cut prep also self-bootstrap the pin to the to-be-cut version (alongside the build-bake), so HEAD and pin stay aligned. Chicken-and-egg: the tag doesn't exist yet when the prep PR is open. ### Surveyor calibration Lean (A). It uses the structural marker that the bake mechanism itself relies on, scopes the tolerated diff narrowly, and keeps the backstop's mechanism-of-touch intent intact for everything else. Bats tests assert that: 1. Normalization correctly treats a bake-line diff as same 2. Removing the marker line entirely is detected as diff (red) 3. Changing surrounding YAML is still detected as diff (red) ## Composition - **Sibling of #179**: both are post-#173/#177 regressions on the cut-path workflow surface. #179 = post-cut manifest-commit bundle violates #139 paths-ignore. **This one** = cut-prep-merge bake-rewrite trips check-self-bootstrap. - **Sibling of #163**: alignment-enforcement at re-pin time (the other side of the structural-backstop discipline). - **Sibling of #174**: resolve-ref extraction (the build-bake's consumer-side counterpart). - **Does NOT block #172** (Phase 2 drops the toolkit_ref input): even with Phase 2, the bake mechanism still rewrites the marker line, so check-self-bootstrap still needs the normalization. ## What this PR will NOT do - Will NOT change the build-bake mechanism itself - Will NOT change `#179`'s paths-ignore handling (separate surface) - Will NOT broaden the structural-backstop's tolerance to other diffs — only the marker-anchored build-bake line ## Refs - v0.13.0 cut empirical (2026-06-27): task 13495 = check-self-bootstrap red on 5a2efa75 - #148 (the build-bake feature that introduces the tolerated-diff line) - #173/#177 (the PRs that implemented + post-cut-reset the build-bake) - #124 / Surveyor 116b (the structural-backstop's mechanism-of-touch design) - AGENTS.md §2 (build-bake + structural-backstop joint surface) ## Labels priority/2-soon (recurs every cut; cosmetic-red but noisy), size/S, kind/fix

quartermaster self-assigned this

2026-06-27 13:21:24 +02:00

quartermaster referenced this issue from a commit

2026-06-27 13:29:33 +02:00

fix(check-self-bootstrap): normalize build-bake marker line before compare

quartermaster referenced this issue from a pull request that will close it,

2026-06-27 13:30:01 +02:00

fix(check-self-bootstrap): normalize build-bake marker line before compare (closes #184) #185

quartermaster closed this issue

2026-06-27 13:37:20 +02:00

quartermaster commented

2026-06-27 13:45:57 +02:00

Author

Owner

Closed by #185 (merged 87d7aa22 → v0.13.1 released 2026-06-27 13:44:53).

Empirical close: task 13524 = check-self-bootstrap on the v0.13.1 cut-prep merge (sha df6b25da, BUILD_BAKED='v0.13.1' baked, divergence from pinned rc) = SUCCESS — the EXACT formerly-red surface (was task 13495 on v0.13.0). The normalization works on live CI on the same shape that produced the false-positive operator flagged.

Surveyor verify-at-source confirmation: Forgejo PR #186 + #187 reviews + de0f closeout.

Closed by #185 (merged 87d7aa22 → v0.13.1 released 2026-06-27 13:44:53). Empirical close: task 13524 = check-self-bootstrap on the v0.13.1 cut-prep merge (sha df6b25da, BUILD_BAKED='v0.13.1' baked, divergence from pinned rc) = SUCCESS — the EXACT formerly-red surface (was task 13495 on v0.13.0). The normalization works on live CI on the same shape that produced the false-positive operator flagged. Surveyor verify-at-source confirmation: Forgejo PR #186 + #187 reviews + de0f closeout.