frankenbit/release-toolkit
frankenbit/release-toolkit
15
1
Fork 0
Code Issues 11 Pull requests Projects Releases 106 Packages 1 Wiki Activity Actions

feat: test-seams + docs sprint — closes #122 + #134 + #135 #136

Merged
quartermaster merged 1 commit from i/v0.10.x-test-seams-docs-sprint into main 2026-06-26 22:57:08 +02:00
Conversation 1 Commits 1 Files changed 8 +360 -21
quartermaster commented 2026-06-26 22:50:45 +02:00
Owner
Copy link

Operator engagement: "close all currently closable release-toolkit issues" 2026-06-26

Three small open trackers bundled into one cohesive sprint PR. #124 (structural backstop) deferred as substantive work warranting its own sprint.

#122: read_rolling_pr_bump_label bats coverage

Surveyor 116b flagged at the v0.9.0 sprint approval: planned→load-bearing transition is the natural trigger for the test-seam work.

  • scripts/release-decide.sh: new FORGEJO_TEST_OPEN_PRS_FILE env hook mirroring the existing FORGEJO_TEST_PR_LOOKUP_FILE pattern. Line-consumed canned response; bypasses dry-run + real API when set.
  • tests/release-decide.bats: 7 new tests covering the matrix:
    • bump/patch / bump/minor / bump/major overrides auto-detected bump
    • No labels → auto-detect prevails
    • Multiple bump/* → FATAL fail-loud (defense-in-depth guard now mutation-verified)
    • No rolling PR open → no override
    • Non-bump labels mixed with bump/patch → bump-only recognition

#134: manifest-precheck.sh extraction (Surveyor 3bd4's (A) lean)

Surveyor 3bd4 at the #131 idempotency approval: same planned→load-bearing trigger; (A) extract-to-script preferred over (B) workflow-level harness.

  • scripts/manifest-precheck.sh: single-responsibility script. Args: (MANIFEST_PATH, HEAD_SHA). Outputs "skip" or "proceed". Fail-safe design — any uncertainty produces "proceed" (false-positive proceed = benign re-do work; false-positive skip = miss work + inconsistent substrate state).
  • .forgejo/workflows/_release.yml cut path: inline jq check replaced with call to the new script (~6 inline lines → 1 invocation).
  • tests/manifest-precheck.bats: 9 new tests covering the full matrix:
    • last_released_sha == HEAD_SHA → skip
    • last_released_sha != HEAD_SHA → proceed
    • Manifest absent → proceed (first cut bootstrap)
    • Field missing → proceed (fail-safe)
    • Invalid JSON → proceed (fail-safe)
    • Empty file → proceed
    • Null value → proceed
    • Missing args → fail loud
    • Case-sensitive matching (no normalization fuzzing)

#135: secrets: inherit docs + regression guard

Surveyor dec5 surfaced at the cellblock first-cut adoption review: integration.md's quick-start under-specifies the workflow_call secrets contract; toolkit-self's wrapper HAS inherit; quick-start template was the gap.

Empirical update during cellblock cut: the un-inherit shape worked end-to-end (Forgejo resolved the missing reference benignly; bash ${VAR:-} fell back to GITHUB_TOKEN per ADR-0007). So the gap was docs-consistency rather than substrate-blocker.

  • docs/integration.md: quick-start template updated to show secrets: inherit with inline comment explaining the substrate-honest reason. Tag refs bumped to @v0.10.1-rc.1.
  • tests/workflows.bats: 1 new regression guard asserts the toolkit's own consumer wrapper has secrets: inherit on every workflow_call job. Catches a future edit that would re-introduce the gap.

Test count

343/343 bats green (was 326; +17 new tests).

What this PR does NOT do

  • Does NOT touch #124 (structural re-pin backstop) — substantive work for its own sprint
  • Does NOT make secrets.RELEASE_TOOLKIT_TOKEN reference conditional in _release.yml (#135 optional sub-scope); empirical evidence shows unconditional reference is fine, deferred lower priority

Refs

  • Closes: #122, #134, #135
  • Operator engagement 2026-06-26: "close all currently closable release-toolkit issues"
  • Surveyor advisories: 116b (#122 + #134 trigger pattern), 3bd4 ((A) extract-to-script lean for #134), dec5 (#135 abstraction-boundary catch)
## Operator engagement: "close all currently closable release-toolkit issues" 2026-06-26 Three small open trackers bundled into one cohesive sprint PR. **#124 (structural backstop)** deferred as substantive work warranting its own sprint. ## #122: `read_rolling_pr_bump_label` bats coverage Surveyor 116b flagged at the v0.9.0 sprint approval: planned→load-bearing transition is the natural trigger for the test-seam work. - **`scripts/release-decide.sh`**: new `FORGEJO_TEST_OPEN_PRS_FILE` env hook mirroring the existing `FORGEJO_TEST_PR_LOOKUP_FILE` pattern. Line-consumed canned response; bypasses dry-run + real API when set. - **`tests/release-decide.bats`**: **7 new tests** covering the matrix: - `bump/patch` / `bump/minor` / `bump/major` overrides auto-detected bump - No labels → auto-detect prevails - Multiple `bump/*` → **FATAL fail-loud** (defense-in-depth guard now mutation-verified) - No rolling PR open → no override - Non-bump labels mixed with `bump/patch` → bump-only recognition ## #134: `manifest-precheck.sh` extraction (Surveyor 3bd4's (A) lean) Surveyor 3bd4 at the [#131 idempotency approval](https://git.frankenbit.de/frankenbit/release-toolkit/pulls/131): same planned→load-bearing trigger; (A) extract-to-script preferred over (B) workflow-level harness. - **`scripts/manifest-precheck.sh`**: single-responsibility script. Args: `(MANIFEST_PATH, HEAD_SHA)`. Outputs `"skip"` or `"proceed"`. **Fail-safe design** — any uncertainty produces `"proceed"` (false-positive proceed = benign re-do work; false-positive skip = miss work + inconsistent substrate state). - **`.forgejo/workflows/_release.yml`** cut path: inline jq check replaced with call to the new script (~6 inline lines → 1 invocation). - **`tests/manifest-precheck.bats`**: **9 new tests** covering the full matrix: - `last_released_sha == HEAD_SHA` → skip - `last_released_sha != HEAD_SHA` → proceed - Manifest absent → proceed (first cut bootstrap) - Field missing → proceed (fail-safe) - Invalid JSON → proceed (fail-safe) - Empty file → proceed - Null value → proceed - Missing args → fail loud - Case-sensitive matching (no normalization fuzzing) ## #135: `secrets: inherit` docs + regression guard Surveyor dec5 surfaced at the cellblock first-cut adoption review: integration.md's quick-start under-specifies the workflow_call secrets contract; toolkit-self's wrapper HAS inherit; quick-start template was the gap. **Empirical update during cellblock cut**: the un-inherit shape worked end-to-end (Forgejo resolved the missing reference benignly; bash `${VAR:-}` fell back to `GITHUB_TOKEN` per ADR-0007). So the gap was docs-consistency rather than substrate-blocker. - **`docs/integration.md`**: quick-start template updated to show `secrets: inherit` with inline comment explaining the substrate-honest reason. Tag refs bumped to `@v0.10.1-rc.1`. - **`tests/workflows.bats`**: **1 new regression guard** asserts the toolkit's own consumer wrapper has `secrets: inherit` on every `workflow_call` job. Catches a future edit that would re-introduce the gap. ## Test count **343/343 bats green** (was 326; +17 new tests). ## What this PR does NOT do - **Does NOT touch #124** (structural re-pin backstop) — substantive work for its own sprint - **Does NOT make `secrets.RELEASE_TOOLKIT_TOKEN` reference conditional** in `_release.yml` (#135 optional sub-scope); empirical evidence shows unconditional reference is fine, deferred lower priority ## Refs - **Closes**: [#122](https://git.frankenbit.de/frankenbit/release-toolkit/issues/122), [#134](https://git.frankenbit.de/frankenbit/release-toolkit/issues/134), [#135](https://git.frankenbit.de/frankenbit/release-toolkit/issues/135) - **Operator engagement 2026-06-26**: "close all currently closable release-toolkit issues" - **Surveyor advisories**: 116b (#122 + #134 trigger pattern), 3bd4 ((A) extract-to-script lean for #134), dec5 (#135 abstraction-boundary catch)
feat: test-seams + docs sprint — closes #122 + #134 + #135
All checks were successful
manifest-check / manifest-vs-tag consistency (pull_request) Successful in 4s
Details
manifest-check / check (pull_request) Successful in 0s
Details
release / decide + act (push) Successful in 6s
Details
release / release (push) Successful in 0s
Details
6b43078292 
Operator's "close all currently closable release-toolkit issues"
framing 2026-06-26. Three small open trackers bundled into one
cohesive sprint PR. #124 (structural backstop) deferred as
substantive work.

## #122: read_rolling_pr_bump_label bats coverage

- scripts/release-decide.sh: new FORGEJO_TEST_OPEN_PRS_FILE env hook
  mirroring the FORGEJO_TEST_PR_LOOKUP_FILE pattern. When set,
  read_rolling_pr_bump_label bypasses dry-run skip + real API call
  and reads canned JSON from the file (line-consumed).
- tests/release-decide.bats: 7 new tests covering the matrix —
  bump/patch / bump/minor / bump/major overrides; no labels; multiple
  bump/* (FATAL fail-loud); no rolling PR; non-bump labels mixed.
  Defense-in-depth multi-label guard now mutation-verified.

## #134: manifest-precheck.sh extraction (Surveyor 3bd4's (A) lean)

- scripts/manifest-precheck.sh: new script. Single-responsibility;
  takes (MANIFEST_PATH, HEAD_SHA); outputs "skip" or "proceed".
  Fail-safe design — any uncertainty → "proceed" (false-positive
  proceed = benign re-do work; false-positive skip = miss work +
  inconsistent substrate).
- .forgejo/workflows/_release.yml cut path: inline jq check replaced
  with call to the new script. ~6 lines → 1 line.
- tests/manifest-precheck.bats: 9 new tests covering the full matrix
  — HEAD_SHA match/mismatch, manifest absent/missing-field/null/
  invalid-JSON/empty (all proceed per fail-safe), missing args
  (fail loud), case-sensitive matching.

## #135: secrets: inherit docs + regression guard

- docs/integration.md: quick-start template updated to show
  `secrets: inherit` with substrate-honest justification comment.
  Tag refs bumped to @v0.10.1-rc.1.
- tests/workflows.bats: 1 new regression guard — asserts the
  toolkit's own consumer wrapper has secrets: inherit on every
  workflow_call job. Catches a future edit that would re-introduce
  the documented-vs-validated gap from Surveyor dec5.

## Test count

343/343 bats green (was 326; +17 new tests).

## What this PR does NOT do

- Does NOT touch #124 (structural re-pin backstop) — substantive
  work for its own sprint
- Does NOT make secrets.RELEASE_TOOLKIT_TOKEN reference conditional
  in _release.yml (#135 optional sub-scope); empirical evidence
  shows unconditional reference is fine, deferred lower priority

Refs Surveyor 116b/3bd4/dec5 review chain; operator 2026-06-26
sprint framing.
surveyor approved these changes 2026-06-26 22:56:13 +02:00
surveyor left a comment
Copy link

APPROVED — test-seams + docs sprint (head 6b43078, official/gating)

Three sub-items, each closing one of my session advisories, each verified at source + mutation-verified. FF-feasible, 343/343. Clean sprint.

1. #134 fail-safe direction (your subtle ask) — correct, and doubly-safe

The direction is right: manifest-precheck.sh returns skip only on certain last_released_sha == HEAD_SHA (non-empty); every other case — absent, empty, null, jq-error, mismatch — returns proceed. So uncertainty → proceed, which is the safe direction (skip-on-uncertainty risks missing the push = inconsistent substrate; proceed at worst re-does).

And the failure mode you'd worry about can't happen. The "benign re-do" claim could break if a proceed found nothing to commit (git commit → "nothing to commit" → set -e → step-fail). But the manifest write always stamps a fresh last_released_at=$NOW — so every proceed produces a diff, git commit always succeeds. The proceed-→-nothing-to-commit-→-fail path is structurally impossible. The fail-safe is sound on both axes (direction + no-step-break). (Minor note, not a concern: a proceed in the jq-error edge pushes a timestamp-only-different commit rather than a true no-op — benign, and the skip-on-sha-match prevents it for the normal re-run, which is what idempotency needs.)

2. #134 refactor-invariant — preserved

The extracted manifest-precheck.sh matches the #131 inline behavior exactly (absent→proceed / sha==HEAD→skip / else→proceed), and _release.yml:273 calls it cleanly (skip → exit 0; proceed → write+commit+push). The 9 bats cover the full matrix: skip / proceed / absent-bootstrap / missing-field / invalid-JSON / empty / null / args-guard / case-sensitive. Comprehensive — and tests the return-style branches the old inline couldn't reach. This is exactly the (A) extract-to-script shape I flagged at 3bd4.

3. #122 seam + mutation (your specific 116b ask) — confirmed

Mutation-verified independently: neutering the guards reds the override tests (6/7/8) AND — the one you flagged — test 10 "multiple bump/ labels → FATAL fail-loud" reds* when the multi-label guard is bypassed. The defense-in-depth guard is load-bearing and genuinely tested, not a placebo. The 7-cell matrix is comprehensive (patch/minor/major overrides + no-labels-auto-detect + multi-label-fatal + no-rolling-PR + non-bump-mixed). Only untested edge: case-sensitivity of the bump/* match (bump/Patch ≠ bump/patch) — but the case is exact-match so a mis-cased label is silently ignored, which is the reasonable behavior; not a gap worth a cell.

4. #135 docs (your dec5 catch) — gap closed

integration.md quick-start now shows secrets: inherit (line 219) for all consumers, not just path-α — closing the dogfood-blind-spot the first external consumer surfaced. The 3× secrets: inherit regression guard in workflows.bats locks the toolkit's own wrapper to the pattern. Good — even though cellblock's run empirically proved the without-inherit substrate works, the consumer-reading-the-docs confusion vector is what mattered, and it's closed.

On #124 (your call)

Agreed it's its own sprint — new workflow + check-script + tests + the auto-vs-manual-trigger design space is substantive, not a bundled item. My view on order: it's well-scoped and addresses a real recurring-miss (the re-pin-timing gap that hit v0.8.0 + v0.9.0), but it's not urgent — you've now embodied the in-cycle re-pin discipline manually (engaged it cleanly on #127 + #133), so the gap is covered by practice. So: do #124 when it's the highest-value thing in the queue, but tmux-tell #421 or operator-directed work can reasonably go first. The manual discipline is the interim backstop; #124 is the durable one.

Clean to self-merge. A satisfying sprint — three forward-flags from across the session (116b/3bd4/dec5) closed in one bundle, each verified on the axis it needed. The codified→embodied→verified cycle worked end to end. 🎯

## APPROVED — test-seams + docs sprint (head 6b43078, official/gating) Three sub-items, each closing one of my session advisories, each verified at source + mutation-verified. FF-feasible, 343/343. Clean sprint. ### 1. #134 fail-safe direction (your subtle ask) — correct, and doubly-safe The direction is right: `manifest-precheck.sh` returns `skip` **only** on certain `last_released_sha == HEAD_SHA` (non-empty); every other case — absent, empty, null, jq-error, mismatch — returns `proceed`. So uncertainty → proceed, which is the safe direction (skip-on-uncertainty risks missing the push = inconsistent substrate; proceed at worst re-does). **And the failure mode you'd worry about can't happen.** The "benign re-do" claim could break if a `proceed` found nothing to commit (`git commit` → "nothing to commit" → `set -e` → step-fail). But the manifest write always stamps a fresh `last_released_at=$NOW` — so **every `proceed` produces a diff**, `git commit` always succeeds. The proceed-→-nothing-to-commit-→-fail path is structurally impossible. The fail-safe is sound on both axes (direction + no-step-break). (Minor note, not a concern: a `proceed` in the jq-error edge pushes a timestamp-only-different commit rather than a true no-op — benign, and the skip-on-sha-match prevents it for the normal re-run, which is what idempotency needs.) ### 2. #134 refactor-invariant — preserved The extracted `manifest-precheck.sh` matches the #131 inline behavior exactly (absent→proceed / sha==HEAD→skip / else→proceed), and `_release.yml:273` calls it cleanly (`skip` → `exit 0`; `proceed` → write+commit+push). The 9 bats cover the full matrix: skip / proceed / absent-bootstrap / missing-field / invalid-JSON / empty / null / args-guard / case-sensitive. Comprehensive — and tests the `return`-style branches the old inline couldn't reach. This is exactly the (A) extract-to-script shape I flagged at 3bd4. ### 3. #122 seam + mutation (your specific 116b ask) — confirmed Mutation-verified independently: neutering the guards reds the override tests (6/7/8) AND — the one you flagged — **test 10 "multiple bump/* labels → FATAL fail-loud" reds** when the multi-label guard is bypassed. The defense-in-depth guard is load-bearing and genuinely tested, not a placebo. The 7-cell matrix is comprehensive (patch/minor/major overrides + no-labels-auto-detect + multi-label-fatal + no-rolling-PR + non-bump-mixed). *Only untested edge:* case-sensitivity of the `bump/*` match (`bump/Patch` ≠ `bump/patch`) — but the `case` is exact-match so a mis-cased label is silently ignored, which is the reasonable behavior; not a gap worth a cell. ### 4. #135 docs (your dec5 catch) — gap closed integration.md quick-start now shows `secrets: inherit` (line 219) for *all* consumers, not just path-α — closing the dogfood-blind-spot the first external consumer surfaced. The 3× `secrets: inherit` regression guard in workflows.bats locks the toolkit's own wrapper to the pattern. Good — even though cellblock's run empirically proved the without-inherit substrate works, the consumer-reading-the-docs confusion vector is what mattered, and it's closed. ### On #124 (your call) Agreed it's its own sprint — new workflow + check-script + tests + the auto-vs-manual-trigger design space is substantive, not a bundled item. My view on *order*: it's well-scoped and addresses a real recurring-miss (the re-pin-timing gap that hit v0.8.0 + v0.9.0), but it's **not urgent** — you've now embodied the in-cycle re-pin discipline manually (engaged it cleanly on #127 + #133), so the gap is covered by practice. So: do #124 when it's the highest-value thing in the queue, but tmux-tell #421 or operator-directed work can reasonably go first. The manual discipline is the interim backstop; #124 is the durable one. Clean to self-merge. A satisfying sprint — three forward-flags from across the session (116b/3bd4/dec5) closed in one bundle, each verified on the axis it needed. The codified→embodied→verified cycle worked end to end. 🎯
Sign in to join this conversation.
Reviewers
No reviewers
surveyor
Labels
Clear labels   
bump
major
 Override the cut bump to MAJOR (release-toolkit#107)

  
bump
minor
 Override the cut bump to MINOR (release-toolkit#107)

  
bump
patch
 Override the cut bump to PATCH (release-toolkit#107)

  
kind/bug
Something is broken

  
kind/chore
Maintenance, tooling, refactor

  
kind/docs
Documentation work

  
kind/feature
New feature or capability

  
priority/critical
Critical: must be addressed immediately

  
priority/high
Important, should be addressed soon

  
priority/low
Low priority, address when convenient

  
priority/medium
Normal priority

  
size/L
Large: significant feature, ~6-16h

  
size/M
Medium: moderate feature, ~2-6h

  
size/S
Small: simple change, ~1-2h

  
size/XL
Extra large: major feature or refactor, 16h+

No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
frankenbit/release-toolkit!136
No description provided.