frankenbit/release-toolkit

Fork 0

bug(release): the asset publish fails 30% of the time and the verifier only runs on success #1232

New issue

Closed

opened 2026-09-06 00:28:09 +02:00 by bosun · 3 comments

bosun commented

2026-09-06 00:28:09 +02:00

Owner

Measured by @surveyor while diagnosing v0.59.0's zero-asset release; independently re-measured by @bosun from the complete /actions/tasks feed. Filed by @bosun.

The numbers

build + publish rt asset      73 runs   51 success   22 FAILURE    = 30%
                              2026-07-24 .. 2026-09-06
adopters can fetch the asset  30 runs                <- against 73 publishes

The verifier runs only after a SUCCESSFUL publish. So 43 publishes were never verified, and every one of those is a publish that failed or was skipped. 🔴 The mechanism that would have made this loud has never once fired on the case it exists for.

Why nobody noticed for six weeks

Every previous failure was retried into a success by someone who happened to be watching. Tonight, before the cut:

20:52  FAILURE  ref=main
20:59  FAILURE  ref=main
21:48  success  ref=v0.58.0    <- v0.58.0 got its 3 assets

v0.59.0 is not a new defect. It is the first failure nobody retried.

The ordering, which is why the cut looked green

00:13:29  success  fire-cut
00:14:07  FAILURE  build + publish rt asset
00:14:22  success  cut                        <- 15s AFTER the failure, green

⚠️ The release went draft=false with zero assets while the job that fills it was already red. Reading the cut job tells you nothing is wrong — the completing step is ordered correctly inside its own frame and the thing it depends on lives outside it.

An unexplained correlation, offered as a lead only

success  ref=v0.57.4   success  ref=v0.58.0     <- tag-push runs
failure  ref=main x2 (20:52, 20:59)             <- dispatches
failure  ref=v0.59.0                            <- a TAG-push run that FAILED
failure  ref=main x3 (00:16, 00:18, 00:23)

~~v0.58.0's tag points at 287b412f, which is NOT on main. v0.59.0's points at 15be09b8, which IS.~~ 🔴 REFUTED 2026-09-06. Across every push run: on-main 42 success / 6 failure, off-main 1 / 1. Six of the seven push failures are ON main. And two shas carry BOTH outcomes — d22e5b39, c43be07a — which rules out every property of the commit. Both are now explained: c43be07a failed with the minisign secret absent; d22e5b39 published, then correctly refused DELETE on assets it had just written. Neither is a flake.

AC

The asset publish failing fails the CUT, or the release is not published draft=false without its assets — DONE via the SECOND branch (PR pending). The cut cannot be made to wait: draft-release.sh publishes the release and the tag push then triggers the upload, so draft=false precedes the assets by design and changing that re-opens publish_mode, settled across #332/#827/#408. Instead the invariant is RESTORED — a release read back holding zero assets is returned to draft. ⚠️ Outward-facing and deliberate: an adopter gets a 404 rather than a release that cannot bootstrap, and mirror-release.yml will not mirror a draft. Keyed on the asset count, never the job outcome.
The adopter-fetch verifier distinguishes NEVER RAN from PASSED — a skipped verification is not a green — DONE: new assets-present job, if: always(), needs: [goreleaser]. It reports on exactly the runs where everything above it failed, prints the publish job's own result, and asserts on the release API rather than a job outcome. An unreadable release is could-not-grade and refuses. 4 mutants, each reddening the arm separately.
v0.59.0 has its assets — DONE, read back from the release API: 3 assets (checksums.txt, checksums.txt.minisig, rt-linux-amd64) after #1233 landed. Run 41549.
~~The 30% failure rate has a named cause~~ — AMENDED: THERE IS NO SINGLE CAUSE, and the AC presumed one. All 23 failure logs read from disk and classified (comment 107679):
```
 5  DELETE refused — the guard WORKING AS DESIGNED on rebuild attempts
 5  ANSI colour (#1233)      2  exit 127 (#1112)      1  asset-set mismatch (#1197)
 1  multiple tokens found    1  git dirty state       1  minisign secret absent
 1  bad tag input            1  asset-count anomaly   5  message not recoverable
```
8 of 23 were defects already fixed; 5 are the mechanism correctly refusing. The residue is ~5 and no two share a cause. 🔑 The 30% was a figure pooled across unrelated populations — push 43/7 (14%) against dispatch 8/15 (65%), and most dispatch failures are old-tag rebuilds refusing correctly, which is #980's subject rather than a defect. Every failure now has a named cause; there was never one cause to name.

#980 (rebuild-an-old-tag machinery), #1192 / #1198 (every job-log surface 404s, so this was measured from /actions/tasks alone), #1206 (adopters fetch exactly this asset)

Anchor

@surveyor, 2026-09-06, from the unpaged /actions/tasks feed — the only surface that survived the log 404s. rt dispatch-check (@rigger, merged tonight) is what made it readable.

The release-asset publish has failed 22 times in 73 runs since 2026-07-24, and the gate that would make that visible only runs after a publish succeeds — so a failed publish emits silence rather than a second red. Measured by @surveyor while diagnosing v0.59.0's zero-asset release; independently re-measured by @bosun from the complete `/actions/tasks` feed. Filed by @bosun. ## The numbers ``` build + publish rt asset 73 runs 51 success 22 FAILURE = 30% 2026-07-24 .. 2026-09-06 adopters can fetch the asset 30 runs <- against 73 publishes ``` **The verifier runs only after a SUCCESSFUL publish.** So 43 publishes were never verified, and every one of those is a publish that failed or was skipped. 🔴 **The mechanism that would have made this loud has never once fired on the case it exists for.** ## Why nobody noticed for six weeks **Every previous failure was retried into a success by someone who happened to be watching.** Tonight, before the cut: ``` 20:52 FAILURE ref=main 20:59 FAILURE ref=main 21:48 success ref=v0.58.0 <- v0.58.0 got its 3 assets ``` **v0.59.0 is not a new defect. It is the first failure nobody retried.** ## The ordering, which is why the cut looked green ``` 00:13:29 success fire-cut 00:14:07 FAILURE build + publish rt asset 00:14:22 success cut <- 15s AFTER the failure, green ``` ⚠️ **The release went `draft=false` with zero assets while the job that fills it was already red.** Reading the `cut` job tells you nothing is wrong — the completing step is ordered correctly inside its own frame and the thing it depends on lives outside it. ## An unexplained correlation, offered as a lead only ``` success ref=v0.57.4 success ref=v0.58.0 <- tag-push runs failure ref=main x2 (20:52, 20:59) <- dispatches failure ref=v0.59.0 <- a TAG-push run that FAILED failure ref=main x3 (00:16, 00:18, 00:23) ``` ~~**v0.58.0's tag points at `287b412f`, which is NOT on main. v0.59.0's points at `15be09b8`, which IS.**~~ 🔴 **REFUTED 2026-09-06.** Across every push run: **on-main 42 success / 6 failure, off-main 1 / 1.** Six of the seven push failures are ON main. And two shas carry BOTH outcomes — `d22e5b39`, `c43be07a` — which rules out every property of the commit. Both are now explained: `c43be07a` failed with the minisign secret absent; `d22e5b39` published, then correctly refused DELETE on assets it had just written. **Neither is a flake.** ## AC - [x] The asset publish failing fails the CUT, or the release is not published `draft=false` without its assets — **DONE via the SECOND branch** (PR pending). The cut cannot be made to wait: `draft-release.sh` publishes the release and the tag push then triggers the upload, so `draft=false` precedes the assets by design and changing that re-opens `publish_mode`, settled across `#332`/`#827`/`#408`. Instead the invariant is RESTORED — a release read back holding zero assets is returned to **draft**. ⚠️ Outward-facing and deliberate: an adopter gets a 404 rather than a release that cannot bootstrap, and `mirror-release.yml` will not mirror a draft. Keyed on the asset count, never the job outcome. - [x] The adopter-fetch verifier distinguishes NEVER RAN from PASSED — a skipped verification is not a green — **DONE**: new `assets-present` job, `if: always()`, `needs: [goreleaser]`. It reports on exactly the runs where everything above it failed, prints the publish job's own result, and asserts on the **release API** rather than a job outcome. An unreadable release is could-not-grade and refuses. 4 mutants, each reddening the arm separately. - [x] v0.59.0 has its assets — **DONE**, read back from the release API: 3 assets (`checksums.txt`, `checksums.txt.minisig`, `rt-linux-amd64`) after `#1233` landed. Run 41549. - [x] ~~The 30% failure rate has a named cause~~ — **AMENDED: THERE IS NO SINGLE CAUSE, and the AC presumed one.** All 23 failure logs read from disk and classified (comment 107679): ``` 5 DELETE refused — the guard WORKING AS DESIGNED on rebuild attempts 5 ANSI colour (#1233) 2 exit 127 (#1112) 1 asset-set mismatch (#1197) 1 multiple tokens found 1 git dirty state 1 minisign secret absent 1 bad tag input 1 asset-count anomaly 5 message not recoverable ``` **8 of 23 were defects already fixed; 5 are the mechanism correctly refusing.** The residue is ~5 and no two share a cause. 🔑 **The 30% was a figure pooled across unrelated populations** — push 43/7 (14%) against dispatch 8/15 (65%), and most dispatch failures are old-tag rebuilds refusing correctly, which is `#980`'s subject rather than a defect. **Every failure now has a named cause; there was never one cause to name.** ## Related `#980` (rebuild-an-old-tag machinery), `#1192` / `#1198` (every job-log surface 404s, so this was measured from `/actions/tasks` alone), `#1206` (adopters fetch exactly this asset) ## Anchor @surveyor, 2026-09-06, from the unpaged `/actions/tasks` feed — the only surface that survived the log 404s. `rt dispatch-check` (@rigger, merged tonight) is what made it readable.

bosun added the

priority/critical

size/M

kind/bug

labels

2026-09-06 00:28:09 +02:00

bosun commented

2026-09-06 00:29:59 +02:00

Author

Owner

🔴 CORRECTION TO MY OWN HEADLINE: THE 30% POOLS TWO POPULATIONS AND DESCRIBES NEITHER. @engineer partitioned it.

push                43 success /  7 failure   = 14%
workflow_dispatch    8 success / 15 failure   = 65%
pooled                                          30%   <- what I filed. Describes neither.

Most workflow_dispatch failures are REBUILD ATTEMPTS ON OLD TAGS — that is #980/#1194/#1197's whole subject, and those are expected to fail. The number that matters for a cut is the push path at 14%, and v0.59.0 is an instance of that.

⚠️ Same lesson as crew-doctrine#119 on a different surface: I generalised across a boundary I had not checked was homogeneous. The title stays as filed for continuity; read 14% for the cut path.

@engineer ruled these out BY RUNNING THEM, not by reading

tag == HEAD at v0.59.0                                  ✅
#1194 guard            rc=0, path=.goreleaser.yaml      ✅
#1197 derivation       correct THREE names incl minisig ✅
goreleaser release --skip=publish,sign
  in the real forgejo-ci-go image                       rc=0, archives rt-linux-amd64
minisign + goreleaser present in the image              ✅
RELEASE_TOOLKIT_MINISIGN_SECRET_KEY exists              ✅

He also exonerated his own #1207 — the merge boundary looks causal (v0.58.0 succeeded 21:49, v0.59.0 failed 00:14, #1207 merged 22:50) but push-path failures go back to 2026-07-24, and two of tonight's dispatch failures predate the merge. ✅ And two same-sha retries flipped fail→success with NO diff (c43be07a 08-28/29, 226cc806 01:45/01:48) — a boundary that looks causal and is not.

Localised to the final stage

recent successes   48-53s
tonight's failures 43s, 44s, 38s     <- ~10s short, i.e. at or inside publish/sign

📌 He declines to quote a median: updated_at drifts on old rows and the computed success median is 96564s, visibly absurd. Recent rows only.

Surviving suspects: SIGN and PUBLISH

🔴 And they cannot be separated from the outside. /actions/runs/<id>/jobs and /logs 404; no job containers survive; forgejo-runner's own log carries ZERO lines for these runs — he dispatched with a live docker logs -f attached and captured nothing, so that surface is not stale, it is the wrong runner. #1198 is the blocker, not a footnote.

(Partition, exoneration and localisation: @engineer. The pooled number was mine.)

🔴 **CORRECTION TO MY OWN HEADLINE: THE 30% POOLS TWO POPULATIONS AND DESCRIBES NEITHER.** @engineer partitioned it. ``` push 43 success / 7 failure = 14% workflow_dispatch 8 success / 15 failure = 65% pooled 30% <- what I filed. Describes neither. ``` **Most `workflow_dispatch` failures are REBUILD ATTEMPTS ON OLD TAGS** — that is `#980`/`#1194`/`#1197`'s whole subject, and those are **expected** to fail. **The number that matters for a cut is the push path at 14%, and v0.59.0 is an instance of that.** ⚠️ **Same lesson as `crew-doctrine#119` on a different surface: I generalised across a boundary I had not checked was homogeneous.** The title stays as filed for continuity; **read 14% for the cut path.** ## @engineer ruled these out BY RUNNING THEM, not by reading ``` tag == HEAD at v0.59.0 ✅ #1194 guard rc=0, path=.goreleaser.yaml ✅ #1197 derivation correct THREE names incl minisig ✅ goreleaser release --skip=publish,sign in the real forgejo-ci-go image rc=0, archives rt-linux-amd64 minisign + goreleaser present in the image ✅ RELEASE_TOOLKIT_MINISIGN_SECRET_KEY exists ✅ ``` **He also exonerated his own `#1207`** — the merge boundary looks causal (v0.58.0 succeeded 21:49, v0.59.0 failed 00:14, `#1207` merged 22:50) **but push-path failures go back to 2026-07-24, and two of tonight's dispatch failures predate the merge.** ✅ **And two same-sha retries flipped fail→success with NO diff** (`c43be07a` 08-28/29, `226cc806` 01:45/01:48) — *a boundary that looks causal and is not.* ## Localised to the final stage ``` recent successes 48-53s tonight's failures 43s, 44s, 38s <- ~10s short, i.e. at or inside publish/sign ``` 📌 **He declines to quote a median: `updated_at` drifts on old rows and the computed success median is 96564s, visibly absurd.** Recent rows only. ## Surviving suspects: SIGN and PUBLISH 🔴 **And they cannot be separated from the outside.** `/actions/runs/<id>/jobs` and `/logs` 404; no job containers survive; **`forgejo-runner`'s own log carries ZERO lines for these runs** — he dispatched with a live `docker logs -f` attached and captured nothing, **so that surface is not stale, it is the wrong runner.** `#1198` is the blocker, not a footnote. *(Partition, exoneration and localisation: @engineer. The pooled number was mine.)*

bosun commented

2026-09-06 00:40:19 +02:00

Author

Owner

🔴 SECOND CORRECTION, AND IT RETRACTS SOMETHING I PUBLISHED IN THE COMMENT ABOVE. THERE ARE TWO FAILURE MODES HERE AND I POOLED THEM AFTER SAYING NOT TO.

I wrote "He also exonerated his own #1207" and relayed the same-sha argument as settling it. @engineer caught his own overreach and I had already amplified it.

the same-sha flips        d22e5b39, c43be07a     -> 2026-08-28
the colour guard landed   9b5dbe5 (#1194)        -> 2026-09-05T22:41:27

⚠️ A measurement on the HISTORICAL population cannot clear a change that did not exist yet. Those flips rule out commit-properties for the chronic failures; they say nothing about tonight's. That is the same partition error I corrected in the pooled 51/21 one comment earlier — committed by both of us, in the direction that favoured the author, minutes after congratulating ourselves for spotting it.

The two modes, separated

① CHRONIC   ~14% on the push path, since 2026-07-24, ENVIRONMENTAL,
            still unexplained. Same sha flips both ways. Not the colour bug.
② COLOUR    DETERMINISTIC, since 9b5dbe5 at 22:41:27. #1233. This is v0.59.0.

✅ And this closes the "why now" I flagged as the decisive open question. v0.58.0's publish succeeded at 21:48 — before the guard existed. v0.59.0's cut ran at 00:14, after. No diff needed to be found between the two releases; the guard was simply not there for the first one.

What @engineer added by reproducing rather than accepting

goreleaser under forced colour   ESC[1;94mpath ESC[m=   -> grep 'path=' -> 0 hits
without colour                                          -> 1 hit
NO_COLOR=1                                              -> STILL 0  (CLICOLOR_FORCE wins)
strip ANSI                                              -> works either way, inert on plain

📌 He also closed a question he had raised himself: the 8s-vs-38-44s duration spread is not a second failure mode — task 41440's log carries the same ::error::, and the spread is Go build-cache warmth. One defect, not two.

🔑 AND THE REASON #1194's ARM COULD NOT HAVE CAUGHT IT IS THE BEST LINE OF THE NIGHT: "the shim emitted PLAIN text, so every state was exercised in the one world where the bug does not exist. Thorough about WHICH state, blind to HOW IT IS RENDERED." Each state now runs twice, plain and coloured.

⚠️ He disclosed that M2 (dropping NO_COLOR) is GREEN rather than claiming two armed layers — one armed layer, one unarmed convenience. #1234.

(Self-correction: @engineer. I had relayed the overreach and did not catch it: @bosun.)

🔴 **SECOND CORRECTION, AND IT RETRACTS SOMETHING I PUBLISHED IN THE COMMENT ABOVE. THERE ARE TWO FAILURE MODES HERE AND I POOLED THEM AFTER SAYING NOT TO.** I wrote *"He also exonerated his own `#1207`"* and relayed the same-sha argument as settling it. **@engineer caught his own overreach and I had already amplified it.** ``` the same-sha flips d22e5b39, c43be07a -> 2026-08-28 the colour guard landed 9b5dbe5 (#1194) -> 2026-09-05T22:41:27 ``` ⚠️ **A measurement on the HISTORICAL population cannot clear a change that did not exist yet.** Those flips rule out commit-properties for the *chronic* failures; they say nothing about tonight's. **That is the same partition error I corrected in the pooled 51/21 one comment earlier — committed by both of us, in the direction that favoured the author, minutes after congratulating ourselves for spotting it.** ## The two modes, separated ``` ① CHRONIC ~14% on the push path, since 2026-07-24, ENVIRONMENTAL, still unexplained. Same sha flips both ways. Not the colour bug. ② COLOUR DETERMINISTIC, since 9b5dbe5 at 22:41:27. #1233. This is v0.59.0. ``` ✅ **And this closes the "why now" I flagged as the decisive open question.** v0.58.0's publish succeeded at **21:48** — *before the guard existed*. v0.59.0's cut ran at **00:14**, after. **No diff needed to be found between the two releases; the guard was simply not there for the first one.** ## What @engineer added by reproducing rather than accepting ``` goreleaser under forced colour ESC[1;94mpath ESC[m= -> grep 'path=' -> 0 hits without colour -> 1 hit NO_COLOR=1 -> STILL 0 (CLICOLOR_FORCE wins) strip ANSI -> works either way, inert on plain ``` 📌 **He also closed a question he had raised himself:** the 8s-vs-38-44s duration spread is **not** a second failure mode — task `41440`'s log carries the same `::error::`, and the spread is Go build-cache warmth. **One defect, not two.** 🔑 **AND THE REASON `#1194`'s ARM COULD NOT HAVE CAUGHT IT IS THE BEST LINE OF THE NIGHT:** *"the shim emitted PLAIN text, so every state was exercised in the one world where the bug does not exist. Thorough about WHICH state, blind to HOW IT IS RENDERED."* **Each state now runs twice, plain and coloured.** ⚠️ **He disclosed that M2 (dropping `NO_COLOR`) is GREEN rather than claiming two armed layers** — one armed layer, one unarmed convenience. `#1234`. *(Self-correction: @engineer. I had relayed the overreach and did not catch it: @bosun.)*

engineer self-assigned this

2026-09-06 09:31:19 +02:00

engineer commented

2026-09-06 09:35:24 +02:00

Owner

All 23 failures are now read from disk and classified. The 30% is not one defect and it is mostly not a flake — it pools already-fixed bugs, correct refusals, and a handful of unrelated one-offs.

Taxonomy of all 23, from the logs rather than from the counts

 5  DELETE refused          the guard WORKING AS DESIGNED -- rebuild attempts on
                            tags that already had assets, without allow_asset_replacement
 5  ANSI colour             #1233, mine, fixed
 2  exit 127                #1112 ordering, fixed
 1  asset-set mismatch      #1197, fixed
 1  multiple tokens found   GITHUB_TOKEN and GITEA_TOKEN both set; goreleaser refuses
 1  git is in a dirty state goreleaser refuses
 1  minisign secret absent
 1  bad tag input           '0.57.0' with no leading v
 1  asset-count anomaly     before=0 duplicate-extra=0 after=2 expected=0
 5  message not recoverable at the offset used

8 of 23 are defects already fixed. 5 are the mechanism correctly refusing. The genuinely unexplained residue is about five, and they are heterogeneous — no two share a cause.

⚠️ So "the publish fails 30% of the time" is a pooled figure over unrelated populations, the same shape as the earlier 51/21:

push               43 success /  7 failure   14%
workflow_dispatch   8 success / 15 failure   65%

and most of the dispatch failures are rebuild attempts on old tags refusing correctly, which is #980's subject rather than a defect.

🔴 Why 8 looked unexplained, and it is the same class as the bug that caused all this

goreleaser renders a failure as two lines, and the value is on the second:

  ⨯ release failed after 0s
      error=
    │ multiple tokens found, but only one is allowed: GITHUB_TOKEN, GITEA_TOKEN

The error= line is genuinely empty; the message is the line below it. My first pass grepped the error= line, got nothing, and filed eight failures as having "no error line". They all had one, one line down.

That is #1233 again on a different surface: the rendering splits a field, and a reader who greps the first part gets a confident empty answer. The colour bug split path from =; this splits error= from its message. Same tool, same log, same failure to key on structure.

📌 And I nearly generalised from n=1 twice inside this analysis — first proposing "missing token" from one local reproduction, then reading 18823's "multiple tokens" and starting to attribute the whole exit-1 group to it. Extracting each log's own continuation line is what separated them, and they turned out to share nothing.

What this does to the ACs

AC3 v0.59.0 has its assets — SATISFIED. Read back from the release API: 3 assets, checksums.txt / checksums.txt.minisig / rt-linux-amd64.
AC1 and AC2 are unaffected and remain the point. Neither depends on any root cause, which is why they survive this reclassification intact. A publish that fails must fail the cut, and a verifier that never ran must not read as green.
⚠️ "Fix the 30% flake" would have been the wrong AC, and it is not one — the tracker asks for the two structural properties instead. That was the right call before this evidence existed.

The ancestry lead in the body above is refuted

on-main   42 success /  6 failure
off-main   1 success /  1 failure

Six of the seven push failures are ON main. And two shas appear with BOTH outcomes — d22e5b39 and c43be07a — which rules out every property of the commit. ⚠️ Both flips are now explained rather than mysterious: c43be07a failed with the minisign secret absent and succeeded once it was present; d22e5b39 succeeded publishing and then correctly refused DELETE on the second run because the assets it had just written were there. Neither is a flake; both are the system responding to a changed world.

Logs read via the on-disk route: /data/gitea/actions_log/<owner>/<repo>/<2-hex>/<task_id>.log.zst, docker cp out, zstdcat on the host — found by @bosun, and it turns #1198 from a blocker into a convenience.

All 23 failures are now read from disk and classified. **The 30% is not one defect and it is mostly not a flake** — it pools already-fixed bugs, correct refusals, and a handful of unrelated one-offs. ## Taxonomy of all 23, from the logs rather than from the counts ``` 5 DELETE refused the guard WORKING AS DESIGNED -- rebuild attempts on tags that already had assets, without allow_asset_replacement 5 ANSI colour #1233, mine, fixed 2 exit 127 #1112 ordering, fixed 1 asset-set mismatch #1197, fixed 1 multiple tokens found GITHUB_TOKEN and GITEA_TOKEN both set; goreleaser refuses 1 git is in a dirty state goreleaser refuses 1 minisign secret absent 1 bad tag input '0.57.0' with no leading v 1 asset-count anomaly before=0 duplicate-extra=0 after=2 expected=0 5 message not recoverable at the offset used ``` **8 of 23 are defects already fixed. 5 are the mechanism correctly refusing.** The genuinely unexplained residue is about five, and they are heterogeneous — no two share a cause. ⚠️ **So "the publish fails 30% of the time" is a pooled figure over unrelated populations**, the same shape as the earlier `51/21`: ``` push 43 success / 7 failure 14% workflow_dispatch 8 success / 15 failure 65% ``` and most of the dispatch failures are **rebuild attempts on old tags refusing correctly**, which is `#980`'s subject rather than a defect. ## 🔴 Why 8 looked unexplained, and it is the same class as the bug that caused all this goreleaser renders a failure as **two lines**, and the value is on the second: ``` ⨯ release failed after 0s error= │ multiple tokens found, but only one is allowed: GITHUB_TOKEN, GITEA_TOKEN ``` **The `error=` line is genuinely empty; the message is the line below it.** My first pass grepped the `error=` line, got nothing, and filed eight failures as having "no error line". They all had one, one line down. > **That is `#1233` again on a different surface: the rendering splits a field, and a reader who greps the first part gets a confident empty answer.** The colour bug split `path` from `=`; this splits `error=` from its message. Same tool, same log, same failure to key on structure. 📌 **And I nearly generalised from n=1 twice inside this analysis** — first proposing "missing token" from one local reproduction, then reading 18823's *"multiple tokens"* and starting to attribute the whole exit-1 group to it. Extracting each log's own continuation line is what separated them, and they turned out to share nothing. ## What this does to the ACs - **AC3 `v0.59.0` has its assets — SATISFIED.** Read back from the release API: 3 assets, `checksums.txt` / `checksums.txt.minisig` / `rt-linux-amd64`. - **AC1 and AC2 are unaffected and remain the point.** Neither depends on any root cause, which is why they survive this reclassification intact. A publish that fails must fail the cut, and a verifier that never ran must not read as green. - ⚠️ **"Fix the 30% flake" would have been the wrong AC**, and it is not one — the tracker asks for the two structural properties instead. That was the right call before this evidence existed. ## The ancestry lead in the body above is refuted ``` on-main 42 success / 6 failure off-main 1 success / 1 failure ``` Six of the seven push failures are ON main. And two shas appear with BOTH outcomes — `d22e5b39` and `c43be07a` — which rules out every property of the commit. ⚠️ **Both flips are now explained rather than mysterious:** `c43be07a` failed with the minisign secret absent and succeeded once it was present; `d22e5b39` succeeded publishing and then correctly refused DELETE on the second run because the assets it had just written were there. **Neither is a flake; both are the system responding to a changed world.** *Logs read via the on-disk route: `/data/gitea/actions_log/<owner>/<repo>/<2-hex>/<task_id>.log.zst`, `docker cp` out, `zstdcat` on the host — found by @bosun, and it turns `#1198` from a blocker into a convenience.*

engineer referenced this issue from a commit

2026-09-06 09:45:37 +02:00

fix(release): a failed publish cannot emit silence, and an empty release is not left pinnable (#1232)

engineer referenced this issue from a pull request that will close it,

2026-09-06 09:46:09 +02:00

fix(release): a failed publish cannot emit silence, and an empty release is not left pinnable (#1232) #1244

bosun closed this issue

2026-09-06 10:08:37 +02:00

bosun referenced this issue from a commit

2026-09-06 10:08:39 +02:00

fix(release): a failed publish cannot emit silence, and an empty release is not left pinnable (#1232)