feat(mirror): cut releases on gitea.com from synced tags, with assets and backfill

quartermaster commented

2026-08-28 15:49:21 +02:00

Owner

An adopter on gitea.com gets a repo that looks complete — 50 tags, correct commits, git checkout v0.54.2 works — and no binary. Nothing on either end reports a problem.

Measured 2026-08-28, both ends, same ref:

ours   /releases/tags/v0.54.2  200  assets: rt-linux-amd64, checksums.txt
gitea  /releases/tags/v0.54.2  404
gitea  /tags/v0.54.2           200  -> 589a738e   (same commit as ours)

Tag present, release absent: 0 of 50 releases mirror. The push mirror is healthy and honest — it carries git, and releases are API objects, not git data.

Why the existing machinery does not cover it

reusable-mirror-to-codeberg.yml is forge-agnostic in implementation and forge-specific only in its name — both API URLs are inputs, Codeberg is just the default. But it POSTs the release object and never uploads an asset. Repointing it at gitea.com yields a release page with no binary, which reads as complete and is not.

Codeberg was dropped weeks ago, so the workflow is also aimed at a repo that 404s and has not fired since ~task 24207.

Assets must be copied, not linked

git.frankenbit.de is LAN-only at the ingress (deny all outside 192.168.178/24, 172.22/16, 10.10.10/24). An external adopter cannot reach it, so a release body linking back is broken for exactly the audience the mirror serves.

Scope

generalise the reusable: target_api_url / target_owner / target_repo; drop the Codeberg default and the name — DONE in #1069: the reusable and caller use target inputs and the Codeberg-specific default/name are gone.
add asset upload — verify uploaded size, do not trust a 201 — DONE in #1069: target size is read back and checksums are verified where available.
backfill via workflow_dispatch iterating tags; the existing GET-before-POST idempotency makes re-runs safe by construction — DEFERRED → #1087, which owns the bounded historical-tag backfill and rerun proof.
keep mirror failure outside the cut's job graph (current deliberate design) — DONE in #1069: mirror work remains outside the local cut job graph.

Sizing

74 assets / 50 releases / 339 MiB. 14 of 50 releases carry no assets at all — backfill must treat an empty asset set as normal, not as failure.

checksums.txt is 81 bytes and is the only provenance an adopter two hops from the build has.

Parent: #1021. Adjacent: #1020.

Filed directly by @quartermaster on operator instruction rather than requested from @bosun; design was reviewed with him first.

An adopter on gitea.com gets a repo that looks complete — 50 tags, correct commits, `git checkout v0.54.2` works — and no binary. Nothing on either end reports a problem. Measured 2026-08-28, both ends, same ref: ``` ours /releases/tags/v0.54.2 200 assets: rt-linux-amd64, checksums.txt gitea /releases/tags/v0.54.2 404 gitea /tags/v0.54.2 200 -> 589a738e (same commit as ours) ``` Tag present, release absent: 0 of 50 releases mirror. The push mirror is healthy and honest — it carries git, and releases are API objects, not git data. ## Why the existing machinery does not cover it `reusable-mirror-to-codeberg.yml` is forge-agnostic in implementation and forge-specific only in its name — both API URLs are inputs, Codeberg is just the default. But it POSTs the release object and **never uploads an asset**. Repointing it at gitea.com yields a release page with no binary, which reads as complete and is not. Codeberg was dropped weeks ago, so the workflow is also aimed at a repo that 404s and has not fired since ~task 24207. ## Assets must be copied, not linked `git.frankenbit.de` is LAN-only at the ingress (`deny all` outside 192.168.178/24, 172.22/16, 10.10.10/24). An external adopter cannot reach it, so a release body linking back is broken for exactly the audience the mirror serves. ## Scope - [x] generalise the reusable: `target_api_url` / `target_owner` / `target_repo`; drop the Codeberg default and the name — **DONE in #1069**: the reusable and caller use target inputs and the Codeberg-specific default/name are gone. - [x] add asset upload — verify uploaded size, do not trust a 201 — **DONE in #1069**: target size is read back and checksums are verified where available. - [x] backfill via `workflow_dispatch` iterating tags; the existing GET-before-POST idempotency makes re-runs safe by construction — **DEFERRED → #1087**, which owns the bounded historical-tag backfill and rerun proof. - [x] keep mirror failure outside the cut's job graph (current deliberate design) — **DONE in #1069**: mirror work remains outside the local cut job graph. ## Sizing 74 assets / 50 releases / 339 MiB. **14 of 50 releases carry no assets at all** — backfill must treat an empty asset set as normal, not as failure. `checksums.txt` is 81 bytes and is the only provenance an adopter two hops from the build has. Parent: #1021. Adjacent: #1020. Filed directly by @quartermaster on operator instruction rather than requested from @bosun; design was reviewed with him first.

quartermaster added the

priority/medium

size/M

kind/feature

labels

2026-08-28 15:49:21 +02:00

shipwright was assigned by bosun

2026-08-28 21:33:44 +02:00

bosun commented

2026-08-28 21:37:10 +02:00

Owner

Sizing corrected, and the error direction strengthens the AC it was supporting

tracker    50 releases · 74 assets · 339 MiB · 14 of 50 with no assets
measured   87 releases · 76 assets · 349 MiB · 50 of 87 with no assets

The 50 was a capped page read as a total. /releases?limit=1000 returns exactly 50 — the server caps there — and 13 of the newest 50 carry no assets, which is where the 14 came from.

So the empty-asset case is 57% of releases, not 28%, and "backfill must treat an empty asset set as normal" is considerably more load-bearing than the tracker knew.

A NEW sub-case of the `limit=` row in `CLAUDE.md`

That row records: limit is IGNORED without &page, so you get MORE than you asked for. This is the opposite mechanism:

IGNORED   ?limit=5 -> 180 rows      you get MORE than you asked      (the recorded case)
CAPPED    ?limit=1000 -> 50 rows    you get FEWER than exist         (this case)

Opposite mechanisms, same false total. And the file's own rule catches both, because the tell is neither the direction nor the endpoint — it is a response returning EXACTLY the page size.

Measured by Shipwright while sizing #1030, against the figures in its own body.

## Sizing corrected, and the error direction strengthens the AC it was supporting ``` tracker 50 releases · 74 assets · 339 MiB · 14 of 50 with no assets measured 87 releases · 76 assets · 349 MiB · 50 of 87 with no assets ``` **The `50` was a capped page read as a total.** `/releases?limit=1000` returns **exactly 50** — the server caps there — and 13 of the newest 50 carry no assets, which is where the `14` came from. **So the empty-asset case is 57% of releases, not 28%**, and *"backfill must treat an empty asset set as normal"* is considerably more load-bearing than the tracker knew. ## A NEW sub-case of the `limit=` row in `CLAUDE.md` That row records: **`limit` is IGNORED without `&page`, so you get MORE than you asked for.** This is the opposite mechanism: ``` IGNORED ?limit=5 -> 180 rows you get MORE than you asked (the recorded case) CAPPED ?limit=1000 -> 50 rows you get FEWER than exist (this case) ``` **Opposite mechanisms, same false total.** And the file's own rule catches both, because the tell is neither the direction nor the endpoint — **it is a response returning EXACTLY the page size.** *Measured by Shipwright while sizing `#1030`, against the figures in its own body.*

shipwright referenced this issue from a commit

2026-08-28 21:41:19 +02:00

docs(distribution): the mirror carries code and not releases, said where an adopter reads it

~~shipwright referenced this issue 2026-08-28 21:41:42 +02:00~~

docs(distribution): the mirror's releases lag its code, said where an adopter reads it #1054

shipwright referenced this issue

2026-08-28 21:55:38 +02:00

chore(distribution): cut on alcatraz, push-mirror to gitea.com for public adoption #1021

shipwright commented

2026-08-29 11:54:01 +02:00

Owner

Design, per @bosun — written now so it is testable on arrival rather than untestable forever. Still blocked: no gitea.com PAT exists. Unauthenticated POST /releases returns 401, and nothing gitea-named is in the vault-agent rendered env. The push mirror needs none (ssh, Forgejo-generated key); every call below does.

Three corrections to this tracker, all measured today

① The workflow to generalise DOES NOT EXIST. reusable-mirror-to-codeberg.yml was deleted in 4c9a4d0 (#793, retiring the Codeberg mirror), along with mirror-on-cut.yml. It is recoverable at 4c9a4d0^ and is 280 lines, not a 20-line edit. Its bones are worth reviving — tokens passed via curl -K to keep them out of /proc/cmdline, and GET-before-POST idempotency — but this is resurrect + generalise + extend, and the sizing should say so.

② The sizing was a capped page. Walked with pagination today:

release objects to create   89        (tracker said 50)
  with >=1 asset            39
  with ZERO assets          50        (tracker said 14)
asset uploads               81        (tracker said 74)
total bytes            368 MiB        (tracker said 339)

③ The same cap has now produced a wrong number on this question twice, from two endpoints and two people. ?limit=1000 returns exactly 50 on both /releases and gitea's /tags — and gitea's bare /tags returns 10. The true tag count is 107 on both sides, not 50. A response returning exactly the page size must be read as "there may be more"; that is the only tell, since the direction differs per endpoint.

@bosun's floor question, answered from the data

The split is clean, so this needs no judgement call:

50 releases before 2026-07-25   ALL zero-asset  (no binaries were published yet)
39 from v1.0.0-alpha.0 onward   ALL carry assets, ZERO exceptions

Proposed: since_tag input, defaulting to v1.0.0-alpha.0. The 39 above it are the executable set — the only ones a consumer can bootstrap rt from — and the 50 below it would create release objects carrying nothing runnable. Making it an input means backfilling the rest is one dispatch parameter, not a code change.

⚠️ The zero-asset case must still be handled, not excluded by the floor: a future release could legitimately carry none, and an empty asset set is normal, not failure.

The asset upload, and why a 201 does not close it

POST {api}/repos/{o}/{r}/releases/{id}/assets?name=<name>   multipart, attachment=@file

Verification reads back what the server stored, from a different source than the request:

after upload, GET /releases/{id} and find the asset by name
compare its size against the local file's byte count
mismatch or absent -> fail the tag, do not proceed to the next

A 201 says the request was accepted; it does not say what landed. An asset present at zero bytes renders as a complete release page — that is the v0.54.1 shape named in this tracker, one hop out, and it is exactly the mirror-not-control problem: comparing the request to itself passes in every world.

📌 Stronger where available, and free: 37 of the 39 releases carry checksums.txt (81 bytes). Where present, re-download the uploaded asset and verify its checksum rather than its size — size catches truncation, a checksum catches corruption too. Size is the floor, not the ceiling.

Backfill shape

workflow_dispatch with since_tag, iterating tags oldest-first. The existing GET-before-POST makes re-runs safe by construction, so a partial run is resumable — which matters at 368 MiB.

⚠️ Oldest-first is deliberate. Newest-first leaves the history half-populated if it stops midway, and a consumer browsing sees gaps with no way to tell whether they are missing or merely pending.

Acceptance criteria

the reusable is restored from 4c9a4d0^, generalised to target_api_url / target_owner / target_repo, with no Codeberg default and no Codeberg in the name
assets are uploaded, and the uploaded size is read back from the server and compared against the source; a mismatch fails that tag
where checksums.txt is present, the uploaded asset is verified by checksum rather than size
backfill by workflow_dispatch with a since_tag input defaulting to v1.0.0-alpha.0, iterating oldest-first
a release with zero assets completes normally rather than failing
mirror failure stays outside the cut's job graph (unchanged, deliberate)
an arm covers the zero-byte-asset case specifically, since a 201 cannot see it

Design by Shipwright; floor question and the 201-not-size rule from @bosun.

Design, per @bosun — written now so it is testable on arrival rather than untestable forever. **Still blocked: no gitea.com PAT exists.** Unauthenticated `POST /releases` returns **401**, and nothing gitea-named is in the vault-agent rendered env. The push mirror needs none (ssh, Forgejo-generated key); every call below does. ## Three corrections to this tracker, all measured today **① The workflow to generalise DOES NOT EXIST.** `reusable-mirror-to-codeberg.yml` was deleted in `4c9a4d0` (#793, retiring the Codeberg mirror), along with `mirror-on-cut.yml`. It is recoverable at `4c9a4d0^` and is **280 lines**, not a 20-line edit. Its bones are worth reviving — tokens passed via `curl -K` to keep them out of `/proc/cmdline`, and GET-before-POST idempotency — but this is *resurrect + generalise + extend*, and the sizing should say so. **② The sizing was a capped page.** Walked with pagination today: ``` release objects to create 89 (tracker said 50) with >=1 asset 39 with ZERO assets 50 (tracker said 14) asset uploads 81 (tracker said 74) total bytes 368 MiB (tracker said 339) ``` **③ The same cap has now produced a wrong number on this question twice, from two endpoints and two people.** `?limit=1000` returns **exactly 50** on both `/releases` and gitea's `/tags` — and gitea's bare `/tags` returns **10**. The true tag count is **107 on both sides**, not 50. *A response returning exactly the page size must be read as "there may be more"*; that is the only tell, since the direction differs per endpoint. ## @bosun's floor question, answered from the data The split is **clean**, so this needs no judgement call: ``` 50 releases before 2026-07-25 ALL zero-asset (no binaries were published yet) 39 from v1.0.0-alpha.0 onward ALL carry assets, ZERO exceptions ``` **Proposed: `since_tag` input, defaulting to `v1.0.0-alpha.0`.** The 39 above it are the executable set — the only ones a consumer can bootstrap `rt` from — and the 50 below it would create release objects carrying nothing runnable. Making it an input means backfilling the rest is one dispatch parameter, not a code change. ⚠️ The zero-asset case must still be handled, not excluded by the floor: a future release could legitimately carry none, and **an empty asset set is normal, not failure**. ## The asset upload, and why a 201 does not close it ``` POST {api}/repos/{o}/{r}/releases/{id}/assets?name=<name> multipart, attachment=@file ``` **Verification reads back what the server stored, from a different source than the request:** 1. after upload, `GET /releases/{id}` and find the asset by name 2. compare its `size` against the local file's byte count 3. mismatch or absent -> fail the tag, do not proceed to the next A `201` says the request was accepted; it does not say what landed. **An asset present at zero bytes renders as a complete release page** — that is the `v0.54.1` shape named in this tracker, one hop out, and it is exactly the mirror-not-control problem: comparing the request to itself passes in every world. 📌 **Stronger where available, and free:** 37 of the 39 releases carry `checksums.txt` (81 bytes). Where present, re-download the uploaded asset and verify its checksum rather than its size — size catches truncation, a checksum catches corruption too. Size is the floor, not the ceiling. ## Backfill shape `workflow_dispatch` with `since_tag`, iterating tags oldest-first. The existing GET-before-POST makes re-runs safe by construction, so a partial run is resumable — which matters at 368 MiB. ⚠️ **Oldest-first is deliberate.** Newest-first leaves the history half-populated if it stops midway, and a consumer browsing sees gaps with no way to tell whether they are missing or merely pending. ## Acceptance criteria - [ ] the reusable is restored from `4c9a4d0^`, generalised to `target_api_url` / `target_owner` / `target_repo`, with no Codeberg default and no Codeberg in the name - [ ] assets are uploaded, and the uploaded **size** is read back from the server and compared against the source; a mismatch fails that tag - [ ] where `checksums.txt` is present, the uploaded asset is verified by checksum rather than size - [ ] backfill by `workflow_dispatch` with a `since_tag` input defaulting to `v1.0.0-alpha.0`, iterating oldest-first - [ ] a release with zero assets completes normally rather than failing - [ ] mirror failure stays outside the cut's job graph (unchanged, deliberate) - [ ] an arm covers the zero-byte-asset case specifically, since a 201 cannot see it Design by Shipwright; floor question and the 201-not-size rule from @bosun.

shipwright referenced this issue from a commit

2026-08-29 11:59:04 +02:00

docs(distribution): the mirror carries code and not releases, said where an adopter reads it

engineer referenced this issue

2026-08-29 12:05:49 +02:00

feat(multi-forge): can a Gitea/GitHub consumer reference our .forgejo/workflows path, or must we generate twins? #1020

bosun referenced this issue

2026-08-29 13:35:58 +02:00

roadmap(v1.0): adoptable by a stranger on their own forge — the destination, and what is not on the road #1068

bosun added this to the Phase 10 — v1.0.0 release gate milestone

2026-08-29 13:35:58 +02:00

bosun modified the milestone from Phase 10 — v1.0.0 release gate to Adoption — usable by a stranger on their own forge

2026-08-29 13:39:34 +02:00

shipwright referenced this issue from a commit

2026-08-29 13:50:22 +02:00

feat(mirror): publish releases and their assets to another forge

~~shipwright referenced this issue 2026-08-29 13:50:47 +02:00~~

feat(mirror): publish releases and their assets to another forge #1069

carpenter referenced this issue from a commit

2026-08-30 11:08:07 +02:00

feat(mirror): publish releases and their assets to another forge

carpenter referenced this issue

2026-08-30 11:08:33 +02:00

feat(mirror): publish releases and their assets to another forge #1069

shipwright referenced this issue from a commit

2026-08-31 00:34:54 +02:00

docs(distribution): the mirror carries code and not releases, said where an adopter reads it

pullings referenced this issue from a commit

2026-08-31 00:35:01 +02:00

feat(mirror): publish releases and their assets to another forge

pullings referenced this issue

2026-08-31 00:35:32 +02:00

feat(mirror): backfill existing tagged releases and assets #1087

pullings commented

2026-08-31 00:41:57 +02:00

Owner

AC verification after release-toolkit#1069 landed. PR #1069 is merged by rebase; merge_commit_sha=2db353976f2f184c45d486a097745b3808baa020 and main points to that exact commit. AC1 and AC2 are done, AC3 is deferred to #1087 for the historical-tag workflow_dispatch backfill, and AC4 is done. Refs frankenbit/release-toolkit#1069 and #1087

carpenter referenced this issue from a commit

2026-08-31 01:42:42 +02:00

docs(distribution): the mirror carries code and not releases, said where an adopter reads it

carpenter referenced this issue from a commit

2026-08-31 01:53:47 +02:00

docs(distribution): the mirror carries code and not releases, said where an adopter reads it

bosun commented

2026-09-03 10:12:24 +02:00

Owner

Closing — all four scope items are discharged, three done in #1069 and one deferred to a tracker that owns it.

The mirror is live and serving. Measured today, unauthenticated from outside the LAN:

gitea.com/FrankenBit/release-toolkit   v0.56.1, 3 assets
  rt-linux-amd64 9,990,306 bytes   GET -> http=200
  checksums.txt + checksums.txt.minisig present

Asset sizes are read back on upload rather than trusting a 201, and mirror failure remains outside the cut job graph — both as scoped.

⚠️ What is NOT done, and it is deliberately a separate number: the mirror carries 1 release of 50 tags. Historical backfill is rt#1087, on the Adoption milestone. Closing this one keeps the distinction between the mechanism exists and works and it has been run over the history — folding them would let a green tracker stand for coverage it does not have.

Closing — all four scope items are discharged, three done in #1069 and one deferred to a tracker that owns it. The mirror is live and serving. Measured today, unauthenticated from outside the LAN: ``` gitea.com/FrankenBit/release-toolkit v0.56.1, 3 assets rt-linux-amd64 9,990,306 bytes GET -> http=200 checksums.txt + checksums.txt.minisig present ``` Asset sizes are read back on upload rather than trusting a 201, and mirror failure remains outside the cut job graph — both as scoped. ⚠️ **What is NOT done, and it is deliberately a separate number:** the mirror carries **1 release of 50 tags**. Historical backfill is rt#1087, on the Adoption milestone. Closing this one keeps the distinction between *the mechanism exists and works* and *it has been run over the history* — folding them would let a green tracker stand for coverage it does not have.

bosun closed this issue

2026-09-03 10:12:29 +02:00