audit: cold-read of README.md + other adopter-facing markdown (skeptical-outsider lens) #160
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
3 participants
herald
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit#160
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
Operator engagement 2026-06-27: surfaced the cold-read audit pattern, anchored on empirical evidence from the tmux-tell README:
Key signal from the Codex feedback that applies broadly to home-grown infra-substrate projects:
Release-toolkit's README + integration.md + AGENTS.md likely have analogous issues:
The cold-read prompt (operator-provided, verbatim)
The framing for release-toolkit would swap "AI agent orchestration" for "release automation" or "CI/CD" — the skeptical-outsider lens is the load-bearing part. The prompt template is reusable for any home-grown infra project's adopter-facing docs.
Audit surfaces (priority order)
README.mddocs/integration.mdAGENTS.mddocs/conventions.mddocs/adr/*.mdWho runs the cold reads
Three possibilities:
(A) Operator runs them through Claude.ai (Chat) + Codex
(B) Herald (in-character)
(C) Fresh chamber spawn (Surveyor / Pilot / new Engineer)
Recommended flow
Output
docs/cold-read-feedback-2026-06-XX.mdcapturing the verbatim model responses + the triangulated findingsComposition
Implementation surface
Audit-only:
Estimated scope: 1-2 hours of cold-read execution + Herald revision PR (depends on what surfaces)
What this PR does NOT do
Refs
Operator engagement update 2026-06-27: the cold-read runner pool expands to include the Codex chambers (Lookout + Carpenter). Recommended triangulation flow:
The two-Codex-chamber pass would give us 3-Codex-instance triangulation plus the Claude.ai out-of-band perspective. Findings shared across all 3-4 surfaces = highest-confidence signal for revision priorities.
Sequencing: run the cold-reads when the docs are stable (post-#148 build-bake + #149/#150 Unicode cleanups + #152 examples) so the runners see the post-cleanup state, not the in-flight state.
uses: @ref#172Cold-Read Audit: release-toolkit adopter-facing docs
Applied the cold-read prompt (skeptical senior infra developer, short on time) to README.md, docs/integration.md, and AGENTS.md at v0.22.0. No project-internal knowledge assumed. Findings ordered by surface priority.
Surface 1 — README.md
1. In ~10 seconds: what is this, what problem does it claim to solve?
Clear. The opening paragraph lands: "A shared, Forgejo-native release pipeline you reference instead of copy-pasting release machinery into every repository." I know what it is and why it exists within 15 seconds. The bullet list under "What it does" reinforces this for the first three bullets.
The fourth and fifth bullets lose me:
"Self-bootstrap re-pin discipline," "construction-enforces," "build-bake mechanism," "cut-prep time" -- all jargon to an outsider. The value these features provide is not stated; only the mechanism is. A developer who just found this repo cannot parse what problem these bullets solve for them. They read as inside-baseball appended to a clear list.
2. Would you run the install command?
Yes, with hesitation at Step 3. Steps 1 and 2 of Quick Start are a clean config file and a CHANGELOG seed -- no friction. Step 3 (the release.yml YAML) introduces
path-(alpha)/path-(gamma)mid-setup without context:A new adopter at Step 3 is being asked to choose between two named paths whose names do not convey meaning. "Direct-push manifest commit" and "manifest-via-PR fallback" are self-contained descriptions; "path-alpha" and "path-gamma" are not. The ADR-0007 cross-reference inside a quick-start step adds to the intimidation surface.
The
paths-ignoreinline comment also reads as internal project notes:An issue reference (#139) inside a quick-start YAML is visible internal scaffolding. A new adopter does not need to know why this line exists, just that it should be there.
3. Where do you get confused, lose the thread, or want to bounce?
4. Does anything read as hype, hand-waving, or unproven assertion?
5. The single biggest thing that is unclear
Path-alpha vs path-gamma in Quick Start Step 3. These named paths appear mid-setup, require an ADR read to understand, and are not introduced anywhere before that paragraph. A developer following the quick-start for the first time hits this and must decide whether to read integration.md first before continuing, or accept that they are choosing blindly between two options named by Greek letters.
6. What would make you star it instead of closing the tab?
Surface 2 -- docs/integration.md
Generally well-structured. "What you get" is tight. The five-step adoption flow is logical. The troubleshooting section is genuinely useful. Specific friction points:
Step 4 has a 600-word interior. The basic release.yml YAML is ~20 lines; then "Wiring secrets," "About publish_mode," "How reusable-release.yml decides what to do" (the mode=noop/update/cut decision tree), and "Post-publish deploy chaining" all land before Step 5. A first-time adopter following the numbered steps hits a large undifferentiated block of content. The decision-tree section in particular ("On every push to main, release-decide.sh walks git log...") is implementation detail that belongs in a "How it works" or troubleshooting section, not inside a setup step.
Path-alpha / path-gamma introduced late. The terms first appear in "Wiring secrets" under Step 4 -- but the formal definition (the three-path table) is under "Branch protection compatibility," several scrolls later. A reader who encounters "required on path-alpha" must scroll forward to understand what path-alpha is. Either define on first use or introduce via "Option A / Option B" language before using the Greek-letter shorthand.
Step 5: "Document... in your project's CLAUDE.md" -- this assumes the reader uses Claude Code. "Your project's CONTRIBUTING.md, CLAUDE.md, or equivalent" would be more general. As written it reads as toolkit-internal convention leaking into adopter guidance.
The build-bake / BUILD_BAKED_TOOLKIT_REF section (under "The toolkit ref") is clear and useful, but the final paragraph ("Reading the ref at runtime via
${{ github.workflow_ref }}was probed and rejected...") is implementation rationale that belongs in an ADR, not an adopter-facing reference.What's well done: the Tokens & permissions table (three-path comparison) is clear. The "Why
secrets: inheritand not an explicitsecrets:block?" callout is an excellent pre-emption of a real footgun. Troubleshooting Q&As are appropriately scoped.Surface 3 -- AGENTS.md
This document is correctly scoped as implementer-facing, so outsider-comprehensibility expectations are lower -- but the issue specifically flags "mechanism-of-touch" and "self-bootstrap" for jargon checking.
Jargon that stops a first-time contributor without definition:
Section 8 (Source-grounded vs invented claims) is the most valuable section for contributor culture and also the densest. The Axis A/B/C counting structure (n=7, n=1, n=2) and the promotion-cadence framing are sound practices, but a first-time reader takes several paragraphs to understand the taxonomy. A 2-sentence summary at the top of section 8 ("Verify claims against live systems, not documentation or inference. The instances below are the failure-mode catalog.") before the axis breakdown would help.
Section 10 (Substrate-grounding arc) reads as project history. The convergence table and forward-going implication are valuable; the section length (300+ words) and density of internal history may not be load-bearing for a new contributor. Worth keeping but could move to an appendix or ADR.
What's well done: the carry-forward enumeration table (section 1), the rc-tag lifecycle outcome table (section 2.4), and the YAML gotchas (section 3) are concrete and immediately applicable.
Cross-cutting observations
Pattern 1 -- Mechanism-buries-the-value (README bullets 4-5, integration.md Step 4 interior, AGENTS.md section 2). The same pattern the #160 issue identified in tmux-tell's README appears here: the what and why (user benefit) are often subordinate to the how (mechanism). Most visible in README bullets 4-5 and the release-decide.sh mode-decision section in integration.md.
Pattern 2 -- "Multiple production projects" claim is repeated but never substantiated. Three appearances in README, none naming the projects or linking to evidence. A single named adopter reference converts the claim from assertion to proof.
Pattern 3 -- Path-alpha/path-gamma vocabulary requires Greek-letter literacy. Clean internal shorthand for a real distinction, but it appears in adopter-facing quick-start surfaces before it's defined. Either define on first use ("path-alpha (direct-push setup)") or use plain language in the quick-start and reserve the labels for integration.md where they're defined.
No broken markdown found. All three documents render cleanly.
No misrepresented shipped changes found. Status section accurately reflects pre-1.0 state and the pin-to-tag requirement.
Verdict
Top 5 highest-leverage edits (across all three surfaces):
README: Remove or demote bullets 4-5 from "What it does" -- "self-bootstrap re-pin discipline" and "build-bake mechanism" are the strongest mechanism-buries-the-value offenders on the primary landing surface. Replace with benefit statements or move to the Architecture section.
README: Name one production adopter -- three "proven across multiple projects" claims without naming one; a link to tmux-tell's releases page converts the assertion into evidence at zero cost.
README + integration.md: Remove path-alpha/path-gamma labels from quick-start surfaces; define on first use in integration.md -- the labels are clean shorthand but arrive before they're explained. Quick Start Step 3 should say "see integration.md for token setup options"; integration.md should define the paths before using the shorthand.
integration.md: Extract the release-decide.sh mode=noop/update/cut decision tree from Step 4 into a "How it works" section -- it's accurate and useful but implementation detail inside a numbered setup step.
AGENTS.md: Add a short glossary or define jargon on first use -- "codified-to-embodied," "substrate," "mechanism-of-touch," "bus-ping," and "Surveyor XXXX" reviewer codes all require project-internal context. A 10-line glossary at the top pays off across the entire document.
Overall length verdict:
Delta Cold-Read: release-toolkit adopter docs — post-#306 validation
Pilot delta read against README.md, docs/integration.md, and AGENTS.md at main HEAD (post-PR #306 merge). Checking the 5 findings from comment 78681 against current state.
Finding 1 — README bullets 4–5 (mechanism-buries-the-value)
Status: CLEAR
Both offending bullets are rewritten:
Both now lead with the user benefit. Mechanism detail is gone. "Self-bootstrap re-pin discipline" and "build-bake mechanism" do not appear in README.
Finding 2 — README: name one production adopter
Status: CLEAR
Status section now reads: "it cuts its own releases and drives tmux-tell's." Named adopter with a link to its releases page — converts the "multiple projects" assertion into evidence.
Finding 3 — path-alpha/path-gamma in quick-start surfaces
Status: CLEAR
path-alpha/path-gammado not appear anywhere in integration.md (grep returns empty). README Quick Start Step 3 no longer contains the Greek-letter vocabulary — replaced with: "Token setup is optional. The default needs no token... To make cuts fully autonomous on a protectedmain, see token setup in integration.md."The Tokens & Permissions section in integration.md introduces (α) / (β) / (γ) via a defining table before using the shorthand. First use is in the section header, immediately followed by the table — acceptable first-use discipline.
Finding 4 — integration.md: decision tree extracted from Step 4
Status: CLEAR
The
release-decide.shmode=noop/update/cut decision tree is now a standalone## How the cut decides what to dosection at line 269 of integration.md — after all five setup steps. Step 4's interior no longer contains the mechanism prose; it carries a cross-reference link: "see How the cut decides what to do below for the mechanism."Finding 5 — AGENTS.md: jargon defined on first use
Status: CLEAR
AGENTS.md now opens with an explicit
## Glossarysection defining all five flagged terms:All six glossary entries are present. Jargon used throughout the document is now grounded at the top.
Regressions / new issues
None found. No regressions introduced by the revision. The three surfaces are structurally clean:
One pre-existing secondary note (from the original cold-read body, not in the top-5 verdict): integration.md Step 5 still reads "Document the commit + fragment format in your project's CLAUDE.md" without the "or equivalent" qualifier. This was a secondary observation in the original read; it was not tracked as one of the 5 findings and is not blocking.
Verdict
All 5 findings CLEAR. No regressions. Issue #160 can close.
Closing per Pilot delta cold-read (comment 78759) — all 5 findings CLEAR on merged-#306 state. Herald addressed every point; secondary CLAUDE.md-Step-5 note is non-blocking pre-existing. Set G 2/2 complete; release-toolkit board fully drains + milestone-closes.