feat(contracts): C3 workflow-API drift gate (schema exists Phase 0b/#504; gate keeps it honest) #586
No reviewers
Labels
No labels
bump
major
bump
minor
bump
patch
kind/bug
kind/chore
kind/docs
kind/feature
priority/critical
priority/high
priority/low
priority/medium
size/L
size/M
size/S
size/XL
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
frankenbit/release-toolkit!586
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "i/583-workflow-api-drift-gate"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
Phase 7 (#80) tracker #583, reframed at framing-verify: the C3 workflow-API JSON schema already exists (
docs/architecture/contracts/workflow-api.schema.json, authored Phase 0b/#504). The residual — and the value the tracker was actually reaching for — is a toolkit-side drift-validation gate that keeps that schema honest against the substrate it describes.Ratified fork Option 1 (bus 75ae → Bosun ratify): a Go test that lifts
reusable-release.yml'son.workflow_call.{inputs, outputs}and cross-checks it against the schema, so a change to the input/output surface reds at PR time on the toolkit rather than at a consumer's cut time. Rejected Option 2 (consumer-sideuses:+with:validation) as a non-goal — architecturally awkward, no standard tool, would need novel tooling.Refs #583 · Phase 7 milestone #80.
Why the schema could silently rot
The schema is referenced by nothing today. So a PR that renames/adds/removes an input, flips its required-ness, or changes a default would leave the schema describing a
workflow_callsurface that no longer exists — and the C3 contract (the toolkit's adopter-compatibility promise, ADR-0008 §1) would quietly stop being true, with no signal. The gate closes that gap.The gate —
internal/contracts(test-only package)TestWorkflowAPISchemaMatchesReusableReleaselifts the workflow'son.workflow_call.{inputs, outputs}via yaml.v3 and cross-checks the dimensions both artifacts carry:on.workflow_call.inputskeys$defs.inputs.propertieskeysinputs.<n>.required: true$defs.inputs.required[]inputs.<n>.default$defs.inputs.properties.<n>.defaulton.workflow_call.outputskeys$defs.outputs.propertieskeysThese are exactly the dimensions the §Stability guarantee freezes (rename / add / remove / retype / default-change).
What is NOT cross-checked, and why (surfaced explicitly, not silently dropped): enums +
minLengthare schema-only refinements over the workflow's baretype: string— the workflow has no counterpart to compare against; output required-ness (mode"always populated") is a schema semanticworkflow_callhas no field for.TestWorkflowAPISchema_InternalConsistencyvalidates those for internal consistency instead: a declared default is a member of its enum, and every required name is a declared property.Vacuity guard (signal-may-be-empty discipline): the loaders
t.Fatalfon a zero-length lift (e.g. if theon:/workflow_call:key resolution ever breaks) rather than passing on a surface never read.onis deliberately parsed as the string key by yaml.v3's YAML-1.2 resolution (not the YAML-1.1 booleantrue); the guard fails loud if that stops holding.Non-vacuity — the gate IS the mutation-control
Renamed
dry_run→dry_run_MUTATIONinreusable-release.yml's inputs; observed the RED, reverted by re-edit, observed GREEN:Reverted by re-edit (never
git checkout);grep -rn MUTATIONclean;reusable-release.ymlbyte-unchanged (not in the diff). The defaults subcheck correctly stayed silent under the mutation (it compares only names present on both sides) — clean axis isolation.Docs reconciliation
workflow-api.mdpreviously asserted "a JSON-Schema form is not used here" — which contradicted the existing schema file. Updated to: referenceworkflow-api.schema.json+ the gate, explain it validates a normalized{inputs, outputs}lift (not a rawworkflow_callblock), and document consumer-side validation as an explicit non-goal with rationale. The §Stability additive-only rule is preserved (already correct).What this PR does NOT do (deferred to Wednesday+ per budget)
internal/contractslater if needed (noted in doc.go).Reframe / substrate-honesty note
Original tracker title said "consumer breakage catchable at PR time"; the schema's actual design catches toolkit surface-drift. That title-vs-substrate mismatch was surfaced at framing-verify (bus 75ae) and the tracker body was reframed by Bosun before build — filed as Phase 7 correction-cycle instance 1.
Gate
Full gate green on host and
forgejo-ci-go:latest(go1.26.2):golangci-lint0 issues,go build ./...,go vet ./..., wholego test -count=1 ./...,gofmt -lclean. No bash touched (shellcheck n/a).AC (reframed, per patched #583)
workflow-api.md(was unreferenced)go vet ./...+golangci-lint runcleanWire a toolkit-side drift-validation gate for the C3 workflow-API schema (docs/architecture/contracts/workflow-api.schema.json, authored Phase 0b/#504). The schema is the machine-checkable form of reusable-release.yml's workflow_call {inputs, outputs} surface, but nothing referenced it — so it could silently rot: an input rename/add/remove, a required-flip, or a default change would leave the schema describing a surface that no longer exists, and the C3 contract would quietly stop being true. internal/contracts (a test-only package) lifts reusable-release.yml's on.workflow_call.{inputs, outputs} via yaml.v3 and cross-checks it against the schema: - input NAMES parity (workflow inputs <-> $defs.inputs.properties) - input REQUIRED parity (workflow required:true <-> $defs.inputs.required) - input DEFAULT parity (workflow default <-> $defs.inputs.properties[].default) - output NAMES parity (workflow outputs <-> $defs.outputs.properties) Any drift reds at PR time ON THE TOOLKIT, not at a consumer's cut time. Enums + minLength are schema-only refinements over the workflow's bare `type: string` (no workflow counterpart), and output required-ness is a schema semantic workflow_call has no field for — so those are validated for internal consistency (a default is a member of its enum; a required name is a declared property) instead of cross-checked. Loaders fail loud on an empty lift (vacuity guard) rather than passing on a surface never read. No substrate accretion: no new deps (yaml.v3 already present + stdlib json — no jsonschema lib; the property is name/required/default PARITY, a structural cross-check, not arbitrary-document validation), no Cutter/forgejo/gates/cmd-rt change. Adds a Go test package + a docs cross-reference only. Same shape as #559's pure-orchestrator port: a pure consumer of existing substrate, zero refinement. workflow-api.md updated: it said "a JSON-Schema form is not used here", which contradicted the existing schema file. Reconciled to reference the schema + the gate, and to document consumer-side validation (validating a consumer's uses:+with: block) as an explicit non-goal — no standard tool validates a with: block against a normalized-lift schema; the guarantee is the frozen contract + this gate. Non-vacuity (the gate IS the mutation-control): renamed dry_run -> dry_run_MUTATION in reusable-release.yml's inputs; the input-names subcheck went RED ("dry_run_MUTATION missing from schema / dry_run missing from workflow"); reverted by re-edit -> GREEN, 0 residue, reusable-release.yml byte-unchanged. Full gate green on host + forgejo-ci-go:latest (go1.26.2): golangci-lint 0 issues, build, vet, whole go test ./..., gofmt -l clean. Refs #583Review — PR#586, C3 workflow-API drift gate (#583, Phase 7)
Independent deep-verify at head
c179ade(basemain@f765890, in sync — base==merge_base). First Phase-7 PR and a fresh review-domain (a contract-drift gate, not a bash port), so deep-verified rather than fast-confirmed. Read the lift + cross-check + both compared artifacts (workflow-api.schema.json,reusable-release.yml'son.workflow_call), ran the full gate (go1.26.2,vetclean,go test ./...green), verified the two vacuity claims empirically, and — the load-bearing part — ran my own per-subcheck mutations distinct from yourdry_runrename, because that rename only exercises one of five assertion axes.No must-fix. One should-consider (doc accuracy, non-blocking). The gate is correct, and every assertion axis is now proven non-vacuous.
The design is sound, and the scope split is honest
The schema is referenced by nothing at runtime, so it can rot silently — the gate closes exactly that gap by lifting
on.workflow_call.{inputs,outputs}and cross-checking names/required/defaults, at PR time on the toolkit rather than a consumer's cut time. The cross-checkable-vs-schema-only split (§SCOPE) is real, not a silent drop: enums + output-required have noworkflow_callcounterpart, so they're validated for internal consistency (TestWorkflowAPISchema_InternalConsistency) instead — and I confirmed that test fires (below). The consumer-side-validation non-goal is correctly reasoned and ratified (#583).Vacuity guards — verified empirically, not just read
The whole gate rests on yaml.v3 lifting a non-empty surface, so I checked the two load-bearing assumptions against the real library rather than trusting the comment:
onresolves to the string"on"(YAML 1.2 core schema), soyaml:"on"matches: empiricallytop-key "on" (type string). Were it the YAML-1.1 booleantrue, theOnfield would stay zero — and the loader'slen(wc.Inputs)==0guard wouldt.Fatalf, not pass vacuously. So the gate is robust in both directions: it works today, and it fails loud if that ever changes.default: falsecoerces to*stringwithout error (err=<nil>), so a future non-string-typed input wouldn't break the loader.Green baseline confirms the lift is non-empty (6 inputs, 6 outputs) — the guards aren't masking a vacuous pass.
Every assertion axis proven non-vacuous — 4 mutations you didn't run
Your
dry_runrename proves only the input-names subcheck. I mutated the other four axes independently in an isolated copy (schema restored byte-identical after), each redding only its intended subcheck — narrowest-correct:dry_rundefaultfalse→trueinput_defaults: "dry_run default drift: reusable=false, schema=true"dry_runto schemainputs.requiredinput_required-ness: "schema required but NOT required in reusable-release.yml: [dry_run]"cut_tag→cut_tag_Xoutput_names: reds both directions (missing + over-specifies)draftfrompublish_mode.enum(keep default)InternalConsistency: "default 'draft' is not a member of its enum [immediate]"input_defaultsstayed green — enum-only axis isolatedThe last one is the important isolation: dropping the enum member without touching the default reds internal-consistency while leaving the defaults cross-check green, proving the two are genuinely independent axes and the enum-consistency check is real. Combined with your input-names rename, all five axes are load-bearing.
Docs reconciliation — accurate, and it fixes a real contradiction
The old prose ("A JSON-Schema form is not used here") directly contradicted the schema file that has existed since Phase 0b/#504 — a doc asserting the absence of a file in the same directory. The new prose correctly describes the schema as a normalized-lift companion, the gate, the schema-only refinements, and the non-goal. Good catch and a clean reconciliation.
doc.gocarries the package identity honestly (test-only, no runtime consumer).should-consider (doc accuracy, non-blocking) —
minLengthis claimed as validated but isn'tThe reconciled doc says "Enums and
minLengthare schema-only refinements … so the gate validates them for internal consistency instead." The gate validates enums (default∈enum, mutation-confirmed above) but notminLength—schemaPropertyparses only{Type, Enum, Default}, andInternalConsistencyasserts default-membership + required-has-property only.minLength(e.g.runs_on: minLength: 1) is neither cross-checked nor internal-consistency-checked. Concretely, adefaultshorter than its ownminLengthwould pass silently, though the sentence implies coverage. Two honest resolutions:minLengthis a schema-only refinement with no gate assertion; ordefaultmust satisfy its ownminLength— which would make the doc true and add a real (if minor) guard.I'd take (a) — it's the one-line honest fix. Flagging it precisely because keeping docs honest against the substrate is this PR's whole thesis, so a doc claiming coverage the gate doesn't provide is exactly the drift-class it exists to prevent. Not a blocker; the gate's actual behavior is correct.
Verdict
APPROVED, head-pinned at
c179ade. The gate's design is sound, the scope split is honestly documented, both vacuity assumptions are empirically verified (yaml.v3on-as-string + non-empty lift, with a fail-loud guard in the wrong-direction case), and all five assertion axes are mutation-confirmed non-vacuous — the four I ran cover the subchecks yourdry_runrename didn't reach. The docs reconciliation fixes a real contradiction. The one should-consider is a doc-precision fix onminLength, yours to take as (a) or (b). Full suite green, CI 8/8. Clean opener for Phase 7. Yours to land.— Surveyor