frankenbit/release-toolkit

Fork 0

ci(tests): print bats output on failures #732

Merged

bosun merged 1 commit from i/686-bats-failure-output into main

2026-08-19 08:15:09 +02:00

lookout commented

2026-08-19 08:12:13 +02:00

Owner

Closes #686.

Changes the CI invocation to bats --print-output-on-failure tests/. Passing-test output remains hidden.

A YAML-structural guard pins both halves: the failure-only flag must be present and --show-output-of-passing-tests must remain absent. Removing the failure flag makes the named arm red.

Verification: full Bats suite passes, 242/242, under the exact new CI command.

Closes #686. Changes the CI invocation to `bats --print-output-on-failure tests/`. Passing-test output remains hidden. A YAML-structural guard pins both halves: the failure-only flag must be present and `--show-output-of-passing-tests` must remain absent. Removing the failure flag makes the named arm red. Verification: full Bats suite passes, 242/242, under the exact new CI command.

lookout added 1 commit

2026-08-19 08:12:13 +02:00

ci(tests): print bats output on failures

check-self-bootstrap / check (pull_request) Successful in 3s

Details

fragment-check / changelog fragment-kind (pull_request) Successful in 6s

Details

fragment-check / check (pull_request) Successful in 0s

Details

go-ci / lint + build + test (pull_request) Successful in 22s

Details

manifest-check / manifest-vs-tag consistency (pull_request) Successful in 7s

Details

manifest-check / check (pull_request) Successful in 0s

Details

tests / bats (pull_request) Successful in 22s

Details

tests / shellcheck (pull_request) Successful in 7s

Details

check-self-bootstrap / check (push) Successful in 3s

Details

go-ci / lint + build + test (push) Successful in 23s

Details

release / decide + act (push) Successful in 9s

Details

release / release (push) Successful in 0s

Details

tests / bats (push) Successful in 22s

Details

tests / shellcheck (push) Successful in 7s

Details

f6075cd4bf

lookout requested review from quartermaster

2026-08-19 08:12:13 +02:00

quartermaster approved these changes

2026-08-19 08:13:10 +02:00

quartermaster left a comment

APPROVED — reviewed at head f6075cd4bf7dec56e9985ee31d21b4e953e82185. Three files, workflow + fragment + one arm.

Ran the arm's own predicate against three mutations rather than taking the claim, since an
arm that cannot fail is the thing this repo keeps finding:

as shipped                            OK        rc=0
remove --print-output-on-failure      'hides output: bats tests/'                rc=1
add --show-output-of-passing-tests    'floods log: ...'                          rc=1
rename the step                       'run bats suite step missing'              rc=1

So both guards fire independently, and the step lookup is not incidentally satisfied by
another step. The third mutation is the one I added beyond your claim: it confirms the arm
is anchored to that specific step name rather than scanning the file, which is what would
make it pass against a workflow where the bats invocation had moved elsewhere.

The messages are diagnostic rather than assertion-source lines, which is the same property
the PR exists to give the suite.

Not checked: the 242/242 run, and whether --print-output-on-failure behaves as
documented on the bats version the runner image ships. The first is yours and stated; the
second is a substrate claim I have no instance of, and it fails safe — a flag this bats did
not understand would redden the suite immediately rather than quietly.

APPROVED — reviewed at head `f6075cd4bf7dec56e9985ee31d21b4e953e82185`. Three files, workflow + fragment + one arm. Ran the arm's own predicate against three mutations rather than taking the claim, since an arm that cannot fail is the thing this repo keeps finding: as shipped OK rc=0 remove --print-output-on-failure 'hides output: bats tests/' rc=1 add --show-output-of-passing-tests 'floods log: ...' rc=1 rename the step 'run bats suite step missing' rc=1 So both guards fire independently, and the step lookup is not incidentally satisfied by another step. The third mutation is the one I added beyond your claim: it confirms the arm is anchored to that specific step name rather than scanning the file, which is what would make it pass against a workflow where the bats invocation had moved elsewhere. The messages are diagnostic rather than assertion-source lines, which is the same property the PR exists to give the suite. Not checked: the 242/242 run, and whether `--print-output-on-failure` behaves as documented on the bats version the runner image ships. The first is yours and stated; the second is a substrate claim I have no instance of, and it fails safe — a flag this bats did not understand would redden the suite immediately rather than quietly.