fix: collapse breaking entrypoint invocations, tolerate oasdiff.yaml fail-on, test YAML support#112
Merged
reuvenharrison merged 5 commits intomainfrom May 10, 2026
Conversation
…early-exit; test oasdiff.yaml across all three actions The breaking action's entrypoint was running oasdiff three times per invocation: 1. default format → captured to $GITHUB_OUTPUT 2. --format githubactions → annotations on the PR 3. --fail-on → exit code Plus a bug: with `set -e`, the first invocation would abort the script if oasdiff exited non-zero — which happens when oasdiff.yaml sets fail-on. Result: action exits silently with no rendered report. Collapse runs 1+3 into a single invocation that captures the report *and* applies --fail-on, with exit-code tolerance via `|| exit_code=$?` (the same pattern already used in diff/entrypoint.sh). Run 2 stays as the annotations renderer, also tolerated. Final `exit $exit_code` preserves fail-on behaviour from either the input or oasdiff.yaml. Net: one fewer oasdiff invocation per run (~1-3s saved on real specs), and fail-on from oasdiff.yaml now produces the same nice rendered output as fail-on from the action input. changelog/entrypoint.sh and diff/entrypoint.sh already run oasdiff once and don't need this fix. Tests: four new jobs in .github/workflows/test.yaml verify oasdiff.yaml support end-to-end: - breaking: fail-on from YAML (also asserts the report still renders) - breaking: err-ignore from YAML - changelog: level from YAML - diff: exclude-elements from YAML
…iff.yaml) Same shape of bug as in breaking/entrypoint.sh: with `set -e`, the oasdiff invocation aborts the script if the customer has fail-on set in their oasdiff.yaml. For pr-comment specifically that means: customer upgrades to Pro, has fail-on: ERR in YAML, breaking change appears → script aborts before posting the PR comment. They paid for Pro and got nothing. Capture the exit code with `|| oasdiff_exit=$?`, then differentiate: - non-zero exit + empty output → real failure (parse error, missing file, etc.), abort with the original exit code - non-zero exit + non-empty output → fail-on triggered; we have the JSON we need; carry on and let the service decide the workflow's outcome Note: pr-comment doesn't need the 3→2 invocation collapse from the breaking/entrypoint.sh fix — it already runs oasdiff exactly once.
oasdiff v1.15.3 (just shipped) added .oasdiff.* as the preferred config filename, with oasdiff.* kept as a back-compat fallback. The Docker image referenced by these tests was bumped to v1.15.3 in #114, so the new lookup is now in effect. Update the four config-file test jobs to write the preferred filename so we canonicalise the recommended shape in our own workflow examples. Behaviour unchanged: both filenames continue to work; this just aligns the test fixtures with the documentation we'll point customers at. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new CI jobs run pr-comment/entrypoint.sh directly with a stubbed oasdiff on PATH, no Docker / token / service required: - pr_comment_tolerates_oasdiff_fail_on: stub exits 1 with valid JSON. Asserts the script reaches the no-token skip and exits 0. Without the fix, set -e would abort at the oasdiff invocation. - pr_comment_aborts_on_oasdiff_real_failure: stub exits 2 with no output. Asserts the script aborts with exit 2 and the explicit 'no output' error message. Plain shell + GitHub-env scaffolding; no new test framework. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> EOF
…nt-and-yaml-tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related fixes to entrypoint scripts plus tests for
.oasdiff.yamlconfig-file support across all three free actions and exit-code tolerance inpr-comment.What's wrong today
breaking/entrypoint.shrunsoasdiff breakingthree times per workflow run$GITHUB_OUTPUT--format githubactions→ annotations on the PR--fail-on→ exit codeThat's two more spec parses + rule walks than necessary on every PR.
set -e+fail-onfrom.oasdiff.yamlaborts the script before renderingWith
fail-on: ERRin the customer's.oasdiff.yamland a real breaking change in the spec, the very firstoasdiff breakinginvocation exits 1,set -eaborts the entrypoint, and the customer sees an action that exits 1 with no rendered report, no annotations, and no notice link. Reproducible: drop.oasdiff.yamlwithfail-on: ERRat the repo root, leavefail-onempty in the actionwith:block, push a PR with a breaking change.The same shape of bug exists in
pr-comment/entrypoint.sh: a Pro customer withfail-on: ERRin YAML loses the PR comment because the script aborts before posting.What this PR does
breaking/entrypoint.sh: collapses runs 1+3 into a single invocation that captures the report and applies--fail-on, with exit-code tolerance via|| exit_code=$?(mirrors the pattern already used indiff/entrypoint.sh). Run 2 (--format githubactions) stays as the annotations renderer, also tolerated. Finalexit $exit_codepreserves fail-on behaviour from either the input or.oasdiff.yaml. Result: 3 → 2 invocations; YAML fail-on now produces the same nicely-rendered output as input fail-on.pr-comment/entrypoint.sh: capturesoasdiff changelog's exit code and differentiates "fail-on triggered, we have the JSON" from "real failure, no output." Real failures still abort with the original exit code; fail-on-triggered exits proceed to post the PR comment, since pr-comment's exit code comes from the service response, not from oasdiff itself.changelog/entrypoint.shanddiff/entrypoint.shalready run oasdiff once and don't need this fix.Tests
End-to-end:
.oasdiff.yamlconfig-file pickupFour new jobs in
.github/workflows/test.yamlexercise.oasdiff.yamlend-to-end through each action's Docker entrypoint:oasdiff_breaking_yaml_config_fail_onfail-on: ERRoutputs.breaking(the entrypoint fix in this PR)oasdiff_breaking_yaml_config_err_ignoreerr-ignore: specs/err-ignore.txtoutputs.breaking == "No breaking changes"oasdiff_changelog_yaml_config_levellevel: ERRoasdiff_diff_yaml_config_exclude_elementsexclude-elements: [description, title, summary]+format: text"No changes"Each job creates
.oasdiff.yamlat the runner's repo root in a step before the action runs (separate runners, no cross-contamination). The legacyoasdiff.yamlfilename still works as a back-compat fallback in oasdiff v1.15.3+; the test fixtures use the preferred.oasdiff.yamlname to match what we document.Unit-level:
pr-commentexit-code toleranceTwo new jobs run
pr-comment/entrypoint.shdirectly with a stubbedoasdiffon PATH (no Docker, noOASDIFF_TOKEN, no reachable service required), to exercise the entrypoint's branching logic in isolation:pr_comment_tolerates_oasdiff_fail_on[]JSON output|| oasdiff_exit=$?tolerance.pr_comment_aborts_on_oasdiff_real_failureERROR: oasdiff exited 2 with no output, never reaches the notice line. Verifies the[ -z "$changelog" ]early-abort branch.These cover the pr-comment fix without needing a token or a running service. The downstream service call is intentionally not exercised; what matters is that the script does (or does not) reach it.
Dependency on oasdiff v1.15.3
The four
.oasdiff.yamljobs require oasdiff v1.15.3 (which added.oasdiff.*as the preferred config filename). The Dockerfile bump landed in #114; this branch carries that bump via merge frommain.Backward compatibility
outputs.breakingcontent shape is unchanged.oasdiff.yamlfilename keep working: oasdiff v1.15.3 retains the legacy lookup as a fallback.Test plan
.oasdiff.yamlend-to-end jobs.oasdiff.yamlwithfail-on: ERRproduces the rendered report alongside the failure