CI / GitHub Actions

Run the same suite contract as local: load config, execute goals, fail the job on required failures, surface summaries for PR triage.

Current MVP shape:

  • Composite action under packages/testing-github-action — same run contract as local, with a step summary and sensible defaults (e.g. fork PR skip).
  • Validate in CI before run: validate --config … fails fast on unsupported YAML.
  • Artefacts: each run writes report.json, Markdown, junit.xml, and related files under --artifact-dir (or RESTORMEL_TESTING_ARTIFACT_DIR in the Action) — upload that directory in Actions for triage. Prefer one action step with suites: a,b,c (or CLI --suites) so each suite gets a subfolder under the same base dir; alternatively use a distinct artefact directory per step.
  • Keys — deterministic goals (URL + DOM only) do not need RESTORMEL_KEYS_*. See Keys in CI (checklist) before adding judge_rubric.
  • report.json includes a reproduction object with the exact local rerun command when a goal fails.

Deep dive: Walkthrough — Phase 5 — CI. Sample workflow: examples/testing-github-actions/sample-workflow.yml. Consuming from another repo: docs/testing/oss-consumption.md. CI security: Fork PRs and workflow triggers. Plotbudget-class adoption notes: docs/testing/plotbudget-testing-adoption-feedback.md.