Keys in CI (checklist)

When you need Restormel Keys in GitHub Actions—and when you do not. Deterministic browser goals (URL + DOM) do not require Keys or RESTORMEL_KEYS_* env.

Plotbudget-style MVP: Suites that only use URL checks, data-testid / DOM signals, and structured_checks with supported selectors need no Keys HTTP client in CI. You do not need RESTORMEL_KEYS_* in the workflow or app for those goals. For judge_rubric and Keys-backed resolve, you can store encrypted provider keys in the Keys dashboard (Connections) and copy RESTORMEL_PROJECT_ID from Restormel Testing—see Keys + Testing onboarding.

You do not need Keys env when

  • Every goal is type: browser with success criteria that do not include judge_rubric.
  • CI only runs testing validate / testing run (or the composite Action) with those suites.

You do need Keys env when

  • Any goal uses judge_rubric and you want model execution through Keys (recommended long term).
  • You rely on logical model refs in YAML that resolve via the Keys HTTP API (see config reference).

Typical GitHub Actions env (never log values; use secrets). RESTORMEL_GATEWAY_KEY is a Gateway key (rk_…), not a provider secret. Name your repository secrets however you like; map them to these runtime names. Compatibility: RESTORMEL_KEYS_API_BASE_URL / RESTORMEL_KEYS_API_TOKEN work the same if your workflow already uses them.

RESTORMEL_KEYS_BASE: ${{ secrets.RESTORMEL_KEYS_BASE }}
RESTORMEL_GATEWAY_KEY: ${{ secrets.RESTORMEL_GATEWAY_KEY }}
RESTORMEL_PROJECT_ID: ${{ secrets.RESTORMEL_PROJECT_ID }}

Optional escape hatch (team policy only): RESTORMEL_TESTING_OPENAI_FALLBACK=1 and OPENAI_API_KEY. Prefer Keys for alignment with production.

Forbidden patterns

  • Do not put raw provider API keys or gateway tokens inside restormel-testing.yaml.
  • Do not print RESTORMEL_KEYS_* or token values in CI logs or step summaries.
  • On fork PR workflows, assume secrets are absent unless you have explicitly opted into a safe public-URL path; see Fork PRs and workflow triggers.

Canonical references

CLI doctor: run testing doctor to check Node, Playwright, Keys env hints, and (when URL + token are set) a single resolve probe (HTTP status only). It also flags a missing RESTORMEL_PROJECT_ID when RESTORMEL_KEYS_BASE (or RESTORMEL_KEYS_API_BASE_URL) is set.