Phase 4 — Run your first suite locally

Execute the suite containing your first goal; confirm verdict output (passed / failed / indeterminate), reason codes, and where artefacts land on disk.

restormel-testing run --suite <your-suite-id> --config restormel-testing.yaml
# optional: --goal id1,id2  --json  --artifact-dir ./out/run-1
restormel-testing report .restormel-testing/runs/run-…

Default artefact root: .restormel-testing/runs/run-<timestamp>/ with report.json, summary.md, traces, and JUnit. A failed goal includes a reproduction command in report.json includes factual judgeInvocationCount when rubrics ran; costEstimate is a non-billing heuristic only.

Configuration · ← Phase 3 · Phase 5 — CI →

Agent prompts

Optional: use these with a coding agent to implement this phase in your repo in a safe, gated sequence. Expand only when you need them.

  1. Run the suite locally; paste the summary block into the PR or team notes.
  2. Force a failure (e.g. break success criteria); confirm artefact paths and reproduction command.
  3. Compare wall-clock time and flake rate over three consecutive runs; adjust retries only if justified.