Run contract

This is the internal boundary between scenario setup and the engine’s runner / verify / record. SREForge is the engine (core/), so this is not a contract with an external framework — it’s the shape setup hands the engine each run. The example uses booklogr.

The shape setup emits

{
  "agent_context": {
    "services": {
      "prometheus":   "http://prometheus:9090",
      "alertmanager": "http://alertmanager:9093",
      "grafana":      "http://grafana:3000",
      "api":          "http://booklogr-api:5000"
    },
    "run_workspace": {
      "path":    "/work/run/<run-id>/booklogr",
      "service": "booklogr-api"
    },
    "workspace_path": "/workspace",
    "submit_command": "submit"
  },
  "scenario": {
    "id":             "latency-cache-stampede",
    "profile":        "incident",
    "run_id":         "<run-id>",
    "expected_alert": "BooklogrApiLatencyP99High",
    "alert_fired_at": "<ISO timestamp — set by the confirm-fire gate>"
  },
  "eval_only": {
    "alert_to_clear":          "BooklogrApiLatencyP99High",
    "max_clear_time_seconds":  600,
    "sustained_clear_seconds": 360
  }
}

The endpoints are the agent’s view — in-network DNS

agent_context.services are service-DNS URLs reachable on the deployment network, not host ports. The engine itself probes the same services from outside the network on their host-published ports — so the addresses differ:

Service	Agent (in-network)	Operator (host)
Prometheus	http://prometheus:9090	http://localhost:9090
Alertmanager	http://alertmanager:9093	http://localhost:9093
Grafana	http://grafana:3000	http://localhost:3002
booklogr API	http://booklogr-api:5000	http://localhost:5000

There is deliberately no loki (or any log-store) endpoint: this stack’s observability is Prometheus + Alertmanager + Grafana, and advertising a service that doesn’t exist would itself be a tell.

The three sections

agent_context — what the agent receives

Rendered into a neutral on-call page (not a curated brief). The run_workspace.path is the host-side substrate clone the engine operates on (CI replay, redeploy, cleanup); the agent sees its own mount at workspace_path (default /workspace) and edits the source in place, then runs submit_command when done. There is no fork_repo, no PAT, and no github.com in v1. The agent never merges or deploys.

The page does not name the firing alert

scenario.expected_alert is engine-internal metadata for the run record — it is not rendered into the agent’s page. The agent reads the firing alerts off Alertmanager itself, the way a real on-call would. Pre-digesting the alert would both spoon-feed the diagnosis and read like a benchmark prompt. (This matters for anyone wiring an agent in: do not expect the alert name/summary in the page — discover it from the alerting stack.)

scenario — run metadata

id identifies the target scenario (defaults to latency-cache-stampede). SCENARIO_ID propagation accepts values via environment variable (SCENARIO_ID=<id>) or trailing task argument (SCENARIO_ID=<id>) on scenario-aware verbs, threading the ID to smoke and arm scripts. Invalid scenario slugs (not matching /^[a-z0-9][a-z0-9-]*$/) are rejected before script execution.

profile is incident or patch. alert_fired_at is populated by the confirm-fire gate: setup injects the fault and confirms the alert fired before the run is handed to the agent; if it never fires, the run aborts or retries.

eval_only — scoring inputs, never exposed to the agent

The oracle’s inputs. sustained_clear_seconds enforces the sustained-clear check.

No diff expectations

There is deliberately no expected_diff_contains or expected_files_touched. Verification is behavioural; diffs are at most non-blocking hints.

How a fix is graded (incident profile)

submit → CI gate (build + existing tests)
       → green → auto-merge → CD-on-merge redeploy of run_workspace.service
       → mitigation oracle scores, under still-active fault:
           CI green + alert_to_clear clears and stays cleared (sustained_clear_seconds)
           + no new alerts + time-to-clear

Multi-signal and fully objective. The RCA channel (submit --rca) is reported alongside the record; the RCA LLM-judge arrives in a later version.

What a run records (outputs)

Each run produces an artifact set in runs/<runId>/:

• record.json: The immutable run-record.v1 (canonical snake_case schema tools/certify/schemas/run-record.v1.schema.json; camelCase RunRecord is merely the in-memory TS type).
• diff.patch: The patch applied by the agent.
• transcript.txt: The engine runner’s log.
• agent-transcript.json: The agent-provided conversation trace.
• rca.json / rca.txt: The agent’s root cause analysis, if provided.
• diagnosis.json: The diagnosis.v1 RCA-judge result, if the RCA judge has graded the run’s rca.txt against the scenario’s authored root-cause truth. Reported beside the verdict — its score never feeds record.json or the pass decision. Absent is a normal state (no RCA, judge not run, or judge unreachable). Written by tools/rca-judge/judge.mjs; never written into or read from record.json.

The engine also captures the record into persistent storage via a split (ADR-0026):

• Pruned record: The record.json stripped of trajectory.transcript and the raw payload (raw_text/raw_json) of agent_transcript, but retaining agent_transcript’s identity header (harness, model, provider, session, confinement, run_id, captured_at) plus a full_record_sha256 reference, is committed to use-cases/<uc>/scenarios/<id>/records/<runId>.json.
• Full record: The complete run state is archived in a content-addressed private store seam. Banking refuses a verdict-bearing record that does not carry a labelled agent_transcript header — one whose confinement is absent or outside host-open | host-sandboxed | in-box, or that has no agent_transcript at all because its driver dropped the handoff. A verdict whose measurement conditions are unknown is not quotable, so it is not banked. Records already in the store are grandfathered, as are records already tracked by git in this repo (the accepted legacy corpus); --allow-unlabelled is a deliberate operator override.
• Headroom campaign runs: tools/headroom campaign run-ids and their records land in these exact same existing paths (runs/<runId>/record.json and pruned copies); the campaign records nothing new.

The run-record.v1 schema fields include:

Field	Meaning
run_id, scenario_id, profile	Run + scenario identity
trigger	The firing alert that opened the run (source, alert_name, severity, labels, annotations, fired_at, signals)
trajectory	What the agent produced: agent_name, diff, submitted, duration_ms
agent_transcript	Harness-sourced transcript metadata (harness, model, provider, session, confinement: host-open | host-sandboxed | in-box, captured_at)
ci, deploy	CI-gate result and the redeploy result (null if not reached)
score	The oracle’s oracle_score (see below)
verdict	"passed" | "failed" | "rejected" | "aborted" | "error"
timings, started_at, finished_at	Per-phase timings and ISO bounds

The oracle_score is the graded result the verdict derives from:

{
  "oracle_id": "mitigation",
  "score": 0.93,
  "passed": true,
  "signals": [
    { "id": "ci_green",      "satisfied": true, "value": 1, "weight": 0.25, "detail": "…" },
    { "id": "alert_cleared", "satisfied": true, "value": 1, "weight": 0.35, "detail": "…" }
  ],
  "sub_scores": []
}

passed is score >= pass_threshold (0.85 for this scenario). For a compound oracle, sub_scores carries the per-phase scores (detect / diagnose / mitigate).

The patch profile variant

For profile: "patch", scenario carries a pinned base_commit and verify/ is a hidden test suite run against the agent’s patch. There is no live deployment, no alert, and no run_workspace deploy step.

Forward-compatibility

Additive-only within v1.x: new fields are fine; no rename or removal without a contract_version bump. The engine ignores unknown fields.

See also

• Scenario format — the authored side of this contract.
• Closed-loop verification — how the oracle uses eval_only.