Author an incident on a real, controlled deployment. Hand an agent a neutral on-call page. Grade it on whether its deployed fix actually resolves the incident — verified behaviourally, under the still-active fault.
SREForge stands up a real, lesser-known public app as a lived-in deployment,
authors an organic regression into its existing code so a genuine incident
surfaces as a real alert, hands an agent only a short trigger, and lets it
investigate through the same tools a human on-call would use. It grades the agent
on behavioural recovery — not on matching a reference diff. You cannot bluff a
behavioural oracle.
The fault stimulus keeps running while the fix is verified. An alert clears
only because the deployed change works — not because the harness stopped
poking the system. Diff-matching is at most a hint, never the grade.
Contamination-free by construction
v1 substrates are real apps with no public solution to memorise. Faults are
authored into existing code, and the run workspace’s git remote is a local
forge — so the fault can’t be diffed against the upstream.
Authored, reproducible incidents
A determinism gate confirms the incident has actually reproduced — the target
alert is firing — before the agent is ever handed the page.
Honest, neutral framing
The agent receives a plain page or ticket. No eval language, no production
theatre. It is never told it’s in a harness.