Bug reproduction harness
Thrice
The verdict comes from the state, never from the story.
A harness that decides whether a reported UI bug is still real. It forks a cloud sandbox from a snapshot of the app at a pinned version, seeds backend state directly, runs the reported steps three times in parallel browsers, and checks concrete predicates against that state. There is no model in the run loop: plans are authored by hand, validated against a schema and executed deterministically, so the verdict comes from the state rather than from anybody’s account of what happened.
Method
A plan is a JSON document: the seed payloads, the ordered steps, and two predicates. The first describes the buggy behaviour, the second describes correct behaviour, and a run counts as reproduced only if the first holds and the second fails. Both are evaluated every time, because a predicate that only ever gets checked when it is expected to pass is not a test.
Ground truth is the commit that fixed each issue. Run against the last buggy release the answer should be reproduced; run against the first fixed release it should not. Fixes were mapped to releases by resolving the submodule commit pinned at each release tag and testing ancestry rather than by comparing dates, which caught two releases shipping an identical interface commit and made that pair unusable.
What it got wrong
Two of the five conclusive entries scored incorrect, and both were caused by the harness rather than by the application under test. A false positive came from the predicate evaluator pinning the first match, so two different predicates silently answered a different question on any page where several elements matched. A false negative came from the mechanism meant to keep one plan valid across a release pair: it waits for a control that only shipped in the later release, so the mitigation for version divergence was itself version dependent.
Every attempt is published, including those two and the one that never reached a verdict, each with per-step timings, assertion screenshots and a recording of all three runs. No entry was removed from the corpus after its result was seen.
What did not work
Sixty-three runs across twenty-one attempts produced zero divergence and zero flaky verdicts. The three-run mechanism the project is named after is unvalidated by this corpus: it cost three times the browser seconds and bought no information on any entry.
The honest qualification is that the corpus was selected for small deterministic seeds, which is close to selecting for the absence of flakiness, so this is weak evidence in either direction rather than a result. It is recorded because the alternative is a harness whose central claim has never been put at risk.