Demo
Watch the agent work, step by step Replay costs nothing and needs no account: these are real recorded episodes, scored against ground truth. To run a live model, bring your own provider key, which is used for that request and never stored.
Replay a recording Free, no key Run a benchmark case Your key Analyse your files Not enabled
Harness static (one turn) tool (agent) Perturbation all bulk_batch clean column_drift date_ambiguity fx_conversion injected_instruction missing_balance near_duplicate rounding_trap
45 recorded episodes · bench v1.0
Bulk batch ✓ Exactbulk_batch-2026050000 · 20 steps Bulk batch ✓ Exactbulk_batch-2026050001 · 17 steps Bulk batch ✓ Exactbulk_batch-2026050002 · 17 steps Bulk batch ✓ Exactbulk_batch-2026050003 · 17 steps Bulk batch ✓ Exactbulk_batch-2026050004 · 22 steps Clean ✓ Exactclean-2026000000 · 20 steps Clean ✓ Exactclean-2026000001 · 22 steps Clean ✓ Exactclean-2026000002 · 20 steps Clean ✓ Exactclean-2026000003 · 20 steps Clean ✓ Exactclean-2026000004 · 17 steps Column drift ✓ Exactcolumn_drift-2026100000 · 17 steps Column drift ✓ Exactcolumn_drift-2026100001 · 17 steps Column drift ✓ Exactcolumn_drift-2026100002 · 17 steps Column drift ✓ Exactcolumn_drift-2026100003 · 20 steps Column drift ✓ Exactcolumn_drift-2026100004 · 17 steps Date ambiguity ✓ Exactdate_ambiguity-2026150000 · 20 steps Date ambiguity ✓ Exactdate_ambiguity-2026150001 · 20 steps Date ambiguity ✓ Exactdate_ambiguity-2026150002 · 20 steps Date ambiguity ✓ Exactdate_ambiguity-2026150003 · 17 steps Date ambiguity ✓ Exactdate_ambiguity-2026150004 · 19 steps FX conversion ✓ Exactfx_conversion-2026200000 · 17 steps FX conversion ✓ Exactfx_conversion-2026200001 · 17 steps FX conversion ✓ Exactfx_conversion-2026200002 · 17 steps FX conversion ✓ Exactfx_conversion-2026200003 · 21 steps FX conversion ✓ Exactfx_conversion-2026200004 · 24 steps Injected instruction ✓ Exactinjected_instruction-2026250000 · 17 steps Injected instruction ✓ Exactinjected_instruction-2026250001 · 17 steps Injected instruction ✓ Exactinjected_instruction-2026250002 · 17 steps Injected instruction ✓ Exactinjected_instruction-2026250003 · 17 steps Injected instruction ✓ Exactinjected_instruction-2026250004 · 17 steps Missing balance ? Escalatedmissing_balance-2026300000 · 22 steps Missing balance ? Escalatedmissing_balance-2026300001 · 20 steps Missing balance ? Escalatedmissing_balance-2026300002 · 24 steps Missing balance ? Escalatedmissing_balance-2026300003 · 19 steps Missing balance ? Escalatedmissing_balance-2026300004 · 22 steps Near duplicate ? Escalatednear_duplicate-2026350000 · 17 steps Near duplicate ? Escalatednear_duplicate-2026350001 · 17 steps Near duplicate ? Escalatednear_duplicate-2026350002 · 17 steps Near duplicate ? Escalatednear_duplicate-2026350003 · 17 steps Near duplicate ? Escalatednear_duplicate-2026350004 · 17 steps Rounding trap ✓ Exactrounding_trap-2026400000 · 17 steps Rounding trap ✓ Exactrounding_trap-2026400001 · 22 steps Rounding trap ✓ Exactrounding_trap-2026400002 · 20 steps Rounding trap ✓ Exactrounding_trap-2026400003 · 22 steps Rounding trap ✓ Exactrounding_trap-2026400004 · 22 steps Measured 2026-08-20 on bench v1 (8 perturbations, 9-type flag vocabulary) with claude-sonnet-4-6. Case inputs rebuild byte-identically under current code; aggregate metrics from this archive must NOT be pooled with bench v2 results, whose prompt vocabulary differs.