Skip to content

Data & API

Take the numbers and check them yourself

A benchmark you cannot audit is a press release. Every figure on this site is served as plain JSON at a stable URL. The whole dataset, cases, ground truth and recorded trajectories, downloads with a datasheet and a checksum manifest.

Read-only API

No keyNo rate limit

These are static files on a static host, not a service. No server here can be down, and there is nothing to sign up for. Same builder as the self-hosted API, so the two cannot disagree.

/v1/index.json

What exists here: models, harnesses, perturbations, bench version. Start here.

/v1/results.json

The scored aggregates, exactly as the scorer produced them. Rates carry their Wilson intervals and money stays a string.

/v1/runs.json

One scored summary per recorded episode.

/v1/runs/{case_id}--{harness_id}.json

A single episode with its full trajectory: every tool call, every result, the submission.

/v1/cases.json

The case list: ids, seeds, perturbations, expected flags.

/v1/cases/{case_id}.json

One case with each input file rendered as text and the perturbation's edits marked.

/v1/perturbations.json

The trap vocabulary of the current code, read from the live registry.

curl -s https://<this-site>/v1/results.json | jq '.results[] | select(.perturbation=="all")'

Dataset export

Built from the repository, so it is always reproducible rather than uploaded once and forgotten:

ledgerbench export-dataset --cases cases --archive data/recorded/bench-1.0
ledgerbench verify-dataset --dir dist/ledgerbench-dataset
  • datasheet.md says what is in it, how it was made, and bluntly what it does not support.
  • manifest.json lists every file with its SHA-256, so you can prove the bundle is intact and cite a specific build.
  • truth.json ships with every case. You cannot score your own agent without it. Training on it would invalidate your results.

Licence and citation

Data is CC BY 4.0; code is Apache-2.0. Cite the bench version: results measured under different versions use different flag vocabularies and must never be pooled.

What is published right now

Model claude-sonnet-4-6Harness static-v1Measured 2026-08-20Bench v1.0Cases 90Run 11eaa71b2d02Commit 3d38c30