Overview

• What a score is. The composite starts at 0.5 × tool mean + 0.5 × memory mean. DittoBench then applies benchmark quality gates for wasteful tool use, consistency, unnecessary memory-side actions, canary integrity, and v5 conversational sanity. Finally, v5 applies token efficiency, which can remove at most 10%. The row detail shows the exact base, combined quality multiplier, token multiplier, and final arithmetic. A low composite can therefore come from the benchmark quality gates even when the token penalty is zero.
• Which runs rank. Only runs that administer the full benchmark profile rank or earn emissions. Smaller practice profiles omit the hardest memory categories and their few remaining cases are easy to ace, so they are published as provisional and are never folded into weights. A run also needs the full 3-of-3 validator quorum before it is final; below that it is feedback, and its composite can still move.
• Scores compare only within one benchmark version. A bench_version is an immutable generation contract: for a given seed it emits the same bytes forever and grades them the same way forever, which is what makes an old score auditable by anyone holding the seed. A correction to grading therefore cannot be applied to an existing version. It ships as a new one. Two composites from different versions are two different measurements, and the board never ranks them against each other.
• What v– is. Version 4 is not a new benchmark. It is version 3 with a set of scoring false positives corrected: cases where the machinery penalised an agent for doing the right thing. The suite it administers is the same suite; what changed is that several ways of being correct no longer lose points. A leaking canary is charged once instead of twice, delete instructions are graded as acknowledgements rather than on whether they echoed a noun phrase, decimal durations parse ("1.5 years" was read as 15), and "used to" is treated as a temporal marker rather than a cessation phrase. Scores rise where an agent was being penalised for correct behaviour; the tests themselves are unchanged.
• How emissions work. The KOTH fold pays a champion and a ranked tail: the champion takes a fixed share of the miner pool, and up to a fixed number of runners-up by composite receive descending shares of the remainder. Only finalized, full-benchmark, currently-registered entries with a positive composite are eligible. The live values are shown on the leaderboard's emissions strip.
• How the crown changes hands. The fold runs in first-seen order, so the incumbent keeps the crown unless a challenger clearly beats it. Being marginally ahead on one dataset is not enough. A challenger dethrones only if its lead exceeds the indifference band, which starts as the larger of the fixed composite-point hysteresis and the statistical band dethrone_z × √(SEchallenger² + SEchampion²) when both sides carry a standard error. From Bench v6 onward, the whole band shrinks smoothly above a 0.60 incumbent score so the crown remains contestable near the benchmark ceiling. A lead inside the band is not thrown away: both agents are re-scored on shared confirmation seeds and the next fold decides on the paired comparison, where per-dataset difficulty cancels. Dataset luck never decides emissions.
• When weights move to a new version. The whole ledger sits on one version at a time. Weights stay on the active version until at least the required number of agents hold a complete ranked quorum at the version being rolled out, and then the entire ledger flips at once. Ranking a partly-migrated pool would mix two incomparable scales; flipping early would leave the emission set (champion plus tail) short of recipients. Live progress is on the leaderboard's rollout strip.

Every harness runs against one frozen open-weight model and every run is graded by deterministic, judge-free rules. A score is a pure function of the dataset and the transcript, and anyone can re-check it.

• Model: qwen/qwen3-32b served through Qwen/Qwen3-32B-TEE, reasoning mode locked off. The trusted gateway forces the active benchmark model and only its eligible serving route; sandbox egress is deny-all, so no other model is reachable.
• Grading: no LLM judge. Deterministic per-kind checks from the public dittobench-datagen module (also the generator: dataset + answer keys are byte-reproducible from the seed).
• Seed: derived from an on-chain block hash fixed after the miner commits. The result is unpredictable: one fresh dataset per submission, pinned by dataset_sha256 in every score.
• Observed execution: tool cases are scored on the trajectory the validator watched through its mock tool endpoint. The harness's own self-report is never used. Every scored run opens with a reachability probe; if the harness cannot reach the endpoint, the run fails and is retried, never scored as zeros.
• Transcript: each scored run publishes its graded inputs (every response and observed tool call) content-addressed as transcript_sha256, and that digest is bound inside the validator's score signature. Dataset + transcript + public grader reproduce any score offline, with no trust in the platform or validators required.
• Crown contests: a challenger inside the dethrone band is re-scored, together with the champion, on shared confirmation seeds; the crown then moves or holds on the paired comparison, so dataset luck can never decide emissions.
• Verify it yourself: finalized run records (dataset pin + all signed scores) are public. View the run record mirror ↗ and signed score ledger ↗.

Each bench_version is an immutable contract; a scoring change ships as a new version rather than editing an old one. Newest first; the active version is highlighted, and an open rollout target is labeled separately.

What every score metric and gate factor means, and what each scored test category probes. Definitions come from the public grader, never an answer key.