Accuracy
How Attest was measured, what each figure counts, the two caveats that go with them, and which number to build on.
Last updated 12 September 2026
Accuracy
Attest is measured on 190 hand-labelled examples built in-house. They cover facts spread across several sentences, arithmetic, near-miss paraphrases, quantifier and scope traps, unit prefixes and sources that conflict with each other: the cases a straightforward check gets wrong.
| Measure | Result | |---|---| | Two-way directional accuracy | 98.6% | | Verdict accuracy, all five verdicts | 95.9% | | Overclaim rate | 1.6% | | Grounding-degree accuracy | 96.2% | | Refutation accuracy | 97.9% | | Source-conflict accuracy | 98.9% |
What each figure counts
Two-way directional accuracy counts a verdict as right when it lands on the same side as the human label: SUPPORTED and PARTIALLY_SUPPORTED as backed, everything else as not backed. It is the headline figure on the developer site, and it is the looser of the two accuracy measures.
Verdict accuracy is the stricter exact match on all five verdicts, so a CONTRADICTED called DISPUTED counts against it.
Overclaim rate is how often Attest asserts more grounding than the evidence supports. It is the failure that costs you something, because it is the one that lets a wrong number reach a reader, and it is the number to build on.
Refutation accuracy is how often a claim the evidence contradicts is called CONTRADICTED. Source-conflict accuracy is how often two sources that disagree produce DISPUTED rather than a side being picked.
Two caveats
Run-to-run variation is about 3.7 points at temperature zero, because hosted inference is not bit-reproducible. Treat differences smaller than that as noise, and expect a printed example to come back with a neighbouring verdict now and then.
The figures come from one labelled set built by one author. They describe that distribution, not yours. The evidence you send, the length of your documents and the kind of claims you check all move the numbers, so measure on a sample of your own before you rely on a threshold.
How the measurement is run
Every example is sent through the same request a customer sends, with the explanation on, and the verdict is compared with the label. Confidence intervals, the label audits and every approach that was tried and lost are kept in the repository's measurement notes; ask Quincy for them if you need the detail.
Related articles
Fetch a page in one call
The same work as the extract call, with the request held open until the page is done and the Markdown returned directly.
Watch an extraction
Server-sent events for one job: its state now, every change after it, and the finished page. Closes when the job does.
Read an extraction's progress
One frame of a job's state without its Markdown, for a progress bar that should not carry a page's text with every poll.
Collect an extraction
Poll a job that answered 202. Returns its state while it runs and the Markdown once it is done.
Still need help?
Ask Quincy in the chat bubble below, or write to support@quorumtech.ch and we will help you directly.