News · 2026-10-30

Trakkr’s citation audit traces only 40% of sampled AI statistics to matching studies

Trakkr reports that only 40% of weighted citation trails in its sampled AI-generated statistics reached a study supporting both the number and its context. Its October 28 “Zombie stats” study examined five assistants answering number-seeking questions through search-enabled APIs on one day. The finding concerns provenance in that sample, not the truth of every unresolved number or the reliability of all consumer chatbot answers.

Key facts

• Researchers asked five assistants 200 number-seeking questions on October 25.
• They sampled 1,285 cited statistics and hand-checked 150 source trails.
• In the weighted sample, 40% reached a matching study; 40% could not be checked.
• Primary source: Trakkr’s study and methodology.

The problem is easy to miss because a citation looks like a completed verification step. A reader sees a number, clicks a source, and finds the same number again. That feels like corroboration. But if the source is repeating another page, and that page is repeating a third, the chain may never reach a study that measured the claim.

Trakkr’s researchers followed those chains rather than stopping at the first plausible link. Their procedure traced sources up to five pages, combining a larger sampled set with manual checking. The test used web search through APIs with a United States location. Each question was asked once, so the design captures one set of responses rather than repeated behavior under many phrasings or times.

The study’s title, “Zombie stats,” describes the authors’ concern that numbers can keep circulating after their evidential basis has become detached. The figure may have started in a real survey but lost its population, date, or definition. It may then be cited by a newer article, which makes the same old result appear current. An assistant retrieving that article can continue the chain without inventing a number from scratch.

A useful analogy is a photocopy of a photocopy. The digits remain legible, but the footnote containing who was surveyed and when can disappear at the edge. Several websites displaying the same copy do not create several independent measurements. Likewise, agreement among assistants can reflect repeated retrieval of the same citation chain.

Trakkr separates several outcomes. In the weighted sample, 40% reached a study matching the number and context. Another 8% reached a study but with a changed number or context. A further 40% could not be checked, and 11% led nowhere. The reported categories are rounded, so readers should not force them into a perfectly summed reconstruction or treat them as exact population frequencies.

The distinctions prevent an attractive but incorrect headline. Failure to verify is not proof of falsehood. A source might be inaccessible, insufficiently described, or missing from the trail the researchers could follow. Conversely, a number appearing in a real study is not enough if the answer changes what that study measured. A statistic about a small sample of companies cannot silently become a statistic about all workers.

The dossier contains a useful comparison: Peec researcher Metehan Yesilyurt’s earlier account describes seeding StateGlobe with 50,000 invented statistics and observing AI retrieval and citation. That was a first-person experiment in introducing fabricated material into the information supply. Trakkr’s study instead asks whether real assistant answers lead back to supporting evidence. The mechanisms overlap, but the experiments are distinct and should not be counted as independent replications of one another.

The broader technical background is retrieval-augmented generation. Search can give a model access to fresh material, but freshness and retrievability do not guarantee evidential quality. A system can accurately reproduce a misleading source. The generation may be faithful to the retrieved page while the page itself no longer supports the implied claim.

That is also why hallucination is only part of the story. A completely invented figure and an accurately copied but miscontextualized figure require different remedies. Better next-word generation cannot restore a missing study definition. Source selection, provenance tracking, and preserving population and date all become part of answer quality.

The strongest caveat is sampling. This is an author-reported study of five assistants, 200 prompts, and one search-enabled API run per question on one date. It is not a universal error rate, and independent replication was not established. The dossier also does not provide a verified expert consensus about the result. Those limits restrict the claim without eliminating the practical value of tracing beyond the first citation.

For readers, the actionable question is whether a source actually measured the thing the answer says it measured. For builders, the implication is to evaluate citation support separately from fluent wording and apparent source count. Trakkr supplies a specific provenance audit and a reason to inspect the chain, with unresolved trails kept distinct from demonstrated false claims.

Primary source, verified: read the paper →

Ground Truth. (2026, October 30). Trakkr’s citation audit traces only 40% of sampled AI statistics to matching studies. Ground Truth. https://groundtruth.day/news/trakkr-zombie-stats-citation-trails.html

@misc{groundtruth:trakkr-zombie-stats-citation-trails,
  title  = {Trakkr’s citation audit traces only 40% of sampled AI statistics to matching studies},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/trakkr-zombie-stats-citation-trails.html}
}

Topics: citations · search · evaluation · misinformation · ai-literacy