News · 2026-09-13

VulnCheck says Anthropic's Glasswing bug ledger shows 202 fixes from 26,153 claimed findings, and its numbers don't reconcile

Anthropic's public record of its AI-found software vulnerabilities shows only a small fraction of the findings the company claims, and its own figures disagree with each other, according to an analysis published on 8 September 2026 by Patrick Garrity of the vulnerability intelligence firm VulnCheck. Of 26,153 findings Anthropic claims for Project Glasswing, 2,736 appear in its disclosure ledger and 202 are marked fixed.

Key facts

• Anchor number: “Anthropic claims to have discovered 26,153 findings. Of those, only 2,736 (10.5%) have reached its disclosure ledger,” and 202, under 1%, are marked fixed.
• Severity gap: Claude rated 91.5% of findings critical or high; maintainers rated 51.3% that way.
• When: published 8 September 2026, after Anthropic backfilled its ledger the previous week; SecurityWeek highlighted it on 11 September.
• Primary sources: VulnCheck's analysis and Anthropic's disclosure ledger.

The promise being audited

Anthropic launched Project Glasswing in April 2026 to point its most capable, restricted Claude models at real software and find security holes before attackers do. It has been one of the company's main arguments that frontier AI can strengthen defence, and in August Anthropic said partners had used its restricted model to find more than ten thousand high- or critical-severity vulnerabilities.

Big numbers in vulnerability research need receipts, because a “finding” is only valuable once a maintainer confirms it, fixes it and users can patch. Anthropic published a vulnerability disclosure ledger on 22 May to provide that record. Garrity, who has tracked the program since launch, went through it after the latest update.

What the ledger shows

The gap between claims and records is large. Beyond the 202 fixed findings, Garrity counts 245 withdrawn and 2 duplicates, meaning the ledger holds more withdrawn findings than fixed ones. About 2,096 are marked as reported to maintainers but not confirmed fixed, and 191 appear in the ledger without having been reported.

The 202 fixes span 113 projects, under two per project. And the documents disagree internally: Anthropic's dashboard, Garrity writes, “states 421 findings patched upstream,” while the ledger itself shows 202 fixed, and its separate ledgers of CVE identifiers do not match either, one listing 70 and the main ledger 82.

His summary: “The receipts are starting to trickle in, they just don't reconcile.”

The severity problem

The second finding may matter more to open-source maintainers. Where both parties rated a bug, “Claude determined a critical or high severity for 91.5% of findings, while the maintainer determined only 51.3% as critical or high.”

Picture a smoke alarm that goes off at full volume for burnt toast as often as for a real fire. It may never miss a fire, but a household that hears it constantly stops treating every alarm as urgent. Maintainers of volunteer-run projects already struggle with floods of automated reports, and an AI that labels nearly everything urgent adds to the load even when its findings are real. The problem echoes our lesson on calibration, whether a system's confidence means anything.

Garrity also questions Anthropic's stated true-positive rate of 91.4%, noting the ledger contains more withdrawn or duplicate entries than fixed vulnerabilities. Separately, Ground Truth has reported on AI tools finding real bugs frontier scanners missed, such as six curl vulnerabilities found by AISLE, so the question is not whether AI finds bugs but how accurately programs report the results.

Why it matters

AI-driven vulnerability discovery is becoming a central claim in the policy fight over frontier models: labs cite defensive wins to justify building more capable systems. Public, checkable ledgers are the right idea. VulnCheck's analysis shows that the numbers in them need to add up before they can carry that weight.

The caveat

Disclosure is slow by design. Maintainers take weeks or months to confirm and patch, embargoes delay publication, and a five-month-old program would naturally show a long tail of unresolved reports. The severity comparison covers only findings where both sides gave a rating, and VulnCheck, which sells vulnerability data, has a professional interest in scrutinising other disclosure sources. VulnCheck's post does not include a response from Anthropic.

Primary source, verified: read the paper →

Ground Truth. (2026, September 13). VulnCheck says Anthropic's Glasswing bug ledger shows 202 fixes from 26,153 claimed findings, and its numbers don't reconcile. Ground Truth. https://groundtruth.day/news/vulncheck-says-anthropics-glasswing-ledger-shows-202-fixes-from-26153-findings.html

@misc{groundtruth:vulncheck-says-anthropics-glasswing-ledger-shows-202-fixes-from-26153-findings,
  title  = {VulnCheck says Anthropic's Glasswing bug ledger shows 202 fixes from 26,153 claimed findings, and its numbers don't reconcile},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/vulncheck-says-anthropics-glasswing-ledger-shows-202-fixes-from-26153-findings.html}
}

Topics: cybersecurity · vulnerabilities · ai-security · anthropic · vulnerability-disclosure · open-source