News · 2026-11-06

METR estimates faster Anthropic research progress using company-supplied data

METR estimates that Anthropic’s measured benchmark progress accelerated to roughly twice its earlier rate, using internal data supplied by the company. The November 5 assessment names an unreleased “Model 4,” but METR did not query that model and its public report does not contain enough detail to independently reproduce the internal scores.

Key facts

• METR’s central benchmark-trend estimate is 1.9 times the prior rate.
• Its reported 95% interval for that trend is 1.5 to 2.8 times.
• Anthropic supplied internal records; METR did not directly access the unreleased model.
• Primary source: METR’s assessment.

The assessment examines whether AI is accelerating AI research and development at Anthropic. METR’s engagement drew on selected internal documents, data, interviews and surveys. The organization estimates that a majority of the current acceleration may be attributable to AI use, perhaps around 80%, while stressing considerable uncertainty in that causal attribution.

The reported trend and its proposed cause are different claims. A stopwatch can show that a team finishes a race faster without revealing exactly how much of the improvement came from shoes, coaching or a new course. Similarly, a faster benchmark trend does not directly measure the fraction caused by AI assistants. METR says AI assistance, rather than fully autonomous AI research, is the principal current channel.

The report lists an internally named Model 4 and gives it an index score of 178.6. It also says the model scored above 50% on each of 15 index benchmarks used for it. That is not a score on every possible capability test, and 178.6 is not the task-completion time horizon on METR’s separate public page. The report does not establish the model’s public product identity.

METR describes its statistical method and several sensitivity checks. Those checks matter: assumptions about task difficulty can change the acceleration estimate. The company withheld specific benchmark names and other material, and METR’s own checklist says the report lacks sufficient methodological detail for independent review. Its interval also does not capture every source of uncertainty, including selection of models in the trend.

Anthropic’s own account says AI is helping its work but does not claim full recursive self-improvement. A circulating comparison assigning Model 4 a specific percentage of an unrelated model-generation jump is not established by the METR report. The useful result is an independent organization’s bounded analysis of confidential evidence, accompanied by serious transparency constraints—not a public leaderboard measurement or a forecast of a particular breakthrough date. See measuring AI by task length for the distinct time-horizon concept.

METR's engagement lasted about eleven weeks and used interviews, surveys and selected internal evidence alongside quantitative analysis. The analysts distinguish an estimate of overall progress from an estimate of AI's contribution to it. That distinction matters because an organization can change its research process, staffing, evaluation portfolio or use of compute at the same time it adopts more AI assistance.

For the benchmark trend, METR fitted model snapshots to an index and tested alternative assumptions. The report describes changes to the link between benchmark difficulty and the latent capability scale, as well as different benchmark subsets and regularization choices. Some alternative assumptions produce materially different acceleration estimates. These checks show that the authors examined sensitivity; they also show why the central number should not be treated as exact.

The report says Model 4 finished training in September to October, but does not identify a public product corresponding to that internal codename. Its score of 178.6 is on Anthropic's version of a capabilities index. METR says the model passed the 50% mark on each of 15 benchmarks used for that model, not that it completed every task or achieved the same result on an independently selected universal test set.

The public record contains genuine scrutiny as well as constraints. METR says it retained control of its methods and conclusions and received no payment for the assessment, while Anthropic had unilateral redaction authority over non-public information. The report says a majority of the evidence METR found most informative was redacted. Those facts do not prove bias or invalidate the result; they do prevent readers from inspecting much of the evidence needed to evaluate competing explanations.

The organization also cautions that its index may overstate or understate changes in practical capability. An index aggregates measurements under a model of task difficulty; it is a useful lens, not a direct measurement of every research project.

Primary source, verified: read the paper →

Ground Truth. (2026, November 6). METR estimates faster Anthropic research progress using company-supplied data. Ground Truth. https://groundtruth.day/news/metr-anthropic-research-acceleration-assessment.html

@misc{groundtruth:metr-anthropic-research-acceleration-assessment,
  title  = {METR estimates faster Anthropic research progress using company-supplied data},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/metr-anthropic-research-acceleration-assessment.html}
}

Topics: research · evaluation · industry