News · 2026-11-21
METR reports faster growth in AI task horizons, with important measurement limits
METR estimates that the length of tasks AI agents can complete at a 50% success rate has doubled roughly every 3.6 months on its combined evaluation suites. The finding points to rapid measured progress, but the organization warns that estimated human completion times and the nature of the tasks limit conclusions about real-world autonomy.
Key facts
- METR published its capability update November 20, 2026.
- Its combined-suite trend is approximately a 3.6-month doubling in the 50% task horizon.
- Direct successful human baselines feed only 9% of its human-time estimates.
- Primary source: METR’s capability update and appendices.
A task horizon is not a stopwatch measuring how long an AI can be left alone. METR relates the probability of success to how long a human would need for the task. A 50% horizon summarizes the point at which a model succeeds on half of tasks at that human-time length. As an analogy, a runner’s median race distance in a test series does not promise that every route of that length will be completed, particularly if the terrain changes.
METR combines an older collection with newer tasks that cover optimization, software, games and research judgment. It uses agent setups with tools such as a shell and Python. The organization says models have improved enough that older tasks increasingly saturate, prompting harder tests and changes to the human baseline. It describes the evaluations as “getting harder to measure.” The time-horizon lesson explains the measure’s interpretation.
The headline trend is faster than the roughly seven-month doubling described in METR’s earlier work. But the newer tasks are harder, and the numbers are estimated across a changing mix. A trend across suites is evidence of improvement in the measured setting, not an exact date when an agent will master a particular occupation.
Human comparison is especially difficult. METR recruited more experienced baseliners and allowed them to use an AI model for implementation, yet direct successful baselines inform only 9% of the human-time estimates. For the rest it uses expert judgment, public records and extrapolation, and discusses sensitivity to those choices. If the human-time estimates shift, the apparent horizon can shift too.
METR also flags the possibility that easily checked, containerized tasks increasingly resemble environments used to train AI systems. Performance there may therefore exaggerate transfer to messier problems. On tasks requiring judgment without quick objective feedback, the report finds weaknesses relative to human experts. In a game-learning test, the best human improved from nine to all 21 objectives across six playthroughs, while a leading model improved much less.
The organization also observed that more expensive evaluation tasks tended to be harder: doubling estimated task lifecycle cost corresponded to roughly a 13.8 percentage-point lower success rate in its data. That relationship is observational and should not be treated as a universal law. For organizations deciding where to deploy agents, the lesson is to test the specific task, feedback and oversight conditions rather than convert an aggregate horizon directly into a staffing forecast. METR’s own report is the source for both the trend and its qualifications.
The new suite includes tasks with relatively cheap, objective checks and tasks where evaluating quality requires judgment. Those categories can behave differently. Optimization and programming tasks can sometimes be scored automatically, while an open-ended research proposal or investigation may require an expert rubric and may not have a single clean answer. An average across them cannot tell a manager whether a particular agent can handle a specific ambiguous project.
METR’s human baselines are also not a timeless standard. The organization changed who performed the comparison and allowed an AI assistant for implementation. Those choices can make the benchmark more relevant to present-day work, but they complicate simple comparisons with earlier studies using different people and tools. Researchers estimate missing times rather than pretending that those baselines were directly observed.
The report observes faster apparent improvement on a subset of verifiable newer tasks than on the combined collection, while explaining that more than one interpretation is possible. That is reason to examine the data as it develops rather than assume a single smooth growth rate will continue indefinitely. The next useful evidence will include more direct human measurements, sensitivity checks and performance on tasks where the right answer is not immediately visible.
Key questions
What does METR mean by a 50% task horizon?
Does a longer horizon mean a reliable autonomous worker?
How many time estimates use direct human baselines?
Cite this
APA
Ground Truth. (2026, November 21). METR reports faster growth in AI task horizons, with important measurement limits. Ground Truth. https://groundtruth.day/news/metr-task-horizons-three-point-six-months.html
BibTeX
@misc{groundtruth:metr-task-horizons-three-point-six-months,
title = {METR reports faster growth in AI task horizons, with important measurement limits},
author = {{Ground Truth}},
year = {2026},
month = {nov},
url = {https://groundtruth.day/news/metr-task-horizons-three-point-six-months.html}
}
Comments are replies to this story on Bluesky — reply with any Bluesky account to join in.