News · 2026-11-07

Anthropic describes a layered review system for agent-written code

Anthropic says merged pull requests in its Claude Code repository more than doubled from June to September while the share later reverted fell by more than 40%. The company describes a layered review system for agent-written changes, but the reported comparison does not establish that the review changes caused the improvement.

Key facts

• Anthropic’s engineering post describes risk-based review and staged release.
• The company reports more than doubled merged pull requests in one repository.
• The share reverted fell by more than 40% in its comparison; absolute rates are not published.
• An incident-rate before-and-after comparison is not reported.

The system resembles an inspection line with multiple checkpoints. Author-side agents run tests and gather evidence. A fresh review looks for bugs; another group evaluates findings. A separate security pass, ordinary automated checks, and a merge queue add further gates. Changes judged more consequential require human approvals, while some low-risk changes can pass without one under the internal process.

Anthropic describes “No increase in reverts or incidents” as a guardrail. It should not be mistaken for a published before-and-after incident result. A separate source-control post reports a rise in daily changes in a broader monorepo; that is a different measure from pull requests in the Claude Code repository.

One difficulty was getting automated reviewers to stop. Anthropic says a pull request accumulated more than 50 review-and-fix rounds overnight, and that about 59% of repeatedly flagged code was already present at the first pass. It responded by bounding review rounds and narrowing later reviews. An endless inspector can slow delivery even when some comments are useful.

Other companies describe related tradeoffs. DoorDash emphasizes useful findings and acknowledges evaluation limits; Cloudflare describes multiple specialist reviewers and remaining architectural blind spots. Practitioner Paddo argues that using the same model family for multiple reviews can create correlated misses, based on a small self-reported comparison.

The strongest case for the approach is that more checking can happen before a human is interrupted, with explicit recovery mechanisms after release. The strongest caveat is that comment volume, revert share and raw change counts are imperfect measures of correctness. Without absolute rates, independent controls and incident outcomes, the figures cannot prove the system is safer or that it would transfer to another team.

The multi-stage review pipeline has two jobs. First it tries to catch defects before merging by combining tests, bug review and security checks. Second it limits damage after merging through staged rollout, observability, flags and rollback. A change can pass all pre-merge checks and still reveal a failure only in production. The two halves therefore address different failure modes.

The company says classifiers help determine which changes require human approval. That decision is itself a model or policy decision with potential error. A change labeled low risk might still have an unexpected effect; a high-risk classification can demand time from scarce reviewers. Sensitive paths and untrusted contributions receive additional controls in the process Anthropic describes. The internal rule allowing some automated-only low-risk approvals should not be confused with a claim that the public review product supplies human approvals.

Different metrics also describe different events. Anthropic says the share of pull requests receiving substantive comments rose from 16% to 54% after an internal preview. That is a measure of comment coverage, not the percentage of bugs found or prevented. An increased comment rate can mean better coverage, more noise, or both. Similarly, a lower reverted share is informative but does not capture errors fixed without a revert, vulnerabilities discovered later, or architectural choices that make future work harder.

The most useful outside comparisons are operational rather than numerical. DoorDash describes looking for findings engineers will act on and says its incident-based evaluation set remains small. Cloudflare describes a coordinator, specialist reviews and time limits. Google Cloud describes integration problems even when separate changes have been approved. Those reports support the importance of convergence and human attention, but the systems and populations differ too much to rank them from their published counts.

A team considering the same approach should establish a baseline for its own codebase and measure outcomes over time, including issues found after release.

Primary source, verified: read the paper →

Ground Truth. (2026, November 7). Anthropic describes a layered review system for agent-written code. Ground Truth. https://groundtruth.day/news/anthropic-agent-code-review-revert-rate.html

@misc{groundtruth:anthropic-agent-code-review-revert-rate,
  title  = {Anthropic describes a layered review system for agent-written code},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/anthropic-agent-code-review-revert-rate.html}
}

Topics: coding · agents · software-engineering