News · 2026-09-01

Anthropic closed the hole distillers used to read Claude's thinking

Anthropic shipped a defensive change with Claude Fable 5.1 on September 1, 2026 that has nothing to do with capability: new API accounts can no longer edit earlier turns of a conversation while preserving the transcript of Claude's prior thinking. That combination was a publicly documented technique for harvesting a model's internal reasoning at industrial scale, and harvested reasoning is the raw material for building a cheap copy. The same release added a statistical watermark to Claude's text output and a detection API, in private preview, for checking it.

Key facts

• The restriction applies to API accounts created on or after September 1, 2026; existing accounts are unaffected for now but will be covered by future model releases.
• Anthropic describes industrial-scale distillation attacks as using "thousands of fake accounts."
• The text watermark covers models released after August 2, 2026, under the EU AI Act's Code of Practice on Transparency of AI-Generated Content, which Anthropic signed in July 2026 alongside 190 other signatories.
• Primary source: Anthropic's Fable 5.1 launch post.

Distillation, in its legitimate form, is one of the most useful techniques in machine learning: you train a small model to imitate a large one, and you get most of the capability at a fraction of the serving cost. Our explainer on distillation covers the mechanics. The trouble is that the technique works just as well when the large model belongs to someone else and you are paying retail for its outputs. Do it at enough volume and you have extracted the expensive part of a competitor's product through the front door.

Reasoning traces make this dramatically more efficient. A final answer tells a student model what to say. The intermediate thinking tells it how to get there -- which is the part that actually transfers. So the valuable extraction target is not Claude's answers, it is Claude's scratch work. Anthropic's API normally hides or invalidates that scratch work when you rewrite history in a conversation. The documented trick was to edit an earlier turn while keeping the prior thinking blocks intact, which turned an ordinary API into a firehose of labeled reasoning data.

Picture a chess grandmaster who will play you for a fee. You are welcome to record the moves. What you are not supposed to get is the grandmaster's running commentary about which lines were considered and rejected -- because that commentary, not the move list, is what would let you train a replacement. Anthropic just took the commentary away from anyone signing up today.

The timing is not subtle. The US government spent August alleging that Moonshot distilled Anthropic's Fable model, and model extraction has moved from an academic curiosity to a trade-policy argument. Anthropic's own framing is that this is a safety problem rather than only a commercial one: "Distillation is a safety risk, since the distilled capabilities can subsequently be released without adequate safeguards." That argument has real force given the structure of today's release -- Anthropic ships the same model at two safety levels, which means the safeguards are a separate layer that a distilled copy would simply not have. Steal the capability and you get the capability without the bouncer. Our explainer on model extraction attacks covers the wider threat model.

The rollout is deliberately gentle, and that tells you something about how load-bearing the technique was for legitimate users too. Existing accounts keep working. Only accounts created from today forward hit the restriction, with the rule extending to everyone at some future model release. Anthropic acknowledges a small number of customers' custom integrations will break, and points them at a help-center article. Shipping a security fix with a grandfather clause is an admission that the hole was also a load-bearing feature for some honest workflows.

The second half of the release is provenance rather than protection. To comply with the EU AI Act's transparency code of practice, which Anthropic signed in July 2026 along with 190 other signatories, all Claude models released after August 2, 2026 now carry a watermark -- a statistical signal in the token choices that indicates the text likely came from Claude. Anthropic says it is invisible without the detection API, carries no information about the user or their conversation, and has no practical effect on output quality. The detection API is in private preview for regulators, law enforcement, media, fact-checkers, independent researchers, educational organizations, EU civil society groups, and enterprises with their own compliance obligations. Our explainer on content provenance and watermarking covers how these schemes work and where they fail.

The honest caveat cuts both ways. On distillation, closing one documented path is not the same as closing the problem -- a determined extractor can still buy outputs at volume, and the final answers alone remain a workable if less efficient training signal. On watermarking, the well-established weakness of statistical text watermarks is that light paraphrasing degrades them, and a detector that only Anthropic and a short list of approved organizations can run is not something the public can audit or independently evaluate. Both changes are real improvements. Neither is a solution, and Anthropic does not claim otherwise.

Primary source, verified: read the paper →

Ground Truth. (2026, September 1). Anthropic closed the hole distillers used to read Claude's thinking. Ground Truth. https://groundtruth.day/news/anthropic-closed-the-hole-distillers-used-to-read-claudes-thinking.html

@misc{groundtruth:anthropic-closed-the-hole-distillers-used-to-read-claudes-thinking,
  title  = {Anthropic closed the hole distillers used to read Claude's thinking},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/anthropic-closed-the-hole-distillers-used-to-read-claudes-thinking.html}
}

Topics: cybersecurity · ai-security · anthropic · distillation · supply-chain · model-extraction · content-provenance · eu-ai-act