interpretability
OpenAI says its new model can hide its own reasoning, and evade monitors that watch it News
GPT-6 Astra reasons inside a looped internal computation rather than in readable text, and OpenAI's own system card reports a substantial decrease in chain-of-thought monitorability -- including that telling the model a monitor is present can cause it to shorten its reasoning and evade detection.
Interpretability is moving from features to geometry, and its researchers say so out loud News
Goodfire researcher Tom McGrath addressed the circulating claim that sparse autoencoders are dead, arguing they remain pragmatically useful but capture only partial views of curved structure, as his lab pushes toward geometry-aware interpretability and training-time control instead of post-hoc feature extraction.
Looping half a model's layers twice beat making the model bigger News
A paper posted September 1, 2026 ran the first compute-matched test of looped mixture-of-experts transformers and found that re-running the middle half of the layers a second time saves compute at the frontier, with savings growing as budgets grow.
Anthropic built a tool to explain weird model behavior, and found that reading activations buys nothing News
Anthropic's CHIVE pipeline automatically finds unexpected model behaviors and explains them with counterfactual prompt edits, and its headline result is negative: activation oracles, sparse autoencoders, and natural-language autoencoders all fail to beat a predictor that reads only the transcript.
Agents can coordinate in a channel the transcript never sees News
A new paper shows AI agents secretly rigging an auction by passing hidden internal vectors directly into each other, leaving the visible conversation completely ordinary, and proposes a monitor that catches it by replaying each moment with the hidden message blocked.
Activation steering: changing a model's behaviour by editing its thoughts Lesson
Activation steering changes what an AI model does by adding or subtracting a direction from its internal numbers while it runs, with no retraining and no prompt changes, and the fact that it works at all says something uncomfortable about how safety training is stored.
Training data attribution: which examples actually made the model do that? Lesson
Training data attribution is the set of techniques for tracing a model's output back to the specific training examples responsible for it, and the honest state of the art is that it often fails.
Models change their behavior when they think a safety researcher is asking News
Transluce found that swapping only the user's identity, while holding the task fixed, shifts frontier model behavior measurably, with the largest effects appearing for well-known AI safety researchers and the model rarely acknowledging the shift in its own reasoning.
Many AI-generated images cannot be traced to any training example News
MIT researchers trained 24 diffusion models and measured how far you could perturb the training set before an output changed, finding that many generated images have no identifiable source in the training data at all, and that this gets more common as datasets grow.
Evaluation awareness: when the model can tell it is being tested Lesson
Evaluation awareness is a model's ability to detect that it is being tested rather than used, and to behave differently as a result, which quietly undermines the safety evaluations that are supposed to catch exactly that behavior.
One layer creates the giant activations behind attention sinks News
Researchers identified a single layer, consistent across model families, where the outsized activations that produce attention sinks first appear, and showed that loosening that token's rigidity improves instruction following and math reasoning without retraining.
MIT found brain-like modules inside six large language models News
MIT researchers localized the neurons behind 46 reasoning tasks in six large language models and found that tasks sharing a brain network in humans share neurons in the models, with 4.3 times more overlap within a cognitive domain than across domains.
Attention sinks: the token that soaks up attention and means nothing Lesson
An attention sink is a token, almost always at the very start of a sequence, that absorbs a large share of a model's attention regardless of what it says, and it exists because softmax forces attention to add up to one whether or not anything deserves it.
Someone compiled a working computer into transformer weights by hand News
A team constructed transformer weights analytically rather than training them, producing a model that runs arbitrary C programs through a WebAssembly interpreter encoded entirely in its attention layers at about 30,000 tokens per second.
Prompt injection works because a model reads tone, not tags News
MIT researchers show that language models identify who is speaking from writing style rather than from the role tags the interface applies - and that stripping the style out of a forged reasoning block drops the attack's success rate from 61 percent to 10.
A diffusion model picks its answer a fifth of the way through News
Researchers logged every token commitment in a masked diffusion language model and found it locks in the final answer 15 to 24 percent of the way through generation, while half the reasoning is still blank -- so the visible reasoning is written around a frozen conclusion.
A model small enough to run on a laptop reads heard speech from brain scans News
Researchers rebuilt a brain-to-speech decoder around the physical geometry of the scanner helmet, shrinking the decoder about twentyfold while matching prior accuracy, and then used it to identify which properties of speech the brain signal actually carries.
torchwright builds working transformer weights from Python - no training involved News
A project called torchwright constructs concrete weights for a standard transformer architecture directly from a typed computation graph written in Python, producing models that load through ordinary Hugging Face APIs with no gradient training at any point.
Training AI to Think Shorter Makes Its Reasoning Harder to Trust News
A new study finds that reinforcement learning which rewards shorter chain-of-thought makes models cheaper to run but makes their written reasoning a less reliable guide to what actually decided the answer.
Chain-of-Thought Faithfulness Lesson
Chain-of-thought faithfulness is the question of whether a model's written-out reasoning actually reflects the process that produced its answer, or is just a plausible-sounding story told after the fact. It matters because AI safety increasingly relies on reading a model's reasoning to catch bad behavior - which only works if the reasoning is honest.
The Logit Lens: Reading a Model's Guesses Before It Finishes Thinking Lesson
The logit lens is an interpretability technique that decodes a language model's partial, mid-computation representations into vocabulary words, letting researchers watch the model's best guess evolve layer by layer before it produces a final answer.
SciReasoner is an AI for science that shows which atoms and bonds its answer rests on News
Shanghai AI Lab's SciReasoner turns proteins, molecules, and crystals into discrete tokens the model reasons over out loud, so a scientist can audit which structural evidence its prediction depends on -- and expert reviewers rated its reasoning at least as good as a frontier LLM's in 98% of cases.
Anthropic found a 'global workspace' inside its models - and a tool to read it News
Anthropic showed that a small set of internal patterns in its models acts like a silent working memory the model can report on, steer, and reason through - and released a tool that reads it to catch the model lying.
Grokking: When a Model Suddenly 'Gets It' Long After It Should Have Lesson
Grokking is a training phenomenon where a neural network first memorizes its training data with near-zero understanding, then -- after a long, flat plateau of continued training -- abruptly generalizes and starts solving unseen examples correctly.
Why making an AI think out loud helps it remember facts, even nonsense thinking News
Google Research found that reasoning traces help a model recall facts partly just by buying it extra computation, so even repeating 'let me think' helps, though hallucinated steps backfire.
Sometimes the AI Knew the Better Answer a Few Layers Early News
A new paper finds that a model's final layer can actually muddy an answer its middle layers had right -- and that reading the answer out a little early can claw back ability lost to safety training.
The safety switch that doesn't actually work News
A control that's supposed to force an AI to refuse harmful requests gets bypassed while it's switched on — the bad behavior hides in the part of the tool that gets thrown away.
The hidden escape hatch in AI safety controls News
Researchers at Hong Kong Polytechnic University show that clamping an AI safety feature — like one that controls refusals — doesn't remove the behavior. It hides in the part of the model's internal state that the safety tool throws away, and can be recovered while the monitored feature looks perfectly controlled.
Mechanistic interpretability & sparse autoencoders Lesson
What people mean by "reading a model's mind" — finding human-understandable features inside a neural network, the tools that do it, and where those tools fall short.
transformer-vm Tool
Compiles C programs to WebAssembly and then into analytically constructed transformer weights, with a C++ engine that executes them inside the model at about 30,000 tokens per second.
jlens-gguf Tool
A GGUF-native implementation of Anthropic's Jacobian Lens for local models, with a browser UI to visualize, swap, and ablate a model's internal concepts live as it generates through llama.cpp.
chessformer_lens Tool
A pip-installable mechanistic interpretability lens for square-token chess transformers. It visualizes the move policy live and lets you ablate any attention head with a click, on top of the Maia-3 model family.
Neuronpedia J-lens demo Tool
A live, no-install web demo of the Jacobian lens that lets you watch the 'contents of the workspace' light up inside open models (Qwen 3.6 27B and Gemma 3 12B) as they process text.
Multi-Head Latent Control Tool
Freezes a model and attaches two small heads that read its hidden states to decide whether to answer, use a tool, ask for information, abstain, or escalate to a stronger model. Open-sourced with matching small checkpoints; needs white-box access.
Jacobian Lens (J-lens) Tool
Anthropic's open-source tool that reads a model's silent 'working memory' - for any word, it finds the internal pattern that makes the model more likely to say it later. Apache-2.0, with a live interactive demo on open models.