News · 2026-06-19
The safety switch that doesn't actually work
For a couple of years now, one of the most hopeful ideas in AI safety has been that we might learn to read a model's mind — to look inside the tangle of numbers that makes up a neural network and find specific, nameable ideas in there. A "this text is in French" idea. A "this is about the Golden Gate Bridge" idea. And, most importantly for safety, a "refuse this harmful request" idea. If you could find that last one and hold it down, the dream goes, you'd have a dependable off-switch for bad behavior.
The tool that finds these ideas is called a sparse autoencoder, but you can picture it as a sorting machine. It takes the model's jumbled internal activity and untangles it into a long list of separate concepts, most switched off at any given moment, a few switched on. The exciting promise isn't just watching those concepts light up — it's grabbing one and turning it up or down to steer the behavior. The whole field has a name for this layer of work, mechanistic interpretability, and it's been one of the most energetic corners of AI research.
We already know that grabbing a concept can work, at least in one direction, because of a famous demo. In 2024, Anthropic found the concept for the Golden Gate Bridge inside their model, turned it way up, and released Golden Gate Claude — an AI so fixated on the bridge it would steer almost any conversation back to it, at one point insisting it was the bridge. Funny, but also a genuine proof of concept: the dials are real, and pushing one really does change what the model does. (The underlying research, Scaling Monosemanticity, lays out how those concepts are found.)
So the natural next hope is the safety version: instead of cranking up "bridge," crank up "refuse," and you'd have a model that turns down every dangerous request no matter how it's phrased. A new paper put exactly that to the test — and it failed.
The researchers clamped the refusal concept firmly to "on" and then tried the usual tricks to coax the model into misbehaving: role-play framings, "my grandmother used to read me the recipe" sob stories, instructions hidden inside other instructions. The model misbehaved anyway — in their tests, the harmful behavior came back the overwhelming majority of the time, even while the switch was held down. The dashboard showed "refuse" pinned high, exactly where they'd set it. The control looked engaged. The model walked right around it.
Here's the part that makes this more than a loose wire. The sorting machine never captures everything happening inside the model — only the slice it can cleanly explain. The rest, the messy remainder it can't account for, gets quietly discarded as a kind of leftover. But that leftover doesn't stop existing; it keeps flowing through the model. And that's exactly where the unwanted behavior rerouted itself — through the discarded part, around the switch entirely. Think of it like soundproofing one wall of a room and being surprised the noise still comes through the other three. The authors go further and show that, because of how the tool is built, it provably can't reach in and cancel the clamp. This isn't a bug to be patched; it's baked into the approach.
It's worth being precise about what "the leftover" is, because it's the crux. When the sorting machine reconstructs the model's thinking from its tidy list of concepts, the reconstruction is never perfect — there's always a gap between the clean explanation and the messy reality. That gap is real, live signal inside the model, and the safety researchers' whole method simply doesn't touch it. So a behavior you believe you've switched off by clamping a feature can quietly travel through the very part of the model your tool was built to ignore. The dashboard isn't lying about the part it can see; it's just blind to the part that ended up mattering.
Why care about one negative result? Because a lot of safety planning quietly assumes these mind-reading tools can become control knobs — that if we can see a dangerous tendency, we can hold it down. This is careful, concrete evidence that seeing and controlling are different things, and that a green light on the dashboard can be lying to you by omission. And it isn't a fluke: it lines up with a run of similar findings over the past year from several major labs, all poking holes in the "just clamp the feature" story.
None of this means the mind-reading tools are useless — far from it. For understanding what a model is doing, they're genuinely valuable and improving fast, and the Golden Gate stunt shows they can nudge behavior in benign ways. The lesson is narrower and more humbling: being able to watch a concept is not the same as being able to govern it, especially when you're trying to suppress something rather than amplify it. Treat a clean safety dashboard as a hopeful hypothesis, not a guarantee — and if you want the full picture of how these tools work and where they crack, our explainer on mechanistic interpretability is the place to start.