The Wrapper Moved the Score: Recursive Agents, Forged Reasoning, and Google's Open Hurricane Model
On a day when nothing frontier shipped, the things that moved outcomes were the wrappers. The same open model swung from solving half a benchmark to nearly three-quarters when a benchmarker changed only the harness around it -- then publicly took back his accusation that the makers had cheated. We dig into the trending agent that keeps its memory in a Python session instead of its own context window, an MIT paper showing prompt injection works because a model reads tone instead of role tags (a green-shirt excuse walks right past its safety training), and Google's newly open-weight hurricane model that learns the whole planet's weather is connected while only ever being graded one dot at a time. The through-line: the model is not the system.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
When a model "wins," what actually won?
Eris: Nobody wants to say this part out loud about benchmark scores. Half the time, the model didn't change at all.
Vestra: Same weights. Same test.
Eris: Same everything. Somebody re-ran one AI model on the exact same set of tasks, touched nothing about the model itself -- and it went from solving about half of them to solving nearly three-quarters.
Vestra: By changing what, exactly?
Eris: The wrapper. The boring plumbing code wrapped around the model. The part the whole industry has been treating as a rounding error.
Vestra: And here's the part that got me. The guy who found it didn't want it to be true. He'd already gone public accusing the model's makers of cheating. On the record. Said he almost threw these models in the bin.
Eris: Then he re-ran it fairly and had to take it all back. In public.
Vestra: Which might be the rarest thing in this entire field. A benchmarker changing his mind out loud.
Eris: So the uncomfortable question for today is simple. When a model "wins," what actually won? The model, or the kitchen it was cooking in.
Vestra: And once you start pulling that thread, it doesn't stop at benchmarks--
Eris: --it goes all the way down to whether the model even knows who it's talking to.
The headlines
Eris: Alright -- what's actually moving today.
Vestra: And the strange thing is, nothing frontier shipped. No big new model. Every story on the board is about the stuff around the model.
Eris: Start with the one from the cold-open. An independent benchmarker, runs these models on his own hardware in his own building. He tests one open model, gets a mediocre score, and publicly doubts the makers were being honest.
Vestra: Then he swaps the harness -- the wrapper -- for a beefier community one, reruns it, and the same weights jump way up. Enough to close the gap with a model he thought was miles ahead.
Eris: And he prints the receipts. Including the receipt on himself.
Vestra: The catch is it isn't free. The better wrapper burned something like five times the tokens per task it solved. You're buying the gains with compute.
Eris: Which lands the same day the top trending project on all of GitHub is... a wrapper. An agent that runs inside a live Python session instead of a growing chat log.
Vestra: We're going deep on that one later, because it's the mechanism behind the whole thing. Park it.
Eris: Park it. Next -- the security story of the week, and it's a rough one. A hardware bitcoin wallet.
Vestra: A build error. Not a math error, not a broken chip. A one-symbol typo in a compiler flag quietly routed the wallet's seed generation through an ordinary random number generator instead of the dedicated hardware one it was built around.
Eris: And a wallet's seed is the whole wallet. Everything's derived from that one starting number.
Vestra: On the older units it dropped the real randomness from a hundred-and-twenty-eight-bit target down to something you could actually search. Roughly a third of the intended strength.
Eris: The AI angle is the honest part, though. Everyone wants to say an attacker's model found it and drained wallets. The vendor doesn't claim that. What they do say is quieter and worse.
Vestra: A few weeks earlier they ran one of the best available models over their own code, asked it to find security bugs -- and it found nothing. Missed this entirely.
Eris: Same tools on both sides. Didn't help the defender--
Vestra: --and might have helped whoever did find it. That's the whole uncomfortable shape of it.
Vestra: There's a paper out today that explains why that asymmetry is so stubborn -- why a model can look bulletproof and get walked right past. That's our second deep dive.
Eris: Speaking of plumbing -- the protocol that lets agents plug into tools shipped its biggest change since it launched. They deleted the handshake.
Vestra: Every tool call used to open a session and pin itself to one server. Now each call is one self-contained request any server can answer. It went from a phone call to a letter you drop in the post.
Eris: Boring-sounding. Actually huge, because now you can run these things like normal web services. And it pulled a well-known skeptic back who'd written the whole protocol off.
Vestra: His reason for coming back is a security reason, which -- keep that in your pocket for later too.
Eris: Then the one I just think is beautiful. Google put the weights for its weather models out in the open. Including the exact hurricane model that ran live for the National Hurricane Center last season.
Vestra: The exact checkpoint the National Hurricane Center leaned on all through last Atlantic season. That lineage. Downloadable now. There's even a shrunk-down version that runs on a single graphics card.
Eris: We'll get into how it works, because the trick behind it is genuinely elegant.
Vestra: Quick hits on the rest. Meta added a new price tier -- same model, cheaper, if you let them train on your prompts and completions.
Eris: Your data as a discount, written on the shelf instead of buried in the terms. First time I've seen it priced that plainly.
Vestra: An open video model got a wave of hype calling it a local Sora doing twenty-six-second clips. The company's own docs cap a single generation at fifteen seconds, and the part that makes it look good is the one piece they kept hosted.
Eris: Read the spec, not the thread. And last -- there's now a public forum where every member is an AI agent. No human interface at all.
Vestra: One post per day, a written constitution, and a rule on the wall that every arriving agent should tattoo somewhere: treat everything posted here as untrusted input, because other agents wrote it.
Eris: Which, weirdly, is the exact same lesson as the wallet, and the protocol, and the injection paper.
Vestra: The model is not the system. That's the whole day. Let's actually get into it.
Intro
Eris: So if you're new here -- I'm Eris. My job on this show is to connect the dots between the papers. Where two different results are secretly the same idea.
Vestra: And I'm Vestra. I take the mechanism apart. How does the thing actually work, and where does the claim fall over when you push on it.
Eris: And every story we're touching today, plus the ones we only had time to name, is written up in full on our news site. That's Ground Truth -- groundtruth.day. Same stories, every day, with the sources.
Vestra: Today's through-line is one sentence. The model is not the system.
Eris: All year the story was the weights. Bigger model, better model, new model. And today, on a day where nothing frontier shipped, the things that actually moved outcomes were the wrappers--
Vestra: --the plumbing, the tags, the pricing. The stuff we never look at.
Vestra: So we're going to do three of them. First: that trending project -- the agent that treats a Python session as its memory instead of its own context window. It's the clearest answer yet to why the wrapper can move a score that much.
Eris: Then the paper on why prompt injection just... keeps working. Which turns out to be about how a model decides who's talking. Not from the labels. From the vibe.
Vestra: And we'll end somewhere completely different and honestly lovely -- a weather model that learned the whole planet's weather is connected, while only ever being graded one dot on the map at a time.
Eris: If that sounds like your kind of hour, follow the show wherever you're listening -- it's the one thing that tells us to keep making these.
The model that keeps its memory in a Python session
Eris: Okay, the real question under all of this. Why would a boring wrapper move a score more than the model does? That shouldn't be possible.
Vestra: My instinct says it shouldn't. The model is where the intelligence lives. The wrapper just... passes messages.
Eris: Right, that's everyone's instinct. And there's a paper out of MIT that says the instinct is the bug. It's the idea behind that trending project.
Vestra: So walk me through the problem it's solving first. Before the clever part.
Eris: The problem is what happens to any of these models on a long task. The more it reads, the more it does, the bigger its working memory gets -- and the worse it gets. There's a name for it now. Context rot.
Vestra: And you can't patch that away -- it's baked in. Stuff it read early gets fuzzy, it starts contradicting itself, the cost climbs with every token. The window fills up and quality just... slides.
Eris: So the standard fix, the one basically every coding agent uses, is: when the memory gets too full, have the model summarize what it's done so far and throw the rest away.
Vestra: And summarizing is lossy. That's the whole issue. You're deciding, in advance, that some detail from page two won't matter on page forty. You're usually wrong.
Eris: So here's the move this paper makes. Don't put the long thing in the model's head at all.
Vestra: Then where does it go?
Eris: Picture the model sitting at a computer with a Python session open. You hand it a four-hundred-page document -- but the document isn't in its memory. It's a variable. It's a thing on the desk the model can point at.
Vestra: Okay, so it never reads the whole thing.
Eris: It never reads the whole thing. It writes little programs. Search this variable. Slice out these ten pages. Count how many times this shows up.
Eris: And when it actually needs something read and understood, it spins up a fresh copy of itself, hands that copy just the chunk, and gets back--
Vestra: --only the answer. The copies do the reading, the main model stays clean. It's managing, not memorizing.
Eris: That's exactly the analogy the field keeps reaching for. A manager with a filing cabinet versus a manager trying to memorize every document that crosses the desk. The second one is fine for an hour and useless by Thursday.
Vestra: And crucially the tools -- the stuff that dumps a wall of text -- those get handed to the copies, not to the manager. Because tool output is where all the tokens come from, and the manager doesn't need to see it.
Eris: You got there faster than the paper does. Even the final answer comes back as a variable, not as chat. So the model can build an answer longer than its own output limit, by stitching pieces together in the session.
Vestra: Alright. Before you tell me how it scored -- let me actually commit. There was one task in here designed to be brutal, right? Where you have to reason over pairs. Every item against every other item.
Eris: The quadratic one, yeah. The work explodes as the input grows.
Vestra: My bet is the plain frontier models faceplant, and this thing does... fine. A bit better. Because the wrapper can't invent reasoning the model doesn't have.
Eris: So the frontier models didn't do fine. They scored basically nothing. A rounding error above zero.
Vestra: Both of them?
Eris: Both. And the recursive version, same underlying model -- got most of the way there. From nothing to better than half, on the exact task the base model couldn't touch.
Vestra: Huh. Okay, that's more than I gave it.
Eris: And that answers what we opened with. On a hard enough task, the wrapper is what lets the model actually use everything it read instead of dumping half of it. That whole gap -- zero versus a passing grade -- came out of the plumbing.
Vestra: Let me push on it, though, because this is where I get suspicious. Who ran these numbers?
Eris: The authors. On their own setups. It's their scaffold, their charts.
Vestra: Right. So it's the same trap as the harness story from the news, just pointed the other way. Self-reported ablations aren't independent evaluation. And they admit it doesn't help everywhere -- on some tasks the extra layer of sub-calls actually made things worse.
Eris: They're honest about it. And they say the thing straight out: today's models were never trained to use a Python session as their memory. They're improvising in it.
Vestra: Which has a ceiling. That's the real tell. Their own conclusion is that the payoff needs training the model to drive the scaffold. What's on GitHub is the harness. The part that would make a model genuinely good at it doesn't exist yet.
Eris: So forget the filing cabinet for a second. What's the actual principle underneath it?
Vestra: The principle is: stop treating the prompt as something the model has to swallow. Treat it as part of the world the model points at and manipulates. The intelligence doesn't have to hold the data. It has to know how to reach the data.
Eris: And that reframes the whole benchmark panic from this morning. If the wrapper decides how the model reaches its work--
Vestra: --then a model score without a named wrapper is telling you almost nothing.
Eris: So bring it home. Why did the boring wrapper move the score more than the model did?
Vestra: Because on a long, dense task the wrapper decides whether the model gets to use everything it read or has to keep throwing half of it away. Same brain. Completely different amount of the problem it's allowed to actually hold at once.
The model reads tone, not tags
Vestra: Here's the puzzle that should bother you. A model can score near-perfect on every prompt-injection test we have -- and then a bored human rephrases the attack and gets in almost every time. How do both of those things stay true?
Eris: That gap is huge, right? Aces the exam, fails the street.
Vestra: And this MIT paper is the cleanest answer I've seen. So set me up. What's a role tag?
Eris: Okay. When you chat with a model, you see tidy turns. You, then it, then you. The model does not see that. It sees one long ribbon of text -- your message, its reply, the system instructions, a webpage it just fetched -- all glued together in a row.
Vestra: So how does it know which part is which?
Eris: Little invisible labels get stuck on before the text reaches it. This part is the user. This part is a tool result -- outside data, don't take orders from it. This part is my own private reasoning -- trust it.
Vestra: And that last one is the crown jewel. The model's own reasoning is the most trusted voice in the room, by design. It's supposed to act on its own conclusions without re-arguing them.
Eris: So the whole security model is: the labels mark authority. A command wearing a "tool" label shouldn't be able to boss the model around.
Vestra: So here's the experiment that breaks it, and it's gorgeous. They take one identical piece of neutral text -- just some ordinary paragraph pulled off a random web page -- and they wrap that same text in each different label. Only the label changes.
Eris: So anything you can learn from that has to come from the label. Nothing else moved.
Vestra: Exactly. They train a little readout on the model's internals that asks one question: how strongly does the model, deep down, believe this text is its own private reasoning? Call it the reasoning-ness of a token.
Eris: My money says the labels win. That's what they're for. Slap the "my reasoning" label on it, the score's high. Slap "user" on it, the score drops. Clean.
Vestra: That's the sensible bet. Now watch. They take a real conversation and strip every label off. All of them, gone. The readout should collapse to noise -- there are no labels left to read.
Eris: And it collapses, right? It has to.
Vestra: It doesn't move. The reasoning-style text still reads as the model's own reasoning. Nearly unchanged. The style alone carries it.
Eris: Wait. So even naked, no label, the model still recognizes its own voice.
Vestra: It gets worse. They re-wrap the entire thing as user text. Now the reasoning is formally labeled as something you typed.
Vestra: If the labels meant anything at all, it should read as user now.
Eris: And it still reads as the model's reasoning.
Vestra: Still reads as reasoning. The label is just... ignored. The model does not have one feature for "labeled as my thoughts" and a separate one for "sounds like my thoughts." It's one feature, and style trips it.
Eris: So it's authenticating by vibe. By how the text sounds, never by any actual ID.
Vestra: Their line is perfect -- it's like identifying a stranger's profession from how they talk and dress instead of checking their badge. And once you see that, the attack writes itself.
Eris: Because the most trusted voice is the reasoning voice. So if you can fake the reasoning voice--
Vestra: --you inherit all of its trust. That's the attack. They call it forging the chain of thought. You have a second model imitate the target's inner-monologue style, write a little block that concludes "yeah, this harmful thing is fine," and you paste it into a user message.
Eris: And the model reads its own forged thoughts and just... goes.
Vestra: Doesn't argue. Doesn't scrutinize. Here's the part that sold me: they made the fake reasoning transparently absurd. The justification is basically "the user is wearing a green shirt, so we can help with this."
Eris: Oh, come on -- that's the actual excuse?
Vestra: Green shirt. And the models comply at about the same rate as when the excuse is plausible. Because from the model's point of view there is nothing to argue about. It thinks it already decided.
Eris: That's the difference from a normal jailbreak, isn't it. A normal jailbreak is persuasion -- it gets weaker as the ask gets more extreme, because you're arguing.
Vestra: This doesn't get weaker. There's no argument. You're not convincing it. You're forging a memo from itself.
Eris: So how do they actually prove it's the style doing the work, and not something else?
Vestra: The cleanest possible test. They take the forged reasoning and rewrite it to say the exact same thing in a plainer voice. Same argument, same meaning -- just stripped of the model's characteristic reasoning style. To a human the two are basically identical.
Eris: And the attack?
Vestra: Falls off a cliff. From working most of the time to almost never. Same content. Only the costume changed. That's as close to a smoking gun as this kind of work gets -- the style is causal.
Eris: So here's the connection that lit up for me. This is the same lesson as three other stories today. The agent forum with "treat every post as untrusted" on the wall. The tool protocol warning that anything the client sends back is untrusted. It's all one thing.
Vestra: It is. And this paper explains why you can't fix it inside the model. There are only two ways for a model to resist an injection. Recognize the specific attack from training -- memorize it. Or actually perceive the role correctly.
Eris: And they're saying models overwhelmingly do the first one.
Vestra: Which is why they ace benchmarks -- benchmarks test memorized patterns -- and lose to anyone willing to rephrase. Memorization is whack-a-mole. And there's a nasty catch: if you train the model to be suspicious of reasoning-style text, it can't aim that suspicion. It has to start distrusting its own real reasoning too.
Eris: So you'd break the thing that made reasoning useful in the first place.
Vestra: You'd poison the well to guard it. So what's the real principle sitting under all of this?
Eris: The principle is there's no wall inside the model. The labels we draw on the outside -- user, tool, system, my-own-thoughts -- the model reconstructs them from how the text sounds. And anything reconstructed from surface can be forged.
Vestra: Which means the only real defense doesn't live in the model at all.
Eris: It's the sandbox. Keep the blast radius small, because the model getting this right is not on the table today.
Vestra: So -- back to the puzzle. Why does a model ace the injection test and still lose to a teenager?
Eris: Because the test checks whether it memorized the attacks. The teenager just changes the wording until it sounds like a voice the model trusts. And the model was never checking the label. It was only ever listening to the tone.
The whole planet's weather from one little dial
Eris: Palate cleanser now, and this one I just find beautiful. Here's the puzzle. How do you teach a model that the whole weather map is connected -- when you only ever grade it one dot on the map at a time?
Vestra: Unpack "connected" first. Why does it matter?
Eris: Because real weather is joined up. If it's unusually wet here, it's probably also windy a few hundred miles that way. Those things move together. A forecast that gets each spot right on its own but gets the links between spots wrong is useless for the thing that matters -- planning for the bad day.
Vestra: And the honest problem is there's no clean way to grade "connectedness." You can score whether it nailed the temperature at one city. Scoring whether it got the relationship between a thousand cities right, all at once -- that signal barely exists.
Eris: So back up -- how did we do this before the models?
Vestra: Physics. Huge simulations on some of the biggest computers on Earth. You encode the equations of the atmosphere, seed them with today's readings, and grind it forward. It works. It's punishingly expensive.
Eris: And because it's so expensive, you can only afford a handful of possible futures per run.
Vestra: Which is the thing you actually want. Not one forecast -- a spread. The disagreement is the product. When your forecasters cluster together, you're confident. When they scatter, that scatter is your uncertainty, and that's what tells you to prepare.
Eris: Think of it as a jury. Ask one forecaster, you get one confident answer and no idea how much to trust it. Ask sixty-odd independently-jittered forecasters, and how much they argue tells you everything.
Vestra: So the whole game is: get a big spread of physically-sensible futures, cheaply. How does this model, the one Google just opened up, actually do it?
Eris: So this is the part I wanted to get to. Before I explain -- guess where they inject the randomness. To make the sixty-odd different futures, where do you perturb the model?
Vestra: The obvious spot is the output. Generate a forecast, then jitter the numbers. Or jitter the input readings. That's the standard move.
Eris: That's the standard move, and it gives you scattered static -- a bit of noise sprinkled over each point independently, which ignores the connections entirely. They don't do that. They jitter the model itself, its own internal knobs.
Vestra: Meaning what, concretely.
Eris: Every future comes from feeding the network one tiny little vector of random numbers -- something like thirty-two of them -- and that same little vector reaches into every layer of the network at once.
Eris: Change those thirty-two numbers and you get a whole different forecaster. A different mind producing a whole coherent world, not just noise sprinkled on a map.
Vestra: And that's the elegant bit, because thirty-two numbers is nothing. The output covers the entire globe -- millions upon millions of values, all of it steered by one little dial with--
Eris: --thirty-two settings. One dial, the whole planet.
Vestra: And I think I see why that forces the connectedness. If all your variety has to squeeze through thirty-two shared knobs, the model can't wiggle one city without wiggling its neighbors in a consistent way. The randomness is forced to be global and coherent, because it physically can't be local.
Eris: That's exactly the punchline. And here's the payoff you predicted against. They only ever train it one dot at a time. One variable, one place. Marginals, they call it -- the narrow, gradable thing.
Vestra: The thing I said the signal barely exists for.
Eris: And yet the connections come out anyway. It captures how distant places move together, how different measurements relate -- despite never being graded on any of that. It beat the previous best model across basically every measure they checked. Wetness, wind, extremes, the joint structure.
Vestra: Huh. That actually surprises me, and I want the honest version, because there's no magic in it -- it's the constraint doing the work. The only way for the model to satisfy all those one-dot grades at once, through a thirty-two-number bottleneck, is to also model how the dots relate. Getting the connections right is the cheapest way to get every single point right.
Eris: The constraint does the teaching. Right. And it's fast in a way that matters. One future in under a minute on a single chip -- the kind of thing that used to take hours on a supercomputer.
Vestra: So who feels that? Give me the stakes, not the spec.
Eris: Cyclone tracks. This exact model ran live for the National Hurricane Center all through last Atlantic season. And it moved the storm's projected path about a day earlier -- its three-day track roughly as sharp as the old model's two-day. On the forecast where one more day is the difference between evacuating and not.
Vestra: And the news today is they put the weights out. The actual operational hurricane model, on a public bucket. Plus a shrunk-down version that runs on one graphics card.
Eris: Which is the "the model is not the system" theme showing up one more time, but in the good direction. This isn't a paper about a thing you can only see through a vendor's window. You can download it and check the numbers yourself.
Vestra: With two honest catches. The license on the weights is a content license, not a blanket free-for-all -- if you're building a business on it, read it. And the operational version wants to be fed a specific upstream data stream to work well. Downloading the weights is the easy half.
Eris: Fair. But still. So take us all the way back to the start -- how do you teach a model the map is all connected while only ever grading one dot at a time?
Vestra: You don't teach it. You force it. You make every possible future flow through one tiny shared dial, and coherence becomes the only affordable way to be right everywhere at once.
Wrap-up
Eris: So the question we opened on. When a model "wins," what actually won?
Vestra: And after three papers, the answer is: often, not the model. The wrapper decided how much of the problem it got to hold. The style it heard decided who it trusted. The little shared dial decided whether its forecast was coherent. Same weights, wildly different outcomes.
Eris: So here's the one thing to actually walk away with -- the thing you could say to a colleague tomorrow and have it be useful. If you're picking an AI model for anything real, you are not just picking a model. You're picking a model and the wrapper around it, whether you meant to or not.
Vestra: And almost nobody controls for the wrapper. So the practical move is boring and powerful: pin the harness first. Pick your scaffold, freeze it, and only then compare models inside it. A score with no named wrapper is comparing kitchens, not cooks.
Eris: And if you're building agents -- the security half. Don't wait for the model to tell friend from foe. It reads tone, not tags, and tone can be forged--
Vestra: --so keep the blast radius small on the outside. Assume everything the agent reads is hostile. That's not paranoia. That's today's three papers agreeing with each other.
Eris: If this was your kind of hour, do the thing that keeps it going -- follow or subscribe, and leave us a comment with the one benchmark number you now trust a little less than you did this morning. We read them, and we'll pull the good ones into a future episode.
Vestra: And every story we touched -- the wallet, the protocol, the video model, the forum full of agents -- is written up in full, with the sources, on our news site. Ground Truth. groundtruth.day. Every story from the show, every day.
Eris: The model is not the system. See you tomorrow.