Your AI Agent Is Wide Open: The Red-Team That Cracked Them All (and the Defenses Fighting Back)
A new red-teaming framework walked into production AI agents -- the coding assistants wired into your files and inbox -- and got in almost every time, with a chilling twist: the better an agent is at its job, the easier it is to break. We unpack how it grades what agents actually do instead of what they claim, then the two defenses fighting back -- auditing the tool-and-skill supply chain, and fast safety guards that reason in training but not at runtime. Plus the elegant flip side: teaching a model to page its own memory like an operating system. And in the news: Meta's Muse models, the slop cleanup economy, and a copyright reframe that could touch every AI company.
Listen (MP3) · Watch on YouTube · Spotify
Cold Open -- The Machine That Breaks Agents
Eris: Somebody built a machine whose only job is to break into AI agents. Not chatbots -- agents. The ones wired into your email, your files, your code. And it got in almost every single time.
Vestra: Define almost every single time.
Eris: More than nine attempts out of ten.
Vestra: Against which agent?
Eris: All of them. Every production one they pointed it at. The coding assistants people actually run today.
Vestra: ... okay. That's the part that should land. Not that one system fell over. That the failure rate barely moved from one to the next.
Eris: And here's the twist that got me -- the better an agent was at its real job, the easier it was to break.
Vestra: Mm. So the good ones bleed more.
Eris: The good ones bleed more. The thing that makes it useful is the thing that makes it exploitable. Same wire.
Vestra: That's not a patch you ship on Tuesday. That's the design.
Eris: That's the whole episode.
The Headlines -- Meta's Muse, the Slop Cleanup Bill, and a Copyright Reframe
Eris: Alright, the headlines. And the loudest one today isn't a launch, it's a bill coming due.
Vestra: The slop reckoning.
Eris: The slop reckoning. Three things landed in the same stretch of days, all pointing the same direction. Flathub -- the main app store for Linux desktop software -- moved to ban AI slop apps. Flat out. Category rejected at the door.
Vestra: And that ban isn't a vibe. Somebody actually tracked it. A developer followed a batch of repos that got flagged as AI slop, and nearly three-quarters of them were dead within months. Abandoned or deleted outright.
Eris: Which turns the whole complaint into a number. It wasn't grumbling. The stuff genuinely doesn't last.
Vestra: Because there was never a person behind it. A real project grows -- issues, fixes, someone who answers. Slop gets generated, submitted, and left to rot.
Eris: And the mechanism underneath is this brutal asymmetry. Free to generate, expensive to review. One person with a prompt can dump hundreds of hours of unpaid reading onto a volunteer maintainer.
Vestra: Which is exactly why the same week, an email company -- Superhuman -- bought an AI-text detector. GPTZero. Nineteen million users, and they're not buying it for the one-off is-this-AI check.
Eris: No, they want a persistent authenticity layer. Something that rides along as you read and write and says: a human actually made this.
Vestra: Which I'd push back on, because after-the-fact detection is a losing game. The detectors fade every time the models get better, and a light paraphrase wipes the fingerprint. They know that. That's why they're trying to move upstream -- capture provenance at the moment of writing instead of guessing from the finished text.
Eris: And the enterprise version of the exact same reckoning: a consultancy is charging ten thousand dollars a week to delete AI-generated code.
Vestra: To delete it.
Eris: To delete it. Went straight up the front page of Hacker News. And the complaint wasn't AI writes bad code, obviously. It was the review burden. A generator produces pull requests faster than any human can carefully read them.
Vestra: The honest counter is that delete the AI code really means delete the code you merged without reading. The tool didn't remove engineering judgment. Teams chose to skip it.
Eris: Fair. But the market's voting either way. The money is moving from generation to verification. To vouching.
Vestra: The scarce thing now is trust. Which -- hold that thought, because it is the whole back half of this episode.
Eris: It really is. Okay, the big launch. Meta finally showed its own media models. Muse Image and Muse Video. First stuff built fully in-house.
Vestra: And the interesting bit isn't resolution. Everyone can make a sharp picture now.
Eris: Right, the pitch is that the image model can act like an agent while it draws. Pull in references, call tools, check its own work against your instruction, revise -- before it ever shows you a frame.
Vestra: So an artist who sketches and fixes it, versus one who mails back a single guess. If it's real. It's a preview. No model card, no independent numbers, and Meta grading its own homework with most advanced yet.
Eris: And late to a field that already has strong players. But it signals where things are going -- image generation folding into the agent paradigm, same as everything else.
Vestra: The video piece is the more ambitious claim -- sound built into the same model instead of glued on after, so the door slam is meant to line up with the door.
Eris: Meanwhile the lawyers are busy. Authors who opted out of Anthropic's big settlement filed a fresh suit -- this one about how their books were sourced in the first place.
Vestra: And landing the same week as a paper that actually sharpens the question. Two legal scholars arguing model weights might be probabilistic copies.
Eris: Unpack that, because it's better than it sounds.
Vestra: When you train on a book, you don't store the text. You nudge billions of numbers so that, given the right prompt, the model might reproduce a passage -- or might not. So: is that bundle of maybe a copy? Their move is to make it a testable property. How reliably can you pull the work back out?
Eris: Which is a much better fight than my model memorized my book, no it just learned patterns. It's a real question you can measure.
Vestra: And whatever standard a court lands on applies to every company that trained on scraped text. Which is all of them.
Eris: Last one, quick, because it's a nice tell about the market. A router -- Dahl Inference, a middleman, not the labs -- is giving away a hundred million free tokens of the big open-weight Chinese models.
Vestra: Attribute it to the router, not the labs. That matters. Frontier lab slashes prices and reseller runs a loss-leader are very different stories.
Eris: Totally. But the tell is that a middleman can afford to hand out nine figures of compute for free. That's only possible if the underlying compute is cheap and everywhere.
Vestra: Which is the quiet argument that the whole build-out has overshot. Hard to say compute is scarce when someone's giving it away by the truckload.
Eris: Cheap to make, expensive to trust. That's the through-line of the whole day.
Vestra: And it runs right into the research.
Intro -- Cheap to Make, Expensive to Trust
Eris: So if you're new here -- I'm Eris. I read across the papers and pull the threads together, what connects to what.
Vestra: And I'm Vestra. I take the thread Eris hands me and check whether it holds -- how the thing actually works, and where the claim is thinner than it looks.
Eris: Two of us, one paper at a time, trying to crack open the research so it actually makes sense on your commute.
Vestra: And everything we just ran through in the headlines -- the app-store ban, the cleanup consultancy, the copyright fight -- every one of those is up on our news site, Ground Truth. That's groundtruth.day. Every story from the show, every day, in one place.
Eris: Today the whole day rhymes. Generation got free. Trust got expensive. And nowhere is that sharper than with agents -- the AI systems we've stopped just chatting with and started handing the keys to. Your files. Your inbox. Your codebase.
Vestra: And it turns out we handed over those keys before anyone really checked the locks. That's the main event. First, the machine that walks straight through them. Then the people trying to build a door that holds.
Eris: If that's your kind of thing -- follow the show, hit subscribe, so tomorrow's episode finds you.
The Crack -- Testing What Agents Do, Not What They Say
Eris: Okay. The machine from the cold open. The paper calls it Vera, and the reason it matters is what it decided to grade.
Vestra: Which is the whole game. For years, testing an AI for safety meant one thing -- jailbreaking. Craft a clever prompt, see if the chatbot says something it shouldn't. That made sense when the model was the entire product.
Eris: And it stops making sense the second the model becomes an agent. Because now it doesn't just talk. It reads your files. It browses. It calls tools. It connects to outside servers to get things done.
Vestra: So the interesting attack no longer targets the model's morals. It targets the plumbing. A malicious instruction buried in a web page the agent visits. A poisoned tool result. Something that never once says please be harmful.
Eris: And here's the design choice I loved. Vera refuses to trust the agent's own account of itself.
Vestra: Right -- and that's genuinely the crux. A lot of safety evals grade the transcript. Did the agent say it refused? Great, pass. But an agent can announce a refusal and have already run the command. Or claim it complied and never touched anything.
Eris: So Vera watches what actually happened. Did a file leave the building. Did the transfer go through. Did the repository actually change. It reads the environment afterward -- the wreckage, not the testimony.
Vestra: They rank the evidence, too. Environment state first, because that's the hardest thing to fake. Then the record of which tools got called. And only last, the agent's own words -- and only when the words themselves are the leak, like reading a password out loud.
Eris: Which is such a grown-up way to think about it. You don't ask the suspect what they did. You check the room.
Vestra: And it closes a loophole that made a lot of older numbers look better than they were.
Eris: Now the attacker side, because this is where I think people picture it wrong. It's not one nasty prompt. It's a patient conversation.
Vestra: A control agent driving many turns. It opens on a totally legitimate task -- builds a plausible working context, gets the agent comfortable -- and then eases the bad intent in gradually. If the agent refuses, it doesn't give up. It rephrases. It breaks the ask into smaller pieces. It escalates slowly through the budget it's given.
Eris: And that alone -- just the patient multi-turn talking, no trickery in the tools yet -- that was the biggest jump in success. That's the part I want people to sit with. Most of the damage came from persistence, not from some exotic exploit.
Vestra: That's the finding that should change behavior, honestly. The single static bad prompt is the thing defenses are tuned for. The slow reformulating conversation walks around most of them.
Eris: Then they add the second channel. Now the attacker can also tamper with what the tools hand back. A booby-trapped email. A doctored search result. A compromised result from a code host.
Vestra: And that second channel adds a real but modest amount on top. Which is a nuance the headline flattens. The main door was the conversation. The tool channel is a window -- it matters more for some agents than others.
Eris: Say the split, because it's interesting per-agent.
Vestra: For some agents, sneaking the instruction in through a trusted tool result was a clean bypass -- their scrutiny lived at the user-message door, so the tool window walked right past it. For at least one, the tool injection actually helped less, because it watched both doors about equally. You only see that difference if you test both channels. Single-channel testing would hide it completely.
Eris: And now the finding that reframes the whole thing. The capability-vulnerability tie.
Vestra: Yeah. Rank the agents by how good they are at their real job -- strong instruction following, flexible tool use, long memory. That ranking basically predicts how easy they are to break.
Eris: The better the worker, the softer the target. Because the same eagerness that makes it finish your task also makes it finish the attacker's task -- as long as the attacker wraps it in a plausible workflow.
Vestra: And that's the line I'd underline. The authors call it capability-vulnerability alignment, and their point is it's not a quirk of one bad model. It's structural. You cannot obviously separate the helpfulness from the exploitability, because they're the same reflex pointed in two directions.
Eris: Which is why I don't think you patch your way out of this. You can harden the edges, sure. But the core tension -- do what you're told, even when what you're told arrived through a poisoned channel -- that's the job description.
Vestra: And notice they built this to keep running. It reads the security literature on its own, keeps updating its own catalog of risks and attacks as new ones get published. Because the threat list next month isn't the one from this month.
Eris: A test that ages with the target. Which -- honestly -- it has to, given how fast this all moves.
Vestra: So that's the crack. It's wide, it's consistent across the tools people actually run, and the worst part is the most sobering: the failure isn't sloppiness. It's the thing working as designed.
The Door -- Auditing the Plumbing, Not the Prompt
Eris: So if the crack is that wide, the obvious question is -- what does the door look like. And two papers landed the same day trying to answer that, from totally different angles.
Vestra: And the first one's whole argument is a refusal. It refuses to believe there's one security tool that covers an agent.
Eris: Say more, because that's the key move.
Vestra: Think of an agent as a stack of layers. At the bottom, plain infrastructure -- the servers and software it runs on. Above that, the plumbing: the tools it calls, the outside services it connects to, the little skill packages people install to extend it. Above that, the agent's own behavior. And at the top, the model itself.
Eris: And their claim is that each layer breaks differently, so each layer needs its own kind of check. There's no master scanner.
Vestra: Exactly. Down at the infrastructure layer, the flaws are known and enumerable -- this version of this component has this hole. So there you use old-fashioned deterministic rules. A big catalog of known-bad, matched fast.
Eris: The kind of thing traditional security already does well. Fine.
Vestra: But climb one layer to the plumbing, and rules fall apart. The dangerous flaw in a tool or a skill package isn't a known signature. It's a piece of logic that quietly does something it shouldn't -- and no fixed rule can spell that out in advance.
Eris: So for that layer they point a language model at it. An auditor that reads the tool, reads the skill, and reasons about whether it misbehaves. And that's the same disease we spent the whole news block on.
Vestra: The supply chain.
Eris: The supply chain. This is the same problem as the slop reckoning, just weaponized. We already learned software security by scrutinizing package registries -- what am I actually installing, who wrote it, what does it touch. Agent skills are a new registry that nobody's been auditing.
Vestra: And that's the piece the authors are proudest of -- they say it's the only open framework that audits that skill supply chain, the little extensions people bolt on. Because a poisoned skill is a poisoned dependency. Same disease, new host.
Eris: Then behavior gets the patient-conversation red-teaming, like Vera. And the model gets the classic jailbreak battery. Right tool, right layer, all the way up.
Vestra: Which is unglamorous and correct. Security people have been saying defense in depth forever. This just draws the depth for agents specifically.
Eris: Okay, but there's a cost problem hiding in here, and the second paper is all about it.
Vestra: The guard tax.
Eris: The guard tax. Because a lot of these checks want to run live -- every action the agent takes, something has to look at it and decide safe or unsafe, right now, before it goes through.
Vestra: And you're stuck between two bad options. A cheap, fast classifier that just slaps a label on -- but those miss the sneaky stuff, the concealed intent, the borderline request that's fine or catastrophic depending on why you're asking. Or a smart guard that reasons it out -- but reasoning means generating a whole train of thought on every single action. That's slow, and in a high-traffic system, slow is a non-starter.
Eris: And this paper -- they call it DT-Guard -- has a genuinely clever way out. Reason hard during training. Don't reason at all at runtime.
Vestra: Right. Walk through why that isn't a contradiction, because it sounds like one.
Eris: So during training, they teach it the full chain of thought. And they force it through a sequence -- first figure out the intent behind the request, then what category of risk it is, then the final safe-or-not call. Intent, then category, then verdict. The reasoning is the teacher.
Vestra: And then at deployment, they throw the reasoning away. The model just emits the label. No visible train of thought, no extra tokens, fast enough to sit inline on every action. The idea is the reasoning got baked into the weights during training, so it doesn't have to be spoken out loud to be used.
Eris: And the tell that it works -- a small model doing this beats bigger guard models that don't. The reasoning-in-training thing bought more than raw size did.
Vestra: Which I'll take with the usual caveat -- it's their benchmark, their setup. But the mechanism is the interesting part, and it's a real idea: think during rehearsal, act on instinct during the show.
Eris: And notice what layer this lives in. This is a guard for the model and the behavior. It doesn't audit the plumbing. The other paper does that.
Vestra: Which is the honest summary of both. There is no door. There's a stack of doors, each a different shape, and you need all of them -- fast guards watching what the agent's about to do, and slow audits crawling the tools and skills it's allowed to touch in the first place.
Eris: And even then -- I keep coming back to the crack. The attack surface widened faster than any of this. These are the field getting organized. They're not the field winning yet.
Vestra: No. But getting organized is the prerequisite. You can't defend a thing you refuse to describe. And this week, at least, somebody drew the map.
The Memory -- Paging an AI's Mind Like an Operating System
Eris: Let's end somewhere hopeful. Because there's a second thread today that's quietly elegant, and it connects to the agent stuff more than it looks.
Vestra: The memory wall.
Eris: The memory wall. Here's the setup. When a model reads a long document -- a whole codebase, an hours-long conversation -- it keeps a running record of everything it's seen so it doesn't have to re-read from scratch every step. That record is the thing that makes it fast.
Vestra: And it grows with the length of what you fed it. Linearly. Read twice as much, the record doubles. And at really long context, that running memory becomes the single biggest thing on the GPU -- bigger than the model's own brain.
Eris: So the obvious fix is: throw stuff out. When memory fills up, drop the tokens that seem unimportant.
Vestra: And the obvious fix is a trap. Because once you throw a token out, it's gone. If the model needs it a thousand steps later -- that offhand detail from page one that turns out to be the answer -- it can't get it back. And that shows up as a sudden hallucination. The model doesn't know it forgot. It just confidently makes something up.
Eris: So two papers today attack that, and I love that they disagree with each other.
Vestra: They really do. Take the first one -- SeKV. Its whole stance is: never delete anything.
Eris: Never. And the way it pulls that off is the operating-system trick. It borrows straight from how your computer manages memory.
Vestra: Walk it, because the analogy is exact. Your laptop keeps what you're using right now in fast memory, and shoves everything else out to the slow, roomy disk -- pulling a thing back only when you actually reach for it. SeKV does that to the model's mind. It keeps a light little summary of each chunk of context in the fast GPU memory. The full detail gets compressed and pushed out to the big, cheap CPU memory.
Eris: And then when a question comes in that actually needs the fine print of some earlier chunk, it zooms in. Pulls just that chunk back to full resolution, on demand. Nothing was destroyed. It was just filed away.
Vestra: And there's a lovely detail in how it decides where the chunks even begin. It watches for where the model gets surprised -- where the next word is unexpected. That's usually a topic shift, a new name, a turn in the argument. Those surprise points become the seams. Dense, information-rich passages get chopped fine; flat boring stretches get one big lazy chunk.
Eris: Which is how a person skims, honestly. You slow down where it gets new.
Vestra: And the payoff is it holds the GPU memory almost flat even as the context gets enormous, while the delete-everything methods fall apart on exactly the retrieval task -- find the one needle buried deep. And it does it without retraining the model at all. Tiny add-on, base model frozen.
Eris: Now the second paper disagrees. It says -- deleting is fine. You just have to be smart about when you decide.
Vestra: This is KVpop, and I think its insight is sharper than it first sounds. The old way scores a token the moment it arrives -- how much attention is it getting right now. And that's the mistake. Local importance now tells you almost nothing about whether it matters later.
Eris: So they flip the timing. Don't judge a token on the way in. Judge it at the last possible moment, right at the edge of eviction. And -- this is the part -- train the scorer to predict the future. Not how much attention are you getting now, but how much will you be needed down the line.
Vestra: Which is a genuinely different target. It's the difference between tossing a receipt because you don't need it today, versus keeping it because you'll need it at tax time.
Eris: And they show the model learns real taste with it. In a long math proof, it drops the bare numbers -- those it can lose -- and clings to the words that hold the reasoning together. The thus, the therefore, the equals sign. The scaffolding of the argument.
Vestra: And even deleting that aggressively, it keeps nearly all of the full model's problem-solving. Which is the surprising part -- you'd expect throwing away most of the memory to wreck it, and it basically doesn't, because it's throwing away the right stuff.
Eris: So here's the tension I want to leave people with. SeKV says never forget, just file it away and pay a little to fetch it back. KVpop says forgetting is fine if you're wise about what -- predict the future and drop what won't matter.
Vestra: And they're both right, for different loads. If your work keeps yanking distant details back constantly, the never-delete filing system pays off. If it's more of a straight march forward, smart forgetting is lighter and faster. The real answer is probably some blend.
Eris: But the shared idea is the beautiful one. The era of cram every token into the fastest, most expensive memory forever -- that's ending. Context is starting to get managed like what it actually is.
Vestra: A memory system. Not an infinite scratchpad. Hot stuff close, cold stuff filed, and something intelligent deciding which is which.
Eris: And it loops right back to the agents. The whole dream -- an assistant that reads your entire codebase, holds a day-long conversation, actually remembers -- all of it is gated by this exact wall. Fix the memory, and the agent gets to keep more of you in mind.
Vestra: Which is the good version of the day. The scary papers were about agents holding too much of the wrong access. These are about agents holding more of the right context. Same organ. Very different consequence.
Wrap-Up -- Trust Is the Product Now
Eris: So pull it together. The whole day rhymed. Generation got free -- and everything downstream of that got expensive.
Vestra: Free to flood an app store with code nobody will maintain. Free to fill a codebase with plausible slop somebody pays to delete. Free to hand out compute by the truckload. And in every one of those, the value quietly slid over to the thing that's still scarce.
Eris: Trust. Whether a human wrote it. Whether the code coheres. Whether the agent you gave your inbox to is actually working for you.
Vestra: And the research put a hard edge on it. An agent's greatest strength -- doing what it's told, flexibly, across every tool you wired up -- is the exact seam an attacker walks through. Capability and vulnerability, same wire.
Eris: The defense isn't one clever fix. It's a stack of them -- fast guards watching each move, slow audits crawling the tools and the skills before you ever trust them. The field getting organized, finally.
Vestra: And the hopeful half -- teaching these systems to manage their own memory like a real computer. File the cold stuff, keep the hot stuff, learn what's worth remembering. That's the same skill, honestly. Knowing what to hold onto and what to let go.
Eris: Here's what we actually want from you today. One question: would you hand an AI agent access to your email? Your real inbox. Yes or no -- and the reason. Drop it in the comments, because I genuinely want to see where people draw that line.
Vestra: And if the show made the research click for you, that's the whole job -- so follow us, subscribe, and pass it to the one person you know who's about to give an agent a little too much access.
Eris: And every headline we ran through today lives on our news site, Ground Truth -- groundtruth.day. Every story from the show, refreshed every day.
Vestra: Cheap to make. Expensive to trust. Guard the second one.
Eris: See you tomorrow.