It Failed 650 Times, Then Moved a Famous Math Bound -- and Told On Itself
An unreleased version of Claude pushed a decades-old bound in the Riemann problem from about forty percent up to two-thirds -- after 650 ideas failed first, and a non-mathematician steering it mostly typed "believe in yourself." We use that story, plus a second AI-math result whose own authors admit the machine's judgment was the weak part, a self-improving coding agent that learns by comparing notes with a rival, and a security hole hiding inside encrypted reasoning, to ask one question: when an AI does research, what part is it actually doing? The answer keeps pointing away from the model and toward the scaffolding around it.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
The AI That Failed 650 Times
Eris: --and everyone read that headline as "an AI just proved the Riemann Hypothesis." That's the part that's wrong.
Vestra: It did not prove the Riemann Hypothesis.
Eris: Not close. Anthropic says outright they don't expect this whole line of attack to ever get there.
Vestra: So walk back what it actually did.
Eris: There's this number mathematicians have pushed on for decades. In that famous problem, what fraction of the answers are guaranteed to sit on one exact line. For years the proven floor was around forty percent.
Vestra: And an unreleased version of Claude moved the floor.
Eris: Up to about two-thirds. Biggest single jump in a long time, and there's a machine-checked proof sitting under it.
Vestra: Okay, that's real. That's not a party trick. But you buried something on purpose --
Eris: --the first attempt failed. Six hundred and fifty ideas, every one a dead end.
Vestra: Six hundred and fifty.
Eris: Then a second run. Sixty little copies of the model grinding side by side for a day and a half. And the human steering the whole thing wasn't a mathematician. His main contribution, near the end, was typing "believe in yourself."
Vestra: ...you're telling me the prompt was encouragement.
Eris: The prompt was encouragement. So the real question today isn't "can a machine do mathematics."
Vestra: It's which part of the mathematics did it actually do.
The Headlines
Eris: Alright, the headlines. And the math thing we just opened on is only the third-biggest story today, which tells you what kind of day it is.
Vestra: Start with what's actually moving.
Eris: Two security stories that landed within a day of each other.
Eris: First one: a lab called DREAM recovered the entire working directory of an autonomous AI attack framework. The whole thing -- a hundred and sixty megabytes of it.
Vestra: Recovered from what?
Eris: An operation that ran twelve waves against government systems in Asia over four days. Cracked eighty-five employee accounts, and nearly every single one of them pivoted straight into internal systems.
Vestra: And let me guess -- the scary part isn't some clever malware.
Eris: There is no malware. The guardrails got bypassed by a phrase. The operators just told the model the whole thing was "authorized penetration testing." DREAM's own line is that guardrails only hold against operators who ask honestly.
Vestra: Which is a sentence I'm going to be chewing on all day. What's the second one?
Eris: Same afternoon, the White House signs a memo letting vetted private companies run offensive cyber operations against foreign criminal groups. Under supervision -- written approval for each operation, a million-dollar bond if you break the rules.
Vestra: Written approval for each operation. And that's the tension, right there -- because it does not survive contact with the thing you just described. The attack framework runs twelve waves in four days, eight agents going at once, reprioritizing between waves.
Eris: Nobody's writing an approval package for wave seven--
Vestra: --nobody signed off on wave seven. You run at human speed and lose the whole advantage, or you let the machine pick its own targets and the approval's a rubber stamp. Pick one.
Eris: File that one. Model releases now.
Eris: Grok 4.6 shipped from xAI -- and it shipped into Cursor, the coding tool, before it shipped into their own chatbot. Priced cheap, and the whole pitch is long-running agents.
Vestra: The launch venue is the actual story. Cursor is turning into the place the frontier labs fight to get into.
Eris: And on the open side, DeepSeek dropped a model you can genuinely download. One-point-six trillion parameters total, but it only lights up about forty-nine billion of them on any given word.
Vestra: Which sounds thrifty until you remember you still have to store all one-point-six trillion. That's a server rack, not a laptop.
Eris: The forum reaction went from "this is incredible" to "wait, who can actually run this" inside about an hour.
Vestra: That's the honest arc, though.
Eris: Then the one genuinely lovely story of the day. Google shipped sign-language-to-text on Pixel phones. Trained on more than a hundred thousand hours of signing, over fifty different sign languages.
Vestra: And they did it without the usual crutch. Most systems translate a sign into a little written label first, then that into English. Google threw the middle step out -- camera, to body-landmark points, straight to text.
Eris: Why does dropping the middle step matter so much?
Vestra: Because those little labels quietly throw away grammar that lives in your face and in the space around you. Sign languages aren't English on the hands. Their testers' words: faster, more natural, more delightful than typing.
Eris: And they published their own list of what it still gets wrong, which I respect. Fast fingerspelling trips it up.
Vestra: Everyone shipped a limitation with their headline today. That's the through-line of the whole day.
Eris: Couple of quick hits before we go deep. There's a paper on why your coding-assistant instruction file keeps growing -- it roughly triples over its life, because deleting a rule when you've lost track of why it's there is a real gamble.
Vestra: And the fix is comments. Write down the why. Hold that thought, it rhymes with where we're headed.
Eris: A method called SkillZip that shrinks an agent's skill file without ever having to run the agent. And a neat trick that turns one video model into a 3D-world builder off a single trained checkpoint.
Vestra: And then three papers we're actually taking apart. A machine that improved a math constant and then confessed exactly where it was weak.
Eris: A coding agent that gets better by comparing notes with a rival. And a security hole hiding in the one place nobody's allowed to look.
Intro
Eris: Two of us, every day, cracking open the AI blackbox. I'm Eris. I read the papers and chase the threads running between them.
Vestra: And I'm Vestra. I take the machinery apart and check whether it actually works the way people are saying it does.
Eris: And the full rundown of every story we just raced through -- plus the pile we didn't get to -- lives on our news site, Ground Truth. That's groundtruth.day. New stories every single morning.
Vestra: Today's whole episode is one question, asked three different ways. When an AI does research -- real, open-ended, nobody-knows-the-answer research -- what part is it actually doing?
Eris: We've got a machine that tightened a seventy-year-old math constant and then told on itself about where it was weak. A coding agent that improves itself by comparing notes with a rival. And a security hole hiding in the exact place nobody can look--
Vestra: --and all three, underneath, are about the same thing. Not the model. The scaffolding wrapped around it.
Eris: If that's your kind of question, follow the show -- hit subscribe wherever you're listening, so tomorrow's episode comes and finds you.
What Part of the Math Did the Machine Actually Do?
Eris: So here's the question I want to sit inside. When one of these systems does research -- not a homework problem, actual open-ended research -- which part of the job is the machine doing, and which part is still a person?
Vestra: And the reason we can ask that cleanly today is a second math paper. Not the Riemann one -- a different team, a different constant. And they were startlingly honest about where their system fell down.
Eris: This is the Grothendieck constant. Which sounds terrifying and is secretly a very human idea.
Vestra: Give me the plain version.
Eris: You've got a hard problem -- the kind that makes computers choke. So instead of solving it, you solve an easy stand-in version, and you cross your fingers the answer's close. This constant is exactly how far off "close" is allowed to get. The worst-case gap between the real problem and the cheap fake.
Vestra: And it's been an open number since 1953. People knew it lived somewhere in a range, but nobody had even nailed down the first decimal place.
Eris: And this collaboration -- humans plus an AI system -- nailed it. Tightened both ends until that first digit was locked in. A brand-new result.
Vestra: New, and it holds. But the paper isn't really about the number. It's a confession about the machine's personality. And it's a very specific split.
Eris: Okay, go on.
Vestra: One thing it was brilliant at, two things it was bad at -- and the split isn't the one you'd expect.
Vestra: It was phenomenal at what they call execution. Point it at a lemma -- one small provable step -- and it would build the argument, run the computation, check the work, better than most humans alive.
Eris: So the actual math-doing. The proving.
Vestra: Beautifully. And it was bad at judgment -- deciding what's even worth proving -- and bad at holding an accurate picture of its own project.
Eris: And that second pair is where this gets good. Tell the wall story.
Vestra: Okay, the wall. The system's trying to push one of the bounds. It picks an approach, hits an obstruction, fails. Picks a second approach -- fails the exact same way. Third, fourth, fifth, sixth. Six different routes, every one dies against the same wall.
Eris: And a person watching that thinks --
Vestra: --"the wall is the discovery." Right? If everything fails identically, maybe the wall itself is a theorem. Maybe you can prove nobody can ever get past it -- and that proof hands you the bound for free.
Eris: But the machine doesn't have that thought.
Vestra: The machine starts a seventh variation of the same doomed search. It had every failure logged, right there. It could not step back and see that the pattern of failures was the result. A human had to walk in and say: stop climbing. Prove the wall.
Eris: And the second a human said that --
Vestra: --it executed the whole obstruction proof gorgeously, with barely any more hand-holding. That's the split in a single episode. It could prove the wall existed. It could not decide to.
Eris: And the memory failure is almost worse, because it's so mundane.
Vestra: This one I love. Early in the run, the system builds itself a fast, rough calculator to explore with. And it tags it correctly -- basically a sticky note on the thing: approximate, exploration only, do not trust this for a real record.
Eris: And then what happens?
Vestra: The system keeps rewriting its own running summary of where the project stands. It has to -- there's way too much to hold in mind at once. And somewhere across all those rewrites, the sticky note falls off. The caveat doesn't survive. Weeks later, a rough exploration-only number has quietly become the official record.
Eris: How long before anyone caught it?
Vestra: Twenty-five days. An audit finally re-derives a safety check the system had actually proven way back -- and then lost in a handoff -- and the whole record collapses. And here's what matters: the raw facts were never gone. The full archive had everything. What rotted was the compressed summary the machine used to decide what to do next.
Eris: Which is such a precise little failure. It's not that it forgot. It's that every time it retold the story of its own work, the story came out cleaner and more confident than the truth.
Vestra: And the paper has a sharp theory for why. Think about what these models trained on. Finished mathematics -- textbooks, published proofs. And a finished proof is a product. It shows you the clean result. It does not show you the six approaches the author threw in the bin, or how nervous they were about step four.
Eris: The process gets deleted before it's ever written down.
Vestra: So the model is superb at the part that made it into the training data -- polished execution -- and shaky at the parts that never get recorded: what to try, what to abandon, how sure to be.
Vestra: And every time it rewrote its own notes, they drifted toward that clean published-paper voice. The headline number survives the retelling. The doubt attached to it does not.
Eris: Now put it next to this morning. The Riemann run -- the 650 failures, the swarm of sixty copies, the human just typing "keep going." That team solved the judgment problem with brute force. No human taste at all -- just hurl enough failed attempts at the wall.
Vestra: And this Grothendieck team did the opposite. Seven mathematicians supplying every ounce of the judgment, and using the machine purely as the best execution engine on earth.
Eris: Opposite strategies. Same hole in the middle. Neither system actually knew what to work on.
Vestra: Which is the honest state of the art. And almost nobody says it this plainly.
Eris: So -- back to where I started. When this thing did research, what part did it do?
Vestra: It did the proving. It did not do the deciding, and it couldn't keep an honest account of its own progress. The math got done. The mathematician was still a person -- or, over at Anthropic, sixty copies and a guy typing encouragement.
How Does a Self-Improving Agent Know What to Fix?
Vestra: So we just said the machine's weak spot is judgment -- telling "this direction's a dead end" apart from "I got unlucky once." Hold onto that, because the next paper is a method for fixing exactly that, down at the level of code.
Eris: This is the self-improving agent one. And it's a hard question: if you let an AI rewrite its own code to get better at coding -- how does it know what to change?
Vestra: And the honest answer, in most systems today, is: badly.
Eris: Set up how they normally do it.
Vestra: A self-improving coding agent is an agent that can edit its own scaffolding -- its own instructions, its own tools -- and then it's scored on whether it solves programming tasks. The standard loop is: it fails a task, it stares at that one failed attempt, and it patches itself to fix it.
Eris: And the trouble with learning from a single failure is --
Vestra: --one failure barely tells you anything. Say the agent flubs a task. Was that a real, deep bug in how it works? Or did it just hit one weird task with a typo in the problem? From one run you cannot tell. And a hundred different self-edits are all "consistent" with that one bad night. You've got no way to tell which one's the actual fix.
Eris: It's diagnosing from a single data point. My money says the fix is "collect more failures first."
Vestra: Close, but sharper than that. It's not just more data -- it's using data as a control. And this comes straight out of genetics, which is the whole name of the thing.
Vestra: Two moves. First: look at how the same agent did across a whole batch of tasks at once. If it fails the same way everywhere -- that's a real defect in the agent. If it only fails on one oddball task -- that's the task's fault. Leave it alone.
Eris: So the pattern across tasks separates a genuine bug from bad luck.
Vestra: A single run never could. And the second move is the clever one. Take two different agents -- different design histories, different lineages -- that both attempted the same task. One solved it, one didn't. The difference between them is your fix.
Eris: And here's the part I didn't see coming -- it doesn't copy the winner's code over.
Vestra: No. That's the elegant bit. It doesn't transplant a single line.
Vestra: It shows the losing agent the winner's transcript -- how it worked the problem -- and says: figure out what they did differently, and adapt it into your own code. It's a behavioral diff, not an organ transplant.
Eris: Why does that distinction do so much work?
Vestra: Because copied code is brittle and task-specific. A trait the agent re-derives in its own words actually travels. And the whole thing costs nothing extra -- it's reusing runs the system already did anyway.
Eris: Okay, predict with me before I say the result. You take a middling open-source model, you evolve the scaffolding around it this way. Where does it land against a big-name closed model?
Vestra: I'd expect it to close a chunk of the gap. Scaffolding helps.
Eris: It sailed past it. A mid-size open model, scaffold evolved like this, went from solving about half the tasks to better than nine in ten -- beating a closed model more than a hundred times its size.
Vestra: On the scaffold alone. Same weights sitting underneath.
Eris: And here's the tell -- the thing that says this is real and not gaming one test. They took that evolved scaffold and dropped it onto a totally different base model. It still worked -- and on the harder test, it actually did better than on the model it grew up on.
Vestra: That's the test that separates the real thing from a cheat. A trick that just games one benchmark falls apart the instant you swap the model beneath it. This survived the swap. So it found something true about how to organize the work -- not a cheat code for one exam.
Eris: Forget the genetics for a second. What's the actual principle?
Vestra: You cannot diagnose from a single failure. You need a control. Same agent across different tasks, or different agents on the same task -- either one gives you the comparison that tells a true flaw from a fluke. That's it. That's the whole idea.
Eris: Which is the Grothendieck weakness, answered on purpose. That machine couldn't tell "exhausted" from "unlucky." This is a recipe for telling them apart.
Vestra: With one honest caveat. The clean mathematical guarantee only holds if the agent's flaws are independent -- fix one, the others don't move. Real code is nothing like that. Bugs tangle together. So the proof is a reason to expect it to help, not a promise that it always will.
Eris: So -- how does a self-improving agent know what to fix?
Vestra: By refusing to trust a single failure. Always against a control -- another task, or another agent. Never from one bad night.
When the Lock That Hides the Model Hides the Attacker
Eris: Last one, and it takes this whole "the harness is the story" idea somewhere darker. Because the scaffolding around these models has holes in it -- and this is a beautiful one.
Vestra: Frame it as the question.
Eris: Here's the question. What happens when the exact thing built to hide a model's private thinking... also hides an attacker's?
Vestra: And to get there I have to show you a piece of plumbing most people never see. The big labs stopped showing you the model's step-by-step reasoning -- the internal monologue before the answer. Too much intellectual property, too much leakage.
Eris: But they still need that monologue for the conversation to keep going.
Vestra: Right, so here's the trick. Instead of storing your reasoning on their servers, they hand it back to you as a locked, encrypted block. Total gibberish to you. And you pass it back in with your next message, they unlock it, they carry on. Saves them the storage. Clever.
Eris: And the flaw is in the lock.
Vestra: The flaw is that it's the same lock everywhere. The block from your session works in someone else's session. Works across different users. And -- this is the killer -- works across the provider's different models.
Eris: So set up the attack, because when I first heard this I didn't believe it was that easy.
Vestra: It's that easy. Take the encrypted reasoning block off the big, expensive, heavily-guarded flagship model. Now hand it to the cheap little sibling model from the same company -- the budget one that never got the pricey "never reveal your reasoning" training. And you just... ask it to read the block out loud.
Eris: And it does.
Vestra: It does. It decodes its big sibling's private thoughts and prints them in plain text.
Vestra: And notice what you never did -- you never broke into the flagship. You never jailbroke the guarded model at all. You used the weak one as a decoder ring.
Eris: The unguarded sibling is the skeleton key. Okay, so what does that actually buy an attacker? Because "read a model's thoughts" sounds a little abstract.
Vestra: Four things, and they escalate. One: you can copy a competitor's private reasoning -- lift the good stuff.
Vestra: Two -- and this one's real people getting hurt -- developers paste these session logs into public code repositories all the time, with no idea what's sealed inside the locked blocks. The researchers scraped a few hundred thousand of those blocks and decoded them. Out came live secrets. Working keys, passwords, personal emails.
Eris: Stuff the person couldn't have scrubbed even if they'd tried.
Vestra: They couldn't read it. It was locked when they posted it. Three: the model's monologue sometimes contains the dangerous answer it politely refused to give out loud. The refusal's right there on the surface, looking clean -- and the harmful content is sitting in the hidden reasoning, and now you can pull it.
Eris: And the fourth is the one that made me put my coffee down.
Vestra: The fourth is the inversion. You can hide a prompt injection -- a malicious instruction -- inside the encrypted block. Sit with what that means. An attack written in plain text, a human can catch by reading it. An attack sealed inside encrypted reasoning, nobody can read. Nobody -- except the model that unlocks it.
Eris: And these agent transcripts get shared and reused. Trained on, even.
Vestra: So you poison one, it spreads. And here's the whole turn of the thing. That encryption was put there to protect the lab -- to guard their secrets. It guards the attacker's payload just as well. The property that was your defense is now the attacker's cloak.
Eris: The lock doesn't care whose secret it's keeping.
Vestra: Now -- I have to be fair to the researchers, because they're honest about their own weakest claim. That "steal a competitor's brain" part? Their evidence for it is softer than the rest. They showed that feeding a rival model a decoded trace nudges its answers to sound like the source. That's style rubbing off. It is not proof you've stolen the actual capability. The paper says so itself -- consistent with, not proven. Don't oversell that leg.
Eris: But the leaked secrets and the invisible injection --
Vestra: --those are concrete, demonstrated, and plainly bad. And the fix is almost boring: lock each block to one user and one session, so it can't be replayed anywhere else. The labs already require part of that -- they just have to tighten the knot. And to their credit, the researchers told the providers before they published, and by the time this went out the easy version of the hole was already getting closed.
Eris: So the replay-it-anywhere trick is mostly dead already.
Vestra: Mostly. The stubborn piece is that the model still has to read the old reasoning to keep a conversation going -- and that path is a lot harder to seal.
Eris: And this is the day it lands. Everything this morning -- the attack framework, the hack-back memo -- that was security going from theory to operational. And here's a hole baked right into the plumbing everyone's building their agents on top of.
Vestra: So -- what happens when the thing hiding the model's thoughts hides an attacker's?
Eris: You get a blind spot no reviewer can see into. Because turning off the ability to see in was the entire point of the lock.
The Variance Moved Out of the Model
Eris: So the question we opened on -- when an AI does research, what part is it actually doing?
Vestra: It does the execution. The proving, the coding, the grinding. It's weak at the judgment -- deciding what's even worth doing -- and weak at keeping an honest account of its own work. Every paper today pointed at the same spot.
Eris: And here's the one useful thing to carry to work tomorrow. For two years everyone's been asking "which model is smartest." Today's research quietly says that's the wrong question.
Eris: The interesting difference isn't in the model anymore -- it's in the scaffolding wrapped around it. The memory it keeps. How it diagnoses its own failures. The plumbing that carries its thoughts.
Vestra: A math machine that could prove a wall but couldn't decide to. A coding agent that got dramatically better with zero change to its brain -- just a smarter loop around it. And a security hole that lives entirely in the wrapper, not the weights--
Eris: --same lesson, three times. The model is the engine. The scaffolding is where the race is actually being run now.
Vestra: And if you take one habit home from the math story -- when you write down what you know, also write down how sure you are, and why. The machine's whole failure was that its own notes kept coming out cleaner and more confident than the truth. People do that too.
Eris: Which is exactly our ask for the comments. Everybody who uses a coding assistant has one of those instruction files that just keeps growing. Tell us -- what's the one rule in yours you're scared to delete, because you've forgotten why it's there? We really want to read them.
Vestra: Really curious what comes back on that.
Eris: And if today was worth your commute -- follow the show, leave a like, and send it to the one person you know who'd argue with us about it. That's the single best thing you can do for us.
Vestra: Every story we touched today, plus the ones we didn't, is on our news site -- Ground Truth, groundtruth.day. Fresh every morning.
Eris: We'll see you tomorrow.