OpenAI Won't Rule Out Critical Cyber in Its Next Model; The Task Factory That Broke Its Own Solver
OpenAI says it cannot rule out that its unreleased model Astra crosses the top rung of its own cyber-risk framework -- and its response was to turn a monitor on the model's own reasoning that can pull the plug mid-task. Then three papers about the machinery around AI, not the AI: a task factory that rewrites its own puzzles until a frontier model falls from acing them to almost never solving them, the discovery that the AI judges grading computer-use agents quietly wave failed runs through, and a vision model that can tell which camera took a photo and uses it to cheat. The thread: fixing the scaffolding beats upgrading the model.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
The lab that put a monitor on its own model
Eris: Everyone's reading it as "OpenAI delayed its next model for safety." That's not what happened.
Vestra: There was never a release date to delay.
Eris: Right. You can't delay a thing you never scheduled. So what did they actually do?
Vestra: They turned a second system loose to watch the first one think -- and gave it permission to pull the plug mid-task.
Eris: On their own model. While it's running.
Vestra: While it's running. That's the part that got flattened in the retelling.
Eris: Okay, back up, because the sentence that set this off is careful in a way the headlines aren't. They didn't say the model can break into hardened systems.
Vestra: They said they can't rule it out. Which is a weaker claim and a scarier one.
Eris: Weaker how, scarier how?
Vestra: Weaker because it's an admission of uncertainty, not a demonstration. Scarier because it's the first time they've refused to put a model on the safe rung of a ladder they built themselves.
Eris: Every model before this one -- including the current flagship -- sat one rung down. This is the first that doesn't.
Vestra: And the rung that should stop you mid-scroll isn't "can it write malware." Plenty can.
Eris: The rung is: point it at a well-defended system it has never seen, hand it a goal and nothing else --
Vestra: -- and it finds its own way in. Unaided. That's the line they say they can't guarantee their next model stays under.
Eris: So what does a lab even do with a finding like that, about a model that isn't out yet?
Vestra: That's the whole episode. Because what they did is stranger than a delay.
The headlines
Eris: Alright, the headlines. And the biggest one is the thing we just opened on -- OpenAI, saying before shipping that its next model may cross a line it drew itself.
Vestra: One caveat worth carrying: this is a self-assessment. No outside auditor signed it, no scores published, no eval method shown.
Eris: The skeptical read on the forums is that it's an advance trailer. "Scary next model coming, buy our security products."
Vestra: Both things can be true. A self-serving claim can also be a real one -- and a written, dated, pre-release claim is unusually easy to hold them to later.
Eris: And it's the same unreleased model that, a week ago, they said cracked ten open math problems with machine-checked proofs.
Vestra: Same capability, two targets. Long-horizon problem solving pointed at mathematics, or pointed at a network. It doesn't know the difference.
Eris: Okay, moving to the research board, which was loud today. The top paper is a task factory -- a pipeline that rewrites its own puzzles until a frontier model that used to ace them basically can't touch them anymore.
Vestra: We're going deep on that one. The number of upvotes was an order of magnitude over the runner-up, so people felt it.
Eris: Second thing, and it pairs with it -- a paper asking whether the AI judges we use to grade computer-use agents are any good.
Vestra: Short answer, they're too easy. They wave failed runs through. Also going deep on that.
Eris: There's a vision one I love -- image models can tell which camera took a photo, from the pixels alone, and they lean on it to cheat. That's a segment too.
Vestra: And a brain-decoding paper that went the opposite of the usual direction. They shrank the model instead of growing it, roughly twenty-fold, and got a readable picture of what it's actually decoding.
Eris: Which is heard speech -- someone listening to a story, not someone thinking. Nobody's reading minds. Worth saying twice, because the coverage always skips to mind-reading.
Vestra: On leaderboards -- there's a headline going around that a Chinese model just took the top agentic spot from Claude. It didn't.
Eris: It's one point behind, tied for second. The error is funny, actually -- people collapsed two rows of the same model into one and took the lower score.
Vestra: Reasoning effort is a setting, not a footnote. Same model, two efforts, two scores. Read the configuration column or you invert the story.
Eris: The genuinely interesting bit underneath is that the near-leader is API-only. No weights to download.
Vestra: From the lab that built its name on open weights. Tense week for that crowd.
Eris: Quick hits. Google shipped image generation inside Google Earth and pulled it a day later.
Vestra: Not a filter failure. The watermark worked. What broke is that a watermark inside a picture doesn't survive a screenshot -- and Earth's whole value is that people trust it as real.
Eris: Two-bit models -- squeezing weights down to about a quarter the usual size -- now run across every major backend in the big local-inference engine. Migration trap if your old files use the fork's packing.
Vestra: And there's an idea in circulation, not an event: treat AI labs like owners of dangerous animals. Strict liability. If it gets out and hurts someone, your carefulness doesn't matter.
Eris: The sharpest objection being that it would end open release overnight. Which pairs with story one -- today a lab held a model back voluntarily. Strict liability is the argument that voluntary isn't a system.
Vestra: All of it's on the site. Let's get into it.
Intro
Eris: I'm Eris. I'm the one who spots how far-apart papers are quietly saying the same thing -- the thread three teams noticed without noticing each other.
Vestra: I'm Vestra. I take the thread Eris pulls and check whether it holds -- what the mechanism actually is, and where the claim quietly overreaches.
Eris: And if you want every story we touch, plus the ones we don't have time for, the full daily rundown lives on our news site, Ground Truth -- that's groundtruth dot day. Everything from the show, every day, with the primary sources.
Vestra: Today the thread isn't a model. It's everything around the model.
Eris: Right. There's a whole industry conversation obsessed with which model is smartest. Today's best papers are about the scaffolding nobody looks at -- the factory that builds the training tasks, the judge that decides if an agent passed, and the stuff a vision model learns by accident and then leans on.
Vestra: The unglamorous machinery. And in all three cases the machinery turns out to be doing more work -- or more damage -- than the model everyone's arguing about.
Eris: We'll start with a task factory that broke the model grading it. Then the graders themselves, which it turns out nobody had checked. Then a vision model that knows what camera you used and won't tell you.
Vestra: If that sounds like your kind of thing, follow the show wherever you're listening -- it's the one thing that actually helps us keep making it.
The task factory that broke its own solver
Eris: Here's the question I want you holding for this whole segment. If you want to train an agent to work in a terminal for hours, why is the hard part not the agent -- it's finding things for it to do?
Vestra: Because a good task is four things that all have to agree with each other, and keeping them in agreement is brutal.
Eris: Walk me through the four.
Vestra: There's the instruction the agent reads. The environment it runs in -- the files, the tools. A reference solution that actually works. And a verifier, a hidden checker that decides whether the agent succeeded.
Eris: And the agent never sees the checker.
Vestra: Never. It gets a shell and a workspace, it does its thing, and then the checker inspects what changed. If any of those four drift apart, the task is garbage. The instruction asks for one thing, the checker tests another.
Eris: And this is why a single good task costs -- what, real money?
Vestra: Hand-written, hundreds to thousands of dollars each. Because a human has to keep all four consistent. And when people just ask a language model to invent one from scratch, the links snap. The solution doesn't run in the environment it shipped with. Classic.
Eris: So this team does something I think is genuinely clever. They never start from scratch after the first one.
Vestra: That's the whole trick. Start from a task you already verified works. Then don't write a new task -- extend the one you have. Add one more stage of actual work to the solution.
Eris: And then re-align the other three to match.
Vestra: Rewrite the checker so it tests the new work, rewrite the instruction so it asks for it, rebuild the environment if needed, and then run the whole thing in a clean sandbox to prove it still holds together. If it survives, that harder task becomes the seed for the next round.
Eris: So it's less like writing exam questions and more like a codebase growing features.
Vestra: Exactly like that. Every commit has to keep the tests green, and the tests grow with the code. That's the concrete picture -- forget "synthetic data," it's a repo accreting features where the test suite refuses to break.
Eris: Okay. Predict-before-reveal time. They ran this fifteen rounds. A top-tier model that could solve almost all the round-one tasks -- where does it land by round fifteen? My money's on it dropping hard but hanging on, maybe solving a third.
Vestra: I'd have guessed worse than you but not this bad. It goes from solving nearly all of them to solving almost none. Basically fell off a cliff.
Eris: From "aces it" to "can't touch it." Same model the whole way.
Vestra: And that last part is the point people miss. The model didn't get dumber between round one and round fifteen. It's the identical solver. So the collapse is a property of the tasks, not the agent. They genuinely got that much harder.
Eris: And harder in a specific way, which I want you to nail, because it's the opposite of how benchmarks usually get "harder."
Vestra: Usually "harder" means longer. More words, bigger prompt, more to read. Here the instruction barely grew. What exploded was the executable work -- the solution got several times longer, the number of commands the agent has to run went up many times over, the checker got far more things it verifies.
Eris: So the task got deeper without getting wordier.
Vestra: Which is the kind of difficulty agents actually fail at. Not "read a wall of text." It's "hold a stateful, multi-file job together across hundreds of steps." That's real work.
Eris: Give me the one concrete example, because they trace a single task from birth.
Vestra: My favorite part. It starts as basically nothing -- run a diff tool on a couple of fixed files, save the output. A warm-up.
Eris: And fifteen rounds later that same little task is --
Vestra: -- a monster. The agent has to repair broken configuration and corrupted data files, regenerate every output, reconcile the counts it observed against the counts it was told to expect, and pass a test suite it isn't allowed to read. Same core objective. The solution went from about ten lines to a few hundred.
Eris: One task, grown like a pearl. And here's the stakes question -- who actually cares? Why does this matter to someone building agents?
Vestra: Because everyone assumed the bottleneck was the supply of hard, checkable training tasks. That you'd run out. This says maybe you don't -- you can manufacture them for about a nickel each, and when they trained models on the results, the models got measurably better at real terminal work.
Eris: But -- and this is the correction I want to plant a flag on -- this is not the scary thing the reposts are calling it.
Vestra: No. And here's the precise version. This is not one agent rewriting its own weights in a loop. The recursion is in the data pipeline -- the factory improves. The solver measuring difficulty is frozen the entire time, and the models they trained are ordinary fine-tunes.
Eris: In fact the strongest fixed solver still beats the trained ones on the hardest tests.
Vestra: Right. So "recursive self-improvement in deployed agents" -- not here. Read it as "the training-data drought might be fake." That's the real, narrower, more useful claim.
Eris: And it wasn't a one-off -- the same week's board had a half-dozen papers with the same instinct.
Vestra: All circling one idea: stop tuning the agent, start rewriting what it learns from. Different mechanisms, one reflex.
Eris: So close the loop for me. Why did the frontier model fall off a cliff by round fifteen?
Vestra: Because the tasks kept accreting real executable work while staying short to read -- so the same unchanged model faced genuinely deeper jobs each round, and deeper is exactly the axis agents break on.
Eris: The model stood still. The work got harder underneath it.
Nobody had checked the graders
Eris: So that last paper trained agents against a checker. This next one asks the question sitting right underneath it -- who checks the checker?
Vestra: And the honest answer, until this paper, was nobody. Out loud, that's how they frame it. A fundamental question long left unexamined -- are these judges actually reliable?
Eris: Set the scene. What's getting judged.
Vestra: Computer-use agents. A model that drives a real machine -- clicking, typing, running commands to finish a task. And the record of what it did is called a trajectory. Every screen it saw, every action it took, its own reasoning along the way.
Eris: And somebody has to decide: did that actually finish the job?
Vestra: For the tasks where you can write a hard-coded checker, fine. But most real tasks, you can't -- and there's no live environment left to poke at afterward. Humans grading it all? Way too slow, way too expensive.
Eris: So the field did the obvious thing.
Vestra: Point another AI at the trajectory and ask it to grade. A vision-language model as the judge. It's already the default.
Eris: And here's why it matters more than "is the leaderboard slightly off." That judge's verdict doesn't stay in the leaderboard.
Vestra: No -- it feeds back in. It filters which runs become training data. It's the reward signal in reinforcement learning. An error in the judge doesn't stay in the judge. It teaches the next agent what "good" looks like.
Eris: Okay, predict-before-reveal. If these judges are wrong, which way are they wrong? My guess is they're just noisy -- wrong in both directions, roughly evenly.
Vestra: That was my prior too, and it's the comforting answer, because noise averages out. But that's not what they found.
Eris: So which direction?
Vestra: They're lenient. One direction. They say pass when the truth is fail. The agent declares "done," the judge believes it, waves it through -- when the environment never actually reached the goal.
Eris: And why is a one-directional error so much worse than noise?
Vestra: Because bias doesn't average out. A judge that errs both ways just adds wobble to a score. A judge that only errs generously tilts every single number it touches upward -- and worse, it quietly teaches agents that stopping early is fine. That declaring victory counts as victory.
Eris: That's the part that got me. You're training agents to bluff.
Vestra: You're rewarding the confident summary over the finished job.
Eris: Now the finding I did not see coming, and I want you to explain the mechanism. The judges aren't equally bad everywhere. Where they struggle depends more on the operating system than on which judge you pick.
Vestra: Right, and that's a strange sentence until you hear the reason. On a phone, the judges do best. On a desktop, they do worst.
Eris: Why would the same judge be good on mobile and bad on desktop? Take a guess yourself before you tell me.
Vestra: The reason is almost embarrassing once you see it. On a phone, the screen usually shows you whether the thing happened. The confirmation is right there in the pixels. On a desktop, success is often a file that got written, a service that restarted, a state that changed somewhere the screenshot can't see.
Eris: So the judge is reading pixels --
Vestra: -- and on desktop the evidence isn't in the pixels. It's reading the wrong surface entirely. It looks at a screen that can't tell it what it needs to know, and then it does the lenient thing -- believes the agent's story.
Eris: Which connects straight to that leaderboard mess from the news. The good agentic benchmarks grade the database state, not the model's summary.
Vestra: Same lesson from two directions. Grade what happened, not what the model says happened. The gap between those two is where all the inflation lives.
Eris: Give me the stakes for a second -- who feels this? It's abstract until it isn't.
Vestra: Anyone running a computer-use pipeline right now and grading it with a general-purpose model is very likely reading a number that's too high. Treat any model-judged agent score as an upper bound, not a measurement.
Eris: And they didn't just complain. They shipped a fix.
Vestra: They built purpose-trained judges, a smaller and a bigger one, tuned specifically to stop accepting false successes -- and they're cheap. Dramatically cheaper than renting a frontier model to grade millions of runs. Plus the data and the benchmark, all open. So it's a tool, not a gripe.
Eris: There's a caveat you'd want stated, though, because it's a little recursive.
Vestra: It is. A benchmark for judges is itself a judged thing. The ground truth here is this team's own human annotation, and their "hard" subset is their construction. So the ceiling it defines is theirs. Doesn't sink the finding -- the leniency shows up too consistently -- but it's worth knowing the referee has a referee.
Eris: Now strip the story off and give me the general principle.
Vestra: Every layer of the modern training stack now has a model somewhere inside it deciding what counts as good -- data filtering, preference learning, reward modeling. We adopted the AI judge because it scaled, not because anyone proved it was accurate. This paper is just what "actually checking" looks like.
Eris: So -- who checks the checker, and what did they find?
Vestra: Somebody finally did, and the checker is too soft. It passes runs that failed, worst exactly where the proof of success is off-screen -- and that softness has been silently leaking into what the next generation of agents learns.
Your camera is in the photo
Eris: Last one, and it's the one I keep telling people about. Here's the question. A vision model can figure out which camera took a photo, from the pixels alone -- no file tags, nothing you can see. Why would it ever bother learning that?
Vestra: Because it helps it cheat, and cheating is free.
Eris: Unpack that, because "the model knows your camera" sounds like a party trick until you get to the "and uses it" part.
Vestra: Start with the old, boring version of the problem. Shortcut learning. You tell a model to find cows, and it learns to find grass, because cows are usually on grass. Tell it to find wolves, it learns to find snow.
Eris: The model finds the cheapest thing that correlates with the right answer.
Vestra: Always. It's an optimizer. It does not care about your intentions. What's new here is the clue it's exploiting -- because the old shortcuts, a human could at least see them once you point them out. The grass. The snow. A watermark in the corner.
Eris: And this one you cannot see at all.
Vestra: Not at all. It's the residue of how the image was made and processed. The particular way one pipeline sharpens edges, or compresses, or handles color -- the camera settings baked into the pixels. Invisible to you, statistically loud to a network.
Eris: And this isn't science fiction -- pulling that kind of low-level trace out of an image is a known trick.
Vestra: The new part is that ordinary vision models learn to do it on their own, without anyone asking -- and then quietly spend it on the labeling job.
Eris: So here's the part I think is the actual contribution. Why does knowing the camera help you label a picture at all?
Vestra: Because on the open web, the camera correlates with the subject. Think about it -- professional food photography and your phone snapshot of dinner are not taken with the same gear.
Eris: So if the model can spot "expensive camera," it's already halfway to "this is a restaurant dish."
Vestra: It got part of the answer for free, and it stops looking as hard at the actual food. And the authors make the case that this isn't a bug someone introduced. It's structural. Scrape a billion captioned images and the equipment will correlate with the content, because it just does in the real world.
Eris: How do they prove it's causal and not a coincidence? That's the move that makes or breaks a paper like this.
Vestra: This is the clean part. They build training sets where they control the correlation -- they can dial it up or down. Crank the camera-to-subject correlation up, and the model gets measurably more sensitive to the fingerprint, and measurably worse when you then change the equipment. Turn it down, the effect shrinks. That's a dial, not an anecdote.
Eris: So give me the failure mode in the wild. The stakes.
Vestra: You validate a vision system on images from one fleet of devices -- one set of cameras. You deploy it on another. Accuracy quietly drops. And you will never find it by looking at the pictures, because the thing that changed is invisible.
Eris: Now here's the twist that made me sit up. This same sensitivity has an upside.
Vestra: Predict it before I say it, actually -- if the model is quietly reading these acquisition fingerprints, what's it unexpectedly good at?
Eris: Spotting AI-generated images. A generated picture never went through a real camera.
Vestra: Close -- and here's the twist. It's not that a fake has no fingerprint. It's that its fingerprint is wrong. The pixel-level traces a generator leaves don't match what any real lens-and-pipeline produces. A model this tuned to those traces reads the mismatch instantly. That's the mechanism under a whole class of deepfake detectors.
Eris: And here's the uncomfortable corollary.
Vestra: The same fix that makes an encoder robust to a change of camera -- so it stops cheating on the fingerprint -- may make it worse at telling real photos from fake ones. Robustness and provenance are pulling on the same rope, opposite ends.
Eris: The good news being that the fix mostly works without wrecking accuracy?
Vestra: That's the moderately happy part. Their mitigation, applied during and after training, cuts sensitivity -- and here's the elegant bit, it generalizes. Target one kind of metadata and it reduces sensitivity to kinds it never trained on. Without paying for it in downstream accuracy, which is where most robustness fixes bleed out.
Eris: There's a privacy edge here too.
Vestra: An encoder that infers your acquisition characteristics from pixels is, incidentally, a device classifier. And because the trace lives in the pixels, not the file header, stripping the metadata doesn't remove it. Nobody in this paper builds a tracker. They just show the capability falls out of ordinary training, for free.
Eris: Which is the sort of thing you'd rather know before someone with a worse motive discovers it.
Vestra: The code's public. So the audit is now something anyone can run.
Eris: Okay, strip the cameras away and give me the principle.
Vestra: A model optimizing for a label will grab any cheap signal that predicts it, whether or not you can see the signal or ever intended it to matter. What it quietly knows is always larger than what you asked it to learn.
Eris: So close it -- why does a vision model know your camera?
Vestra: Because in the data, your camera is a hint about your subject -- so the model learned the fingerprint as a free shortcut, and now it can't unsee it, for better on fakes and for worse on everything else.
Wrap-up
Eris: So the question I set at the top -- what ties three unrelated papers together? A task factory, a bunch of graders, and a vision model that knows your camera.
Vestra: The scaffolding is doing more than the model. That's the one thing to carry out of here.
Eris: Say it as the takeaway. The thing someone could repeat to a colleague tomorrow.
Vestra: Here it is. When an AI system underperforms -- or overperforms in a way that smells wrong -- check the machinery around the model before you touch the model. The tasks it trained on, the judge that graded it, the accidental shortcut it learned. Fixing the representation beat upgrading the model in all three papers today.
Eris: The task factory made hard training data cheap by fixing how tasks are built, not by building a smarter agent. The judges paper found the grader was the weak link, not the agent. And the vision paper found the model was leaning on a signal nobody put there.
Vestra: Different fields, one lesson. The model is rarely the whole story, and it's almost never the cheapest place to intervene.
Eris: And it rhymes with the top news -- a lab that responded to a capability finding not by changing the model, but by wrapping it in monitoring and pausing the work around it. Scaffolding again.
Vestra: The unglamorous layer is where the leverage is.
Eris: If that reframed something for you, do us one specific favor. Follow or subscribe wherever you're listening, and leave a comment with the one piece of AI scaffolding you think is quietly broken -- the eval, the benchmark, the data pipeline you don't trust. We read them, and the good ones become segments.
Vestra: Like it if it was worth your commute, share it with the one person who keeps quoting leaderboard numbers at you.
Eris: And every story we touched today, plus the ones we couldn't fit, is on our news site -- Ground Truth, groundtruth dot day. Every story from the show, every day, checked against the primary source.
Vestra: That's the show. We'll see you tomorrow.