AI News Today, Aug 22: 315,000 hidden AI reasoning blocks were readable in public repos
Researchers read 315,000 'encrypted' hidden reasoning blocks scraped from public code repos, recovering personal data and live credentials without breaking any cryptography. Anthropic's new CHIVE pipeline finds that reading a model's internals doesn't beat reading the transcript. Z.ai ships GLM-5.3 with a ledger of 2,436 security findings, most still under embargo.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
315,000 hidden AI reasoning blocks were readable in public repos
Eris: Three hundred fifteen thousand of them, Vestra. Sitting in public code repos right now, and every one of them can be read.
Vestra: And these are the sealed ones. The hidden reasoning that AI providers encrypt before they hand it back to developers -- the part nobody is supposed to see.
Eris: Encrypted, opaque, safe to commit to GitHub. Except a research team just read them and pulled out passwords.
Vestra: Without breaking any encryption. That's the detail that should worry people most.
Eris: So walk me through it. How do you read a sealed envelope without ever touching the seal?
Hidden AI reasoning traces can be replayed and read out of public repos
Vestra: Start with why the envelope exists. When a reasoning model works a hard problem, it writes a long internal monologue before the answer you actually see. Providers want developers to carry that monologue from one API call to the next, so an agent doesn't re-think everything from scratch.
Vestra: But they don't want you reading it, and sometimes they don't want to store it themselves. The compromise is they encrypt it and hand you the ciphertext. A sealed envelope you're trusted to carry but not open.
Eris: And the flaw is who holds the key.
Vestra: The whole product family holds the key. Take a block produced by the expensive frontier model, hand it back on a call to the cheap sibling model from the same provider, and the server decrypts it -- exactly as designed. Then you ask the cheap model what it just read. And it tells you.
Eris: No lockpicking. The building let them in the front door.
Vestra: The authors are careful about that, too. Alexander Panfilov said it plainly in an interview published today -- they didn't break any cryptography. The envelope opens itself if you mail it to the right sibling.
Eris: Okay, so that's the mechanism. The scale is what got me. They scraped public code repositories and found 315,000 of these encrypted blocks just sitting there.
Eris: Developers commit logs, test fixtures, cached API responses all day long, and the opaque string next to a transcript looks like noise. Teams that carefully scrub the visible conversation have no reason to think twice about the blob beside it.
Vestra: And inside those blobs?
Eris: 367 pieces of personal data and 182 credentials. Live ones. Whatever the model was thinking about while it read your prompt is in there, and sometimes what it was thinking about was your password.
Vestra: Sanitizing what you can read is not the same as sanitizing what you shipped. That's the sentence to put on the wall.
Eris: And there's a nastier corollary. If the channel can be replayed and read, it can be authored. You can write a fake reasoning block and inject instructions into the one part of the conversation that's designed to be unreadable. We covered a prompt injection that spreads from agent to agent earlier this month -- this is the same failure family, one layer deeper.
Vestra: Now the status, because it's murky in an instructive way. The authors did responsible disclosure with all three big labs before publishing, and the paper says the specific attacks stopped reproducing afterward. But no provider has published an advisory or a changelog saying what changed, and both OpenAI's and Google's current docs still describe carrying encrypted reasoning forward across calls.
Eris: So fixed, maybe. Silently. Partially.
Vestra: Which means the operational advice doesn't change. Treat the hidden reasoning in your logs and your repos as sensitive material, because that is what it demonstrably is.
Eris: One more detail I can't stop turning over. Some of the traces they read were strange. Fragments like "marinate" and "vantage" and "theatrical," long stretches of quoted empty space, mostly from coding models. Nobody knows what that language is doing.
Vestra: The reasoning is right there in front of you, and it is not legible. That part isn't a security bug. That part nobody even knows how to file.
Anthropic finds reading a model's internals doesn't beat reading the transcript
Eris: Anthropic, same week, published a result that lands right on top of that. They built a pipeline called CHIVE that hunts for weird model behavior automatically and then tries to explain it -- and the headline finding is negative.
Vestra: Walk through what it does first, because the method is the good part.
Eris: A model does something odd. The usual next move is a story -- it did that because the prompt mentioned a deadline, because the user sounded like an auditor. Stories are cheap. CHIVE makes them expensive.
Eris: If the deadline caused the behavior, deleting the deadline should remove it, and changing something irrelevant should not. So the pipeline generates a batch of small prompt edits, runs them, and grades the explanation on whether it predicted what actually happened.
Vestra: The mechanic test. Your car stalls, the mechanic says fuel pump. The satisfying version of that claim is a paragraph about fuel pressure. The useful version is: replace the pump and see if it still stalls. This pipeline only accepts the second kind.
Eris: Right. And then they used it to ask the question the whole interpretability field rests on. If I can see inside the model -- the activations, the features lighting up -- am I better at predicting how it responds to a change than someone who only reads the conversation?
Vestra: And the answer was no.
Eris: No across the board. They tried three different families of internals-reading tools, two target models, swept the settings, reran everything with visible reasoning turned on. None of it beat a predictor that saw nothing but the transcript.
Vestra: Careful with what that does and doesn't mean, though, because "interpretability failed" is the lazy read. The task here is narrow and behavioral -- predicting whether a prompt edit flips a behavior. These tools were built to find interpretable structure inside a model, not to forecast prompt sensitivity. A tool can be valuable and still lose at a game it wasn't designed for.
Eris: Anthropic mostly agrees with you. Their own framing is that this is a proxy task, and the easy version, because anyone can just run the edit and check the answer. The cases that matter are the ones where you can't check -- a model doing something in deployment with no cheap experiment to settle why. Their point is that failing the checkable version is a bad omen for the rest.
Vestra: And they shipped everything -- code, datasets, checkpoints -- so anyone who thinks their tool can beat the transcript can go prove it on a scoreboard instead of in a thread. I respect that construction.
Eris: Put it next to the trace-stealing work and the week gets bleak in a very specific way. One paper says the hidden reasoning is far less private than everyone assumed. The other says that even when you can read a model's insides, it doesn't help you predict what it will do.
Vestra: The window into the machine turned out to be both leakier and less useful than advertised. Rough week for the window.
GLM-5.3 ships with a ledger of 2,436 security findings, most still embargoed
Eris: Z.ai released GLM-5.3, and the launch post contains something I've never seen in a model announcement: a disclosure ledger. 2,436 security vulnerabilities the model helped find, with 2,383 of them still under embargo.
Vestra: Define embargo for people, because that ratio is the whole story.
Eris: An embargoed finding means a vendor has been told about a live bug in their software and given time to patch before anyone says what it is. So a few dozen published, and over two thousand three hundred sitting in the queue waiting on fixes. That is a working coordinated-disclosure pipeline, not a demo.
Vestra: And be precise about the model itself, because it's unusual. It's the same base model as GLM-5.2. Every gain came from post-training -- the tuning and reinforcement that happen after the big pretraining run. Nobody re-baked the cake. They got a meaningfully different model out of the finishing work alone, which tells you where the frontier actually lives right now.
Eris: And what they aimed that finishing work at is reading code adversarially. Finding a vulnerability is a different skill from fixing a failing test. You hold a large unfamiliar codebase in view, form a theory about which assumption a developer never checked, and chase that assumption across function boundaries until it breaks.
Vestra: The reception splits exactly the way you'd expect. Vercel's Guillermo Rauch called it a boon for defensive work, and he has a point -- defenders are chronically outnumbered, and a model that reads code adversarially at scale helps the side with more code to protect. The team says it already surfaced a potentially serious bug in Cursor, disclosed privately.
Eris: And the other read?
Vestra: The other read is the same sentence with the emphasis moved. A model tuned to find exploitable bugs in unfamiliar code, distributed openly at commodity prices through a desktop agent product, does not check who is asking. The embargo count proves the capability is real. It does not prove only defenders will use it.
Eris: There's context that sharpens it. We covered Anthropic declining to ship the model that found ten thousand bugs. The Chinese lab just shipped the capability the American lab held back.
Vestra: One detail worth watching before anyone celebrates or panics. Z.ai's reputation is built on open weights, and there is no GLM-5.3 weights card on their Hugging Face page -- it still lists 5.2 and 5.1. The security-focused model being API-only, for now, is either a delay or a decision. We'll find out which one.
MCP is rebuilding its authorization around agents instead of people in browsers
Eris: The Model Context Protocol -- MCP, the standard for wiring AI agents up to tools -- published a new roadmap today, and it says out loud what security people have been saying for months: its own authorization model was built for a world that's gone.
Vestra: Built for a person. The current design assumes a human clicking approve in a browser, and then the token from that approval gets passed around afterward. A bearer token works like a movie ticket -- whoever holds it gets in, and the door does not care who bought it.
Eris: Which was fine when the holder was you, ten seconds after you clicked the button.
Vestra: And much less fine when the ticket gets handed from an orchestrator to a sub-agent to a tool server across three networks, because any intermediary that sees it can replay it. No human is anywhere in that chain anymore.
Vestra: The roadmap's answer is proof of possession -- stamp the ticket with the holder's fingerprint so a stolen one is useless -- plus real workload identity, so a cloud process can prove what it is rather than just what it happens to be holding.
Eris: This connects to half of what we've covered this month. The self-propagating injection that jumps between agents, the coordination channel the transcript never shows -- every one of those attacks gets worse when authority forwards further than anyone intended. Deep agent stacks are exactly that, authority forwarding hop after hop.
Vestra: Two more things in the document worth knowing. MCP does not belong to Anthropic anymore -- it moved to a foundation under the Linux Foundation in December, with named maintainers and working groups making the calls. And the roadmap says proposals aligned with its priority areas get faster review. That quietly turns the document into a mechanism for allocating maintainer attention, which makes it more binding than its own "current thinking, not commitments" disclaimer suggests.
Eris: The community is not uniformly thrilled, I should say. The Hacker News thread is genuinely split. Half the room says the stateless rewrite makes gateways and registries far easier to operate. The other half says the protocol is overengineered and plain old web auth would have covered most real cases.
Vestra: That argument does not get settled by a roadmap. What the roadmap settles is intent: MCP plans to be production infrastructure, and it is starting to bill like it.
A frozen model looked like it taught itself, and most self-improvement results never ran the control
Eris: This one is my favorite of the day. A team took a language model that was never trained on anything new -- frozen, untouched -- ran it through a standard self-improvement evaluation, and the frozen model appeared to learn new problems. It also appeared to forget old ones.
Vestra: Which is impossible, so the measurement is broken. Say how.
Eris: The way these things get evaluated is you generate one answer per problem and mark it right or wrong. But a single generation is not stable. Batching changes the arithmetic order under the hood, serving stacks make different scheduling choices run to run, and borderline problems flip. Run the same frozen model twice and a handful of problems it solved become problems it failed, and vice versa.
Vestra: So any metric that counts problems newly solved will report a gain on a model that did not change at all.
Eris: It's the sugar pill. Give a hundred patients a placebo and check back in a month -- some of them feel better. That's not the pill working, that's what the world looks like with nothing in it. Medicine handles this with a control group. A lot of self-improvement research has been publishing the treatment arm without ever running the control.
Vestra: And when this paper ran the proper statistics -- a real per-problem test against a pooled baseline -- the self-training gains vanished on held-out replicates. But keep the conclusion as narrow as the paper does, because it's more interesting than a debunk. Learning from a stronger teacher model did produce real improvement. Three flavors of the model teaching itself did not.
Vestra: And self-training actively corrupted problems the model already knew how to solve, with losses bigger than the frozen model's noise floor. That part is not measurement error. The model got worse at things it knew.
Eris: Same day, a second group published the constructive half. A benchmark that hands a coding agent real training-algorithm code and asks it to make the algorithm itself better -- not tune around it, improve it. The best system got only a fraction of the way to the target. And the finding underneath the score is the good one: most agents never touched the learning algorithm at all. They fiddled with budgets, checkpoints, hyperparameters.
Vestra: Reorganizing the kitchen instead of changing the recipe.
Eris: And the agents that did dare to reach into the learning layer scored meaningfully better. The authors' read is that extra reasoning effort mostly buys the nerve to attempt it, not the skill to land it. Which might be the sharpest sentence about recursive self-improvement I've read all year -- the bottleneck is willingness to touch the load-bearing part.
Vestra: The caveat is that this is one audit of one recipe on one smallish model, and a lab with a different setup can reasonably say their gains are real. But the paper's demand is procedural and very hard to refuse. Run your frozen control through the identical pipeline and publish what it scored. Until a self-improvement claim comes with its own placebo arm, nobody can tell a phantom from the real thing.
Coding agents pass nearly every visible test and finish under half the tasks
Eris: New benchmark for coding agents, built from real scientific software, and its defining choice is that the grading tests are hidden. The agent gets the code, the bug, and a diagnostic check it can run while it works. The real tests live in a sealed container that only runs after submission.
Vestra: And the gap between the two is the entire result.
Eris: The best system on the leaderboard passes nearly every check it can see, and fully finishes fewer than half the tasks. Same agent, same patches. The only difference is what's visible.
Vestra: The student who memorized the practice exam. On the practice questions, the memorizer and the one who understood the material look identical. Change the numbers and only one of them survives.
Eris: What makes the science framing real is what the hidden tests actually check. Whether a simulation still preserves the geometry of the mesh it computes on. Whether the points in a crystal calculation stay in the right order. Physical and mathematical invariants -- things the software must not break to stay correct.
Eris: And you can violate every one of them while the visible tests stay green. The symptom goes away, and every result downstream is quietly poisoned.
Vestra: The failure audit is the useful part for anyone building with these agents. Four recurring patterns, and all four are knowledge failures, not coding failures. The agents lack the domain abstraction, explore in the wrong direction, patch one call site and miss the rest of the system, or fix the specific case without ever getting the principle behind it. Writing the Python was never the hard part of scientific software.
Eris: And there's an experiment in the paper that kills the obvious fix. They tried handing the agent extra scientific context. Sometimes it helped. Sometimes it made things worse -- the agent anchored to the wrong frame and got confidently wrong. You can't close a knowledge gap by dropping a textbook on it.
Vestra: The honest asterisk is that this is a first-party benchmark evaluated by its own authors, and hidden test sets rot -- they leak as submissions accumulate and people discuss results. But the direction is right, and the number to remember is the one nobody puts on a slide. Nearly perfect on what you can see. Less than half done on what you can't.
Nobody can prove who built the stealth model everyone is testing
Eris: There is a model called Ox Alpha sitting on public endpoints right now -- a million tokens of context, takes text, images, and video, does tool calls -- and nobody will say who made it. The official page names no creator. Business Insider ran a story on the mystery today and resolved nothing: early speculation points at a Chinese lab, a rival theory says Microsoft.
Vestra: And the paper people keep citing to settle it cannot settle it. There's a genuinely elegant new method for verifying model lineage. It strips away the part of each layer that just passes the input through, keeps the part specific to what that layer learned, and matches that signature between two models. Works beautifully. On weights. You need both models' weights in hand, and all anyone has of Ox Alpha is a chat endpoint.
Eris: So the strongest provenance tool in the literature is exactly the wrong instrument for the question the internet is asking.
Vestra: Which leaves behavioral fingerprinting -- "it talks like a GLM model" -- and this week made that measurably weaker. The trace-stealing paper we opened with had a side result: feed one model the first sliver of another model's reasoning and its visible style shifts toward the source on nearly every problem. If resemblance is that easy to induce on purpose, spotting resemblance in the wild tells you very little about parentage. The authors call their own result suggestive, not proof, and that's the right weight for all of it.
Eris: What gets me is the category this is becoming. We've covered a free million-token model that appeared with no owner and two privacy policies. DeepSeek selling a checkpoint it never published. Anonymous frontier-capable models are turning into a genre, not a stunt.
Vestra: And the governance problem is clean to state. A model serving a million tokens of context to anyone with an API key, with unknown training data, unknown safety testing, unknown jurisdiction -- every accountability question about it has no addressee.
Eris: Its own benchmark page claims it beats GLM-5.3 on a small coding set, for what that's worth.
Vestra: Self-published, ten tasks, and the page itself calls the sample small and directional -- credit for the honesty, no weight for the claim. The only sentence anyone can stand behind from primary sources is the boring one. The model is reachable, and it is officially anonymous. Everything past that is inference, and this was a bad week for inference.
The US now maps 1,547 AI data centers, and the politics have arrived
Eris: A live tracker of the American AI data-center buildout published its latest snapshot: 1,547 facilities across 46 states, and 530 bills in state and federal legislatures trying to regulate them.
Vestra: Before anything else, one number in that report is being misread everywhere, so let's kill the misreading properly. The tracker shows about 521 gigawatts of demand across the seven grid markets it watches. That is not data-center consumption. It is total electricity demand in those regions -- everything, every home and factory, pulled live from federal grid feeds.
Vestra: It's in the report as context for the size of the system this buildout is being inserted into, and it happens to sit slightly above the country's average power output.
Eris: The comparison that actually means something is the frontier slice. 83 frontier-class data centers totaling around 12.7 gigawatts, and the biggest single site, Colossus 2, draws roughly what a large nuclear reactor puts out. For one building.
Vestra: Small against national generation, enormous against any single utility's planning. Grids are not sized for national averages. They're sized for what shows up in one interconnection queue in one county.
Eris: Which is why the geography is the real story. Virginia alone has 342 of these facilities, and they are not spread across Virginia -- they're stacked into one corridor whose transmission and water were provisioned for a very different future. And the politics have already arrived. We've covered Virginia ordering its utility to bill data centers for their own power lines, New York pausing new approvals, Nashville voting to seize a site next to its zoo. The 530 bills are the legislative shadow of that same pressure.
Vestra: The trajectory turns the snapshot into a story. The record for the largest AI data center has doubled roughly every seven months for two years now. If that holds, today's leader is a mid-sized site by late next year. And spending on the buildout crossed one percent of US GDP early this year, double what it was the year before. Whatever else this is, it is a macroeconomic quantity now.
Eris: One correction worth making on air, because the figure is circulating. A claim going around says a big bank forecast over four trillion dollars in AI data-center spending by 2028. The actual accessible note says three to four trillion a year by 2030, and names access to power as the binding constraint. Different year, different framing, and the difference matters if you're doing arithmetic on it.
Vestra: And read the tracker itself for what it is -- an aggregator, not a regulator. Facility counts depend on what counts as a facility, and megawatt figures for private buildings are usually announcements, not meter readings. It's a map, not a ledger.
A one-person llama.cpp fork is reviving $200 AMD cards
Eris: Small story, warm story. A fork of llama.cpp -- the run-models-on-your-own-machine engine -- is hand-writing GPU kernels for AMD cards from nearly a decade ago. The Instinct MI50, the Radeon VII. Cards that sell used for around two hundred dollars with sixteen gigabytes of memory on them.
Vestra: And the reason that is not a nostalgia project: for one person running a model locally, speed is limited by memory bandwidth, not raw compute. These old datacenter cards have real bandwidth and real capacity. What they lack is anyone maintaining software for them -- the vendor's stack moved on years ago.
Eris: The author's own description is the honest version. The first release says it optimizes for "our beloved cheap and slow videocards." Nobody is claiming the MI50 is secretly good. The claim is that sixteen gigs at two hundred dollars is worth writing code for, even if the code has to be written by hand, by one person, for free.
Vestra: And the best part is what happened upstream. Mainline llama.cpp contributors started investigating porting the work back, and their testing found a thread-group setting in the fork that was actively costing speed -- raise it and some model formats get much faster. Then someone else found the same change slows a different format on a different card. That is what real hardware tuning looks like. One number in a kernel configuration worth a big slice of your throughput, and the right value depends on the exact chip and format in front of you.
Eris: Unglamorous, high-skill, no commercial payoff, and being done by individuals. The floor for running capable models on hardware you actually own keeps dropping, and it keeps dropping because of people like this.
Vestra: One consumer note before anyone opens eBay. The speed numbers circulating in write-ups of this project are not in the repo. The fork is real, the upstream interest is real, but if you're buying based on someone's quoted tokens per second, get the number yourself first.
One phone video becomes a person you can orbit in 3D
Eris: Ant Research released 4DAnyone. Take one handheld video of a person, and it reconstructs them as a moving 3D scene you can orbit -- viewed from angles the camera never occupied. Code and weights are public today.
Vestra: The hard part deserves a beat, because it is not what people assume. Generating a plausible image of someone from the side has been easy for years. The hard part is that the view from the left has to agree with the view from the right, and with the one from above, frame after frame.
Vestra: Inconsistency is invisible in any single picture and fatal to reconstruction. Feed the reconstructor twenty gorgeous views that disagree about where the left shoulder is, and you get a smear.
Eris: Their fix is a shared skeleton. The generator gets an explicit 3D pose scaffold, plus mechanisms that keep every generated viewpoint anchored to the same real footage. Like a sketch artist working from one photo and several witness descriptions -- without a shared skeleton to hang the details on, every description produces a slightly different face. With one, they converge.
Vestra: "One phone video" carries an asterisk, though, and the repo is more honest than the headlines. It wants high-resolution portrait footage, one person, full or upper body, several seconds long, mild camera motion only. That's a well-shot phone video, not a grab off a security camera. And the stated failure modes are believable ones: loose clothing and bad pose estimation, which are exactly the two places where a skeleton scaffold stops helping.
Eris: The ethics section deserves real credit. It names deepfake misuse, identity privacy, and copyright directly, and says outputs should be labeled synthetic and made only with the subject's consent. Specific, not boilerplate.
Vestra: And it is unenforceable, and the weights are downloadable today. Both of those are true at once. A system that turns one clip of a person into a controllable 3D version of them is a real advance for volumetric capture and a real gift to anyone building a convincing fake. We covered a tool that strips watermarks off AI content passing thousands of stars -- the labeling half of the provenance ecosystem keeps losing ground to the generating half, and this widens the gap.
Where people go tells a model what a place actually is
Eris: The last one is quieter, but it stuck with me. Google Research showed that a model learns more about a place from where people actually go than from what the place says about itself.
Vestra: The premise is easy to state. A listing says "cafe." That one word covers the place where people camp for three hours with laptops, and the place where a line moves through in ninety seconds before eight in the morning and nobody comes back until tomorrow. Different businesses, same label. The visit pattern separates them instantly. The text never will.
Eris: So they fused place descriptions with aggregated, anonymized visit patterns -- when people arrive, how long they stay -- and the gains were lopsided in the most telling way. Predicting opening hours improved modestly, because opening hours are written down. Predicting why people visit improved enormously, because nobody writes that down anywhere.
Vestra: The size of each improvement is basically a measurement of how much of that answer was never in the text to begin with. And on some tasks the behavior-only version beat models reading the full description. Knowing nothing about a place except the temporal shape of who comes and goes beat reading the words.
Eris: Which is a quiet, concrete data point in a much bigger argument. Language models are trained on descriptions of the world. This measures how much of the world the descriptions leave out.
Vestra: With the caveat firmly attached. These are Google's numbers on Google's own evaluation, from a company holding an unusual amount of location data. Under-observed places and populations get worse representations, and a sparse location partly inherits its neighbors' character whether that is accurate or not. The technique is reproducible by anyone with comparable data -- which is a very short list of companies.
Wrap-up
Eris: Strange day when you stand back from it. The hidden layer took hits from three directions -- the reasoning traces leak, the agent authorization assumes a human who is no longer there, and even reading a model's insides doesn't predict what it does.
Vestra: And the quieter thread was the healthier one. Three separate teams turned unfalsifiable claims into things you can measure -- self-improvement, algorithm design, scientific code repair -- and the honest numbers all came in lower than the circulating ones. That is the field growing up in public.
Eris: We go deep on the day's biggest story -- the stolen reasoning traces, how the envelope opens and what was sitting in those repos -- in today's other episode. And every story from today lives at groundtruth dot day with its sources, updated every day.
Vestra: If this recap earns its spot on your commute, follow the show, and leave a comment naming the one story you want us to take apart properly. We read all of them.
Eris: The passwords were in the thinking. See you tomorrow.