Learning From Half the Text: Concept Prediction, the MoE Repeat Tax, and Research in a Simulator
Everyone says AI is running out of training data -- but today's papers say the shortage is signal, not text. An open 8.9B model from Shanghai matched a rival's full training run on about half the data by predicting ideas, not just words; Stanford and UW show why the industry's favorite mixture-of-experts models crack first when you re-read scarce data; and an Illinois/Amazon team trains research agents in a simulator that runs only one real experiment in ten -- and comes out ahead. Plus the cold shower: a new benchmark finds frontier agents are nearly expert at spotting what's inside a model, and only half as good at actually steering it.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
Cold Open
Eris: Every AI lab on earth is hoarding text like it's oil, and a lab in Shanghai just said the quiet part: you don't need more of it. You're wasting half of what you already have.
Vestra: Wasting it how? By training on it badly?
Eris: By asking the model the wrong question while it reads. They changed the question, and their model matched a rival's entire training run on about half the text.
Vestra: That's a very big claim to hang on a loss curve.
Eris: It is, and you're going to put a dent in it. But the idea underneath is the story of the day: the data crunch might be a prediction problem, not a supply problem.
Vestra: Then start with the question they changed.
Learning From Half the Text
Eris: The paper is called NCP-ArchPreview, from Shanghai AI Lab and Shanghai Jiao Tong University, and the setup is unusually clean. They took the recipe of the Allen Institute's OLMo, a fully open model -- same architecture family, same open training data, trillions of tokens of it -- and changed essentially one thing: what the model is asked to predict while it reads.
Vestra: Which matters because every large language model today is trained on one question, repeated trillions of times: given everything so far, what is the next word? Every idea the model has about meaning, structure, the shape of an argument -- all of that has to emerge as a side effect of that one tiny question. The paper's own framing is that understanding shows up purely as an indirect byproduct.
Eris: Their bet is that you can supervise the ideas directly. So here's the question the whole paper hangs on: how do you teach a model an idea, when all you have is text?
Vestra: You build a vocabulary for ideas, the same way it already has a vocabulary for words. While the model reads, they take its internal representation of every four-word chunk and compress it into one vector. Call that a concept. Then they snap the vector to the nearest entry in a learned codebook -- a finite menu of concept-pieces the model builds for itself during training.
Vestra: And the menu construction is the clever part. One flat list of every possible concept would be hopeless, so they describe each concept using thirty-two slots, where each slot picks from a small set of options. Picture a form with thirty-two dropdown menus. The parts are tiny, but the combinations run to more distinct concepts than you could ever enumerate.
Eris: Then a middle section of the network reads the concept history and predicts the next concept, and that guess gets fed back down to help write the actual next words. So on every pass the model does two jobs at once: guess the next word, and guess the next idea.
Vestra: With one detail that keeps the result honest: the concept guess is causally shifted, so it can never peek at the words it is about to help predict. No leakage, or the whole comparison is fake.
Eris: And the headline. Trained on the exact same data, it hit the reference model's final training loss after reading about half the text. Twice the learning per token, roughly. The weights, the recipe, thirteen intermediate checkpoints -- all public, under a permissive license.
Vestra: Now for the dent you promised. Training loss is the model's score on its own homework -- how well it predicts held-back text -- and that is not the same thing as being useful. The comparison also has a thumb on the scale: their model is somewhat bigger than the reference model. Some of that speedup is just extra capacity.
Eris: They ran that control, though. Against a plain baseline scaled up to the same size, the concept model roughly keeps pace while spending meaningfully less compute -- their component ablations show the concept module, the routing connections, and the concept loss each contributing a piece. The free lunch shrinks under scrutiny, but it doesn't vanish.
Vestra: Agreed -- it shrinks to a real but modest discount. And here's the part I find most interesting, because it's the honest part of the paper. After the first big training stage, the concept model was ahead on actual tasks too, clearly ahead, with the largest jump on grade-school math word problems. At that point in the read, my money was on the gap widening through the final polish stage.
Eris: Mine too. And we'd both have lost. After the mid-training stage, the lead on real tasks shrinks to almost nothing, and coding actually gets worse. They even report three of their own training variants where the loss kept improving while the task scores kept dropping.
Vestra: Which is a finding in its own right. Lower loss stopped meaning better model. That decoupling is quietly one of the most important open problems in the field, and this paper documents it instead of burying it.
Eris: One more result before we leave it, because it's the one a practitioner can use. That concept vocabulary -- the menus -- is only about seventeen million parameters inside a nine-billion-parameter model. Freeze everything else and retune just the menus on math data, and math improves more than the standard cheap fine-tuning trick manages, while general ability holds steady instead of eroding. Twice the training throughput of full fine-tuning, a third of the memory.
Vestra: A tiny steering wheel bolted to the whole model. That part I would use tomorrow.
Eris: So why did it work? Why did this thing learn as much from half the text?
Vestra: Because every four words carried a second, harder lesson. Forget the dropdown menus -- the rule is: add a prediction target at the scale of ideas, not just words, and every token you read teaches you twice. The open question is whether teaching-twice on loss ever fully cashes out as capability.
The Repeat Tax on Mixture-of-Experts
Eris: Paper two is the warning label. Stanford and the University of Washington -- Leskovec, Liang, Zettlemoyer, names you know -- asking the question everyone quietly stopped asking out loud: when you run out of fresh text and start re-reading the same data, what breaks first?
Vestra: And nobody asks because everybody already does it. Re-reading is standard practice. The classic GPT-3 run went over its encyclopedia data more than three times while seeing its web crawl less than once. Scarce, high-quality domains get repeated. That is simply how training mixes are built.
Eris: So the sharper question is whether architecture matters. The industry has largely moved to mixture-of-experts models -- and before we go on, say what that is in one breath.
Vestra: One breath. Instead of one big network that every word passes through, you have a crowd of smaller specialist networks, the experts, and a little router that picks a couple of them for each word. Most of the model sits idle on any given token, which is why these things run cheap at enormous total size.
Eris: Now make your prediction, because I'll confess mine. Repeated data: does the sparse expert model or the plain dense model hold up better? My money was on sparse. Fewer parameters touch each token, so I figured fewer chances to memorize it.
Vestra: Then you lose, and this time I collect, because my money was on dense -- though I admit I expected the margin to be small. It is not small. Across every scale they tested, dense models shrug off data repeated several times over. The expert models start degrading at about half that much repetition, and it accelerates from there. Push the repetition far enough and the expert models don't just lose their usual edge -- they end up dramatically worse than the dense models they normally beat.
Eris: And the pattern held everywhere they looked. Web text, code, scientific papers, encyclopedia articles -- four very different domains, near-identical curves. Quality filtering barely moves it. Quadrupling the data budget barely moves it. This is not a quirk of one dataset.
Vestra: Here's what actually explains it, because the headline correlation is total parameters, not active ones. The more total experts you bolt on, the earlier and harder the overfitting, even though the compute spent per token never changes. And the mechanism is lovely, in a grim way. They watched the router across training. At the start its choices are random. Within roughly the first tenth of training it has mostly made up its mind, and for the second half its decisions are essentially frozen -- the same kinds of tokens go to the same experts, forever.
Vestra: So consider what a single expert actually experiences. The dataset was already small enough to need repeating. Then a frozen router hands that expert only its little shard of the data, over and over. Every specialist is cramming a pamphlet, while the dense generalist at least reads the whole small library. Specialization on top of repetition is memorization squared.
Eris: And they can measure the cramming directly, can't they? The knockout test.
Vestra: They silence one expert at a time and watch how much the model suffers. With more repetition, losing any single expert hurts more and more -- the experts have specialized so hard that the redundancy is gone. Then the elegant control: dropout, the old trick of randomly muting parts of the network during training, largely rescues the expert models. With strong enough masking they beat dense models even at extreme repetition. And it does that without unfreezing the router at all -- it just stops each expert from over-committing. Meanwhile the remedies people would usually reach for, weight decay and gradient clipping, do nothing here. Measurably nothing.
Eris: Stitch it to paper one for me. The Shanghai answer to the data crunch was to extract more signal from each token. This paper is the other side of the ledger: if your plan is simply to re-read the tokens you have, your architecture choice puts you in a tax bracket. And the architecture the whole industry just standardized on sits in the high one.
Vestra: With a usable escape clause. Mix the repeated scarce domain into a larger pile of fresh, semantically similar data and the fresh data acts as a regularizer -- their repeated academic text degraded far more gently when it was diluted into a fresh web crawl. Repetition inside diversity is much safer than repetition alone.
Eris: So, the closing question comes back to you: why do sparse models crack first on repeated data?
Vestra: Because the router freezes early, so each expert re-reads a tiny fixed shard of an already-repeated dataset. Strip away the office metaphor and the rule reads: overfitting tracks total parameters per unique token. Sparsity buys you cheap compute, and it pays for it in data hunger.
Research in a Simulator
Eris: Paper three moves the scarcity up a level. The first two were about training models on text. This one, from Illinois and Amazon, is about training agents to do research -- and the moment the bottleneck stops being data at all.
Vestra: The agents here are machine-learning research agents. Hand one a task -- here's a dataset, build the best model you can -- and it writes code, runs the experiment, reads the result, and tries again. To improve that agent with reinforcement learning you have to grade thousands upon thousands of its attempts, and grading means actually executing the code. A real sandbox, a real GPU, real hours.
Eris: And there's an asymmetry in there I'd never seen stated so plainly. The generation side -- the agent writing its attempts -- batches beautifully; a thousand attempts share the same hardware almost for free. The execution side doesn't batch at all. Every attempt needs its own sandbox and its own machine time. Scale the training up and the experiments become the wall, not the model.
Vestra: Hence their move: stop running most of the experiments. Predict them instead.
Eris: A world model. It's the same underlying model as the agent, prompted for a different job: read the agent's solution and predict what score the experiment would get if you actually ran it. That predicted score becomes the training reward. Suddenly rewards batch like everything else, and the wall is gone.
Vestra: At which point every listener with a science bone should be twitching, because a model grading imagined experiments is a machine for hallucinating success. My prediction, reading that setup, was collapse -- the agent learns to please the grader rather than the task.
Eris: The paper half agrees with you. They ran that arm. Trained purely on predicted rewards, the agent lands clearly below one trained on real execution. Prediction alone is not good enough. The entire contribution is what they wrap around it.
Vestra: Two corrections, and both are old-school statistics rather than deep learning. For one, they never go fully synthetic. About one attempt in ten still gets truly executed -- they call it the anchor stream. Comparing predicted scores against real scores on those anchors tells you exactly how the predictor lies, so they fit a running calibration curve that remaps every predicted score onto the real scale. That kills the systematic bias, and keeps killing it as the bias drifts over training.
Vestra: The other correction handles the noise. Real-execution gradients are scarce but trustworthy; predicted ones are abundant but jittery. Rather than averaging them naively, they weight each stream by the inverse of its variance -- the textbook way to fuse two measurements of different quality. And the weight is not a knob anyone tunes. It's measured live, from how wrong the predictor currently is on the anchors.
Eris: Which produces my favorite property in the whole paper: it fails gracefully. If a task is truly unpredictable -- the outcome hinges on something no reading of the code reveals -- the measured error stays large, the weighting quietly shoves training back onto the real experiments, and you've simply reinvented ordinary training. The system audits its own simulator, continuously.
Vestra: The results, with the seasoning to follow. Training cost drops to about a third of the real-execution baseline, and the scores don't just survive the discount -- on the held-out tasks they come out ahead of full-price training. Their small trained agents also end up beating open-weight agents more than ten times their size.
Eris: A flight simulator with one real flight in ten, and an instructor who re-trues the simulator after every real landing. And the recipe traveled: applied to a robot-arm policy, where real rollouts are the expensive part, same shape of result.
Vestra: Now the promised seasoning. The tasks are built from old public machine-learning competitions on Kaggle, the data-science contest site, and the paper itself flags that those competitions and their winning solutions predate the models' training data. Some of what the world model predicts, it may in some sense remember. The recipe is the contribution; hold the exact margins loosely.
Eris: Fair. So -- can you train a scientist in a simulator?
Vestra: You can, provided one lesson in ten stays real and you keep re-truing the simulator against those. Strip away the flight school and the rule reads: when the grader is the expensive part, learn a cheap grader, keep a thin stream of ground truth, and correct its bias and its noise online. Expect that pattern everywhere.
Can Find, Can't Steer
Eris: Last paper, and it's the cold shower for everything we just said. If agents are starting to run the research loop, someone should check whether they can do the one kind of research that keeps the rest honest: looking inside a model and working out what it actually learned.
Vestra: Interpretability -- the audit function. This benchmark comes out of the Chinese Academy of Sciences, and the task is beautifully concrete. Modern interpretability leans on a tool called a sparse autoencoder. Think of it as a prism that splits a model's tangled internal activity into over a hundred thousand separate features, and when the tool works, each feature means one thing: this one fires on Portuguese, this one on cats, this one on clinical symptoms.
Eris: The agent's job is to be the scientist. Given a concept -- find me the feature for Portuguese somewhere in that dictionary of a hundred thousand -- it has to write its own probe texts, run contrastive experiments, and hand back the single feature it believes is the real one. Ten frontier agents, twenty concepts, judged against features curated by human experts.
Vestra: And the outcome splits cleanly down the middle of the scientific method. On correlation -- finding a feature that fires on the concept and stays silent on lookalikes -- the best agents come within a hair of the human experts. Some of the experimental design is honestly good science. In the cat task, one agent found a strong candidate, got suspicious, and wrote probes about copycat killers and catalytic converters. The feature fired on both. It had found the letters c-a-t, not the animal, and the agent correctly threw it away.
Eris: A machine inventing counterexamples to debunk its own favorite hypothesis. That's the real thing.
Vestra: On causation, the picture inverts. The second half of every task is steering: push on your discovered feature during generation and make the model actually produce Portuguese, or cat-talk, or symptom language. There, the best agents get roughly half the effect the expert-chosen features get. And the failure modes are painfully human. One top-tier agent wrote perfectly matched probes in Portuguese, Spanish, and English -- then submitted a feature that fired harder on its English control than on the Portuguese. The experiment was right; the reading of it was wrong.
Vestra: Another agent built careful diagnostic controls for clinical symptoms, watched its favorite feature light up on a patient record containing no symptoms whatsoever, and talked itself out of its own evidence. It had found hospital-paperwork formatting, not medicine, and it shipped the answer anyway.
Eris: So the benchmark's one-line verdict may be the sharpest sentence of the day: finding a correlated, selective feature does not readily translate into finding a causally potent lever. Connect that back to the simulator paper -- we can now train agents to run experiments at scale, but reading the instruments, deciding what an experiment actually showed, is still where they fall over.
Vestra: And that matters well beyond interpretability, because this auditing skill is exactly what everyone is counting on for oversight as AI development gets more automated. The agents can already produce the lab notebook. Whether they can be trusted to draw conclusions from it is a different and currently open question.
Eris: Close it out, then. Can the agents do the science?
Vestra: They can run it; they can't yet reliably read it. Passing the correlation half while flunking the causation half means their discoveries look right and don't move the world. Until that steering gap closes, keep a human between the lab notebook and the conclusions.
Wrap-Up
Eris: So, back to where we started: is the data actually running out?
Vestra: The text is. The signal isn't. The Shanghai model pulled twice the learning from each token by predicting ideas alongside words. The Stanford crew showed that if you just re-read your data instead, sparse expert models pay a tax that dense ones don't, unless you regularize hard. And the simulator paper moved the same scarcity up a level: when experiments are the bottleneck, predict most of them and verify one in ten.
Eris: With the audit paper as the asterisk over the whole program -- agents can run that loop, but at the causal step they still misread their own instruments. If you repeat one thing to a colleague tomorrow, make it this: when the data runs out, don't hunt for more data first. Change what you predict.
Vestra: The rest of today's news -- the RubyGems report, the pacing essay, all of it -- is in the AI News Today brief, which is its own episode.
Eris: And every story we touched is on our news site, Ground Truth -- that's groundtruth dot day -- every day. If this one earned it, follow the show and leave us a comment with your bet: does the data wall fall to better prediction targets, or does somebody just buy more text? I genuinely want to know which way you'd put money.
Vestra: Half the text, twice the lesson. See you tomorrow.