The Self-Improvement Loop Arrived as a Server Log -- the Day a Researcher Quit Over It
The day a pretraining researcher publicly quit Anthropic warning that labs are 'racing straight to self-improving superintelligence,' the most-upvoted paper in AI was a working prototype of that exact loop. NeoHorse-1 turns a routing harness's own server logs -- every prediction, dispatch, and outcome -- into the next round of training, and its small model closes most of the gap to one twice its size. Eris and Vestra crack open how the loop works, whether 'recursive' is earned after a single turn of the crank, and why Tencent's MIT-licensed AuK speech model quietly learned tricks nobody taught it.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
Cold Open
Eris: A researcher walked out of Anthropic today saying the labs are racing straight to self-improving superintelligence. The same day, the most-upvoted AI paper in the world was a working prototype of exactly that.
Vestra: And here's the twist -- it looks nothing like what people imagine. No model rewriting its own code at three in the morning.
Eris: Then what does the first draft of self-improvement actually look like?
Vestra: A server log. The most boring object in computing.
Eris: The system watches which of its models handled your request, whether it went well, and trains the next round on that.
Vestra: Which is either mundane or the start of something -- and the paper can't tell you which yet.
Eris: That tension is the whole episode. Let's open the log.
The Self-Improvement Loop Is a Server Log
Eris: The question that got this paper to the top of the charts: how does an AI system find out what it's bad at -- without a human sitting down to write the test?
Vestra: Because that's the missing piece in every self-improvement story. Learning is easy once you know what to learn. Knowing is the hard part.
Eris: And the NeoHorse team's answer is almost insultingly simple. You already have the test. It's your traffic.
Vestra: Let me build the machine, because the machinery is the paper. This isn't one model. It's a pool of different-sized models sitting behind a router. Every request that comes in, the router reads it and predicts how much capability it needs -- a quick bounded question, a general one, a multi-step job with tools, or something that needs the biggest model available. Four tiers. Then it dispatches.
Vestra: And it writes down three things. What it predicted, which tier actually got the job, and everything that happened next. Every single turn, forever.
Eris: The prediction is the trick, right? That's the part I had to sit with for a minute.
Vestra: The prediction is everything. Picture a taxi dispatcher who, before every job, writes in a ledger: "this looks like an easy airport run, sending the new driver." Then the job either goes fine or the driver calls in lost. After a month, that ledger isn't a record of trips anymore. It's a map of exactly which jobs the fleet can handle and which it can't -- and nobody had to design an exam. The exam wrote itself.
Eris: So ordinary serving traffic arrives pre-labelled. The system committed to a guess, and reality graded the guess.
Vestra: Then they get careful, which I'll give them real credit for. The raw logs go through a gauntlet before anything is trained on. A structural check first -- is this interaction complete, does every tool call have its result, did it end properly. Then a semantic pass that judges the execution on six separate dimensions: did it reach the goal, follow the instructions, use tools sensibly, stay consistent with its own evidence, recover from errors, and finish cleanly. Anything ambiguous gets quarantined rather than guessed at.
Eris: And what survives becomes training data -- but not tidy question-answer pairs. Whole working traces. The reasoning, the tool calls, the surrounding context, kept together in one piece.
Vestra: Which matters because an agent's job was never producing a final answer. The job is the sequence -- think, act, read the result, adjust. Train on the answer alone and you've thrown away the actual skill.
Eris: Right, and then the routing signal gets spent a second time.
Vestra: Twice more, actually. The predicted-demand scores order the training itself -- three stages, starting on easier interactions and shifting toward harder ones. A curriculum, where the difficulty labels came free from the router. Then the same progression drives a second phase where the small model generates its own responses and a bigger teacher grades them token by token. The student practices on its own mistakes, in order of difficulty, on a schedule written by the traffic.
Eris: And then it wraps around. They evaluate the new model, see where it's still weak, and that reshapes the recipe of the next training mix. What the system learns to do shapes what it learns from next. That's their phrase, and it's the best sentence in the paper.
Vestra: Now, before results -- commit to a guess. They post-train their small model this way, four billion parameters, and their pool includes a base model more than twice that size. How much of the gap between small and big does this training close?
Eris: Small models are stubborn. I'll say a quarter of the gap. Maybe a third on tool use, since that's where the logs are richest.
Vestra: Most of it. Across a broad suite -- agent tasks, tool use, coding, instruction following -- the trained small model closes most of the aggregate distance to the untrained model more than double its size. On several of the agent tests it matches or beats it outright.
Eris: Most of the way to doubling your parameter count, without the parameters. That's a commercial result whatever you think of the framing.
Vestra: And the detail that moved me off "more data helps" was the control. Same recipe, same curriculum, same budget -- but fed a large public dataset of synthetic agent interactions instead of their own harness logs. The harness data wins everywhere, by the widest margins exactly where you'd predict: multi-turn tool work, the messy interactive stuff. The logs aren't just cheap. They're better, because they were generated under the same conditions the model will face when deployed.
Eris: There's a trace in the paper I keep thinking about. One test scenario drops the model into a data-analysis task, and the library it reaches for isn't installed. The small model just flails -- tries to install it, tries to hand-roll a parser, patches scripts, digs itself deeper, never delivers the report.
Vestra: While the larger trained model recognizes the approach is dead, abandons it, redoes the job with the plain standard tools that are available, and finishes -- using about a third of the model calls and roughly a sixth of the tokens. That's the behavior all those scores are proxying for: knowing when to stop digging.
Eris: Forget the taxi ledger for a second. What's the rule underneath it?
Vestra: Every prediction a system is forced to make about its own workload is a free label. If your infrastructure commits to guesses and observes outcomes, evaluation stops being something you build and becomes something you harvest.
Eris: So -- one more time -- how does the system find out what it's bad at?
Vestra: By betting on itself in writing, on every request, and then reading the ledger. The router's guess is the question, the outcome is the answer sheet, and the gap between them is next month's curriculum.
Is "Recursive" Earned?
Eris: We have to deal with the word in the title, because the word is why this paper and a resignation letter collided on the same day. "Towards recursive self-improvement." How much weight is "towards" carrying?
Vestra: Nearly all of it, and the authors mostly admit that. Start with what the loop is not. Nothing modifies itself during use. No weights change while the system is running. The loop is offline: collect logs for a while, filter them, train a new checkpoint, evaluate it, put it back in the pool. People run the pipeline. People decide when to turn the crank.
Vestra: And they've turned it roughly once. Everything we just covered is a single pass of the evaluate-select-update cycle. The actual claim of "recursive" -- that gains compound, that generation two improves faster because generation one got better -- is precisely what a single pass cannot show. They say so themselves: whether the gains can keep accumulating across iterations remains to be tested.
Eris: The compounding is the entire question, though. One pass of "we trained on good logs and got better" is post-training. Every lab does that on Tuesdays.
Vestra: And there's a structural reason to doubt the second turn. The system trains on traffic its own router selected -- a diet chosen by the thing being fed. Get better at the requests you already see, and your logs concentrate even harder on those requests. That's the classic way a self-training loop narrows instead of climbs. The paper gestures at preserving broad coverage in the mixture, but a mechanism that guarantees it isn't there yet.
Eris: Worth saying who "they" is, too. Not a frontier-lab flag-planting exercise -- the team is fronted by a startup called TokenRhythm, with an inference company and several university groups behind it. Serious names on the list, none of the giant labs. Which cuts both ways: less compute to brute-force a result, and also less incentive to undersell it.
Vestra: Now say the part you've been holding back, about Tuesday.
Eris: The same day this hit number one, a pretraining researcher publicly quit -- three years across OpenAI and Anthropic -- saying, quote, they are "racing straight to self-improving superintelligence and gambling with our lives." Hours later Anthropic's own alignment science lead replied, agreeing, and added that the company does not yet have a plan to solve alignment for superintelligence and is not clearly on track to get one. His words, posted while still employed there, still running that team.
Vestra: So the loudest warning of the day and the loudest paper of the day were the same subject, approached from opposite ends. Neither one cites the other.
Eris: And I want to hold both thoughts without flinching. Reading this paper and panicking is wrong -- it's a slow, supervised, human-cranked flywheel that has turned once. Reading it and relaxing is also wrong, because the resignation wasn't about this paper. It was about the direction, and the direction now has a published, reproducible first step with its code sitting on GitHub.
Vestra: For fairness -- the researcher who quit wasn't uniformly grim. He also said coordination between labs is becoming more viable, that warning shots have made pacing agreements more plausible than they used to be. That detail travelled a great deal less far than the scary quote.
Eris: It always does. So, verdict on the word. Earned or not?
Vestra: Not yet. What's earned is "repeatable." They built the machinery that makes the second turn of the crank cheap -- the logs keep flowing, the pipeline keeps filtering, the next mixture writes itself. Whether turn two beats turn one, that result does not exist anywhere. The day it does, in either direction, is a day worth marking on the calendar.
One Model Replaces the Studio Rack
Eris: Second paper, and this one Tencent gave away outright -- MIT license, weights on the hub, a speech model called AuK. The puzzle that makes it worth your commute: they trained one model to do five jobs whose definitions of success contradict each other.
Vestra: Contradict is the right word. Cloning a voice means creating speech that never existed. Editing one word in a recording means changing almost nothing. Denoising means deleting parts of the signal. Separating two speakers means keeping one and throwing the other away. Those aren't variations on a task -- they're mutually exclusive ideas of what a good output is.
Eris: Then how does one model hold all five in its head at once?
Vestra: You move the contradiction out of the model and into the sentence. Every task becomes the same shape: a plain-language instruction, maybe some input audio, and a waveform out. "Say this in her voice." "Take out the second speaker." "Same line, but wearier." The instruction carries the definition of success, so the model never needs a fixed one.
Eris: A recording studio used to be a rack of boxes -- one compresses, one denoises, one pitch-shifts -- and you had to know which box, in which order, with which knobs. This fires the rack and hires an engineer who takes requests in English.
Vestra: And what makes it work isn't architectural cleverness. It's manufacturing. The training set is on the order of three billion instruction-audio pairs -- nearly two million hours of audio, a couple of centuries of nonstop sound. Almost none of it occurs in nature. You cannot scrape "the same voice, angry" paired with "the same voice, calm."
Vestra: So they built the pairs. Pitch-shifted and time-stretched copies of real speech for the acoustic edits. Voice systems performing the same sentence in eight different emotions. Recordings with one word masked out and re-infilled for the content edits. Simulated rooms, telephone filters, even underwater coloration layered over clean speech, so the model learns to undo each corruption on request.
Eris: The supervision is synthetic, and the capability comes out real.
Vestra: Constructed rather than fake -- every pair is a true example of the transformation, even though no human ever spoke both halves. And that's the pattern worth keeping from this paper: when paired data doesn't exist in nature, build the transformation as a factory and teach the model its inverse.
Eris: Now the section that made me sit up -- the discovery notes at the end. Do the accent one.
Vestra: Your guess first, that's the rule. Their accent-reduction training data is Chinese only. Thirteen regional dialect and accent categories, nothing else. Someone hands the finished model English -- English with a strong Indian or Japanese accent -- and asks it to soften the accent. What happens?
Eris: It should fail politely. Accent reduction learned on Chinese dialects is a mapping between Chinese sound systems. I'd expect it to do nothing, or mangle the speaker trying.
Vestra: It softens the accent. In a language the task was never trained on, while keeping the speaker recognizably themselves. Same story with whispering -- the model only ever saw whisper as an editing operation, normal recording in, whispered version out. The finished model can whisper from a cold text prompt. No input audio at all.
Eris: So the transformation came loose from the language and the task it was taught through. "Accent" and "whisper" ended up living somewhere independent of "Chinese" and "editing." That's the flavor of generalization text models showed us years ago, now arriving in audio.
Vestra: With the asterisk the authors themselves attach: those are qualitative observations, not benchmarked claims. And one more limit, tucked near the end. Free-form requests still mostly pass through a rewriting layer -- a front end that figures out which task you meant and reformats your sentence into the phrasing the model was trained on. Ask in your own sloppy words and you often miss the model's ceiling. Possessing a capability and invoking it reliably through unconstrained language are still two different things.
Eris: The gap between what it can do and what you can ask for. Every generative field keeps rediscovering that one.
Vestra: Which leaves the license, and that's the actual news event. MIT. No use restrictions, no non-compete clauses, commercial use welcome. Open speech models usually ship under bespoke community licenses with strings attached, and the obvious applications here -- dubbing, accessibility, audio production -- are overwhelmingly commercial. The license may matter more than any quality edge.
Eris: And as two people who talk for a living, the other edge is personal. This is also a capable voice cloner anyone can download, and the report describes no watermark, no provenance signal in the output. The friction that used to make voice impersonation hard keeps dropping, and this release removes another chunk of it.
Vestra: Both edges sharp, shipped anyway, and now simply in the world. So bring it home -- what made this model general, the architecture or the interface?
Eris: The interface. Once every audio operation is a sentence, the boundaries between tasks dissolve, and the model finds structure underneath that nobody explicitly taught it. The rack of boxes was hiding the fact that it was all one job.
Wrap-Up
Eris: End of the day, then. What does the first draft of self-improvement actually look like?
Vestra: A logbook and a training schedule. A router that bets on its own fleet in writing, and a pipeline that turns the ledger into the next curriculum. Human-cranked, offline, turned exactly once -- and cheap to turn again, which is the part to watch.
Eris: If you repeat one thing to a colleague tomorrow, make it this: every prediction a system is forced to make about its own workload is a free training label. That's useful far below the frontier -- if your infrastructure commits to guesses and logs outcomes, you're already sitting on evaluation data you never had to build.
Vestra: And the day that idea hit number one, people at the frontier were saying out loud that there's no plan for where it ends. Both things are true at once. Hold them together.
Eris: The resignation itself, the government advisory on model copying, the git-config attack on coding agents -- the rest of today's news is in the AI News Today brief, which is its own episode. And every story we touched is on our news site, Ground Truth -- groundtruth.day -- every story from the show, every day.
Vestra: If this one earned it, follow the show and leave us a comment with your answer to the question we couldn't settle: is "recursive" earned, or what word should that paper have used instead?
Eris: Best word gets stolen and used on air. See you tomorrow.