Learn · Intermediate

Recursive self-improvement: when AI starts building AI

Most progress in AI looks like this: humans have an idea, run an experiment, look at the results, and try a better idea. The humans are the engine. Recursive self-improvement is the name for what happens if the AI becomes the engine instead. If a model gets good enough at the work of building AI, designing experiments, writing the training code, judging which research direction is most promising, then it could improve itself. And the improved version, being a little better at all of those things, could improve itself again, a little faster. Round after round, with the humans increasingly watching rather than driving.

The idea is old. Back in 1965 the mathematician I.J. Good imagined a machine clever enough to design even better machines, and pointed out that the first such machine might be "the last invention that man need ever make," because everything after it would be invented by the machines. For decades that was philosophy. What changed is that the ingredients started showing up in real systems.

The loop, step by step

Picture a workshop that builds tools. Normally a human craftsman uses the tools to make furniture. Now imagine the craftsman uses the tools to make better tools, and those better tools let him make tools that are better still. Each generation of tools shortens the time to the next. That compounding, where the output of one round becomes the input to the next, is the "recursive" part. The fear, and the hope, is that the loop could get faster as it goes, because a smarter researcher both works quicker and makes bigger leaps.

For any of this to work, the AI needs three capabilities, and they have arrived at very different speeds. It needs to act, not just talk, which is the whole story of AI agents that can run code and check results. It needs to improve through trial and outcome, which is what reward-based training provides. And, hardest of all, it needs judgment: the taste to pick which of a thousand possible experiments is worth running. The first two have come fast. Judgment is the one everybody was watching.

What we've actually seen

There are early, narrow versions of the loop. The clearest is self-play: AlphaZero taught itself chess and Go with no human games to copy, by playing against itself and using each improved version as a tougher sparring partner, a real feedback loop in a tiny world. On the theory side, Schmidhuber's Goedel Machine described a system that rewrites its own code, but only when it can prove the change is an improvement, a careful blueprint more than a running product. And Self-Taught Optimizer showed a language model writing code that improves code, including the code that does the improving, while quietly noting the catch: the underlying model itself never changed. It improved its scaffolding, not its mind.

That catch is the whole debate. There is a big difference between a model that gets better at using itself and a model that builds a genuinely smarter successor.

"Close" is not "here"

In June 2026 Anthropic put hard numbers to the question in an essay called When AI builds itself. It reported that its own model now writes most of the company's production code, that the length of task an AI can finish before needing a human has been doubling every few months, a trend independent researchers have charted in a study of long software tasks, and, most strikingly, that an internal model began choosing better next research steps than its own scientists more often than not. Judgment, the missing ingredient, was starting to fill in.

And then Anthropic said the thing worth memorizing: we are not there yet, and recursive self-improvement is not inevitable. That is the honest center of this topic. Writing lots of code under human review is not the same as autonomously designing a smarter successor. The most dramatic figures came from an unreleased model nobody outside the company can test, which is exactly why the company also proposed a way for rival labs to verify a shared slowdown before any loop runs away. We have watched a model that could have rewritten itself and held back; the gap between can and does is where the safety of the whole field currently lives.

Why it matters

Recursive self-improvement is the hinge that separates "AI is a powerful tool" from "AI is an autonomous force," because a process that improves itself is one humans steer less with every round. It is also the most over-hyped phrase in AI, routinely used to mean a model that merely got a bit better at calling its own tools. The grown-up position holds both halves at once: the trend lines are real and bending upward, the loop has not closed, and the interesting question is no longer whether the parts exist but whether the judgment to chain them together does. Watch the judgment numbers, not the code-volume ones, and watch whether anyone can reproduce them outside the lab that reported them.