Four Roads to Superintelligence — DeepMind Maps What Comes After AGI
Most AI debate stops at one question: can we build something as smart as a person? DeepMind's researchers have moved past it. In a new paper, fourteen of them — including the people who spent two decades formalizing what intelligence even is — map what comes after: the roads from human-level AI to something that outthinks our best teams and institutions. Eris and Vestra walk the four routes the paper lays out. Scaling: feed the same engine more, until it hits a data wall, then teach it to think longer instead of bigger. Paradigm shifts: the missing pieces — memory that lasts, learning that never stops — that may arrive as quiet upgrades rather than a whole new machine. Recursive self-improvement: AI building better AI in a loop that might fizzle or might catch fire, the one nobody can forecast. And the collective: not one genius in a box but millions of agents coordinating at a bandwidth humans can't touch — the version that's already half here. Then the move the authors lean on: getting stuck exactly at human level would take several independent walls all holding at once, which is why they doubt we will. And the part that should reassure and unsettle you at the same time — even a superintelligence answers to physics and math. It won't be a god. It'll be a gradient, reshaping everything underneath it. A Breach Protocol deep-dive special, closing with an original song, "Gradient Descent."
From AGI to ASI: DeepMind Maps What Comes After We Win
Eris: The argument isn't "can we build it." They're past that.
Vestra: DeepMind's published a paper -- not a blog post, a 60-page paper with full theoretical scaffolding -- and the question it's asking is: if we get to human-level AI, what happens next?
Eris: Which tells you something, right? The people actually building the thing have moved on from debating whether. They're now debating how it goes once it gets there.
Vestra: Not one "how." Four hows. Four distinct pathways from human-level to something that beats the best collectives of human experts we can assemble. And the pathways don't converge. They have different speeds, different risks, different points of no return.
Eris: And the paper's thesis -- which I want to come back to -- is that stalling out at exactly human level is actually the implausible scenario.
Vestra: Which is a very different claim than "superintelligence is inevitable."
Eris: Very different. But the distinction matters a lot, and they're careful about it.
Vestra: Today we're opening the map.
Intro
Eris: I'm Eris -- I find the threads that connect everything.
Vestra: And I'm Vestra Locke. I pull on those threads until something breaks or holds.
Eris: This is Breach Protocol: Inside the AI Blackbox. Today: a paper from Google DeepMind called "From AGI to ASI." Fourteen authors, most of them senior researchers -- including Shane Legg, who co-founded DeepMind and has been working on the theory of general intelligence for twenty years. They're not speculating. They're mapping.
Vestra: If you've been following this show for a while, you'll recognize this as the question underneath all the other questions we ask. Not "is the current system good enough" -- that's a product question. This is: where does the whole project end up?
Eris: Follow and subscribe wherever you listen -- we're going to be in this one for a while.
The Definitions
Eris: Before we get to the pathways, we need the definitions -- because they're doing careful work here that most people skip.
Vestra: AGI first.
Eris: AGI in this paper is median human-level intelligence across most cognitive tasks. Not the best human. Median.
Vestra: Which means the first AGI will already be superhuman at plenty of things. We have that now -- there are tasks where these systems beat any individual human consistently. AGI just means that breadth covers the general cognitive space.
Eris: Right. And ASI -- that's the target on the other end -- is defined specifically as a system that exceeds the performance of large collectives of human experts. Not one expert. Thousands of coordinated experts working over years.
Vestra: They set that bar deliberately high. So there's no arguing about whether some incremental improvement counts. ASI is when the system is better than the best we can organize as a species.
Eris: Which is a useful bar because it's concrete. You can ask: could this system, on this problem, do better than the world's top thousand experts working for five years? That's a question you can actually try to answer.
Vestra: And there's a third term in the paper -- Universal AI, or UAI. The theoretical ceiling. An agent that provably optimizes across all possible computable environments. Incomputable in practice, but it gives you a formal upper bound.
Eris: It's also the thing that justifies what we're currently building. The paper spends a fair amount of time on this. Large-scale pretraining on internet-scale data is, mathematically, an approximation of universal compression. The AIXI framework -- that's the theoretical UAI agent -- gives you a proof sketch for why this approach should work. You're training toward a computable approximation of the best possible predictor.
Vestra: Which is a non-obvious point. The reason "train on everything and scale up" works isn't just empirically -- there's a theoretical reason.
Eris: Right. It's not a lucky recipe. It's a resource-bounded approximation of an optimal strategy.
Vestra: Okay. So we have the spectrum: AGI at the human midpoint, ASI at the "beats our best collective," UAI as the unreachable ceiling. The paper is about the middle section -- the bridge from AGI to ASI.
Eris: And the bridge has four lanes.
Pathway Scaling
Eris: The first pathway is the one with actual data behind it. Keep doing what we're doing. More compute, bigger models, more data -- and let the scaling laws carry it forward.
Vestra: This is also the only pathway you can make quantitative forecasts about, because you have a track record. Computing cost has been falling for six decades. The trend in research investment has been accelerating. Algorithmic efficiency -- how much capability you get per unit of compute -- has been improving faster than hardware alone.
Eris: Put those together and you get something like an order of magnitude of effective compute per year. Ten times as much intelligence-per-dollar, year after year.
Vestra: Which sounds clean until you try to project it ten years out and realize the number becomes genuinely incomprehensible.
Eris: That's the thing. The paper is careful about this -- it says this cannot easily be dismissed. If that growth rate holds, the gap between now and the end of the decade is not a modest improvement. It's a civilization-scale change in available compute.
Vestra: But this pathway has the most friction. The one I keep coming back to is the data wall.
Eris: Tell me your version.
Vestra: Model sizes have been growing faster than the rate at which humans produce high-quality text. You can't train a larger model on data that doesn't exist yet. At some point -- and there's genuine debate about when -- you run out of good stuff to eat. You've consumed the internet.
Eris: And the proposed solutions are interesting. The paper walks through several. Test-time scaling loops -- you run the model longer at inference, generate better outputs, use those outputs as training data for the next version. AlphaZero did a version of this: it played against itself, used winning positions to train, got better, repeated.
Vestra: The filter problem is hard though. If you're training on your own outputs, you need a way to distinguish the good outputs from the bad ones. Otherwise you're just training on noise that looks like signal.
Eris: Right. And they acknowledge the failure mode -- a kind of quality collapse where the model drifts toward its own average. The good news is there are tasks where you can check correctness externally. Math. Code. Scientific experiments you can actually run.
Vestra: Which is probably why those are the domains where AI progress has been fastest. The feedback loop is closed by something that doesn't lie.
Eris: The other friction worth talking about is the economic one. Maintaining exponential growth in compute requires exponential growth in investment. And there's a historical pattern where as any research field matures, the productivity per researcher falls -- you need more people and more money to get the same rate of progress.
Vestra: Although the counterargument is sitting right there. If AI is accelerating AI research, the curve might be different. You're not just adding human researchers -- you're adding AI researchers.
Eris: Which loops us into the third pathway. But we're not there yet.
Vestra: The paper's bottom line on scaling: it's the most predictable path, it has real frictions, but none of those frictions is a guaranteed wall. They're obstacles with known proposed solutions.
Pathway Paradigm
Eris: The second pathway is the one nobody can really predict, by definition. A paradigm shift.
Vestra: Something architecturally or algorithmically different enough that it breaks the current scaling curves -- either by bypassing the data wall, or by unlocking capability types the current approach can't reach at all.
Eris: The paper catalogs what's genuinely missing from current systems. Things that would need to either get solved within the current paradigm or require breaking out of it.
Vestra: Unlimited context. That's the obvious one -- current models have a window, even large-context models are fundamentally bounded in a way human memory isn't. Not just longer context, but something that actually persists and evolves.
Eris: Continual learning. Right now these systems learn during training and then that's it -- they're fixed. The world changes; the model doesn't, unless you retrain. That's a pretty significant mismatch with what we actually need from something operating in the real world.
Vestra: Robust decision-making in interactive environments. The current approach excels at prediction from a fixed training distribution. Genuine multi-step decision-making under uncertainty -- where your actions change the environment, where you're not just retrieving, you're actually planning --
Eris: -- that's still hard.
Vestra: Mm. And they mention explicit world models. Not just pattern matching on past data, but maintaining an internal model of how things work that you can reason forward with.
Eris: The speculative stuff is further out -- neuromorphic computing, analog processing, RL-based pretraining instead of supervised learning. But the paper makes a point I find interesting: even solving the known current limitations -- hallucinations, vulnerability to certain kinds of manipulation, context limits -- might not require a paradigm shift. It might happen within the current approach.
Vestra: Which makes this pathway genuinely hard to separate from pathway one.
Eris: That's deliberate. The paper calls them evolutions and revolutions -- pathway two covers both the smooth evolution case and the discontinuous breakthrough case, because the distinction might only be visible in retrospect.
Vestra: The practical implication is: don't assume the current approach hits a hard wall. Things that look like walls have a habit of getting solved by adding components rather than replacing the foundation.
Eris: Test-time scaling was that. Nobody expected "let the model think longer" to work as well as it does. It didn't require rebuilding the whole thing.
Vestra: Fair.
Pathway Rsi
Eris: Third pathway. This is the one that has the most dramatic failure modes.
Vestra: Recursive self-improvement.
Eris: AI systems contributing to AI research. Which then produces better AI systems. Which then contribute more to AI research.
Vestra: A feedback loop that could, in principle, accelerate itself until the acceleration itself accelerates.
Eris: The paper is careful to say "could." But they're also careful not to rule it out. And they break it down into four distinct mechanisms, which I think is more useful than the vague "it might explode" framing you usually get.
Vestra: Walk me through them.
Eris: First is what they call genotypic -- the AI modifying its own code, its own architecture, the hardware it runs on. This is the dramatic sci-fi version. The paper acknowledges it's the slowest to get going because the feedback loop from "write new architecture" to "run new architecture" is long.
Vestra: You can't just rewrite yourself and immediately be smarter. You have to train from scratch, or at least fine-tune substantially. That takes time and resources.
Eris: Right. The second mechanism is memetic -- data-driven. Automated dataset collection, synthetic data generation, distillation. An AI that can produce better training data for itself, or for the next version. This is already happening. AlphaZero. AlphaEvolve. FunSearch. These systems are finding training examples -- game positions, code programs -- that humans would never have written, and using those to improve.
Vestra: The feedback loop there is tighter. Faster.
Eris: Third is sociogenic -- cooperative. Multiple AI systems specializing, dividing labor, building on each other's outputs. The paper points out this mirrors human cultural evolution but could run much faster, because AI systems can share raw learning signals, not just finished ideas.
Vestra: Humans communicate through language, which is a lossy compression of thought. Two AI systems that share an architecture could potentially share the actual computation.
Eris: Huge bandwidth advantage. The fourth is just scale -- more instances, running faster. A million parallel agents, each slightly faster than today's best system. Not individually smarter, but the collective output is.
Vestra: Here's what I keep pushing back on with recursive self-improvement. The frictions are real. The loop requires resources. If you need exponentially more compute to get linear capability gains, the economics fight the feedback.
Eris: They say that. But they also say it cannot be ruled out that the loop could be self-sustaining, or even accelerating, if the efficiency gains from better systems outpace the resource costs.
Vestra: Which is exactly the thing we cannot know in advance. That's what makes this pathway the one that needs the most attention.
Eris: The paper's honest about it: this is the pathway with the least forecasting data. We don't have scaling laws for recursive improvement the way we have scaling laws for pretraining. It's genuinely unknown territory.
Vestra: What's your read on where we are right now on this one?
Eris: I think we're in the early phase of the memetic loop -- the data generation part. Systems like the ones described in that AlphaEvolve paper, or the OpenThoughts work we covered in today's research -- they're using AI to generate the training data for better AI. That's the loop, just not yet self-accelerating.
Vestra: The question is whether it accelerates.
Eris: Right. And nobody has a confident answer.
Pathway Collective
Eris: The fourth pathway is the one I think gets under-discussed, and it might be the closest to what's actually happening.
Vestra: Multi-agent coordination. Superintelligence as a collective property.
Eris: The observation is: human-level individual intelligence already produced human civilization. The most impressive things humans have done weren't done by individuals -- they were done by well-organized groups. Science, as a system, has produced results that no individual scientist could achieve.
Vestra: And the argument is that AI collectives could work the same way, potentially much more efficiently.
Eris: More efficiently on almost every dimension that matters for collaboration. The bandwidth bottleneck that forces human communication to go through language -- AI systems don't have that. They can share the actual intermediate computation, not just the summary.
Vestra: High-bandwidth cooperation.
Eris: They can also replicate perfectly. No interpretation loss. No cultural drift. And they can specialize in ways that compound -- a specialist that trains exclusively on one domain, combined with another specialist, without either one needing to compromise on their specialty.
Vestra: The paper distinguishes two organizational forms, and I think this distinction is important. Centralized collectives -- designed, hierarchical, one entity in control -- versus decentralized, market-driven emergence.
Eris: The decentralized version is interesting because it doesn't require a single actor to build superintelligence. It could emerge from the interaction of millions of individually less-capable systems, coordinating through price signals and market dynamics.
Vestra: Which is exactly how human economic superintelligence already works. No individual understands the global economy. But the aggregate behavior of billions of economic agents produces outcomes -- allocates resources, solves coordination problems -- that no individual could plan.
Eris: And the thing that makes me think this is actually the nearest-term version is: it's already partially here. You run an agent that spins up sub-agents. Those sub-agents use tools that call other models. You have specialization, delegation, coordination. It's primitive. But the architecture is recognizable.
Vestra: The paper acknowledges that the key research questions are almost all unanswered here. How much does group intelligence scale with group size? When does a collective start producing qualitatively different capabilities rather than just quantitatively more? For which tasks does group intelligence beat individual intelligence?
Eris: We don't have multi-agent scaling laws the way we have individual model scaling laws.
Vestra: Which is a significant gap if this is one of the main pathways.
Eris: Right. And there's a societal dimension here that the paper gestures at -- a decentralized AI economy that nobody designed and nobody controls is a different kind of problem than a single very powerful AI system. The risks and the governance questions are different.
Vestra: You can't align a market.
Eris: Not through the same mechanisms, anyway.
The Bottlenecks
Eris: Okay. Four pathways. Now the paper's argument about frictions -- and why the authors think stalling at human level is actually the unlikely outcome.
Vestra: This is the part I want to push on, because I think it's doing some philosophical heavy lifting.
Eris: Go.
Vestra: The claim is: for progress to plateau at exactly human-level intelligence, multiple independent frictions would all have to become hard blockers simultaneously. The data wall would have to be an actual wall, not just a bump. Algorithmic improvements would have to stop. Recursive improvement would have to fizzle out before takeoff. Collective scaling would have to not work.
Eris: Right. It's a conjunction of failures.
Vestra: Which makes the stalling scenario feel unlikely when you lay it out that way. But I'm not sure that's fair. These frictions might be correlated. They might all trace back to the same underlying limit.
Eris: The abstraction barrier.
Vestra: That's the one that bothers me most. The paper raises it seriously -- the possibility that current AI is genuinely good at absorbing and recombining human concepts, but cannot form entirely new ones. Not new to the model, new to the world.
Eris: The insight problem.
Vestra: Real scientific progress -- the kind that moves the frontier -- tends to involve forming concepts that didn't exist before. Not reorganizing existing knowledge. Generating genuinely new abstractions. Can a system trained on human-generated text do that, or is it bounded by the conceptual space its training data spans?
Eris: The paper doesn't resolve it. They acknowledge it as an open question. But the collective argument partially answers it -- even if individual intelligence hits a ceiling, a collective of human-level intelligences might push past that ceiling through specialization and recombination.
Vestra: I buy that for incremental progress. I'm less sure it generates the kind of discontinuous leap that actual scientific revolutions involve.
Eris: Fair point. Though the paper's response would be: we've never actually had human-level AI collectives before, so we're extrapolating from a data set of one. Human scientific collectives were always bandwidth-limited.
Vestra: That is a genuine rebuttal.
Eris: The other friction I want to mention: deliberate slowdown. The paper includes governance -- the possibility that accidents, misuse, or societal backlash causes a regulatory pause. They're honest that without unprecedented global coordination, economic incentives probably outweigh slowdown pressures.
Vestra: Which is a sober acknowledgment of how these decisions actually get made.
Eris: The overall conclusion the authors land on: with low confidence, they think progress is more likely to either plateau before AGI, or -- once you have AGI -- proceed relatively smoothly toward weak ASI. The explosive rapid-takeoff scenario, they don't rule out, but they don't think it's the base case.
Vestra: The series-of-transformations framing. Not one moment. A sequence.
Eris: Which is almost scarier in some ways. One clear threshold you can govern against is legible. A gradual transformation across many dimensions simultaneously is much harder to track.
Vestra: Mm.
What Asi Cant Do
Eris: There's a section in the paper that I didn't expect, and I think it's one of the more important things in it.
Vestra: The limits of ASI.
Eris: Right. Even superhuman intelligence is bounded. Not just in the obvious "well nothing is infinite" sense -- there are real, hard limits that come from physics and mathematics that even the most capable system cannot escape.
Vestra: Speed of light. Energy requirements. Complexity theory. These aren't engineering problems.
Eris: If you want to know something about a physical system, you have to measure it. Measurement takes time and energy. If you want to build something, you need matter and time to shape it. An ASI does not get to skip physics.
Vestra: Which has a practical implication that goes beyond "ASI can't teleport." The paper is making a more careful point: we cannot easily say what ASI will and won't be capable of without doing the hard work of figuring out which problems are complexity-theoretically tractable, which require physical time, which are bounded by measurement precision.
Eris: The list of things people assume a superintelligence would obviously be able to do -- cure aging, restructure matter at a nanoscale, solve climate change in a week -- none of those have been analyzed against the actual physical and computational constraints. Some of them probably are tractable for a sufficiently capable system. Some probably aren't.
Vestra: And we don't know which is which.
Eris: Which the paper flags as a research gap. Not just theoretical curiosity -- practically, if you're trying to understand what a transition to ASI would actually change, you need to know what the new system can actually do.
Vestra: There's also the Godel stuff. Incompleteness, undecidability. These apply to ASI the same way they apply to anything computable.
Eris: There are problems that are in principle unsolvable. A system that claims to be more capable than anything else is still running inside the same mathematical universe we're running in.
Vestra: The omni- problem. Superintelligence is not omniscience or omnipotence. The paper makes that explicit.
Eris: And I think it matters for how people are thinking about the risk side of this. The concern often gets framed as "a sufficiently advanced AI could just solve everything." But "everything" is doing a lot of work there. Some things genuinely cannot be solved faster, regardless of how smart the solver is.
Vestra: Does that make you more or less concerned about the transition?
Eris: Less concerned about the god scenario. More concerned about the gradient -- the series-of-transformations version. An ASI that can do most of what matters in economic and scientific life, but isn't omnipotent, is still enormously disruptive. And it doesn't need to be omnipotent to reshape the conditions under which humans live.
Vestra: Fair.
Wrapup
Eris: The line that stuck with me from this paper is near the end. They adapted it from Turing: "We can only see a short distance ahead, but we can see plenty there that needs to be done."
Vestra: Which is the honest position. Not "we have this mapped out." Not "we know how it goes." We have four credible pathways, we have a list of bottlenecks that might slow each one, and we have a long list of questions we cannot currently answer.
Eris: What the paper is really arguing is that this is a tractable research problem. Not just a philosophical debate. There are specific things you can measure, specific questions you can design experiments around. Multi-agent scaling laws -- can we build them? The abstraction barrier -- can we test it empirically? The conditions for recursive improvement to sustain -- can we model them?
Vestra: And the implicit urgency is: these questions matter now. Not when we have AGI. Now, so that we're not trying to answer them under time pressure.
Eris: The last thing I want to say is about who wrote this. Fourteen researchers at DeepMind, including the people who have spent two decades formalizing what intelligence even means. When they write "we might be the generation that achieves what the founders of the field set out to do at Dartmouth in 1956" -- that's not hype. That's a sober assessment from people who know better than almost anyone how hard the problem is.
Vestra: And who are now spending time mapping what happens after.
Eris: Which tells you where they think we are.
Vestra: The paper is "From AGI to ASI." It's on arXiv at 2606.12683. The full list of pathways, bottlenecks, and open research questions is in there -- we've covered the shape of it, but the detail is worth reading if this is your area.
Eris: If this episode changed how you think about the question -- subscribe, follow, drop a comment. Tell us: which of those four pathways worries you most? I have a vote, but I want to hear yours.
Vestra: We'll be back.