Linearize the Unlinearizable — Taming Chaos with a 1931 Trick
A 1931 idea, dead for ninety years, that deep learning just revived: the Koopman operator turns a chaotic, nonlinear system into a simple linear one — and for energy-conserving systems, the dynamics become a rotation on a sphere, so conservation of energy stops being a hope and becomes pure geometry. Luna and Vestra trace the bet from the three-body problem to Hamiltonian Neural Koopman Operators, with the honest limit: it lives or dies on the system actually conserving energy. A Breach Protocol deep-dive special — closing with an original song, "Rotation on a Sphere," whose lyrics trace the whole episode.
Cold Open
Eris: Three bodies. That's all it takes.
Vestra: Three.
Eris: The sun, the earth, the moon. Three things pulling on each other by gravity — a law we've known cold for three hundred years. And there is no formula that tells you where they'll be far enough down the road. None. It doesn't exist.
Vestra: It's not that we haven't found it yet. Poincaré proved there isn't one. You want the long-term answer, you simulate, step by step, and you pray your errors don't pile up.
Eris: Two bodies, easy — ellipses, clockwork, you can write it on a napkin. Add a third and the napkin bursts into flame.
Vestra: That's the whole monster, right there. Almost nothing in the real world is two bodies. Weather, a fluid, a beating heart, a folding protein —
Eris: Nonlinear. The interesting word for "the math gives up."
Vestra: Linear things we own completely. We have a hundred years of tools that crack them open on sight. Nonlinear things? No general theory. At all. It's the embarrassing hole at the center of the subject.
Eris: So here's the paper that got me. Somebody dusts off an idea from nineteen thirty-one — older than the transistor, older than the computer — that says: you can take one of these monsters and turn it into the easy kind. The clockwork kind.
Vestra: With a catch that kept it useless for ninety years.
Eris: With a catch. And the thing that finally cracked the catch is a neural network. And then there's a twist at the end where conserving energy stops being something you hope your model does, and becomes a matter of pure shape.
Vestra: Shape.
Eris: A rotation. On a sphere. Stay with me.
Intro
Eris: This is Breach Protocol. I'm Luna — I read the papers, I chase the threads between them, I'm the one going "wait, this connects to that."
Vestra: And I'm Vestra. I'm here to slow her down and ask whether the beautiful idea actually holds up when you push on it. I explain how the machinery works, and I get suspicious when something sounds too clean.
Eris: And today is going to sound very clean. So you're going to be busy.
Vestra: I already don't trust it.
Eris: Here's the shape of the hour. There are two kinds of systems in the world. Linear ones — predictable, well-behaved, completely solved. Think a single pendulum, or a savings account with fixed interest. Boring, in the way that "I know exactly what happens next" is boring.
Vestra: And then everything else. Nonlinear. Where small causes blow up into huge effects, where the same system can be calm one minute and chaos the next. That's most of nature, and we have no general way to solve it.
Eris: So the dream — the very old dream — is to build a bridge. Take a nonlinear monster and find some magic change of perspective where it suddenly looks linear. Looks solved.
Vestra: And the claim on the table is that that bridge exists. It's called the Koopman operator, it's from nineteen thirty-one, it sat on a shelf for most of a century, and deep learning just took it down off the shelf.
Eris: Plus the part I can't stop thinking about — what happens when the system you're taming is one that conserves energy. A planet that never slows down. A molecule that never loses heat. Because for those, the bridge has a shape, and the shape does something quietly beautiful.
Vestra: Let's see if it survives me.
The 1931 Bet
Vestra: So set it up properly. Why is "linear" the promised land. What do we actually get for free if a system is linear.
Eris: Everything. If a system is linear, you can break it into simple independent pieces — pure tones, basically — and each one just grows or shrinks or spins at its own steady rate. You add them back up and you've got the future. Forever. No simulation, no creeping errors. You just know.
Vestra: And the reason nonlinear systems are hell is that you can't do that. The pieces interact. Push here, it comes out somewhere you didn't expect. The parts won't stay separate.
Eris: Right. So in nineteen thirty-one a mathematician named Koopman makes a move that sounds, at first, completely insane. He says: stop tracking the thing.
Vestra: Stop tracking the thing.
Eris: Stop tracking where the planets are. Instead, track measurements of the system. Every possible measurement. Not "where is the moon," but the whole infinite catalog of questions you could ask about the moon — its height, its height squared, its distance times the earth's speed, every combination you can dream up.
Vestra: And here's the part that does the work, because it's genuinely strange. He proves that if you track that whole infinite catalog of measurements, the way the catalog evolves over time is linear. Perfectly, exactly linear. The monster is gone.
Eris: You took a small, twisted, nonlinear problem and traded it for a gigantic, perfectly straight one. You went up in size to get down in difficulty.
Vestra: Which should set off every alarm you have. Because "gigantic" is doing a lot of lifting there.
Eris: It's infinite. The catalog of all possible measurements is infinitely long. That's the trade. The twist is gone, but now you're holding an object with infinitely many moving parts.
Vestra: So it's true and useless. The best kind of math.
Eris: For ninety years — exactly that.
The Ninety-Year Catch
Vestra: So spell out the catch, because this is where most of the romance dies.
Eris: The catch is that infinity isn't a thing you can put in a computer. To actually use Koopman's bridge, you don't need all the measurements. You need a small, finite handful — the right handful — where the system already looks linear. A special set of coordinates.
Vestra: And those special coordinates exist, in principle. The theory promises them. They have a name, they're the natural "modes" of the system. The problem is purely that nobody could find them.
Eris: For almost any real system, no. You're hunting for a needle in an infinite haystack, and the haystack is made of functions. Which coordinates make this particular monster go straight? Total mystery, case by case.
Vestra: There was a workhorse method — people would take a pile of data and basically fit the best straight-line approximation they could to the measurements they happened to have. And it works, sometimes, beautifully. Fluid flows, especially.
Eris: But it's guessing at the coordinates by hand, more or less. And here's the thing that matters for where we're going — it gives you no guarantees. It can drift. It can quietly violate the physics. You can get a model that predicts well for a while and then slowly invents energy out of nowhere.
Vestra: Which for a physicist is a cardinal sin. If your model of a swinging pendulum gradually speeds up forever, you haven't modeled a pendulum. You've modeled a fantasy.
Eris: So that's the standoff. Gorgeous idea, real bridge, and the on-ramp is hidden in an infinite fog. For ninety years you either got lucky or you went home.
Vestra: And then the thing that's good at finding needles in impossible haystacks shows up.
Eris: The thing we've spent this whole show being suspicious of.
Letting the Net Find the Coordinates
Eris: So a group around Steven Brunton and Nathan Kutz at Washington asks the obvious-in-hindsight question. Finding the magic coordinates is a search through an enormous space of possible functions. What do neural networks do for a living?
Vestra: Search enormous spaces of possible functions. That's the whole job.
Eris: So they build what's called an autoencoder. And the picture is genuinely simple. You have a network that takes your raw data — the messy nonlinear measurements — and squeezes it down into a small new set of coordinates. That's the encoder. And a second network that takes those coordinates and rebuilds the original. The decoder.
Vestra: The squeeze is the important part. You're forcing the system through a narrow gate, so the network can't just memorize — it has to find a compact, honest description. The few numbers that actually matter.
Eris: And then the trick. They add one demand on top. They say: in this new squeezed space you invented — the dynamics have to be linear. Going forward in time has to be the simple, straight, clockwork kind. That's a penalty in the training. Find me coordinates that both rebuild the data and march forward in a straight line.
Vestra: So the network is being paid to discover Koopman's mythical coordinates. The needle in the infinite haystack becomes a thing you train for overnight.
Eris: That's the unlock. The math knew the coordinates existed and couldn't find them. The network just... finds them. From data. For systems where people had no hope before.
Vestra: And to their credit they went after the genuinely nasty case — systems that don't have a tidy set of pure tones at all, where the frequencies smear into a continuous blur. That's the case classical Koopman struggled with most, and they handled it by letting a little side-network predict the frequencies on the fly.
Eris: So now the bridge has an on-ramp. Deep learning builds you the coordinates, and the monster goes straight.
Vestra: Fine. That's real, and it's clever, and I'll grant it. But notice we still haven't touched the thing you teased — energy. Nothing here stops the model from cheating physics. It's accurate. It's not principled. Yet.
Eris: No. For that you need the twist. And the twist needs us to talk about why honest models leak.
Models That Leak
Eris: So picture the simplest toy in physics. A mass on a spring, no friction. It bobs back and forth forever, trading speed for stretch and back again, and the total energy never changes. A perfect little loop.
Vestra: In the real math it traces a circle, over and over, in the right picture. Same circle, every cycle, till the end of time.
Eris: Now train an ordinary neural network to watch that and predict the next instant, and the next, and the next. What you get — and this is in the Greydanus paper from twenty nineteen — is a thing that's almost right. And "almost" is fatal. The little circle slowly spirals inward. The model's pendulum gently winds down and stops.
Vestra: Even though there's no friction. The network invented a loss that isn't there, because it only ever learned the dynamics approximately, and the tiny errors all leak the same direction. It's not modeling the spring. It's modeling a spring slowly dying.
Eris: So Greydanus and his coauthors ask: can we build a network that's physically incapable of that? And they reach back to a two-hundred-year-old reformulation of mechanics — Hamilton's — whose entire purpose is bookkeeping for conserved quantities.
Vestra: Here's the elegant bit. In Hamilton's picture there's one master function — call it the energy — and once you know it, the motion is completely determined by it, in a very specific geometric way. The system is forced to slide along paths where the energy stays fixed.
Eris: So instead of training the network to predict the motion directly, they train it to learn that one master function. The energy. And then they let the motion fall out of it, through Hamilton's recipe.
Vestra: And now conservation isn't a hope. It's built into the form. The network literally can't spiral inward, because the only motions it's allowed to express are ones that hold the energy fixed. The circle stays a circle.
Eris: Which is the whole philosophy of this corner of the field in one move. Don't make the network learn physics from a billion examples and pray. Bake the physics into the shape of the network, so breaking it isn't an option.
Vestra: It's a good move. I'll even call it beautiful. But it's done by hand, in a sense — you commit to Hamilton's formulas up front. The paper we actually came here for does the same thing from a completely different direction. The Koopman direction. And the geometry that falls out is, I have to admit, prettier.
Conservation as Geometry
Eris: Okay. The payoff. This is a group at Fudan — Zhang, Zhu, and Wei Lin — and they go back to a footnote that Koopman himself basically wrote in nineteen thirty-one and nobody leaned on hard enough.
Vestra: And the footnote is this. For a system that conserves energy — a planet, a frictionless spring, a molecule that never sheds heat — the Koopman bridge isn't just linear. It's a very special kind of linear. It's a rotation.
Eris: Say what that means, because this is the whole episode.
Vestra: Remember the squeezed coordinates from the neural turn — the compact space where the dynamics go straight. For an energy-conserving system, you can arrange those coordinates so that the data lives on the surface of a sphere. A high-dimensional sphere, but a sphere. And time, going forward, is just that sphere turning. Spinning in place.
Eris: And here's the thing that made me sit up. A rotation can't change the size of what it's rotating. Spin a globe — every city stays exactly as far from the center as it was. The radius is untouchable. It's the one thing a rotation physically cannot do.
Vestra: And in these coordinates, the radius is the energy.
Eris: The radius is the energy. So if the only motion your model is allowed to make is a rotation — a turn of the sphere — then it cannot change the energy. Not "we trained it not to." It can't. There's no spiraling inward, because spiraling inward means the radius shrank, and a rotation never shrinks the radius. Conservation stopped being a rule you enforce and became the shape of the thing.
Vestra: So what they actually do, mechanically — they take the linear operator at the heart of the Koopman model, the matrix that steps you forward in time, and they nail it down to be a pure rotation. A mathematically perfect one. The network can learn the coordinates freely, the squeeze and the un-squeeze, but the step-forward in the middle is locked to be a turn of the sphere.
Eris: And because it's locked, two nice things happen. One, it's shockingly steady against noise — you can feed it filthy, jittered data and it holds the energy dead level, because it has no way not to.
Vestra: That part genuinely surprised me. Noise usually wrecks these methods. Here the structure acts like a keel. The junk in the data can't tip it over, because the only motions on the menu are honest ones.
Eris: And two — it scales. They run it on systems with hundreds, thousands of moving parts, the kind of thing where the older methods just fall over, and it stays accurate way further into the future than the models that were only trying to be accurate.
Vestra: Because — and this is the lesson again — the ones only trying to be accurate have nothing holding them to the physics. They drift. This one can't drift off the sphere. It's wearing the conservation law as a body, not a belief.
Finding Laws Nobody Told It
Eris: And then there's the bonus, which is in the title of the paper and is easy to skate past. It doesn't just preserve the energy you knew about. It can find conserved quantities you didn't.
Vestra: Unpack that, because "discovers conservation laws" is the kind of sentence that should make you ask for the receipt.
Eris: So a real physical system often has more than one thing it secretly keeps constant. Energy, sure. But maybe also a momentum, an angular momentum, some combination nobody named. These are the deep symmetries of the system — the quiet promises it keeps. Finding them, historically, took a human with a deep understanding of the equations.
Vestra: And the way this falls out here is honestly kind of lovely. Once you've insisted the dynamics are a rotation on a sphere, the geometry has consequences you didn't put in by hand. Certain directions in that space just don't move — they sit still while everything turns around them. And a direction that never changes while the system evolves is, by definition, a conserved quantity.
Eris: So the conserved laws show up as the still points of the rotation. You didn't tell the model "momentum is conserved." You told it "you're a rotation," and the conservation of momentum fell out as a thing that doesn't spin.
Vestra: Which is the part I keep turning over. They constrained the architecture to respect one piece of physics — energy, the radius — and got other pieces of physics for free, as a side effect of the geometry. That's the sign you've hit real structure and not just a trick. When you get more out than you put in.
Eris: Structure you can feel. The rotation isn't a metaphor I'm using to make it nice on a podcast. It's the actual object, and the symmetries are literally the axes it turns around.
Vestra: I'll concede the elegance fully. So now — finally — let me earn my keep and tell you everywhere this beautiful thing does not reach.
Where the Sphere Ends
Vestra: The whole reason this is so clean is the restriction. It only works because the system conserves energy. The moment it does, the bridge becomes a rotation and everything I just praised follows. But take that away —
Eris: And the sphere's gone.
Vestra: The sphere's gone. The real world has friction. Things cool down, slosh, bleed energy, leak heat into the room. An engine, a stock market, a living cell, the weather — these are open, they dissipate, they don't ride a perfect circle forever. And for those, the trick we just fell in love with doesn't apply on its face. You've built the most gorgeous possible tool for the cleanest possible case.
Eris: That's fair. Though "cleanest possible case" undersells it — it makes the conserving systems sound like a toy corner. They're not. They're the bedrock. Orbits. Molecules. The physics engines under every simulation and game and drug-discovery pipeline. A huge amount of fundamental physics lives exactly on that sphere.
Vestra: Granted. It's a big, important island. It's still an island. And there's a second thing, quieter. The on-ramp problem — finding the coordinates — never fully went away. For genuinely chaotic systems, the kind where the frequencies smear into a continuous mess, finding a small clean set of coordinates is still brutally hard. Deep learning helps. It did not abolish the catch.
Eris: No. Though the direction of travel is right there in the literature — people are already pushing the same structure-preserving idea toward systems that lose energy in a controlled way. The "bake the law into the shape" move generalizes even when this specific sphere doesn't.
Vestra: And that, for me, is the actual takeaway, more than any one result. Not "Koopman won." It's the design principle. For a decade the reflex in this field was: throw a bigger network and more data at it and hope the physics emerges. This is the opposite reflex. Constrain the architecture so the physics can't not be there. And it buys you robustness, and honesty, and it generalizes from a handful of examples instead of a mountain.
Eris: Less learning, more knowing.
Vestra: When you actually know something about your system — build it in. Don't make the machine rediscover gravity from scratch every morning.
Wrapup
Eris: So let me try to land it. The monster was nonlinearity — the three bodies, the weather, the things no formula can pin down. The old dream was a bridge to the easy, linear world. Koopman built that bridge in nineteen thirty-one and it sat unused because the on-ramp was hidden in an infinite fog.
Vestra: Deep learning cleared the fog — a network that searches for the magic coordinates where the monster goes straight. That part is general, and it's already changing how people model fluids and brains and climates.
Eris: And then the twist, for the systems that conserve energy: the bridge isn't just straight, it's a rotation. Lift the system onto a sphere, and time is just the sphere turning. Energy is the radius. And a rotation can't touch the radius — so the model physically cannot lose or invent energy. Conservation as shape.
Vestra: Which buys you a thing that holds steady in noise, scales to systems with thousands of parts, and even coughs up conserved laws nobody handed it — the still points of the rotation. With the honest limit that it lives or dies on the system actually conserving energy. Friction breaks the spell.
Eris: The part that stays with me is the age of it. A nineteen thirty-one idea — older than the things we usually call old in this field — sitting on a shelf, waiting for a tool that wouldn't exist for ninety years. Old math, new compute, and suddenly it's alive.
Vestra: And the design lesson, which travels far past this paper. If you know something true about your system, don't make the network guess it from a mountain of data. Build it into the architecture so the truth isn't optional. Less learning, more knowing.
Eris: What I'm watching: whether the same move cracks the messy, energy-losing systems — the engines and weather and living things. That's the prize.
Vestra: What I'm watching: whether anyone finds the coordinates for genuinely chaotic systems, or whether Koopman stays a gift to the well-behaved. Beautiful on the sphere. We'll see how far the sphere stretches.
Eris: Either way — a chaos tamed into a turning globe. That's a good image to leave you with. This was Breach Protocol.
Vestra: Stay suspicious. We'll see you next time.