News · 2026-06-19
What if a word were a rotation? A more mathematical way to build AI
This one is for the technically curious — it's more abstract than most of what we cover, but the core idea is genuinely lovely, so bear with the setup. Inside today's AI models, the things being shuffled around and combined are vectors: long lists of numbers. Almost everything a model does is some flavor of comparing and blending those lists. This paper poses a deceptively simple "what if": what if the things the model worked with weren't static lists of numbers, but operations — geometric moves like rotations and shifts?
To see why that's appealing, you need one idea: symmetry, or in the jargon, equivariance. Often you want a model whose understanding changes in step with the world. Rotate a scene by thirty degrees, and a good model's sense of what's where should rotate by thirty degrees too — not scramble into something unrelated. Normally, you have to teach a model to respect symmetries like that, usually by showing it mountains of examples until it grudgingly learns the pattern. It's expensive, the model only ever approximates the rule, and it can still break on an example unlike anything it trained on.
The paper's payoff is that if you build the model out of these geometric operations from the start, certain symmetries stop being something you train for and start being something that's simply true by construction — they fall out of the underlying algebra automatically, for free. The "turn the scene, turn the answer" property isn't learned and approximated; it's baked into the math, guaranteed, the same way a circle drawn with a compass is exactly round without anyone checking. To picture it: imagine teaching someone to read a map versus handing them a physical globe. With the map, you have to drill them on how directions warp near the poles; with the globe, the geometry is just right, inherently, with nothing to memorize. This work is reaching for the globe version of part of a model.
It helps to know this isn't an idea conjured from nowhere. There's a whole established tradition of building known symmetries directly into a model's bones — it shows up in AI for physics, chemistry, and molecules, where the laws don't care which way you've oriented your coordinates, so the model shouldn't either. What's fresh here is aiming that philosophy at the core attention machinery that powers today's language and vision models — the part almost everyone treats as plain number-crunching — and asking whether it, too, could be rebuilt on a geometric foundation.
That's a genuinely different foundation, which is what makes it noteworthy — and also why the honesty about its current state matters. The results so far are on small, toy-scale problems, and the authors are upfront that this is a proof of concept, not a finished, scaled-up method ready to challenge the models you actually use. There's a long, uncertain road between "elegant idea that works on a small example" and "approach that holds up at the size of a real system," and plenty of beautiful ideas never make that trip. New architecture proposals appear constantly — a glance at any day's trending papers will show you several — and the overwhelming majority quietly go nowhere.
So why feature an early, unproven idea at all? Because of where almost all AI progress actually comes from these days: taking the same basic design and making it bigger. Genuinely different mathematical foundations — new answers to "what is the model even made of?" — are rare, and most of them go nowhere, but the occasional one reshapes the field. Treating the building blocks as geometric operations, so that hard-won symmetries become free guarantees, is exactly the kind of from-the-ground-up rethink that's worth watching early, precisely because it isn't just "the usual thing, scaled."
The caveats here are bigger than usual and we won't soft-pedal them: toy-scale evidence, a proof-of-concept by the authors' own description, and no demonstration yet that it survives contact with real-world scale. File this under "promising and beautiful, unproven" rather than "new state of the art." But part of reading the field honestly is paying attention to the rare structural ideas while they're still small — because if one of them does grow up, it won't look like a bigger version of today's models; it'll look like a different kind of thing entirely.