Learn · Intermediate

Neural operators

A neural operator is a neural network that learns a mapping between functions instead of between fixed-size vectors. Where an ordinary network trained on a 64-by-64 grid produces garbage on a 256-by-256 grid, a neural operator trained at one resolution can be evaluated at another, because what it learned was the underlying transformation, not the pixels. That property is why learned weather models now run on a single consumer graphics card in minutes instead of on a supercomputer for hours, and why the same architecture family shows up in climate emulation, fusion plasma modelling and chip lithography.

The idea was formalized by Anima Anandkumar's group at Caltech together with collaborators at Purdue and NVIDIA, in Neural Operator: Learning Maps Between Function Spaces. A parallel line from George Karniadakis's group at Brown, DeepONet, arrived at operator learning from the universal approximation theorem for operators. They differ in construction and agree on the goal.

The problem, stated plainly

Enormous amounts of science are governed by partial differential equations: how heat spreads through a metal plate, how air moves over a wing, how plasma churns inside a tokamak, how the atmosphere evolves over the next six hours. Solving one numerically means chopping space into a grid, chopping time into steps, and grinding forward. It is accurate and it is expensive, and the crucial waste is that you throw the answer away. Change the initial conditions even slightly and you start from scratch.

The machine-learning framing is: rather than solving the equation, learn the solution operator -- the function that takes an initial state and returns the state later. Learn it once, apply it forever.

The obstacle is that inputs and outputs here are not numbers, they are fields. The temperature across a plate is a function defined at every point. Any real computer must sample it onto a grid, but the grid is an artifact of your budget, not a property of the physics. A model that bakes in a specific grid has learned something about your budget.

Discretization invariance

This is the technical heart of the idea. A neural operator is constructed so that it approximates a map between infinite-dimensional function spaces, and any particular grid is just a way of evaluating it. Train on coarse data, run on fine data. Train on a uniform mesh, evaluate on an irregular one.

The analogy that fits: a lookup table of square roots is tied to the numbers in the table. A square-root algorithm works on any number you hand it. Ordinary networks learn tables; operators learn algorithms.

This has an economic consequence. High-resolution simulation data is brutally expensive to generate, so you usually have plenty of coarse examples and very few fine ones. An operator can be trained mostly on the cheap data and deployed on the expensive regime -- which is why these models get away with sample counts that would be laughable in language modelling.

Why Fourier

The most influential concrete instance is the Fourier neural operator, from Li et al., 2020. Each layer does three things: transform the input field into frequency space with a fast Fourier transform, multiply the low frequencies by learned weights while discarding high ones, and transform back -- then add a local pointwise transformation and a nonlinearity.

Why this works is worth sitting with. A multiplication in frequency space is a convolution in ordinary space, and a convolution with an unrestricted kernel connects every point to every other point. So an FNO layer gets global receptive field, which a convolutional network only achieves after stacking many layers, and it gets it at the cost of a fast Fourier transform rather than the quadratic cost of attention in a transformer.

The picture: attention is a room where everyone shouts at everyone individually, and the noise grows with the square of the crowd. A Fourier layer is a room where everyone contributes to a handful of shared frequencies and then listens to the mix. Cheaper, and for waves and fluids -- phenomena that are made of frequencies -- a far more natural basis. Truncating high frequencies is not merely efficiency; it is a smoothness prior that happens to be true of most physical fields.

Where it has actually worked

FourCastNet applied adaptive Fourier neural operators to global weather at 0.25-degree resolution, reaching accuracy close to conventional numerical weather prediction while running orders of magnitude faster, and it was open-sourced permissively before comparable models from other labs. FourCastNet 3 added the geometric and probabilistic layer -- crucially, treating the Earth as a sphere rather than a flat rectangle, which is what keeps long rollouts from drifting into nonsense -- and produces 60-day forecasts in under four minutes on a single GPU.

On climate, the Allen Institute for AI's ACE2 emulator (paper) runs subseasonal-to-decadal variability while conserving dry air mass and moisture, at roughly 1,500 simulated years per day of wall clock. In fusion, Fourier neural operator surrogates for magnetohydrodynamic plasma report a six-orders-of-magnitude speedup over traditional solvers. The reference implementation is the open-source neuraloperator library.

The argument this is part of

Anandkumar's broader claim, made at length in a recent interview we covered, is that transformers are not merely inefficient here but categorically mismatched. Industrial simulation runs at roughly a thousand grid points per spatial dimension in three dimensions, plus time. Treat each grid point as a token and the context window required lands in the hundreds of billions to a trillion. Her verdict: all of the world's compute would not be enough.

The honest limits

Neural operators are surrogates. They learn from simulator output or observations, so they inherit the biases of whatever generated their training data, and they have no built-in guarantee of respecting conservation laws unless you add one. Long autoregressive rollouts can accumulate error, which is exactly why the geometry fix in FourCastNet 3 mattered so much. And they are excellent on the smooth, wave-like phenomena the Fourier basis suits, and less obviously advantaged on sharp shocks and discontinuities where high frequencies carry the physics you just truncated. Knowing which regime you are in is most of the skill.

Ground Truth. (2026, August 26). Neural operators. Ground Truth. https://groundtruth.day/learn/neural-operators.html

@misc{groundtruth:neural-operators,
  title  = {Neural operators},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/neural-operators.html}
}

Topics: neural-operators · scientific-ml · architecture · physics · weather · fourier