climate

The case against using transformers for physics News

Caltech's Anima Anandkumar argues that simulating the physical world at industrial resolution implies hundreds of billions to a trillion tokens of context, putting it permanently out of reach for transformers -- and that neural operators, which learn maps between functions rather than sequences, are already outrunning supercomputers on weather, climate and fusion.