stability

Gradient clipping: the one-line fix that keeps big models from blowing up mid-training Lesson

Exploding gradients happen when the correction signal in a neural network grows enormous on a single unlucky batch, and one huge update destroys weights that took days to learn. Gradient clipping caps the size of that update, keeping the direction and throwing away the magnitude, and it is why frontier training runs survive at all.