generalization

Grokking: When a Model Suddenly 'Gets It' Long After It Should Have Lesson

Grokking is a training phenomenon where a neural network first memorizes its training data with near-zero understanding, then -- after a long, flat plateau of continued training -- abruptly generalizes and starts solving unseen examples correctly.