Learn · Intermediate

Grokking: When a Model Suddenly 'Gets It' Long After It Should Have

Grokking is a strange and revealing training phenomenon: a neural network first memorizes its training data -- hitting perfect training accuracy while performing no better than chance on unseen examples -- and then, after a long, flat stretch of seemingly pointless extra training, abruptly generalizes, suddenly solving new problems it had been failing all along. The name is borrowed from science fiction, where to 'grok' something is to understand it deeply and completely. It matters because it cracks open one of the central mysteries of deep learning: how and when a model stops parroting and starts understanding.

The phenomenon was reported in 2022 by Alethea Power and colleagues at OpenAI, who trained small networks on tidy algorithmic tasks -- things like modular arithmetic, learning to compute expressions like (a + b) mod 97. Early in training the model did the obvious lazy thing: it memorized the answer table. Training accuracy shot to 100%; test accuracy stayed near zero. A normal person watching the loss curves would call it overfit and stop. But the researchers kept training -- far past the point of apparent convergence -- and then something remarkable happened. Long after memorization was complete, test accuracy suddenly climbed from near-zero to near-perfect. The model had, belatedly, discovered the actual rule.

To feel why this is surprising, contrast it with the usual story of overfitting. Normally a model that memorizes its training set generalizes worse the longer you train, not better -- it clings ever tighter to the specific examples. Grokking flips the timeline: generalization arrives late, suddenly, and long after the training loss went quiet. There are two learning phases hiding in one curve -- a fast memorization phase and a slow, delayed understanding phase -- separated by a plateau where nothing visible seems to be happening.

So what is happening on that plateau? The best answer came in 2023 from Neel Nanda and coauthors, who used mechanistic interpretability -- reverse-engineering the network's internal weights -- to watch a grokking model from the inside. They found the network was not idle during the plateau. Beneath the memorized lookup table, it was slowly, smoothly assembling a genuine general algorithm; for modular addition, it literally learned to use trigonometric identities, representing numbers as rotations around a circle (a close cousin of how rotary position encodings work in real language models). The sudden jump in test accuracy is the moment that quietly-built general circuit finally takes over from the memorized one and the memorization scaffolding gets cleaned away.

Why would the network bother building the general solution at all, once memorization already got perfect training accuracy? The leading explanation is regularization pressure, chiefly weight decay -- a training penalty that nudges the model to keep its internal numbers small. A memorized lookup table is a bulky, high-magnitude solution; the clean general algorithm is far more compact. Both fit the training data perfectly, so the memorized version wins first (it's easier to find), but weight decay keeps applying steady pressure toward the smaller solution until, eventually, the general algorithm becomes the cheaper way to be right. Grokking is that slow tug-of-war resolving in favor of understanding. This connects grokking to a deep theme in learning theory: among all solutions that fit the data, simpler ones tend to generalize better, and part of training is a search for simplicity.

It's worth being honest about the limits. Grokking was discovered on small, clean, algorithmic problems with heavy regularization; it is not a claim that if you train any model long enough it will magically 'get it,' and it's not a standard recipe you apply to production training runs. Its real value is as a microscope. Because grokking cleanly separates memorization from generalization in time, it gives researchers a controlled setting to study how generalization forms -- and even inspires 'progress measures' that can detect the general circuit developing before the accuracy jump becomes visible. That is why grokking keeps earning attention at the top conferences: a 2026 ICML honorable mention, 'To Grok Grokking,' even reproduced the effect in plain ridge regression, showing the phenomenon isn't unique to deep networks at all. Understanding grokking is understanding the difference between a model that has stored the answers and a model that has learned the rule -- the same difference that separates a system that passed the test from one that actually knows the material.

Ground Truth. (2026, July 5). Grokking: When a Model Suddenly 'Gets It' Long After It Should Have. Ground Truth. https://groundtruth.day/learn/grokking.html

@misc{groundtruth:grokking,
  title  = {Grokking: When a Model Suddenly 'Gets It' Long After It Should Have},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/grokking.html}
}

Topics: training · generalization · interpretability · deep-learning · grokking