deep-learning
Batch normalization: grading every layer on a curve so deep networks train faster Lesson
Batch normalization standardizes each layer's intermediate values using the current training batch's average and spread, then lets the network learn its own scale and offset; introduced in 2015, it let an image classifier reach the same accuracy in 14 times fewer training steps.
Convolutional neural networks: how machines learned to see Lesson
A convolutional neural network learns small reusable filters that slide across an image, so the same edge or texture detector works anywhere in the frame - the idea that made computer vision practical and still runs inside modern image, audio and video systems.
Residual Connections: The Shortcut That Made Deep Networks Possible Lesson
A residual connection is a shortcut that adds a layer's input directly to its output, so the layer only has to learn the change rather than rebuild everything from scratch — a simple trick that lets networks be hundreds of layers deep without collapsing, and the reason modern transformers can be stacked as deep as they are.
Grokking: When a Model Suddenly 'Gets It' Long After It Should Have Lesson
Grokking is a training phenomenon where a neural network first memorizes its training data with near-zero understanding, then -- after a long, flat plateau of continued training -- abruptly generalizes and starts solving unseen examples correctly.
GANs: the two-network duel that taught AI to imagine Lesson
A generative adversarial network trains two neural networks against each other -- a forger trying to create fake data and a detective trying to spot it -- until the forger's output becomes indistinguishable from the real thing, the breakthrough that first made AI image generation convincing.
Transformers: the engine inside almost every modern AI Lesson
The neural-network design behind GPT, Claude, and nearly every modern AI model, and the one idea, attention, that made it work.