residual-connections

Residual Connections: The Shortcut That Made Deep Networks Possible Lesson

A residual connection is a shortcut that adds a layer's input directly to its output, so the layer only has to learn the change rather than rebuild everything from scratch — a simple trick that lets networks be hundreds of layers deep without collapsing, and the reason modern transformers can be stacked as deep as they are.