dropout

Regularization: why deliberately handicapping a model makes it better Lesson

Regularization is the family of techniques that make a model perform worse on its training data in order to make it perform better on data it has never seen, with dropout and weight decay the two that shaped modern deep learning.