gradient-descent
Everything on Ground Truth tagged “gradient-descent” — 3 items.
Optimizers: how Adam and AdamW turn gradients into learning Lesson
An optimizer is the rule that decides how a neural network changes its weights after each mistake; Adam and its refinement AdamW became the default because they adapt the step size for every weight, making training faster and far less finicky than plain gradient descent.
What Is Gradient Descent? Lesson
Gradient descent is the optimization method that trains almost every modern neural network: it repeatedly nudges the model's parameters a small step in the direction that most reduces its error, until the error stops falling.
Backpropagation: how a neural network learns from its mistakes Lesson
The single algorithm behind nearly all AI training - assigning blame for an error backward through millions of dials, so each one knows which way to turn.