Learn · Intermediate

Straight-through estimators: hard choices forward, useful gradients backward

A straight-through estimator lets a neural network make a hard choice in its forward computation while using a different, useful derivative during learning. It matters when rounding, selecting, or emitting a discrete symbol would otherwise give gradient-based training little or no signal about how to improve. The usual estimator is a deliberate approximation, so useful learning is possible without pretending the hard operation is smooth.

Consider a network that must output either zero or one. It computes a score x, then outputs one if x is positive and zero otherwise. During actual use, that crisp decision may be exactly what the system needs. During training, however, a small change to x usually leaves the answer unchanged. The derivative of the threshold is zero away from the switching point and undefined at the switch.

Ordinary backpropagation follows a chain of local derivatives. Multiplying by zero at that decision cuts off the learning signal to earlier computations. The system can know its final answer was bad while receiving no useful local instruction for moving the score toward a better choice.

The analogy is a thermostat with a dial connected to an on-off heater. The heater has only two states, but a technician still wants directional guidance about the dial. A straight-through rule supplies that guidance during adjustment while leaving the heater’s actual on-off behavior intact. It is a rule for learning through the switch, not an extra intermediate heater setting.

Yoshua Bengio, Nicholas Léonard, and Aaron Courville discuss this family of approaches in Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. The simplest version treats the hard operation as the identity function on the backward pass. An incoming derivative of three is passed backward as three, even though that is not the threshold’s true derivative. Other variants restrict or rescale the surrogate derivative, for example to keep updates concentrated near the switching region.

It helps to separate three choices. The forward operation determines what the model actually computes. The loss measures how desirable that result is. The backward rule determines how training changes earlier parameters. A straight-through estimator changes the third choice. It does not by itself change the loss, make a discrete result continuous, or guarantee that every update improves the actual forward computation.

A common expression makes the separation explicit. Let stop_gradient(z) return z as a value but contribute zero derivative. Then y = x + stop_gradient(round(x) - x) has the forward value round(x), because the x terms cancel numerically. Its derivative with respect to x is one, because the stopped correction contributes no derivative. The expression is a compact way to request hard values forward and an identity derivative backward.

That intentional mismatch explains an important testing detail. A finite-difference check perturbs x and measures changes in the actual rounded output. Automatic differentiation follows the substituted backward rule. Those answers can disagree by design. Developers should check that the forward values are correct and that the implemented surrogate derivative matches the intended rule; a mismatch with the true finite-difference derivative does not automatically mean the code is broken.

One established use appears in Aaron van den Oord and colleagues’ Neural Discrete Representation Learning. Its encoder chooses a nearest codebook vector, a discrete selection that is awkward to differentiate. A straight-through path carries a reconstruction-learning signal back to the encoder. The codebook and commitment terms have their own roles, explained in the existing vector quantization lesson. The estimator is the backward bridge, rather than the whole representation-learning method.

A related use occurs when training models whose forward computation rounds values to simulate low-precision arithmetic. The forward path exposes the model to rounding effects while a chosen backward rule allows updates to continue. This helps distinguish training behavior from the storage and deployment choices discussed in model quantization. Merely saving weights with fewer bits does not imply that a straight-through estimator was used to train them.

The same programming ingredients can also implement more specific gradient identities. Today’s dossier describes Jayden Teoh’s Multi-Token Prediction Done Right and its Soft-MTP implementation. It routes gradients to cancel a particular residual under a shared-output-head setup. That application should not be reduced to the claim that all straight-through gradients are exact: the paper’s exactness concerns its specified algebraic cancellation. The general hard-choice estimator remains a surrogate, and the dossier’s paper review was partial.

The main limitation is bias: the substituted derivative need not equal the derivative of the true expected objective. A useful direction can still be imperfect, unstable, or sensitive to scale and clipping. Evaluation must therefore measure the real forward task, compare appropriate baselines, and disclose the backward rule. A smooth relaxation changes the forward problem; a straight-through estimator retains the hard computation and changes the training signal. Choosing between them requires knowing which mismatch the application can tolerate.

Ground Truth. (2026, October 31). Straight-through estimators: hard choices forward, useful gradients backward. Ground Truth. https://groundtruth.day/learn/straight-through-estimators.html

@misc{groundtruth:straight-through-estimators,
  title  = {Straight-through estimators: hard choices forward, useful gradients backward},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/learn/straight-through-estimators.html}
}

Topics: training · gradients · discrete-optimization · fundamentals