Learn · Intermediate

Training data attribution: which examples actually made the model do that?

Training data attribution asks a specific counterfactual question: if this training example had not been in the dataset, would the model have produced this output? Techniques that answer it, from influence functions to datamodels, are the technical foundation under copyright arguments, data valuation, and debugging. The uncomfortable finding of the last few years is that for large models trained on large datasets, the honest answer for many outputs is that no single example mattered enough to detect.

Start with why the naive approach fails. The clean way to test whether an example mattered is to remove it, retrain the model, and compare. This is called leave-one-out retraining, and it is unambiguously correct. It is also absurd at any real scale. A dataset with a million examples would require a million retrainings to build a complete attribution map, and modern datasets have billions of examples and cost millions of dollars per training run. Every technique in this field exists to approximate that answer without paying that price.

The first serious attempt came from statistics. Pang Wei Koh and Percy Liang's 2017 paper Understanding Black-box Predictions via Influence Functions adapted a classical robust-statistics tool to deep learning. The idea: instead of removing an example entirely, imagine reducing its weight in the loss by an infinitesimal amount, and use calculus to estimate how the model's parameters would shift in response. Because you are working with derivatives rather than retraining, you can compute it. The mathematics involves inverting a Hessian matrix, which for a model with billions of parameters is its own nightmare, so most of the practical work in this area has been about approximating that inversion cheaply.

The intuition is a supply chain. If a factory stops receiving one shipment of screws, does the product change? For a screw that appears in every unit, yes, obviously. For one of ten thousand interchangeable screws from redundant suppliers, the factory does not notice, and no audit of the finished product will point back at that shipment.

A second family sidesteps the Hessian entirely. TracIn works by watching training itself: every time the optimizer takes a step on a batch containing example X, the model's loss on your test output changes a little. Sum those changes across the whole run and you get a measure of how much X pushed the model toward or away from that output. It is more of a bookkeeping approach than an analytical one, and it requires having saved checkpoints during training, which not everyone does.

A third family gave up on approximating the counterfactual analytically and decided to learn it. Datamodels trains many models on many random subsets of the data, then fits a simple predictor that maps "which examples were included" to "what the model does." Surprisingly, a linear predictor works well. TRAK combined this insight with random projections to make it tractable at larger scale, and is currently the most practical option for real attribution work.

Now the part that matters more than any of the methods. All of these techniques measure the same underlying quantity, and that quantity gets smaller as datasets get bigger. When a model sees a hundred examples of some pattern, removing one changes what it learned. When it sees a hundred million, removing one changes nothing measurable, because the signal was massively redundant. This is not a flaw in the estimators; it is a property of the trained model.

Work published in 2026 by MIT researchers made this precise for image generation. Their ablation-based counterfactual method trained 24 diffusion models and measured, for each output, the largest distance between what the full model produced and what any model trained on ablated data could produce. They call this the counterfactual radius, and outputs with a radius of zero are unattributable: no removal from the training set would have prevented them. Crucially, the radius shrinks as training sets grow. Scale dissolves attribution.

This cuts in two directions, and it is worth resisting the urge to pick the convenient one. It weakens the argument that every generated output traces back to identifiable source works, because for many outputs no such work is findable. It equally weakens any promise of provenance on demand, because a method that returns nothing for many outputs cannot certify that an output is clean either. Unattributable is not the same as original, and it is not the same as safe. It just means the question has no answer this method can find.

Attribution also has uses far from copyright. If a model has learned a bad behavior, attribution tells you which data to remove, which is the entry point for machine unlearning. If a dataset has been poisoned, attribution is how you find the poison after the fact, which matters given how few malicious documents it takes to implant a backdoor. And in data markets, attribution is the only principled basis for deciding what a contributor's data was worth.

The related idea worth knowing is ablation studies, which apply the same remove-and-observe logic to architecture components rather than training examples. The difference is scale: you can ablate a dozen components, and you cannot ablate a billion examples one at a time. That gap is the entire field.

Ground Truth. (2026, August 19). Training data attribution: which examples actually made the model do that?. Ground Truth. https://groundtruth.day/learn/training-data-attribution.html

@misc{groundtruth:training-data-attribution,
  title  = {Training data attribution: which examples actually made the model do that?},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/training-data-attribution.html}
}

Topics: interpretability · attribution · copyright · training-data · provenance · fundamentals