Learn · Intermediate
Calibration: whether a model's confidence means anything
A model is calibrated when its confidence means what it says. If it labels a hundred answers with 70% confidence, roughly seventy of them should be correct. Calibration is a completely separate property from accuracy, and confusing the two is behind a large share of the trouble people have deploying AI systems: a model can be right most of the time and still give you no usable signal about which answers to check.
The reason this matters is that almost every practical use of AI is a routing decision. Do I ship this answer or send it to a human? Do I let the agent execute the command or ask first? Do I trust this extraction or re-read the document? Every one of those questions needs a number that means something. Accuracy tells you how often the system is right on average. It tells you nothing about this answer.
The forecaster test
The cleanest way into this is weather. A forecaster who says "70% chance of rain" and sees rain on 70% of those days is perfectly calibrated, and is genuinely useful - you know when to carry an umbrella. A forecaster who says "99% chance" and is right 60% of the time is worse than useless, because their confidence has been decoupled from reality and you cannot act on it in either direction.
Now notice that the second forecaster might be more accurate overall than the first. Being right more often and being honest about uncertainty are different skills, and optimising for one does not deliver the other.
How you measure it
The standard instrument is a reliability diagram. Take a large set of predictions, bucket them by stated confidence - everything the model said with 50-60% confidence in one bin, 60-70% in the next - and for each bucket plot the actual success rate against the claimed one. A perfectly calibrated model traces the diagonal line. A model whose curve sags below the diagonal is overconfident; above it, underconfident. Expected calibration error, or ECE, averages the gap across buckets to compress the picture into one number.
Nearly every modern system sags. Chuan Guo and colleagues documented this in On Calibration of Modern Neural Networks, showing that as image classifiers got deeper and more accurate through the 2010s, they simultaneously got substantially more overconfident. The fix they proposed is charmingly simple: temperature scaling, which divides the model's raw scores by a single learned number before converting them into probabilities - flattening the distribution without changing which answer ranks first. One parameter, fit on a held-out set, recovers most of the lost calibration and costs nothing at inference.
Language models make it stranger
For a language model, "confidence" can mean two different things, and they do not agree.
The first is the internal probability the model assigns to the tokens it produced - the same machinery behind how a model picks its next word. The second is what the model says when you ask how sure it is. Both are measurable, and the gap between them is instructive.
Stephanie Lin, Jacob Hilton and Owain Evans showed in Teaching Models to Express Their Uncertainty in Words that a model can be trained to state calibrated confidence in plain English, not just to have it buried in its token probabilities. Saurav Kadavath and colleagues at Anthropic found in Language Models (Mostly) Know What They Know that large models can often predict whether their own answer is correct - the information is present internally, even when the model's prose does not reflect it.
The uncomfortable finding sits alongside these. Base models trained purely to predict the next token tend to be reasonably calibrated, and the post-training that makes a model pleasant - reinforcement learning from human feedback - tends to damage it. Human raters prefer confident, decisive answers. Optimising for that preference teaches the model to sound certain, which is exactly the wrong lesson, and it is closely related to sycophancy: both are cases where the model learns to produce what feels good rather than what is warranted.
Where calibration shows up in practice
Once you look for it, calibration is load-bearing in a surprising number of places. Speculative decoding schedules verification based on how likely a draft is to survive - a miscalibrated confidence estimate there wastes real compute. Model routing sends easy queries to a cheap model and hard ones to an expensive one, and that decision is a confidence estimate wearing a different hat. Any system with an abstain option - "I do not know" - is only as good as the threshold it abstains at.
The practical advice is short. Do not treat a model's stated percentage as meaningful until you have plotted it against outcomes on your own data. If it sags, temperature scaling on a held-out set is the cheapest fix in machine learning. And treat overconfidence as the default assumption rather than the exception - it is what the training process rewards.
On Calibration of Modern Neural Networks (Guo et al., 2017)
Calibration of Pre-trained Transformers (Desai and Durrett, 2020)
Teaching Models to Express Their Uncertainty in Words (Lin, Hilton and Evans, 2022)
Language Models (Mostly) Know What They Know (Kadavath et al., 2022)
Key questions
What is the difference between accuracy and calibration?
Why did instruction-tuned chat models get worse at this?
How do you measure calibration?
Cite this
APA
Ground Truth. (2026, July 31). Calibration: whether a model's confidence means anything. Ground Truth. https://groundtruth.day/learn/calibration-and-confidence.html
BibTeX
@misc{groundtruth:calibration-and-confidence,
title = {Calibration: whether a model's confidence means anything},
author = {{Ground Truth}},
year = {2026},
month = {jul},
url = {https://groundtruth.day/learn/calibration-and-confidence.html}
}