Learn · Beginner

Precision and recall: finding the right things without missing too many

Precision and recall describe different kinds of classification success. Precision asks how often a flagged case is actually positive; recall asks how many of all real positives the system catches. They matter because a model can be excellent at one and poor at the other, especially when the event being sought is rare.

Imagine a security team screening 10,000 incoming alerts. There are 100 genuine attacks. A detector catches 80 of them but also flags 900 harmless events. Its recall is 80 divided by 100, or 80%. Its precision is 80 divided by 980, about 8.2%. Most flagged alerts are false alarms even though the detector catches most attacks.

The accounting uses four outcomes. A true positive is a genuine attack that is flagged; a false positive is a harmless case flagged as an attack. A false negative is an attack missed by the detector; a true negative is a harmless case left alone. Precision equals true positives divided by all flagged positives. Recall equals true positives divided by all actual positives. Both formulas depend on which outcome is designated positive. If there are no predicted positives, precision is undefined unless an evaluation convention specifies otherwise; if there are no actual positives, recall is likewise undefined.

The threshold changes the tradeoff

Most models produce a score rather than a decision. To turn that score into an alert, a team chooses a cutoff. Lowering it ordinarily flags more cases, often increasing recall and increasing the number of false positives. Raising it ordinarily catches fewer real cases and often improves precision, although precision need not rise monotonically on a finite dataset. A spam filter’s operator can choose an aggressive cutoff if missed spam is costly or a conservative one if losing a legitimate message is worse.

Tom Fawcett’s introduction to receiver operating characteristic analysis explains how a threshold creates different operating points. The related ROC lesson looks at true-positive and false-positive rates over thresholds. A precision–recall curve instead plots precision against recall as the cutoff moves. The curves answer related questions but place different emphasis on the fraction of flagged cases that are useful.

Consider a stricter detector that flags 30 cases, 27 of them genuine. Its precision is 90%, but its recall is just 27% if there are still 100 attacks. Is it better? That depends on whether the team can investigate only a few alerts and on what happens to the 73 attacks it misses. There is no universally best score without a cost or capacity decision.

Why the base rate changes the answer

Suppose the same detector catches 80% of real attacks and falsely flags 9.1% of harmless events. On the 10,000-case example, with 100 attacks and 9,900 harmless events, that produces about 80 true alerts and 900 false alerts. Now suppose attacks are only one in 1,000, while those conditional rates remain unchanged. Out of 10,000 cases, about 8 are true alerts and about 909 are false alerts. Precision falls below 1%. The detector’s sensitivity to attacks did not necessarily change; the population did.

This is why an artificially balanced benchmark can overstate the usefulness of a production alert stream. In its identity-resolution experiment, Hightouch deliberately made half of an easier synthetic set over-merged even though the company says such cases are rare in a healthy graph. That design helps test errors, but the reported overall accuracy cannot by itself predict the fraction of real-world flags that deserve investigation. The Bayesian updating lesson develops the underlying base-rate calculation.

Choosing and reporting a metric

Start with the action. If a human must investigate each flag, precision and total alert volume affect workload. If missing a harmful event is costly, recall matters. If both matter, teams sometimes summarize them with the F1 score: twice precision times recall divided by their sum. F1 gives them equal weight in a particular mathematical sense, but it does not encode the real cost of an error, the capacity of a review team, or the benefits of true negatives. A single score cannot replace those decisions.

Report the threshold, the number of examples, prevalence, confusion-matrix counts and uncertainty alongside precision and recall. For a small test, a handful of changed labels can move the score substantially. Test on representative data and check for distribution shifts; neither high precision nor high recall on one benchmark guarantees the same result on another. A model that changes the ranking of cases can also change which thresholds are operationally useful.

Finally, labels can be wrong. An unreviewed alert is not automatically a harmless alert, and an audited sample selected by the model may miss the attacks it failed to flag. Establishing recall requires evidence about cases the system did not select. Measurement is part of the work, not a property the model supplies on its own.

Ground Truth. (2026, November 7). Precision and recall: finding the right things without missing too many. Ground Truth. https://groundtruth.day/learn/precision-and-recall.html

@misc{groundtruth:precision-and-recall,
  title  = {Precision and recall: finding the right things without missing too many},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/learn/precision-and-recall.html}
}

Topics: evaluation · classification · metrics