Learn · Beginner
Selection bias: when the sample tells the wrong story
Selection bias occurs when the people or examples included in a dataset differ systematically from the population a claim is meant to describe. It matters because an accurate calculation on the observed sample can still give a misleading answer about the wider world.
Imagine a restaurant asks diners leaving on a rainy Tuesday whether they enjoyed dinner. The responses may accurately describe those diners. They do not automatically describe people who stayed home, chose another restaurant or visit on weekends. The problem is not arithmetic; it is the path by which observations entered the dataset.
The same problem appears when evaluating an AI model. A public leaderboard may consist of tasks that are easy to grade, written in English, and contributed by enthusiasts. A model's measured score answers how it performed on those selected tasks under that test's rules. Applying that score to medical records, multilingual customer support or a software team's private code requires an additional argument that the test resembles those uses in relevant ways.
Start with the target population
A useful measurement begins with a specific question: performance for whom, on what tasks, and under what conditions? “How good is the model?” is too broad. “How often does this assistant correctly summarize incoming English-language support tickets at this company during the next quarter?” defines a population, task and period that can guide sampling.
Next ask how examples entered the sample. Were they randomly sampled from incoming tickets, donated by a few enthusiastic teams, collected only after users complained, or generated by another model? Each route can leave out different cases. A volunteer sample can contain many observations and still overrepresent people motivated to respond. More observations reduce random noise around the selected sample's value; they do not necessarily repair systematic selection.
The November HN hiring analysis is a concrete example. Its reported 51.6% is the fraction of 244 posts in a particular Hacker News hiring-thread sample that mention AI. It is useful evidence about that forum. Employers who choose that forum are not a random cross-section of employers, and a mention is not the same thing as worker adoption. Moving from “posts mentioning AI” to “all jobs using AI” changes both population and measurement.
Selection and measurement are different problems
Selection bias concerns which cases are observed. Measurement error concerns what the recorded variable actually means. A perfectly representative sample of ads would still not prove that a keyword measures a worker's day-to-day AI use. Conversely, a precise measure of tool use can still mislead about the economy if it comes only from an unusually AI-intensive company. Both questions matter and should be checked separately.
There is another subtle trap: selecting the most promising result after many experiments. Gavin Cawley and Nicola Talbot describe overfitting in model selection: repeatedly choosing models on the same evaluation data can make the chosen result look better than it will on fresh data. This is selection among model candidates rather than selection of people into a survey, but both illustrate how the selection procedure affects the inference. The existing lesson on cross-validation and holdout sets explains the model-selection case in detail.
What can improve the inference?
First, document who could enter the dataset and who actually did. Compare observed groups with the target population when credible population data exist. If a sample underrepresents a known group, analysts can sometimes weight observations to match known population shares. For example, a survey with too few small firms might assign their responses greater weight.
Weighting is not magic. If no small firms were sampled, or the small firms that responded are unusual in ways the analyst cannot observe, matching the known firm-size distribution will not necessarily remove the bias. Large weights can also increase uncertainty. Reporting results across plausible assumptions is more honest than presenting a weighted estimate as automatically representative.
Second, sample directly from the intended population when feasible. A random sample of incoming tickets with clear inclusion rules offers a stronger basis for estimating performance on those tickets. Keep an untouched test set and report how cases were excluded. Ron Kohavi's work on cross-validation and accuracy estimation is a classic reference for estimating predictive accuracy; valid splitting still depends on whether the data themselves represent the intended use.
Third, examine subgroups and conditions. A system may look strong overall because the sample contains many easy cases and few difficult ones. Compare performance across relevant languages, time periods, user groups and task types, while recognizing that small subgroups have wide uncertainty. The lesson on distribution shift covers what happens when deployment data change after evaluation.
Finally, state the limit in the result itself. “Fifty-one percent of the sampled posts mentioned AI” is a defensible observation. “Half of all jobs use AI” is a different claim requiring different evidence. Careful boundaries make the data more useful: they show what has been learned and identify the next observation needed.
On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Key questions
Why can a large sample still be biased?
Is selection bias the same as measurement error?
Can weighting always fix a biased sample?
Cite this
APA
Ground Truth. (2026, November 16). Selection bias: when the sample tells the wrong story. Ground Truth. https://groundtruth.day/learn/selection-bias-in-samples.html
BibTeX
@misc{groundtruth:selection-bias-in-samples,
title = {Selection bias: when the sample tells the wrong story},
author = {{Ground Truth}},
year = {2026},
month = {nov},
url = {https://groundtruth.day/learn/selection-bias-in-samples.html}
}