Learn · Intermediate
Missing data: why an empty cell can bias an AI system
Missing-data mechanisms describe why some values are absent and whether that absence can distort a statistical analysis. The distinction matters because filling in blank cells does not automatically repair the bias created when the missing values differ systematically from the observed ones.
Imagine a hospital collecting patients’ recovery scores. Some surveys are lost in the mail; some patients with long hospital stays are less likely to respond; some patients with particularly poor recovery avoid responding because of their score. All three datasets contain blank cells, but the missingness tells three different stories. Donald Rubin’s foundational paper Inference and Missing Data formalized distinctions that help determine which conclusions can be supported.
Three ways data can be missing
Missing completely at random, often abbreviated MCAR, means the probability that a value is missing depends neither on observed values nor on missing values. A genuinely random logging failure that independently drops a small fraction of records can approximate this condition. Removing incomplete records then reduces sample size, but does not systematically favor one kind of record over another under the assumption. The assumption concerns the actual missingness process, not whether the gaps look scattered across a spreadsheet.
Missing at random, or MAR, is a conditional statement: after accounting for observed information, the probability of missingness does not additionally depend on the missing value itself. Suppose older patients are less likely to return a survey, but age is recorded for everyone, and within each age group response does not depend on the unreported score. Missingness can depend strongly on age and still satisfy MAR conditional on age. An analysis that uses age appropriately may then adjust for the imbalance. If important predictors of nonresponse are not recorded, that assumption is less plausible.
Missing not at random, or MNAR, means the probability of missingness still depends on the unseen value after conditioning on observed information. If patients with worse unreported recovery are less likely to respond even among people with the same recorded characteristics, missingness is MNAR. The observed answers alone usually cannot tell us exactly how the nonrespondents would have answered. Different plausible assumptions about their scores can fit the same observed data yet lead to different conclusions.
These categories refer to a statistical model and the variables observed, not a permanent label attached to a dataset. Measuring an additional variable related to response can make a conditional MAR assumption more credible, though it cannot automatically prove it. The causal graphs lesson explains why conditioning on the wrong variable can itself create a misleading association.
Why a blank is not just an empty value
Dropping incomplete rows, called complete-case analysis, can be reasonable under some assumptions, including MCAR, but wastes information. When response depends on observed or unobserved characteristics, the remaining rows may be unrepresentative. For example, if an AI benchmark contains only tasks on which a system returned an answer, reporting accuracy on those answers alone can hide a high failure or refusal rate. The missing result is part of the evaluation story. The existing selective prediction lesson discusses systems that choose when to answer.
Replacing each blank with the overall average is also tempting. It makes the table complete, but suppresses variation and usually treats an invented value as if it were measured. Relationships between variables can be distorted, and uncertainty can be understated. A sophisticated model is not a cure if it assumes away the reason observations are missing.
Multiple imputation and uncertainty
Multiple imputation offers a more careful workflow under suitable assumptions. Instead of filling each blank once, an analyst creates several plausible completed datasets using an imputation model that reflects uncertainty and relationships in the observed variables. The intended analysis is run on each completed dataset. Results are then combined so the reported uncertainty includes variation both within and between imputations. Rubin’s original missing-data framework motivates treating missing values as uncertain quantities rather than facts.
Suppose a patient’s recovery score could plausibly be 40, 55 or 70 given their observed characteristics. Several imputed datasets can reflect that range. But if the imputation model leaves out a predictor of both response and recovery, or the score itself drives nonresponse in a way the model ignores, repeated plausible-looking guesses can still be systematically wrong. Multiple imputation is a method for propagating uncertainty under a model, not a machine for discovering hidden data without assumptions.
What to do in practice
First report the amount and pattern of missingness by variable, group and time. Investigate the collection process: did a sensor fail, did an option not apply, or did participants choose not to respond? Preserve an indicator of whether a value was observed when it is meaningful. For predictive systems, fit imputers using training data only, then apply them to validation and test data; otherwise the evaluation can leak information across the split. The holdout lesson explains that boundary.
Next state the assumption that makes the chosen analysis interpretable, compare reasonable approaches, and run a sensitivity analysis. For a survey, ask how the result changes if nonrespondents are modestly or substantially worse off than comparable respondents. If the conclusion reverses under plausible departures from MAR, say so. The most rigorous result may be a range of possible conclusions rather than one overly precise number.
Finally, distinguish prediction from explanation. A missingness indicator can be predictive because the collection process encodes useful information, but it can fail after a hospital changes its forms or a product changes its logging. A missing-data strategy must therefore be evaluated in the environment where it will be used, with its assumptions and limitations visible to the reader.
Rubin, Inference and Missing Data (1976)
Key questions
What is the difference between missing at random and missing completely at random?
Does multiple imputation recover the true missing values?
Can observed data prove that missingness is at random?
Cite this
APA
Ground Truth. (2026, November 15). Missing data: why an empty cell can bias an AI system. Ground Truth. https://groundtruth.day/learn/missing-data-mechanisms-and-imputation.html
BibTeX
@misc{groundtruth:missing-data-mechanisms-and-imputation,
title = {Missing data: why an empty cell can bias an AI system},
author = {{Ground Truth}},
year = {2026},
month = {nov},
url = {https://groundtruth.day/learn/missing-data-mechanisms-and-imputation.html}
}