Learn · Beginner

Bayesian updating: how a belief should change when evidence arrives

Bayesian updating is the rule for revising a belief when new evidence arrives: start with a prior probability, weigh how much more likely that evidence would be under one hypothesis than under another, and arrive at a posterior probability. It is the only way of combining prior knowledge with new observations that does not contradict itself, which is why it shows up everywhere from spam filters to the planning loops inside modern AI agents. The rule is named for Thomas Bayes, whose essay on the problem was published posthumously in 1763.

Three quantities

Every application uses the same three pieces.

The prior is what you believed before the evidence arrived. The likelihood is how probable the evidence would be if a given hypothesis were true. The posterior is what you should believe afterward.

Bayes' rule says the posterior is proportional to the prior times the likelihood. In words: how plausible something was, multiplied by how well it explains what you just saw.

The version that is easiest to use in your head works in odds. Take the odds you started with, multiply by the likelihood ratio, which is how much more probable the evidence is under one hypothesis than the other, and you have your new odds. Two-to-one odds against, times a piece of evidence three times more likely if the thing is true, gives you three-to-two odds in favor. That is the entire mechanism.

The example that fixes the intuition

A disease affects one person in a thousand. A test for it is 99 percent accurate in both directions. You test positive. What are the odds you have it?

Most people say 99 percent. The right answer is about 9 percent.

Work through a population of 100,000. About 100 people have the disease, and the test correctly flags 99 of them. About 99,900 do not, and the test wrongly flags one percent of those, which is 999 people. So 1,098 people test positive and only 99 of them are sick. Your odds are 99 in 1,098.

Nothing was wrong with the test. What went wrong was ignoring the prior. The disease is rare, so most positives are drawn from the enormous healthy group, and a small error rate on a large group beats a high accuracy rate on a tiny one. This is the same arithmetic behind why a highly accurate content filter still generates mostly false accusations when the thing it is looking for is rare, and it is worth knowing before you ship one.

Priors are not cheating

The standard objection to Bayesian reasoning is that the prior is made up. Sometimes it is. But the alternative is not neutrality, it is a hidden prior. Refusing to state one usually means assuming everything is equally likely, which is itself a strong and often absurd claim.

The more practical answer is that priors wash out. Two people who start with very different beliefs and see the same stream of evidence converge, as long as neither began by assigning a probability of exactly zero or exactly one. Assigning zero is the one unrecoverable error: no amount of evidence can move a belief that starts at impossible. That is a reasonable working definition of dogmatism, and it falls out of the mathematics rather than being imposed on it.

Where it lives in AI

The naive Bayes classifier that ran spam filtering for a decade is the rule applied directly, treating each word as independent evidence and multiplying likelihood ratios together.

Variational autoencoders are Bayesian inference wearing a neural network. The model maintains a distribution over latent causes of the data and updates it, with a tractable approximation standing in for the exact posterior. The distance measure that governs that approximation is covered in our note on forward and reverse KL divergence.

Bayesian neural networks keep a distribution over weights rather than a single number for each, so the model can say how uncertain it is rather than just what it predicts. Since the exact computation is hopeless at scale, the field runs approximations. Yarin Gal and Zoubin Ghahramani showed that ordinary dropout at inference time is itself an approximation to Bayesian inference, which quietly made uncertainty estimates available to anyone already using a standard regularizer.

Thompson sampling picks actions by sampling from the posterior over which action is best, and it is how recommendation systems and A/B testing infrastructure decide how much to explore. And calibration, the question of whether a model's stated 70 percent confidence means it is right 70 percent of the time, is the same idea turned into a diagnostic.

It runs attacks too

A concrete and uncomfortable example landed this week. An autonomous attack framework analyzed by security firm DREAM scored every vulnerability it found starting from an uninformative prior of 0.50, updated with explicit likelihood ratios as evidence arrived, and used the resulting posterior to decide what to do next. It then assembled confirmed findings into multi-step attack chains carrying their own probability scores, and reprioritized between waves. The result was an intrusion campaign that ran twelve waves in four days and predicted its own success rate before acting.

That is Bayes' rule as a targeting system. It is a useful reminder that the rule is neutral machinery: it makes any agent that uses it better at concentrating effort where the payoff is, regardless of what the payoff is.

The honest limits

Exact Bayesian inference is computationally intractable for anything the size of a modern model, which is why the field runs approximations and why those approximations have their own failure modes. A badly chosen prior can dominate the evidence for a long time. And the rule assumes your hypothesis space contains the truth; if the right explanation is not on your list, Bayes will confidently distribute belief among the wrong ones forever. It tells you how to update, not what to consider.

Key papers
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
Weight Uncertainty in Neural Networks
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
A Tutorial on Thompson Sampling

Key questions

What problem does Bayesian updating solve?

It answers how much a new piece of evidence should move your belief, given what you already believed. Without a rule like this, people either ignore prior information entirely or refuse to update at all, and both failures are common enough to have names.

Why does the prior matter so much?

Because evidence tells you how the world would look under different hypotheses, not how likely those hypotheses were to begin with. A test that is right 99 percent of the time still produces mostly false alarms when it hunts for something genuinely rare, and only the prior captures that rarity.

Is a Bayesian neural network the same thing as Bayesian updating?

It is one application of it. A Bayesian neural network keeps a distribution over its weights rather than a single value, and updates that distribution as it sees data, which is Bayes' rule applied to millions of parameters at once. Because the exact computation is intractable at that scale, practical systems use approximations.

Cite this

APA

Ground Truth. (2026, August 12). Bayesian updating: how a belief should change when evidence arrives. Ground Truth. https://groundtruth.day/learn/bayesian-updating.html

BibTeX

@misc{groundtruth:bayesian-updating,
  title  = {Bayesian updating: how a belief should change when evidence arrives},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/bayesian-updating.html}
}

Topics: fundamentals · probability · uncertainty · statistics · decision-making