Bayes' Theorem

Bayes’ theorem calculates a conditional probability by combining an initial probability with new evidence. It is especially useful when you know how likely the evidence is under a hypothesis, but want to know how likely the hypothesis is after observing that evidence.

For example, a factory may know how often an inspection flags a defective item. Bayes’ theorem answers a different question: if an item is flagged, how likely is it to be defective? Those two probabilities are generally not equal.

Bayes’ theorem formula

For events A and B, with \(P(B)>0\),

\[P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}.\]

\(P(A)\): the prior probability of A, before conditioning on B.
\(P(B\mid A)\): the likelihood of observing B if A occurs.
\(P(B)\): the overall probability of the evidence.
\(P(A\mid B)\): the posterior probability of A, after conditioning on B.

The vertical bar means “given.” Thus, \(P(A\mid B)\) is read as “the probability of A given B.” For a refresher on events and probability rules, see our introduction to probability.

A worked example: a flagged item

Suppose a factory produces 10,000 items. The following figures are hypothetical:

2% of the items are defective.
An inspection flags 90% of defective items.
It also flags 5% of items that are not defective.

Let D mean “defective” and F mean “flagged.” We want \(P(D\mid F)\).

1. Count the defective items that are flagged

There are \(10{,}000\times0.02=200\) defective items. The inspection flags \(200\times0.90=180\) of them.

2. Count the nondefective items that are flagged

The other 9,800 items are not defective. A 5% false-positive rate produces \(9{,}800\times0.05=490\) additional flags.

3. Divide by all flagged items

There are \(180+490=670\) flagged items, of which 180 are defective. Therefore,

\[P(D\mid F)=\frac{180}{670}=\frac{18}{67}\approx0.2687.\]

A flagged item has about a 26.9% probability of being defective under these assumptions. The inspection detects most defective items, but nondefective items are much more common, so false positives make up most of the flags.

Using the formula directly

The total probability of a flag includes two mutually exclusive possibilities: a defective item is flagged, or a nondefective item is flagged.

\[P(F)=P(F\mid D)P(D)+P(F\mid D^{c})P(D^{c}).\]

Here, \(D^{c}\) means “not defective.” Substituting the numbers gives

\[P(F)=(0.90)(0.02)+(0.05)(0.98)=0.067,\]

and hence

\[P(D\mid F)=\frac{(0.90)(0.02)}{0.067}=\frac{18}{67}\approx26.9\%.\]

The formula and the count of 10,000 items describe the same calculation. Counting often makes the denominator easier to understand.

Why Bayes’ theorem works

Conditional probability gives two ways to write the probability that both events occur:

\[P(A\cap B)=P(A\mid B)P(B)=P(B\mid A)P(A).\]

Dividing by \(P(B)\) gives Bayes’ theorem. The result does not require A and B to be independent.

Common mistakes

Reversing the condition. A 90% detection rate is \(P(F\mid D)\), not \(P(D\mid F)\).
Ignoring the base rate. The proportion of defective items affects how informative a flag is.
Using only true positives in the denominator. All ways of observing the evidence must be included.
Treating a model as a guarantee. The calculation is exact for the stated probabilities; whether those probabilities describe a real production line is an empirical question.

Bayes’ theorem is a rule for updating probabilities, not a way to repair unreliable data automatically. In practical applications, check how the prior probability and error rates were estimated and whether they apply to the population being studied.

Bayes’ Theorem