Bayes' Theorem

Suppose we know $P(B \mid A)$ but want the reverse conditional probability $P(A \mid B)$. Bayes' theorem lets us flip the conditioning:

$$P(A \mid B) = \dfrac{P(B \mid A) \, P(A)}{P(B)}.$$

This follows from writing the joint probability $P(A \cap B)$ two different ways using the definition of conditional probability:

$$P(A \cap B) = P(A \mid B) \, P(B) = P(B \mid A) \, P(A).$$

Solving the second equality for $P(A \mid B)$ gives Bayes' theorem.

The factor $P(A)$ is called the prior — our belief about $A$ before observing $B$. The result $P(A \mid B)$ is the posterior — our updated belief after observing $B$.

For instance, if $P(A) = 0.4,$ $P(B) = 0.5,$ and $P(B \mid A) = 0.25,$ then

$$P(A \mid B) = \dfrac{0.25 \cdot 0.4}{0.5} = \dfrac{0.1}{0.5} = 0.2.$$