Extending Bayes' Theorem

Suppose the sample space is partitioned into mutually exclusive and exhaustive events $A_1, A_2, \ldots, A_n$, and we observe an event $B$. Ordinary Bayes' theorem says

$$P(A_i \mid B) = \dfrac{P(B \mid A_i)\, P(A_i)}{P(B)}.$$

Using the extended law of total probability, the denominator can be rewritten as

$$P(B) = \sum\limits_{j=1}^{n} P(B \mid A_j)\, P(A_j).$$

Substituting, we obtain the extended form of Bayes' theorem:

$$P(A_i \mid B) = \dfrac{P(B \mid A_i)\, P(A_i)}{\sum\limits_{j=1}^{n} P(B \mid A_j)\, P(A_j)}.$$

For instance, with three partition events $A_1, A_2, A_3$:

$$P(A_2 \mid B) = \dfrac{P(B \mid A_2)\, P(A_2)}{P(B \mid A_1)\, P(A_1) + P(B \mid A_2)\, P(A_2) + P(B \mid A_3)\, P(A_3)}.$$

In practice, we know the priors $P(A_j)$ and the likelihoods $P(B \mid A_j)$; the formula combines them into the posterior $P(A_i \mid B)$, updating our belief about which $A_i$ occurred once we observe $B$.