Likelihood Functions for Continuous Probability Distributions

For a discrete random variable, we defined the likelihood function as the joint probability of the observed data:

$$L(\theta) = P(X_1=x_1,\,\ldots,\,X_n=x_n;\theta) = \prod_{i=1}^n P(X_i = x_i;\theta).$$

This definition breaks down for a continuous random variable, because $P(X_i = x_i;\theta) = 0$ for any specific value $x_i$. To repair it, we simply replace the pmf with the probability density function $f(x;\theta)$.

Given $n$ independent observations $x_1, x_2, \ldots, x_n$ from a continuous distribution with pdf $f(x;\theta)$, the likelihood function is

$$L(\theta) = \prod_{i=1}^n f(x_i;\theta).$$

Unlike the discrete case, $L(\theta)$ is not a probability — pdf values are not capped at $1$, so $L(\theta)$ can be any nonnegative number. However, its interpretation is the same: parameter values $\theta$ that produce a larger $L(\theta)$ are more consistent with the observed data.

For instance, suppose $X \sim \text{Exponential}(\lambda)$ with pdf $f(x;\lambda) = \lambda e^{-\lambda x}$ for $x \geq 0$. If we observe $x_1 = 0.5$ and $x_2 = 1.5$, then

$$\begin{align*} L(\lambda) &= \bigl(\lambda e^{-0.5\lambda}\bigr)\bigl(\lambda e^{-1.5\lambda}\bigr) \\[3pt] &= \lambda^2 e^{-2\lambda}. \end{align*}$$