Log-Likelihood Functions for Discrete Probability Distributions

Introduces the log-likelihood function $\ell(\theta) = \ln L(\theta)$ for iid samples from discrete distributions. Students learn to write the log-likelihood as a sum of log-PMF terms, derive closed-form expressions for Bernoulli and Poisson samples, and evaluate the log-likelihood numerically at a specified parameter value.

Step 1 of 119%

Tutorial

Introduction to the Log-Likelihood

The log-likelihood function is defined as the natural logarithm of the likelihood function:

\ell(\theta) = \ln L(\theta).

Why work with the logarithm? For an iid sample, the likelihood $L(\theta)$ is a product of PMF values, and products are awkward to differentiate. The logarithm converts products into sums:

\ln \prod_{i=1}^n a_i = \sum_{i=1}^n \ln a_i.

Sums are far easier to manipulate and differentiate. Furthermore, since $\ln$ is strictly increasing, the value of $\theta$ that maximizes $L(\theta)$ also maximizes $\ell(\theta)$ -- the two functions attain their maximum at the same point.

To illustrate, suppose we observe two independent Bernoulli( $\theta$ ) trials with outcomes $x_1 = 1$ and $x_2 = 0$ . The likelihood is

L(\theta) = \theta \cdot (1-\theta).

Taking the natural logarithm of both sides,

\ell(\theta) = \ln \theta + \ln(1-\theta).