Maximum Likelihood Estimation

Define the maximum likelihood estimate (MLE) of a parameter, and compute MLEs by differentiating the log-likelihood and solving. Apply the procedure to the rate parameter of an exponential distribution and to the mean and variance of a normal distribution.

Step 1 of 119%

Tutorial

Introduction

The maximum likelihood estimate (MLE) of a parameter $\theta$ from an i.i.d. sample $x_1, x_2, \ldots, x_n$ drawn from a distribution with density $f(x;\theta)$ is the value $\hat\theta$ that maximizes the likelihood

L(\theta) = \prod_{i=1}^n f(x_i;\theta).

Since $\ln$ is strictly increasing, $\hat\theta$ also maximizes the log-likelihood

\ell(\theta) = \ln L(\theta) = \sum_{i=1}^n \ln f(x_i;\theta),

which is almost always easier to work with — sums are much friendlier than products. To find $\hat\theta,$ we differentiate, set the derivative equal to zero, and solve:

\frac{d\ell}{d\theta} = 0.

For instance, suppose $X \sim \mathrm{Exp}(\lambda),$ so that $f(x;\lambda) = \lambda e^{-\lambda x},$ and we observe a single value $x = 4.$ Then $\ell(\lambda) = \ln \lambda - 4\lambda,$ and

\begin{align*} \frac{d\ell}{d\lambda} = \frac{1}{\lambda} - 4 &= 0 \\[3pt] \hat\lambda &= \frac{1}{4}. \end{align*}