Logarithmic Differentiation

When we observe data $x_1, x_2, \ldots, x_n$ drawn independently from a distribution with parameter $\theta$, the likelihood function is the product

$$L(\theta) = \prod_{i=1}^{n} f(x_i; \theta).$$

Products are awkward to differentiate. So instead of maximizing $L(\theta)$ directly, we maximize the log-likelihood

$$\ell(\theta) = \ln L(\theta) = \sum_{i=1}^{n} \ln f(x_i; \theta).$$

Since $\ln$ is strictly increasing, the value of $\theta$ that maximizes $L(\theta)$ is the same value that maximizes $\ell(\theta)$. The technique of taking the log before differentiating is called logarithmic differentiation.

The log turns products into sums via the rules

$$\ln(ab) = \ln a + \ln b, \qquad \ln(a^k) = k\ln a, \qquad \ln(e^x) = x.$$

For example, if $L(\theta) = \theta^4 e^{-6\theta}$, then

$$\ell(\theta) = \ln(\theta^4) + \ln(e^{-6\theta}) = 4\ln\theta - 6\theta.$$