The Method of Moments

Suppose we have IID samples $X_1, X_2, \ldots, X_n$ drawn from a distribution with one unknown parameter $\theta$. We want to estimate $\theta$ from the data.

The method of moments equates the sample mean to the population mean and solves for $\theta$.

The population mean is determined by $\theta$ — write it as $E[X] = \mu(\theta)$. The sample mean is

$$\bar{X} = \dfrac{1}{n}\sum\limits_{i=1}^n X_i.$$

The method-of-moments estimator $\hat{\theta}$ is the solution to

$$\mu(\hat{\theta}) = \bar{X}.$$

For example, if $X \sim \text{Bernoulli}(p)$, the population mean is $E[X] = p$. Setting $p = \bar{X}$ gives the estimator

$$\hat{p} = \bar{X}.$$

If we observe $\{1, 0, 1, 1\}$, then $\bar{X} = \dfrac{3}{4}$, so $\hat{p} = \dfrac{3}{4}.$