Hypothesis Tests for Two Means: Unequal and Unknown Population Variances

When comparing two population means with unknown variances that we cannot assume are equal, we use Welch's $t$-test. Given two independent samples with sizes $n_1, n_2$, means $\bar{x}_1, \bar{x}_2$, and standard deviations $s_1, s_2$, the test statistic for $H_0\colon \mu_1 - \mu_2 = \Delta_0$ is

$$t = \dfrac{(\bar{x}_1 - \bar{x}_2) - \Delta_0}{\sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}}}.$$

Under $H_0$, this statistic approximately follows a $t$-distribution. The degrees of freedom come from the Welch–Satterthwaite formula:

$$\nu = \dfrac{\left(\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}\right)^2}{\dfrac{(s_1^2/n_1)^2}{n_1-1} + \dfrac{(s_2^2/n_2)^2}{n_2-1}}.$$

We round $\nu$ down to the nearest integer to be conservative.

For example, suppose $n_1=5,\, s_1=2,\, n_2=8,\, s_2=3$. Then

$$\dfrac{s_1^2}{n_1} = \dfrac{4}{5} = 0.8, \qquad \dfrac{s_2^2}{n_2} = \dfrac{9}{8} = 1.125.$$

The numerator of the Welch–Satterthwaite formula is $(0.8+1.125)^2 = 3.706$, and the denominator is

$$\dfrac{(0.8)^2}{4} + \dfrac{(1.125)^2}{7} = 0.160 + 0.181 = 0.341.$$

Therefore

$$\nu = \dfrac{3.706}{0.341} \approx 10.87 \;\Longrightarrow\; \nu = 10.$$