Linear Regression

Given paired data $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$, the least-squares regression line is the line $\hat{y} = b_0 + b_1 x$ that minimizes the sum of the squared vertical distances from the data points to the line.

The slope $b_1$ and intercept $b_0$ are given by

$$b_1 = \dfrac{\sum\limits_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum\limits_{i=1}^n (x_i - \bar{x})^2}, \qquad b_0 = \bar{y} - b_1 \bar{x},$$

where $\bar{x}$ and $\bar{y}$ are the sample means of the $x$- and $y$-values.

For instance, consider the three points $(1, 2), (2, 2), (3, 5)$. We have $\bar{x} = 2$ and $\bar{y} = 3$, so

$$\begin{align*} b_1 &= \dfrac{(-1)(-1) + (0)(-1) + (1)(2)}{(-1)^2 + 0^2 + 1^2} = \dfrac{3}{2} = 1.5, \\[3pt] b_0 &= 3 - 1.5 \cdot 2 = 0. \end{align*}$$

The least-squares regression line is $\hat{y} = 1.5\,x.$