Multiple Linear Regression With Matrices

Extends the matrix formulation of linear regression to the case of multiple predictors. Students learn to build the design matrix when there are $p$ predictors, apply the normal equation $\hat{\boldsymbol\beta} = (X^T X)^{-1} X^T \mathbf{y}$ to compute least-squares estimates, and use the fitted model to predict the response at a new observation.

Step 1 of 119%

Tutorial

The Multiple Linear Regression Model in Matrix Form

In multiple linear regression, we model a response $y$ as a linear combination of $p$ predictors plus an intercept:

y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_p x_p + \varepsilon.

Given $n$ observations $(x_{i1}, x_{i2}, \ldots, x_{ip}, y_i)$ , we stack the equations into matrix form:

\mathbf{y} = X\boldsymbol\beta + \boldsymbol\varepsilon,

where the design matrix $X$ has a leading column of $1$ s (for the intercept) followed by one column per predictor:

X = \begin{bmatrix} 1 & x_{11} & x_{12} & \cdots & x_{1p} \\ 1 & x_{21} & x_{22} & \cdots & x_{2p} \\ \vdots & \vdots & \vdots & & \vdots \\ 1 & x_{n1} & x_{n2} & \cdots & x_{np} \end{bmatrix}, \qquad \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, \qquad \boldsymbol\beta = \begin{bmatrix} \beta_0 \\ \beta_1 \\ \vdots \\ \beta_p \end{bmatrix}.

This is the same matrix form used for simple linear regression. The only change is that $X$ now has $p+1$ columns instead of $2$ , and $\boldsymbol\beta$ has $p+1$ entries instead of $2$ .