The Covariance Matrix

For a random vector with multiple components, the covariance matrix collects all pairwise variances and covariances into a single symmetric square matrix. This lesson defines the covariance matrix, builds 2x2 and 3x3 examples from given variances and covariances, and reads variances, standard deviations, covariances, and correlation coefficients back out of a given covariance matrix.

Step 1 of 157%

Tutorial

Introduction

Often we want to track the variability and pairwise relationships of several random variables at once. The covariance matrix packages all of this information into a single square matrix.

Given a random vector $\mathbf{X} = (X_1, X_2, \ldots, X_n)^T,$ its covariance matrix is the $n \times n$ matrix $\Sigma$ whose entry in row $i$ and column $j$ is the covariance between $X_i$ and $X_j{:}$

\Sigma_{ij} = \text{Cov}(X_i, X_j).

Written out, this is

\Sigma = \begin{bmatrix} \text{Cov}(X_1, X_1) & \text{Cov}(X_1, X_2) & \cdots & \text{Cov}(X_1, X_n) \\ \text{Cov}(X_2, X_1) & \text{Cov}(X_2, X_2) & \cdots & \text{Cov}(X_2, X_n) \\ \vdots & \vdots & \ddots & \vdots \\ \text{Cov}(X_n, X_1) & \text{Cov}(X_n, X_2) & \cdots & \text{Cov}(X_n, X_n) \end{bmatrix}.

Two facts make this matrix easy to read off:

Diagonal entries are variances. Since $\text{Cov}(X_i, X_i) = \text{Var}(X_i),$ the main diagonal lists the variances $\text{Var}(X_1), \text{Var}(X_2), \ldots, \text{Var}(X_n).$
The matrix is symmetric. Since $\text{Cov}(X_i, X_j) = \text{Cov}(X_j, X_i),$ the entry in row $i,$ column $j$ equals the entry in row $j,$ column $i.$

For two random variables $X$ and $Y$ with $\text{Var}(X) = 2,$ $\text{Var}(Y) = 8,$ and $\text{Cov}(X, Y) = -1,$ the covariance matrix is

\Sigma = \begin{bmatrix} \text{Var}(X) & \text{Cov}(X, Y) \\ \text{Cov}(Y, X) & \text{Var}(Y) \end{bmatrix} = \begin{bmatrix} 2 & -1 \\ -1 & 8 \end{bmatrix}.