Introduction to Order Statistics

Given a random sample $X_1, X_2, \ldots, X_n$ of iid continuous random variables with common CDF $F$ and PDF $f,$ the order statistics are the sample values rearranged in nondecreasing order:

$$X_{(1)} \le X_{(2)} \le \cdots \le X_{(n)}.$$

The $k$th order statistic $X_{(k)}$ is the $k$th smallest value in the sample. In particular,

$$X_{(1)} = \min(X_1, \ldots, X_n), \qquad X_{(n)} = \max(X_1, \ldots, X_n).$$

The distribution of the sample maximum is easy to derive. Notice that $X_{(n)} \le x$ holds if and only if every $X_i \le x.$ By independence,

$$F_{X_{(n)}}(x) = P(X_{(n)} \le x) = \prod_{i=1}^n P(X_i \le x) = F(x)^n.$$

Differentiating gives the PDF of the maximum:

$$f_{X_{(n)}}(x) = n\,F(x)^{n-1} f(x).$$

For example, if $X_1, X_2, X_3$ are iid Uniform$(0,1),$ then $F(x) = x$ on $[0,1],$ so

$$F_{X_{(3)}}(x) = x^3, \qquad f_{X_{(3)}}(x) = 3x^2.$$