2.5b Second-Order Moments

Variance, Covariance, Correlation (Second-Order Moments)

The variance of a random variable is its second central moment, and measures the spread of values around its expected value, such that
$\mathrm{Var}(X)=\mathbb{E}[(X-\mathbb{E}[X]{)}^2]=\mathbb{E}[X^2]-E[X{]}^2.$

The square root of variance, ${\sigma }_X=\sqrt{\mathrm{Var}(X)}$, is known as the standard deviation of $X$.

Given random variables $X,Y$, the variance of their sum can generally be expressed as
$\mathrm{Var}(\alpha X+\beta Y+k)={\alpha }^2\mathrm{Var}(X)+{\beta }^2\mathrm{Var}(Y)+2\alpha \beta \mathrm{Cov}(X,Y),$
where $\mathrm{Cov}(X,Y)$ is the second mixed central moment of $X$ and $Y$, such that
$\mathrm{Cov}(X,Y)=\mathbb{E}[(X-\mathbb{E}[X])(Y-\mathbb{E}[Y])]=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y],$
and serves as a measure of the linear association between them (i.e., negative correlation indicates inverse relationship and vice-versa). Note that if $X,Y$ are independent, then $\mathrm{Cov}(X,Y)=0$ (but not the other way around).

A measure for covariance normalized by the product of their standard deviations is given by the unit-less correlation coefficient. Formally,
${\rho }_{X,Y}=\frac{\mathrm{Cov}(X,Y)}{\mathrm{Var}(X)\mathrm{Var}(Y)},$
where $-1\le {\rho }_{X,Y}\le 1$, such that ${\rho }_{X,Y}=0$ indicates no relationship between $X$ and $Y$.

Non-Negativity of Variance $(X-\mathbb{E}[X]{)}^2\ge 0$ for all $X$, the variance of any random variable is always non-negative.

Since

Intuition

Properties

(Cond. Variance) Any $X,Y$ satisfy $\mathrm{Var}(X\mid Y=y)=\mathbb{E}\left[(X-\mathbb{E}[X\mid Y=y]{)}^2\mid Y\right]$

(Total Variance) Any $X,Y$ satisfy $\mathrm{Var}(X)=\mathbb{E}[\mathrm{Var}(X\mid Y)]+\mathrm{Var}(\mathbb{E}[X\mid Y])$.

(Linearity of Covariance).

Examples

(Easy) Optimal Hedge Ratio

You just bought one share of stock $A$ and want to hedge it by shorting stock $B$. A short is an investment strategy where you borrow stock and sell it immediately rather than buy it, in the hopes that you will be able to buy it and return it at a lower price in the future. Assume that the variance of stock $A$‘s return is ${\sigma }_A^2$, the variance of stock $B$‘s return is ${\sigma }_B^2$, and their correlation coefficient is $\rho$.

How many shares of $B$ should you short to minimize the variance of the hedged position? Denote $P=A-kB$ as the value of your portfolio. We are trying to solve for $k^{\ast }=\arg \underset{k}{\min }\mathrm{Var}(P)$. We proceed as follows:

\frac{\delta}{\delta k}\operatorname{Var}(P)&= \frac{\delta}{\delta k}[\sigma^2_{A}+k^2\sigma^2_{B}-2k\rho \sigma_{A} \sigma_{B}] \
&= 2k\sigma^2_{B}-2\rho\sigma_{A}\sigma_{B} \
&= 0 \iff k^*= \rho\frac{\sigma_{A}}{\sigma_{B}},
\end{align}$$

which is known as the Beta coefficient in portfolio management, and is used to evaluate the relative risk of stock $A$ to stock $B$.

(Hard) Elevator Variance

A building has n upper floors numbered $1,2,\dots ,n$, plus a ground floor $G$. At the ground floor, $m$ people get on the elevator together, and each person gets off at one of the $n$ upper floors uniformly at random and independently of everyone else.

What is the variance of the number of floors the elevator does not stop at? $I_j$ be the indicator that nobody gets off at floor $j$. Then, let $N={\sum }_{j=1}^n{I}_j$ be the number of floors that the elevator does not stop at. The probability that a person exits on each form is uniform with a value of $\frac{1}{n}$. Since each person exits independently, then the probability that none of the $m$ people exit on each floor is ${\left(1-\frac{1}{n}\right)}^m$. As such, $\mathbb{E}(N)={\sum }_{j=1}^n\mathbb{E}(I_j)={\sum }_{j=1}^n{\left(1-\frac{1}{n}\right)}^m=n{\left(1-\frac{1}{n}\right)}^m.$

Let

Then, $\operatorname{Var}(N)=\mathbb{E}(X^2)-E(X)^2=n\left ( 1-\dfrac{1}{n} \right )^m - n^2\left ( 1-\dfrac{1}{n} \right )