3.2 Laws of Large Numbers

Laws of Large Numbers

The Weak Law of Large Numbers is a direct consequence of Chebyshev’s inequality and describes the convergence of sample averages to the expected value. Formally, if $X_1,X_2,\dots$ are i.i.d. with $\mathbb{E}[X_i]=\mu$ and $\mathrm{Var}(X_i)={\sigma }^2<\infty$, then for every $\epsilon >0$,
$P\left(\left|\frac{1}{n}{\sum }_{i=1}^n{X}_i-\mu \right|\ge \epsilon \right)\to 0\text{as}n\to \infty .$
Since the variance of the sample mean is ${\sigma }^2/n$, Chebyshev’s inequality shows that the probability of a deviation of size $\epsilon \to 0$ as $n\to \infty$, establishing convergence in probability.

Proof