5.3 Distribution of Markov Chains

Markov Chain Distribution

The probability mass function ${\pi }_n:\mathcal{X}\to [0,1]$ (or ${\pi }_n\in {\mathbb{R}}^{1\times |\mathcal{X}|}$) of the random variable $X_n$ represents the probability of the Markov chain being in each possible state in $\mathcal{X}$ at time $n$. It can be used in tandem with the transition probability matrix $P$ to derive the $n$-th distribution in the sequence, such that ${\pi }_n={\pi }_{n-1}P$, or equivalently,
${\pi }_n={\pi }_0{P}^n.$

If ${\pi }^{\ast }={\pi }^{\ast }P$, then ${\pi }^{\ast }$ is said to be an invariant distribution. Per the Big Theorem, an irreducible Markov chain has a unique invariant distribution ${\pi }^{\ast }$, such that ${\pi }^{\ast }(x)$ is the long-term fraction of time that $X_n$ is equal to $x$, defined as
${\pi }^{\ast }(x)={\lim }_{n\to \infty }\frac{1}{n}{\sum }_{i=0}^{n-1}{1}_{\{X_i=x\}}=\frac{1}{\mathbb{E}[T_x\mid X_0=x]}.$

Furthermore, if the Markov chain is also aperiodic, then any ${\pi }_n$ converges to $\pi$, such that $\underset{n\to \infty }{\lim }{\pi }_n(x)={\sum }_{y\in \mathcal{X}}^{}\underset{n\to \infty }{\lim }{\pi }_0(x)P^n(y,x)={\pi }^{\ast }(x).$

Properties

(Balance Equation) ${\pi }^{\ast }(x)=\sum _{y\in \mathcal{X}}{\pi }^{\ast }(y)P(y,x)$