12.2 Eigendecomposition

Eigendecomposition

A square matrix $A\in {\mathbb{R}}^{n\times n}$ is diagonalizable if it admits $n$ linearly independent eigenvectors, allowing it to be decomposed as
$A=V\text{Uplambda}{V}^{-1},$
where $V=\left[\begin{array}{cccc}{\vec{v}}_1& {\vec{v}}_2& \dots & {\vec{v}}_n\end{array}\right]$ forms the eigenbasis of $A$, and $\text{Uplambda}=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$ is a diagonal square matrix that holds the strictly real eigenvalues of $A$.

The Spectral Theorem extends this result to symmetric matrices ($A=A^T$), ensuring that they can be diagonalized as
$A={\sum }_{i=1}^n{\lambda }_i{q}_i{q}_i^T={\sum }_{i=1}^n{\lambda }_i{\Pi }_{\mathrm{span}(u_i)}=Q\Lambda Q^T,$ where $Q$‘s columns form an orthonormal eigenbasis of $A$, and $\text{Uplambda}=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$.

Intuition

We use diagonalization to make linear transformations between matrices easier. We do so by changing the basis of a matrix $A$ so that it is strictly diagonal. We then perform our necessary calculations and change it back to our initial basis.

Why eigenvectors and eigenvalues?

Essentially,

Properties

$A^n=V{\text{Uplambda}}^n{V}^{-1}$

$$\begin{array}{rl}{A}^n& =(V\text{Uplambda}{V}^{-1})(V\text{Uplambda}{V}^{-1})...(V\text{Uplambda}{V}^{-1})\\ & =V\text{Uplambda}(V^{-1}V)\text{Uplambda}(V^{-1}V)...\text{Uplambda}{V}^{-1}\\ & =V{\text{Uplambda}}^n{V}^{-1}\end{array}$$

Example

Computing Powers of Matrices

We want to compute the 5th power of $A=\left[\begin{array}{cc}5& 3\\ 0& 4\end{array}\right]$. This is hard, since we will have to perform matrix-matrix multiplication 4 times. Instead, we will find the eigenvalues and eigenvectors of $A$, change its basis, perform the calculation and change it back into standard basis. Using simple calculation we find the eigenvalues ${\lambda }_{1,2}=5,4$. The corresponding eigenvectors are therefore ${\vec{x}}_{{\lambda }_1}=\left[\begin{array}{c}-3\\ 1\end{array}\right]$ and ${\vec{x}}_{{\lambda }_2}=\left[\begin{array}{c}1\\ 0\end{array}\right]$. As such, the eigenbasis of $A$ is $V=\left[\begin{array}{cc}-3& 1\\ 1& 0\end{array}\right]$. Its inverse is $V^{-1}=\left[\begin{array}{cc}0& 1\\ 1& 3\end{array}\right]$ and the diagonal matrix of eigenvectors is $\text{Uplambda}=\left[\begin{array}{cc}4& 0\\ 0& 5\end{array}\right]$. We can then perform the computation extremely easily:
${\text{Uplambda}}^5=\left[\begin{array}{cc}{5}^5& 0\\ 0& 4^5\end{array}\right]=\left[\begin{array}{cc}1024& 0\\ 0& 3125\end{array}\right].$
A quick change of basis to reverse the diagonalization will yield:
$V{\text{Uplambda}}^5{V}^{-1}=\left[\begin{array}{cc}3125& 6303\\ 0& 1024\end{array}\right]=A^5.$