5.1 Eigenvectors

Eigenvector

An eigenvector of a square matrix $A\in {\mathbb{R}}^{n\times n}$ is a nonzero vector $\vec{v}\in {\mathbb{R}}^n$ that does not change direction when $A$ is applied to it. Instead of being rotated or distorted, it is simply stretched or shrunk by both $A$ and the roots of the characteristic polynomial $\det (A-\lambda I)$, called the eigenvalues of $A$. Formally,
$A\vec{v}=\lambda \vec{v},$
where $\lambda$ is an eigenvalue of $A$.

Eigenvector Triviality

Eigenvector equations always have infinitely many solutions (or none). The trick is to find free variables in the system.

Intuition

Suppose we applied a linear transformation $A$ to a known vector $\vec{x}$. However, even with the transformation, the resulting vector $\vec{z}=A\vec{x}$ is just a linear combination of $\vec{x}$ ($\mathrm{Span}(\vec{x})=\mathrm{Span}(\vec{z})$), scaled by the factor $\lambda$. Therefore, $\vec{z}=\lambda \vec{x}$ too! Such a vector $\vec{x}$ that remains on its span after a linear transformation is an eigenvector of $A$, whereas $\lambda$ is the eigenvalue is the scaling factor of that eigenvector when $A$ is applied. However, we are usually tasked with finding the eigenvectors of $A$.

To find such an eigenvector $\vec{x}$, we must solve for $A\vec{x}-\lambda \vec{x}=(A-\lambda I)\vec{x}=\vec{0}$. We do this by calculating $\det (A-\lambda I)=0$, or the characteristic polynomial of $A$. This is because the determinant of a matrix is 0 when the matrix is not in full rank, and therefore squishes the possibilities of the span on a single line with area 0. We can say that the eigenvalues of $A$ “throw it off” from linear independence (if it was ever in such) in order to squish it into the lower-dimensional subspace $\text{span}(\vec{x})$.

Thus, formally, an eigenvector of $A$ is any nonzero vector that is a scalar multiple of itself ($\text{span}(\vec{x})=\text{span}(A\vec{x})$) when multiplied by $A$.

Then, the resulting eigenvalue when $A$ is applied to an eigenvector $\vec{x}$ is essentially the scaling factor of the transformation We simply use eigenvalues to make our lives much easier when applying transformations.

Eigenvalues also do not have to be real or unique. Check out the examples below to see examples of nonstandard eigenvalue cases.

Properties

(Rayleigh Quotient) For some real symmetric matrix $A$, $R(A)=\frac{x^{T}Ax}{\Vert x{\Vert }^2}$, such that ${\lambda }_{\max }^{(A)}=\underset{\Vert x\Vert =1}{\max }R(A)$, and ${\lambda }_{\min }^{(A)}=\underset{\Vert x\Vert =1}{\min }R(A)$.

Examples

Repeated Eigenvalues

Consider the matrix $A=\left[\begin{array}{cc}2& 1\\ 0& 2\end{array}\right]$. Since the diagonal entries of $A$ are equal and the far entries have a product of 0, this matrix will yield a single eigenvalue $\lambda =2$. There are no eigenvectors associated with this matrix.

Complex Eigenvalues

Consider the rotation matrix $$