4.4 Matrix Spaces

Matrix Spaces

Given a matrix $A\in {\mathbb{R}}^{m\times n}$, the range of $A$ (also referred to as the column space) is a vector subspace in ${\mathbb{R}}^m$ the represents the span of its columns, defined as
$\mathcal{R}(A)=\mathrm{span}({\vec{a}}_1,{\vec{a}}_2,\dots ,{\vec{a}}_n)=\left\{{\sum }_{i=1}^n{\alpha }_i{\vec{a}}_i:{\alpha }_i\in \mathbb{R}\right\}\subseteq {\mathbb{R}}^m,$
where ${\alpha }_i$ is the scaling factor (or weight) of the $i$-th column vector of $A$.

Similarly, the row space of $A$, denoted $\mathrm{Row}(A)\subseteq {\mathbb{R}}^n$, is the span of its rows. The row space of $A$ is related to the column space of $A^T$, such that $\mathrm{Row}(A)=\mathcal{R}(A^T)$ (and vice-versa).

Conversely, the null space (or kernel) of $A\in \mathbb{R}$ is a subspace of ${\mathbb{R}}^n$ that includes all vectors that output the zero vector when multiplied by $A$, such that
$\mathcal{N}(A)=\{x\in {\mathbb{R}}^n\mid Ax=0\}\subseteq {\mathbb{R}}^n.$

Per the Fundamental Theorem of Linear Algebra, every matrix $A$ naturally splits the domain (${\mathbb{R}}^n$) and codomain into two complementary parts each. Namely,
\begin{alignat}{0}\mathbb{R}^n = \mathcal{R}(A^T) \oplus \mathcal{N}(A), \quad \text{and} \quad \mathbb{R}^m = \mathcal{R}(A) \oplus \mathcal{N}(A^T).\end{alignat}

Proof

We start by observing that every $x\in \mathcal{R}(A^T)$ in the row space of $A$ is orthogonal to every $z\in \mathcal{N}(A)$. This follows from the fact that for some $x=A^{T}y$, it can be said that
$x^{T}z=y^{T}Az=y^{T}0=0.$
Thus, $\mathcal{R}(A^T)\perp \mathcal{N}(A)$, or equivalently, $\mathcal{N}(A)=\mathcal{R}(A^T{)}^{\perp }$. We know that a direct sum of a subspace and its orthogonal complement equals the whole subspace, and thus,
$\mathcal{R}(A^T)\oplus \mathcal{R}(A^T{)}^{\perp }=\mathcal{R}(A^T)\oplus \mathcal{N}(A)={\mathbb{R}}^n.$
By the same reasoning, we can prove the fact for ${\mathbb{R}}^m$.