13.2 Moore-Penrose Pseudoinverse

Moore-Penrose Pseudoinverse

Any matrix $A\in {\mathbb{R}}^{m\times n}$ with rank $r\le \min (m,n)$ and SVD $A=U\Sigma V^T$ has a Moore-Penrose pseudoinverse, defined as

$$A^{†}=V{\Sigma }^{†}{U}^T\in {\mathbb{R}}^{n\times m},$$

where ${\Sigma }^{†}=\left[\begin{array}{cc}{\text{Upsigma}}_r^{-1}& 0_{r\times (m-r)}\\ 0_{(n-r)\times r}& 0_{(n-r)\times (m-r)}\end{array}\right]\in {\mathbb{R}}^{n\times m}$, with ${\text{Upsigma}}_r^{-1}=\mathrm{diag}(\frac{1}{{\sigma }_1,},\dots ,\frac{1}{{\sigma }_r})\in {\mathbb{S}}_{++}^r$.

Properties

(Left Inverse) If $\mathrm{Rank}(A)=n\le m$ (full col. rank), $A^{†}A=V_r{V}_r^T=V{V}^T=I_n$.

(Right Inverse) If $\mathrm{Rank}(A)=m\le n$ (full row rank), $A{A}^{†}=U_r{U}_r^T=U{U}^T=I_m$.

(Left Decomposition) If $\mathrm{Rank}(A)=n\le m$, then $A^{†}=(A^{T}A{)}^{-1}{A}^T$.

(Right Decomposition) If $\mathrm{Rank}(A)=m\le n$, then $A^{†}=A^T(A{A}^T{)}^{-1}$.

(Reflexivity) $A{A}^{†}A=A$ and $A^{†}A{A}^{†}=A^{†}$.

(Invertibility Equality) If $A$ is invertible, $A^{†}=A^{-1}$.

(Orthogonal Projectors) $P_{\mathcal{R}(A)}=A{A}^{†}=U_r{U}_r^T$ and $P_{}{A}^{†}A=V_r{V}_r^T$.