2.12d.2 Compact SVD

Compact SVD

A matrix $A\in {\mathbb{R}}^{m\times n}$ with rank $r\le \min \{m,n\}$ can be decomposed using the compact form of the SVD as
$A=U_r{\text{Upsigma}}_r{V}_r^T$
where $U_r\in {\mathbb{R}}^{m\times r}$ and $V_r\in {\mathbb{R}}^{n\times r}$ are matrices with orthogonal columns known respectively as the left singular vectors and right singular vectors of $A$, whereas ${\text{Upsigma}}_r\in {\mathbb{R}}^{r\times r}$ is a diagonal matrix whose diagonal entries are positive ordered scalars known as the singular values of $A$.

Intuition

The compact SVD is connected to the outer product SVD as follows:

$$\begin{array}{rl}\left[\begin{array}{ccc}{\vec{u}}_1& \dots & {\vec{u}}_r\end{array}\right]\left[\begin{array}{ccc}{\sigma }_1& & \\ & \ddots & \\ & & {\sigma }_r\end{array}\right]{\left[\begin{array}{ccc}{\vec{v}}_1& \dots & {\vec{v}}_r\end{array}\right]}^T& =\left[\begin{array}{ccc}{\vec{u}}_1& \dots & {\vec{u}}_r\end{array}\right]\left[\begin{array}{ccc}{\sigma }_1& & \\ & \ddots & \\ & & {\sigma }_r\end{array}\right]\left[\begin{array}{c}{\vec{v}}_1^T\\ ⋮\\ {\vec{v}}_r^T\end{array}\right]\\ & =\left[\begin{array}{ccc}{\vec{u}}_1& \dots & {\vec{u}}_r\end{array}\right]\left[\begin{array}{c}{\sigma }_1{\vec{v}}_1^T\\ ⋮\\ {\sigma }_r{\vec{v}}_r^T\end{array}\right]\\ & =\sum _{i=1}^r{\sigma }_i{\vec{u}}_i{\vec{v}}_i^T\end{array}$$

Additionally, the compact SVD has a square and invertible ${\text{Upsigma}}_r$ and is easier than the full SVD to construct by hand, but $U_r$ and $V_r$ aren’t square and do not have an inverse, and there is no characterization of null spaces like in the full SVD.

Properties

We can see that our properties are simply the properties of the outer product SVD in matrix form.

$\mathrm{Col}(U_r)=\mathrm{Col}(A).$

Same as outer product SVD.

$\mathrm{Col}(V_r)=\mathrm{Row}(A)=\mathrm{Col}(A^T).$

Same as outer product SVD.