4.1 Matrix

Matrix

A matrix $A\in {\mathbb{R}}^{r\times c}$ is a rectangular arrangement of elements into $r$ rows and $c$ columns, written in the form

\begin{bmatrix} \vert & \vert & & \vert \\ \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_c \\ \vert & \vert & & \vert \\ \end{bmatrix} = \begin{bmatrix} A_{11} & A_{12} & \dots & A_{1c} \\ A_{21} & A_{22} & ... & A_{2c} \\ \vdots & \vdots & \ddots & \vdots \\ A_{r1} & A_{r2} & ... & A_{rc} \end{bmatrix} \in \mathbb{R}^{r \times c},$$ where each element $A_{r_ic_i}$ represents the element located at column index $c_i$, and row index $r_i$. The transpose of $A$, denoted $A^T$, is obtained by flipping the matrix along its diagonal, such that $$A^T= \begin{bmatrix} \enspace — & \vec{a}_1^T & — \enspace \\

\enspace — & \vec{a}_2^T & — \enspace \

& \vdots & \
\enspace — & \vec{a}_c^T & — \enspace
\end{bmatrix}

\begin{bmatrix}
A_{11} & A_{21} & \dots & A_{r1} \
A_{12} & A_{22} & \dots & A_{r2} \
\vdots & \vdots & \ddots & \vdots \
A_{1r} & A_{2r}& \dots & A_{rc}
\end{bmatrix}
\in \mathbb{R}^{c \times r},

meaning that $A$'s rows become $A^T$'s columns and vice-versa.

Properties

$A{A}^T\in {\mathbb{R}}^{m\times m}$ and $A^{T}A\in {\mathbb{R}}^{n\times n}$ are always symmetric, positive and real for any $A\in {\mathbb{R}}^{m\times n}$.

1. We can easily prove that the product is symmetric by taking $(A{A}^T{)}^T=(A^T{)}^T{A}^T=A{A}^T$. Same can be said the other way.
2. We can prove that the product is real by

Example

Water Reservoirs

Imagine a city with 3 water reservoirs, labeled A, B, and C. Each day, $\frac{1}{2}$ of the water in reservoir A is transferred to reservoir B while the remainder stays in A, $\frac{1}{4}$ of the water in B goes to A while the rest is transferred to C, and the water in C is evenly split between the reservoirs. We can illustrate this change of state in the following Markov chain:

\begin{document}
  \begin{tikzpicture}[->, auto, semithick, node distance=2cm] 
  	% Nodes 
  	\node (A) at (0,0) [circle, draw] {A}; 
  	\node (B) at (2,2) [circle, draw] {B}; 
  	\node (C) at (4,0) [circle, draw] {C}; 
  	% Edges with labels 
  	\draw (A) to[bend left=20] node[above] {$\frac{1}{2}$} (B); 
  	\draw (B) to[bend left=20] node[right] {$\frac{1}{4}$} (A); 
  	\draw (B) to[bend left=20] node[above] {$\frac{1}{3}$} (C); 
  	\draw (C) to[bend left=20] node[below] {$\frac{3}{4}$} (B); 
  	\draw (C) to[bend left=20] node[below] {$\frac{1}{3}$} (A); 
  	\draw (A) to[loop left] node[left] {$\frac{1}{2}$} (A); 
  	\draw (B) to[loop above] node[above] {$\frac{1}{2}$} (B); 
  	\draw (C) to[loop right] node[right] {$\frac{1}{3}$} (C); 
  \end{tikzpicture}
\end{document}

This change of state can be described using the following state-transition matrix,

P = \begin{bmatrix} P_{A \to A} & P_{B \to A} & P_{C \to A} \\ P_{A \to B} & P_{B \to B} & P_{C \to B} \\ P_{A \to C} & P_{B \to C} & P_{C \to C}

\end{bmatrix} =

\begin{bmatrix}
\frac{1}{2} & \frac{1}{4} & \frac{1}{3} \
\frac{1}{2} & 0 & \frac{1}{3} \
0 & \frac{3}{4} & \frac{1}{3}
\end{bmatrix}
.

$$Thecolumnvectorsofthismatrixdescribetheoutflowofeachreservoir\ast \ast (orthecurrentwaterineachreservoir)\ast \ast ,whiletherowvectorsdescribetheinflowofeachreservoir\ast \ast (orthewaterineachreservoirinthenextstep)\ast \ast .Notethateachcolumnvectoraddupto1,whichmeansthatwatercannotbelostintheflowingprocess.$$