This is an old revision of the document!
Warning: Undefined array key "do" in /home/levene/public_html/w/mst10030/lib/plugins/revealjs/action.php on line 14
Table of Contents
Chapter 2: The algebra of matrices
Definition
An $n\times m$ matrix is a grid of numbers with $n$ rows and $m$ columns: \[ A=\begin{bmatrix}a_{11}&a_{12}&\dots&a_{1m}\\a_{21}&a_{22}&\dots&a_{2m}\\\vdots&\vdots&&\vdots\\a_{n1}&a_{n2}&\dots&a_{nm}\end{bmatrix}\]
The $(i,j)$ entry of a matrix $A$ is $a_{ij}$, the number in row $i$ and column $j$ of $A$.
Examples
- If $B=\begin{bmatrix} 99&3&5\\7&-20&14\end{bmatrix}$, then $B$ is a $2\times 3$ matrix, and the $(1,1)$ entry of $B$ is $b_{11}=99$, the $(1,3)$ entry of $B$ is $b_{13}=5$, the $(2,1)$ entry is $b_{21}=7$, etc.
- $\begin{bmatrix}3\\2\\4\\0\\-1\end{bmatrix}$ is a $5\times 1$ matrix. A matrix like this with one column is called a column vector.
- $\begin{bmatrix}3&2&4&0&-1\end{bmatrix}$ is a $1\times 5$ matrix. A matrix like this with one row is called a row vector.
Even though the row matrix and the column matrix above have the same entries, they have a different “shape”, or “size”, so we must think of them has being different matrices. Let's give the definitions to make this precise.
Definition
Two matrices $A$ and $B$ have the same size if they have the same number of rows, and they have the same number of columns.
If two matrices do not have the same size, we say they have different sizes.
Definition
Two matrices $A$ and $B$ are said to be equal if both of the following conditions hold:
- $A$ and $B$ have the same size; and
- every entry of $A$ is equal to the corresponding entry of $B$; in other words, for every $(i,j)$ so that $A$ and $B$ have an $(i,j)$ entry, we have $a_{ij}=b_{ij}$.
When $A$ and $B$ are equal matrices, we write $A=B$. Otherwise, we write $A\ne B$.
Examples
- $\begin{bmatrix}3\\2\\4\\0\\-1\end{bmatrix}\ne \begin{bmatrix}3&2&4&0&-1\end{bmatrix}$, since these matrices have different sizes: the first is $5\times 1$ but the second is $1\times 5$.
- $\begin{bmatrix}1\\2\end{bmatrix}\ne\begin{bmatrix}1 &0\\2&0\end{bmatrix}$ since these matrices are not the same size.
- $\begin{bmatrix}1&0\\0&1\end{bmatrix}\ne \begin{bmatrix}1&0\\1&0\end{bmatrix}$ because even though they have the same size, the $(2,1)$ entries are different.
- If $\begin{bmatrix}3x&7y+2\\8z-3&w^2\end{bmatrix}=\begin{bmatrix}1&2z\\\sqrt2&9\end{bmatrix}$ then we know that all the corresponding entries are equal, so we get four equations:\begin{align*}3x&=1\\7y+2&=2z\\8z-3&=\sqrt2\\w^2&=9\end{align*}
Definition of matrix multiplication
If $A$ is an $n\times m$ matrix and $B$ is an $m\times k$ matrix, then the product $AB$ is the $n\times k$ matrix whose $(i,j)$ entry is the row-column product of the $i$th row of $A$ with the $j$th column of $B$. That is: \[ (AB)_{i,j} = \text{row}_i(A)\cdot \text{col}_j(B).\]
If we want to emphasize that we are multiplying matrices in this way, we might sometimes write $A\cdot B$ instead of $AB$.
If $A$ is an $n\times m$ matrix and $B$ is an $\ell\times k$ matrix with $m\ne \ell$, then the matrix product $AB$ is undefined.
Examples
- If $\newcommand{\mat}[1]{\begin{bmatrix}#1\end{bmatrix}} A=\mat{1&0&5\\2&-1&3}$ and $B=\mat{1&2\\3&4\\5&6}$, then $AB=\mat{26&32\\14&18}$ and $BA=\mat{5&-2&11\\11&-4&27\\17&-6&43}$. Note that $AB$ and $BA$ are both defined, but $AB\ne BA$ since $AB$ and $BA$ don't even have the same size.
- If $A=\mat{1&2\\3&4\\5&6}$, $B=\mat{2&1&1\\1&2&0\\1&0&2\\2&2&1}$ and $C=\mat{1&3&0&7\\0&4&6&8}$, then $A$ is $3\times 2$, $B$ is $4\times 3$ and $C$ is $2\times 4$, so
- $AB$, $CA$ and $BC$ don't exist (i.e., they are undefined);
- $AC$ exists and is $3\times 4$;
- $BA$ exists and is $4\times 2$; and
- $CB$ exists and is $2\times 2$.
- In particular, $AB\ne BA$ and $AC\ne CA$ and $BC\ne CB$, since in each case one of the matrices doesn't exist.
- If $A=\mat{0&1\\0&0}$ and $B=\mat{0&0\\1&0}$, then $AB=\mat{1&0\\0&0}$ and $BA=\mat{0&0\\0&1}$. So $AB$ and $BA$ are both defined and have the same size, but they are not equal matrices: $AB\ne BA$.
- If $A=\mat{1&0\\0&0}$ and $B=\mat{0&0\\0&1}$, then $AB=\mat{0&0\\0&0}$ and $BA=\mat{0&0\\0&0}$. So $AB=BA$ in this case.
- If $A=0_{n\times n}$ is the $n\times n$ zero matrix and $B$ is any $n\times n$ matrix, then $AB=0_{n\times n}$ and $BA=0_{n\times n}$. So in this case, we do have $AB=BA$.
- If $A=\mat{1&2\\3&4}$ and $B=\mat{7&10\\15&22}$, then $AB=\mat{37&54\\81&118}=BA$, so $AB=BA$ for these particular matrices $A$ and $B$.
- If $A=\mat{1&2\\3&4}$ and $B=\mat{6&10\\15&22}$, then $AB=\mat{36&54\\78&118}$ and $BA= \mat{36&52\\81&118}$, so $AB\ne BA$.
Commuting matrices
We say that matrices $A$ and $B$ commute if $AB=BA$.
Which matrices commute? Suppose $A$ is an $n\times m$ matrix and $B$ is an $\ell\times k$ matrix, and $A$ and $B$ commute, i.e., $AB=BA$.
- $AB$ must be defined, so $m=\ell$
- $BA$ must be defined, so $k=n$
- $AB$ is an $n\times k$ matrix and $BA$ is an $\ell\times n$ matrix. Since $AB$ has the same size as $BA$, we must have $n=\ell$ and $k=m$.
Putting this together: we see that if $A$ and $B$ commute, then $A$ and $B$ must both be $n\times n$ matrices for some number $n$. In other words, they must be square matrices of the same size.
Examples 4 and 5 above show that for some square matrices $A$ and $B$ of the same size, it is true that $A$ and $B$ commute. On the other hand, examples 3 and 6 show that it's not true that square matrices of the same size must always commute.
Because it's not true in general that $AB=BA$, we say that matrix multiplication is not commutative.
Definition of the $n\times n$ identity matrix
The $n\times n$ identity matrix is the $n\times n$ matrix $I_n$ with $1$s in every diagonal entry (that is, in the $(i,i)$ entry for every $i$ between $1$ and $n$), and $0$s in every other entry. So \[ I_n=\begin{bmatrix} 1&0&0&\dots&0\\0&1&0&\dots&0\\0&0&1&\dots&0\\\vdots & & &\ddots & \vdots\\0&0&0&\dots&1\end{bmatrix}.\]
Examples
- $I_1=[1]$
- $I_2=\mat{1&0\\0&1}$
- $I_3=\mat{1&0&0\\0&1&0\\0&0&1}$
- $I_4=\mat{1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1}$, and so on!
Proposition: properties of $I_n$
- $I_nA=A$ for any $n\times m$ matrix $A$;
- $AI_m=A$ for any $n\times m$ matrix $A$; and
- $I_nB=B=BI_n$ for any $n\times n$ matrix $B$. In particular, $I_n$ commutes with every other square $n\times n$ matrix $B$.
Proof of the proposition
1. We want to show that $I_nA=A$ for any $n\times m$ matrix $A$. These matrices the same size, since $I_n$ has size $n\times n$ and $A$ has size $n\times m$, so $I_n A$ has size $n\times m$ by the definition of matrix multiplication, which is the same as the size of $A$.
Note that $\text{row}_i(I_n)=[0~0~\dots~0~1~0~\dots~0]$, where the $1$ is in the $i$th place, by definition of the identity matrix $I_n$; and the $j$th column of $A$ is $\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix}$. The (i,j) entry of $I_nA$ is $\text{row}_i(I_n)\cdot \text{col}_j(A)$, by the definition of matrix multiplication, which is therefore \begin{align*} [0~0~\dots~0~1~0~\dots~0]\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix} &= 0a_{1j}+0a_{2j}+\dots+0a_{i-1,j}+1a_{ij}+0a_{i+1,j}+\dots+0a_{nj} \\&= a_{ij}.\end{align*} So the matrices $I_nA$ and $A$ have the same size, and the same $(i,j)$ entries, for any $(i,j)$. So $I_nA=A$.