Definition of matrix multiplication

Examples

1. If $\newcommand{\mat}[1]{\begin{bmatrix}#1\end{bmatrix}} A=\mat{1&0&5\\2&-1&3}$ and $B=\mat{1&2\\3&4\\5&6}$, then $AB=\mat{26&32\\14&18}$ and $BA=\mat{5&-2&11\\11&-4&27\\17&-6&43}$. Note that $AB$ and $BA$ are both defined, but $AB\ne BA$ since $AB$ and $BA$ don't even have the same size.
2. If $A=\mat{1&2\\3&4\\5&6}$, $B=\mat{2&1&1\\1&2&0\\1&0&2\\2&2&1}$ and $C=\mat{1&3&0&7\\0&4&6&8}$, then $A$ is $3\times 2$, $B$ is $4\times 3$ and $C$ is $2\times 4$, so
   • $AB$, $CA$ and $BC$ don't exist (i.e., they are undefined);
   • $AC$ exists and is $3\times 4$;
   • $BA$ exists and is $4\times 2$; and
   • $CB$ exists and is $2\times 2$.
   • In particular, $AB\ne BA$ and $AC\ne CA$ and $BC\ne CB$, since in each case one of the matrices doesn't exist.
3. If $A=\mat{0&1\\0&0}$ and $B=\mat{0&0\\1&0}$, then $AB=\mat{1&0\\0&0}$ and $BA=\mat{0&0\\0&1}$. So $AB$ and $BA$ are both defined and have the same size, but they are not equal matrices: $AB\ne BA$.
4. If $A=\mat{1&0\\0&0}$ and $B=\mat{0&0\\0&1}$, then $AB=\mat{0&0\\0&0}$ and $BA=\mat{0&0\\0&0}$. So $AB=BA$ in this case.
5. If $A=0_{n\times n}$ is the $n\times n$ zero matrix and $B$ is any $n\times n$ matrix, then $AB=0_{n\times n}$ and $BA=0_{n\times n}$. So in this case, we do have $AB=BA$.
6. If $A=\mat{1&2\\3&4}$ and $B=\mat{7&10\\15&22}$, then $AB=\mat{37&54\\81&118}=BA$, so $AB=BA$ for these particular matrices $A$ and $B$.
7. If $A=\mat{1&2\\3&4}$ and $B=\mat{6&10\\15&22}$, then $AB=\mat{36&54\\78&118}$ and $BA= \mat{36&52\\81&118}$, so $AB\ne BA$.

Commuting matrices

Which matrices commute? Suppose $A$ is an $n\times m$ matrix and $B$ is an $\ell\times k$ matrix, and $A$ and $B$ commute, i.e., $AB=BA$.

• $AB$ must be defined, so $m=\ell$
• $BA$ must be defined, so $k=n$
• $AB$ is an $n\times k$ matrix and $BA$ is an $\ell\times n$ matrix. Since $AB$ has the same size as $BA$, we must have $n=\ell$ and $k=m$.

Putting this together: we see that if $A$ and $B$ commute, then $A$ and $B$ must both be $n\times n$ matrices for some number $n$. In other words, they must be square matrices of the same size.

Examples 4 and 5 above show that for some square matrices $A$ and $B$ of the same size, it is true that $A$ and $B$ commute. On the other hand, examples 3 and 6 show that it's not true that square matrices of the same size must always commute.

Because it's not true in general that $AB=BA$, we say that matrix multiplication is not commutative.

Definition of the $n\times n$ identity matrix

Examples

1. $I_1=[1]$
2. $I_2=\mat{1&0\\0&1}$
3. $I_3=\mat{1&0&0\\0&1&0\\0&0&1}$
4. $I_4=\mat{1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1}$, and so on!

Proposition: properties of $I_n$

1. $I_nA=A$ for any $n\times m$ matrix $A$;
2. $AI_m=A$ for any $n\times m$ matrix $A$; and
3. $I_nB=B=BI_n$ for any $n\times n$ matrix $B$. In particular, $I_n$ commutes with every other square $n\times n$ matrix $B$.

Proof of the proposition

1. We want to show that $I_nA=A$ for any $n\times m$ matrix $A$. These matrices the same size, since $I_n$ has size $n\times n$ and $A$ has size $n\times m$, so $I_n A$ has size $n\times m$ by the definition of matrix multiplication, which is the same as the size of $A$.

Note that $\text{row}_i(I_n)=[0~0~\dots~0~1~0~\dots~0]$, where the $1$ is in the $i$th place, by definition of the identity matrix $I_n$; and the $j$th column of $A$ is $\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix}$. The (i,j) entry of $I_nA$ is $\text{row}_i(I_n)\cdot \text{col}_j(A)$, by the definition of matrix multiplication, which is therefore \begin{align*} [0~0~\dots~0~1~0~\dots~0]\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix} &= 0a_{1j}+0a_{2j}+\dots+0a_{i-1,j}+1a_{ij}+0a_{i+1,j}+\dots+0a_{nj} \\&= a_{ij}.\end{align*} So the matrices $I_nA$ and $A$ have the same size, and the same $(i,j)$ entries, for any $(i,j)$. So $I_nA=A$.

Proof of the proposition, continued

2. To show that $AI_m=A$ for any $n\times m$ matrix $A$ is similar to the first part of the proof; the details are left as an exercise.

3. If $B$ is any $n\times n$ matrix, then $I_nB=B$ by part 1 and $BI_n=B$ by part 2, so $I_nB=B=BI_n$. In particular, $I_nB=BI_n$ so $I_n$ commutes with $B$, for every square $n\times n$ matrix $B$. ■

Algebraic properties of matrix multiplication

The associative law

Proposition: associativity of matrix multiplication

Matrix multiplication is associative. This means that $(AB)C=A(BC)$ whenever $A,B,C$ are matrices which can be multiplied together in this order.

We omit the proof, but this is not terribly difficult; it is a calculation in which you write down two formulae for the $(i,j)$ entries of $(AB)C$ and $A(BC)$, and carefully check they are equal using the fact that if $a,b,c$ are real numbers, then $(ab)c=a(bc)$.

Example

We saw above that $\newcommand{\m}[1]{\begin{bmatrix}#1\end{bmatrix}}A=\m{1&2\\3&4}$ commutes with $B=\m{7&10\\15&22}$. We can explain why this is so using associativity. You can check that $B=AA$ (which we usually write as $B=A^2$). Hence, using associativity at $\stackrel*=$, \[ AB=A(AA)\stackrel*=(AA)A=BA.\] The same argument for any square matrix $A$ gives a proof of:

Proposition

If $A$ is any square matrix, then $A$ commutes with $A^2$.■

The powers of a square matrix $A$ are defined by $A^1=A$, and $A^{k+1}=A(A^k)$ for $k\in \mathbb{N}$. Using mathematical induction, you can prove the following more general proposition.

Proposition: a square matrix commutes with its powers

If $A$ is any square matrix and $k\in\mathbb{N}$, then $A$ commutes with $A^k$.■

The distributive laws

Lemma: the distributive laws for row-column multiplication

1. If $a$ is a $1\times m$ row vector and $b$ and $c$ are $m\times 1$ column vectors, then $a\cdot (b+c)=a\cdot b+a\cdot c$.
2. If $b$ and $c$ are $1\times m$ row vectors and $a$ is an $m\times 1$ column vector, then $(b+c)\cdot a=b\cdot a+c\cdot a$.

The proof is an exercise (see tutorial worksheet 5).

Proposition: the distributive laws for matrix multiplication

If $A$ is an $n\times m$ matrix and $k\in\mathbb{N}$, then:

1. $A(B+C)=AB+AC$ for any $m\times k$ matrices $B$ and $C$; and
2. $(B+C)A=BA+CA$ for any $k\times n$ matrices $B$ and $C$.

In other words, $A(B+C)=AB+AC$ whenever the matrix products make sense, and similarly $(B+C)A=BA+CA$ whenever this makes sense.

Proof

1. First note that

• $B$ and $C$ are both $m\times k$, so $B+C$ is $m\times k$ by the definition of matrix addition;
• $A$ is $n\times m$ and $B+C$ is $m\times k$, so $A(B+C)$ is $m\times k$ by the definition of matrix multiplication;
• $AB$ and $AC$ are both $n\times k$ by the definition of matrix multiplication
• so $AB+AC$ is $n\times k$ by the definition of matrix addition.

So we have (rather long-windedly) checked that $A(B+C)$ and $AB+AC$ have the same size.

By the Lemma above, the row-column product has the property that \[a\cdot (b+c)=a\cdot b+a\cdot c.\] So the $(i,j)$ entry of $A(B+C)$ is \begin{align*}\def\row{\text{row}}\def\col{\text{col}} \text{row}_i(A)\cdot \col_j(B+C) &= \text{row}_i(A)\cdot \big(\col_j(B)+\col_j(C)\big) \\ &= \text{row}_i(A)\cdot \col_j(B)+\row_i(A)\cdot\col_j(C).\end{align*} On the other hand,

• the $(i,j)$ entry of $AB$ is $\text{row}_i(A)\cdot \col_j(B)$; and
• the $(i,j)$ entry of $AC$ is $\row_i(A)\cdot\col_j(C)$;
• so the $(i,j)$ entry of $AB+AC$ is also $\text{row}_i(A)\cdot \col_j(B)+\row_i(A)\cdot\col_j(C)$.

So the entries of $A(B+C)$ and $AB+AC$ are all equal, so $A(B+C)=AB+AC$.

2. The proof is similar, and is left as an exercise.■

Matrix equations

We've seen that a single linear equation can be written using row-column multiplication. For example, \[ 2x-3y+z=8\] can be written as \[ \def\m#1{\begin{bmatrix}#1\end{bmatrix}}\m{2&-3&1}\m{x\\y\\z}=8\] or \[ a\vec x=8\] where $a=\m{2&-3&1}$ and $\vec x=\m{x\\y\\z}$.

We can write a whole system of linear equations in a similar way, as a matrix equation using matrix multiplication. For example we can rewrite the linear system \begin{align*} 2x-3y+z&=8\\ y-z&=4\\x+y+z&=0\end{align*} as \[ \m{2&-3&1\\0&1&-1\\1&1&1}\m{x\\y\\z}=\m{8\\4\\0},\] or \[ A\vec x=\vec b\] where $A=\m{2&-3&1\\0&1&-1\\1&1&1}$, $\vec x=\m{x\\y\\z}$ and $\vec b=\m{8\\4\\0}$. (We are writing the little arrow above the column vectors here because otherwise we might get confused between the $\vec x$: a column vector of variables, and $x$: just a single variable).

More generally, any linear system \begin{align*} a_{11}x_1+a_{12}x_2+\dots+a_{1m}x_m&=b_1\\ a_{21}x_1+a_{22}x_2+\dots+a_{2m}x_m&=b_2\\ \hphantom{a_{11}}\vdots \hphantom{x_1+a_{22}}\vdots\hphantom{x_2+\dots+{}a_{nn}} \vdots\ & \hphantom{{}={}\!} \vdots\\ a_{n1}x_1+a_{n2}x_2+\dots+a_{nm}x_m&=b_n \end{align*} can be written in the form \[ A\vec x=\vec b\] where $A$ is the $n\times m $ matrix, called the coefficient matrix of the linear system, whose $(i,j)$ entry is $a_{ij}$ (the number in front of $x_j$ in the $i$th equation of the system) and $\vec x=\m{x_1\\x_2\\\vdots\\x_m}$, and $\vec b=\m{b_1\\b_2\\\vdots\\b_n}$.

More generally still, we might want to solve a matrix equation like \[AX=B\] where $A$, $X$ and $B$ are matrices of any size, with $A$ and $B$ fixed matrices and $X$ a matrix of unknown variables. Because of the definition of matrix multiplication, if $A$ is $n\times m$, we need $B$ to be $n\times k$ for some $k$, and then $X$ must be $m\times k$, so we know the size of any solution $X$. But which $m\times k$ matrices $X$ are solutions?

Example

If $A=\def\m#1{\begin{bmatrix}#1\end{bmatrix}}\m{1&0\\0&0}$ and $B=0_{2\times 3}$, then any solution $X$ to $AX=B$ must be $2\times 3$.

One solution is $X=0_{2\times 3}$, since in this case we have $AX=A0_{2\times 3}=0_{2\times 3}$.

However, this is not the only solution. For example, $X=\m{0&0&0\\1&2&3}$ is another solution, since in this case \[AX=\m{1&0\\0&0}\m{0&0&0\\1&2&3}=\m{0&0&0\\0&0&0}=0_{2\times 3}.\]

So from this example, we see that a matrix equation can have many solutions.

Invertibility

We've seen that solving matrix equations $AX=B$ is useful, since they generalise systems of linear equations.

How can we solve them?

Example

Take $A=\def\mat#1{\begin{bmatrix}#1\end{bmatrix}}\mat{2&4\\0&1}$ and $B=\mat{3&4\\5&6}$, so we want to find all matrices $X$ so that $AX=B$, or \[ \mat{2&4\\0&1}X=\mat{3&4\\5&6}.\] Note that $X$ must be a $2\times 2$ matrix for this to work, by the definition of matrix multiplication. So one way to solve this is to write $X=\mat{x_{11}&x_{12}\\x_{21}&x_{22}}$ and plug it in: \[\mat{2&4\\0&1}\mat{x_{11}&x_{12}\\x_{21}&x_{22}}=\mat{3&4\\5&6}\iff \mat{2x_{11}+4x_{21}&2 x_{12}+4x_{22}\\x_{21}&x_{22}}=\mat{3&4\\5&6}\] and then equate entries to get four linear equations: \begin{align*}2x_{11}+4x_{21}&=3\\2 x_{12}+4x_{22}&=4\\x_{21}&=5\\x_{22}&=6\end{align*} which we can solve in the usual way.

But this is a bit tedious! We will develop a slicker method by first thinking about solving ordinary equations $ax=b$ where $a,x,b$ are all numbers, or if you like, $1\times 1$ matrices.

Solving $ax=b$ and $AX=B$

If $a\ne0$, then solving $ax=b$ where $a,b,x$ are numbers is easy. We just divide both sides by $a$, or equivalently, we multiply both sides by $\tfrac1a$, to get the solution: $x=\tfrac1a\cdot b$.

Why does this work? If $x=\tfrac1a\cdot b$, then \begin{align*} ax&=a(\tfrac1a\cdot b)\\&=(a\cdot \tfrac1a)b\\&=1b\\&=b\end{align*} so $ax$ really is equal to $b$, and we do have a solution to $ax=b$.

What is special about $\tfrac1a$ which made this all work?

1. we have $a\cdot \tfrac1a = 1$,
2. and $1b = b$.

Now for an $n\times k$ matrix $B$, we know that the identity matrix $I_n$ does the same sort of thing as $1$ is doing in the relation $1b=b$: we have $I_nB=B$ for any $n\times k$ matrix $B$. So instead of $\tfrac1a$, we want to find a matrix $C$ with the property: $AC=I_n$. In fact, because matrix multiplication is not commutative, we also require that $CA=I_n$. It's then easy to argue that $X=C\cdot B$ is a solution to $AX=B$, since \begin{align*} AX&=A(CB)\\&=(AC)B\\&=I_nB\\&=B.\end{align*}

Example revisited

If $A=\mat{2&4\\0&1}$, then the matrix $C=\mat{\tfrac12&-2\\0&1}$ does have the property that \[ A C=I_2=C A.\] (You should check this!). So a solution to $AX=B$ where $B=\mat{3&4\\5&6}$ is $X=CB=\mat{\tfrac12&-2\\0&1}\mat{3&4\\5&6} = \mat{-8.5&-10\\5&6}$.

Notice that having found the matrix $C$, then we can solve $AX=C$ easily for any $2\times 2$ matrix $C$: the answer is $X=CC$. This is quicker than having to solve four new linear equations using our more tedious method above.

Definition: invertible

Examples

• $A=\mat{2&4\\0&1}$ is invertible, and the matrix $C=\mat{\tfrac12&-2\\0&1}$ is an inverse of $A$
• a $1\times 1$ matrix $A=[a]$ is invertible if and only if $a\ne0$, and if $a\ne0$ then an inverse of $A=[a]$ is $C=[\tfrac1a]$.
• $I_n$ is invertible for any $n$, since $I_n\cdot I_n=I_n=I_n\cdot I_n$, so an inverse of $I_n$ is $I_n$.
• $0_{n\times n}$ is not invertible for any $n$, since $0_{n\times n}\cdot C=0_{n\times n}$ for any $n\times n$ matrix $C$, so $0_{n\times n}\cdot C\ne I_n$.
• $A=\mat{1&0\\0&0}$ is not invertible, since for any $2\times 2$ matrix $C=\mat{a&b\\c&d}$ we have $AC=\mat{a&b\\0&0}$ which is not equal to $I_2=\mat{1&0\\0&1}$ since the $(2,2)$ entries are not equal.
• $A=\mat{1&2\\-3&-6}$ is not invertible. We'll see why later!

Proposition: uniqueness of the inverse

If $A$ is an invertible $n\times n$ matrix, then $A$ has a unique inverse.

Proof

Suppose $C$ and $C'$ are both inverses of $A$. Then $AC=I_n=CA$ and $AC'=I_n=C'A$. So \begin{align*} C&=CI_n\quad\text{by the properties of $I_n$}\\&=C(AC')\quad\mbox{because }AC'=I_n\\&=(CA)C'\quad\mbox{because matrix multiplication is associative}\\&=I_nC'\quad\mbox{because }CA=I_n\\&=C'\quad\text{by the properties of $I_n$}.\end{align*} So $C=C'$, whenever $C$ and $C'$ are inverses of $A$. So $A$ has a unique inverse. ■

Definition/notation: $A^{-1}$

If a matrix $A$ is not invertible, then $A^{-1}$ does not exist.

Warning

If $A$ is a matrix then $\frac 1A$ doesn't make sense! You should never write this down. In particular, $A^{-1}$ definitely doesn't mean $\frac 1A$.

Similarly, you should never write down $\frac AB$ where $A$ and $B$ are matrices. This doesn't make sense either!

Examples revisited

• $A=\mat{2&4\\0&1}$ has $A^{-1}=\mat{\tfrac12&-2\\0&1}$. In other words, $\mat{2&4\\0&1}^{-1}=\mat{\tfrac12&-2\\0&1}$.
• a $1\times 1$ matrix $A=[a]$ with $a\ne 0$ has $[a]^{-1}=[\tfrac1a]$.
• $I_n^{-1}=I_n$.
• $0_{n\times n}^{-1}$ does not exist
• $\mat{1&0\\0&0}^{-1}$ does not exist
• $\mat{1&2\\-3&-6}^{-1}$ does not exist

Proposition: solving $AX=B$ when $A$ is invertible

If $A$ is an invertible $n\times n$ matrix and $B$ is an $n\times k$ matrix, then the matrix equation \[ AX=B\] has a unique solution: $X=A^{-1}B$.

Proof

First we check that $X=A^{-1}B$ really is a solution to $AX=B$. To see this, note that if $X=A^{-1}B$, then \begin{align*} AX&=A(A^{-1}B)\\&=(AA^{-1})B\\&=I_n B \\&= B. \end{align*} Now we check that the solution is unique. If $X$ and $Y$ are both solutions, then $AX=B$ and $AY=B$, so \[AX=AY.\] Multiplying both sides on the left by $A^{-1}$, we get \[ A^{-1}AX=A^{-1}AY\implies I_nX=I_nY\implies X=Y.\] So any two solutions are equal, so $AX=B$ has a unique solution. ■

Corollary

If $A$ is an $n\times n$ matrix and there is a non-zero $n\times m$ matrix $K$ so that $AK=0_{n\times m}$, then $A$ is not invertible.

Proof

Since $A0_{n\times m}=0_{n\times m}$ and $AK=0_{n\times m}$, the equation $AX=0_{n\times m}$ has (at least) two solutions: $X=0_{n\times m}$ and $X=K$. Since $K$ is non-zero, these two solutions are different.

So there is not a unique solution to $AX=B$, for $B$ the zero matrix. If $A$ was invertible, this would contradict the uniqueness statement of the last Proposition. So $A$ cannot be invertible. ■

Examples

• We can now see why the matrix $\def\mat#1{\left[\begin{smallmatrix}#1\end{smallmatrix}\right]}A=\mat{1&2\\-3&-6}$ is not invertible. If $X=\mat{-2\\1}$ and $K=\mat{2\\-1}$, then $K$ is non-zero, but $AK=0_{2\times 1}$. So $A$ is not invertible, by the Corollary.
• $A=\mat{1&4&5\\2&5&7\\3&6&9}$ is not invertible, since $K=\mat{1\\1\\-1}$ is non-zero and $AK=0_{3\times 1}$.

$2\times 2$ matrices: determinants and invertibility

Question

Which $2\times 2$ matrices are invertible? For the invertible matrices, can we find their inverse?

Lemma

If $A=\mat{a&b\\c&d}$ and $J=\mat{d&-b\\-c&a}$, then we have \[ AJ=\delta I_2=JA\] where $\delta=ad-bc$.

Proof

This is a calculation (done in the lectures; you should also check it yourself). ■

Definition: the determinant of a $2\times 2$ matrix

Theorem: the determinant determines the invertibility (and inverse) of a $2\times 2$ matrix

Let $A=\mat{a&b\\c&d}$ be a $2\times 2$ matrix.

1. $A$ is invertible if and only if $\det(A)\ne0$.
2. If $A$ is invertible, then $A^{-1}=\frac{1}{\det(A)}\mat{d&-b\\-c&a}$.

Proof

If $A=0_{2\times 2}$, then $\det(A)=0$ and $A$ is not invertible. So the statement is true is this special case.

Now assume that $A\ne0_{2\times 2}$ and let $J=\mat{d&-b\\-c&a}$.

By the previous lemma, we have \[AJ=(\det(A))I_2=JA.\]

If $\det(A)\ne0$, then multiplying this equation through by the scalar $\frac1{\det(A)}$, we get \[ A\left(\frac1{\det(A)}J\right)=I_2=\left(\frac1{\det(A)}J\right) A,\] so if we write $B=\frac1{\det(A)}J$ to make this look simpler, then we obtain \[ AB=I_2=BA,\] so in this case $A$ is invertible with inverse $B=\frac1{\det(A)}J=\frac1{\det(A)}\mat{d&-b\\-c&a}$.

If $\det(A)=0$, then $AJ=0_{2\times 2}$ and $J\ne 0_{2\times2}$ (since $A\ne0_{2\times2}$, and $J$ is obtained from $A$ by swapping two entries and multiplying the others by $-1$). Hence by the previous corollary, $A$ is not invertible in this case. ■