Differences

This shows you the differences between two versions of the page.

--- lecture_10 [2015/02/19 11:07] – rupert
+++ lecture_10 [2017/02/21 10:02] (current) – rupert
@@ Line 1: / Line 1: @@
-=== Proof of the proposition ===
-. We want to show that $I_nA=A$ for any $n\times m$ matrix $A$. These matrices the [[same size]], since $I_n$ has size $n\times n$ and $A$ has size $n\times m$, so $I_n A$ has size $n\times m$ by the definition of [[matrix multiplication]], which is the same as the size of $A$.
- Note that $\text{row}_i(I_n)=[0~0~\dots~0~1~0~\dots~0]$, where the $1$ is in the $i$th place, by definition of the [[identity matrix]] $I_n$; and the $j$th column of $A$ is $\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix}$. The (i,j) entry of $I_nA$ is
-$\text{row}_i(I_n)\cdot \text{col}_j(A)$, by the definition of [[matrix multiplication]], which is therefore
-\begin{align*} [0~0~\dots~0~1~0~\dots~0]\begin{bmatrix}a_{1j}\\a_{2j}\\\vdots\\a_{nj}\end{bmatrix} &= 0a_{1j}+0a_{2j}+\dots+0a_{i-1,j}+1a_{ij}+0a_{i+1,j}+\dots+0a_{nj} \\&= a_{ij}.\end{align*}
-So the matrices $I_nA$ and $A$ have the same $(i,j)$ entries, for any $(i,j)$. So $I_nA=A$.
-. To show that $AI_m=A$ for any $n\times m$ matrix $A$ is similar; the details are left as an exercise.
-. If $B$ is any $n\times n$ matrix, then $I_nB=B$ by part 1 and $BI_n=B$ by part 2, so $I_nB=B=BI_n$. In particular, $I_nB=BI_n$ so $I_n$ commutes with $B$, for every square $n\times n$ matrix $B$. ■
-===== Algebraic properties of matrix multiplication =====
-==== The associative law ====
-=== Proposition ===
-Matrix multiplication is //associative//. This means that $(AB)C=A(BC)$ whenever $A,B,C$ are matrices which can be multiplied together in this order.
-We omit the proof, but this is not terribly difficult; it is a calculation in which you write down two formulae for the $(i,j)$ entries of $(AB)C$ and $A(BC)$, and carefully check they are equal using the fact that if $a,b,c$ are real numbers, then $(ab)c=a(bc)$.
-=== Example ===
-We saw above that $\newcommand{\m}[1]{\begin{bmatrix}#1\end{bmatrix}}A=\m{1&2\\3&4}$ commutes with $B=\m{7&10\\15&22}$. We can explain why this is so using associativity. You can check that $B=AA$ (which we usually write as $B=A^2$). Hence, using associativity at $\stackrel*=$,
-\[ AB=A(AA)\stackrel*=(AA)A=BA.\]
-The same argument for any square matrix $A$ gives a proof of:
-=== Proposition ===
-If $A$ is any square matrix, then $A$ commutes with $A^2$.■
-Using [[wp>mathematical induction]], you can prove a more general fact:
-===Proposition===
-If $A$ is any square matrix and $k\in\mathbb{N}$, then $A$ commutes with $A^k$.■
-====The distributive laws====
-=== Proposition ===
-If $A$ is an $n\times m$ matrix and $k\in\mathbb{N}$, then:
-  - $A(B+C)=AB+AC$ for any $m\times k$ matrices $B$ and $C$; and
-  - $(B+C)A=BA=CA$ for any $k\times n$ matrices $B$ and $C$.
-In other words, $A(B+C)=AB+AC$ whenever the matrix products make sense, and similarly $(B+C)A=BA+CA$ whenever this makes sense.
-===Proof===
-. First note that
-  * $B$ and $C$ are both $m\times k$, so $B+C$ is $m\times k$ by the definition of [[matrix addition]];
-  * $A$ is $n\times m$ and $B+C$ is $m\times k$, so $A(B+C)$ is $m\times k$ by the definition of [[matrix multiplication]];
-  * $AB$ and $AC$ are both $n\times k$ by the definition of matrix multiplication
-  * so $AB+AC$ is $n\times k$ by the definition of matrix addition.
-So we have (rather long-windedly) checked that $A(B+C)$ and $AB+AC$ have the [[same size]].
-Recall that in tutorial 4 we saw that if $a$ is a $1\times m$ row vector and $b$ and $c$ are $m\times 1$ column vectors, then the [[row-column product]] has the property that \[a\cdot (b+c)=a\cdot b+a\cdot c.\]
-So the $(i,j)$ entry of $A(B+C)$ is
-\begin{align*}\def\row{\text{row}}\def\col{\text{col}}
-\text{row}_i(A)\cdot \col_j(B+C) &= \text{row}_i(A)\cdot \big(\col_j(B)+\col_j(C)\big)
-\\ &= \text{row}_i(A)\cdot \col_j(B)+\row_i(A)\cdot\col_j(C).\end{align*}
-On the other hand,
-  * the $(i,j)$ entry of $AB$ is $\text{row}_i(A)\cdot \col_j(B)$; and
-  * the $(i,j)$ entry of $AC$ is $\row_i(A)\cdot\col_j(C)$;
-  * so the $(i,j)$  entry of $AB+AC$ is also $\text{row}_i(A)\cdot \col_j(B)+\row_i(A)\cdot\col_j(C)$.
-So the entries of $A(B+C)$ and $AB+AC$ are all equal, so $A(B+C)=AB+AC$.
-. The proof is similar, and is left as an exercise.■
 ===== Matrix equations =====
@@ Line 73: / Line 10: @@
 We can write a whole [[system of linear equations]] in a similar way, as a matrix equation using [[matrix multiplication]]. For example we can rewrite the linear system
-\begin{align*} 2x-3y+z&=8\\ y-z=4\\x+y+z=0\end{align*}
+\begin{align*} 2x-3y+z&=8\\ y-z&=4\\x+y+z&=0\end{align*}
 as
 \[ \m{2&-3&1\\0&1&-1\\1&1&1}\m{x\\y\\z}=\m{8\\4\\0},\]
@@ Line 79: / Line 16: @@
 \[ A\vec x=\vec b\]
 where $A=\m{2&-3&1\\0&1&-1\\1&1&1}$, $\vec x=\m{x\\y\\z}$ and $\vec b=\m{8\\4\\0}$. (We are writing the little arrow above the column vectors here because otherwise we might get confused between the $\vec x$: a column vector of variables, and $x$: just a single variable).
 More generally, any linear system
@@ Line 86: / Line 24: @@
 where $A$ is the $n\times m $ matrix, called the **coefficient matrix** of the linear system, whose $(i,j)$ entry is $a_{ij}$ (the number in front of $x_j$ in the $i$th equation of the system) and $\vec x=\m{x_1\\x_2\\\vdots\\x_m}$, and $\vec b=\m{b_1\\b_2\\\vdots\\b_n}$.
-More generally still, we might want to solve a matrix equation like \[AX=B\] where $A$, $X$ and $B$ are matrices of any size. Because of the definition of [[matrix multiplication]], if $A$ is $n\times m$, we need $B$ to be $n\times k$ for some $k$, and then $X$ must be $m\times k$, so we know the size of the solution(s) $X$. But which $m\times k$ matrices $X$ are solutions?
+More generally still, we might want to solve a matrix equation like \[AX=B\] where $A$, $X$ and $B$ are matrices of any size, with $A$ and $B$ fixed matrices and $X$ a matrix of unknown variables. Because of the definition of [[matrix multiplication]], if $A$ is $n\times m$, we need $B$ to be $n\times k$ for some $k$, and then $X$ must be $m\times k$, so we know the size of any solution $X$. But which $m\times k$ matrices $X$ are solutions?
 === Example ===
-If $A=\m{1&0\\0&0}$ and $B=0_{2\times 3}$, then any solution $X$ to $AX=B$ must be $2\times 3$.
+If $A=\def\m#1{\begin{bmatrix}#1\end{bmatrix}}\m{1&0\\0&0}$ and $B=0_{2\times 3}$, then any solution $X$ to $AX=B$ must be $2\times 3$.
 One solution is $X=0_{2\times 3}$, since in this case we have $AX=A0_{2\times 3}=0_{2\times 3}$.
@@ Line 97: / Line 35: @@
 So from this example, we see that a matrix equation can have many solutions.
+===== Invertibility =====
+We've seen that solving matrix equations $AX=B$ is useful, since they generalise systems of linear equations.
+How can we solve them?
+==== Example ====
+Take $A=\def\mat#1{\begin{bmatrix}#1\end{bmatrix}}\mat{2&4\\0&1}$ and $B=\mat{3&4\\5&6}$, so we want to find all matrices $X$ so that $AX=B$, or \[ \mat{2&4\\0&1}X=\mat{3&4\\5&6}.\] Note that $X$ must be a $2\times 2$ matrix for this to work, by the definition of [[matrix multiplication]]. So one way to solve this is to write $X=\mat{x_{11}&x_{12}\\x_{21}&x_{22}}$ and plug it in:
+\[\mat{2&4\\0&1}\mat{x_{11}&x_{12}\\x_{21}&x_{22}}=\mat{3&4\\5&6}\iff \mat{2x_{11}+4x_{21}&2 x_{12}+4x_{22}\\x_{21}&x_{22}}=\mat{3&4\\5&6}\]
+and then equate entries to get four linear equations:
+\begin{align*}2x_{11}+4x_{21}&=3\\2 x_{12}+4x_{22}&=4\\x_{21}&=5\\x_{22}&=6\end{align*}
+which we can solve in the usual way.
+But this is a bit tedious! We will develop a slicker method by first thinking about solving ordinary equations $ax=b$ where $a,x,b$ are all numbers, or  if you like, $1\times 1$ matrices.
+==== Solving $ax=b$ and $AX=B$ ====
+If $a\ne0$, then solving $ax=b$ where $a,b,x$ are numbers is easy. We just divide both sides by $a$, or equivalently, we multiply both sides by $\tfrac1a$, to get the solution: $x=\tfrac1a\cdot b$.
+Why does this work? If $x=\tfrac1a\cdot b$, then
+\begin{align*} ax&=a(\tfrac1a\cdot b)\\&=(a\cdot \tfrac1a)b\\&=1b\\&=b\end{align*}
+so $ax$ really is equal to $b$, and we do have a solution to $ax=b$.
+What is special about $\tfrac1a$ which made this all work?
+  - we have $a\cdot \tfrac1a = 1$,
+  - and $1b = b$.
+Now for an $n\times k$ matrix $B$, we know that the identity matrix $I_n$ does the same sort of thing as $1$ is doing in the relation $1b=b$: we have $I_nB=B$ for any $n\times k$ matrix $B$. So instead of $\tfrac1a$, we want to find a matrix $C$ with the property: $AC=I_n$. In fact, because matrix multiplication is not commutative, we also require that $CA=I_n$. It's then easy to argue that $X=C\cdot B$ is a solution to $AX=B$, since
+\begin{align*} AX&=A(CB)\\&=(AC)B\\&=I_nB\\&=B.\end{align*}
+==== Example revisited ====
+If $A=\mat{2&4\\0&1}$, then the matrix $C=\mat{\tfrac12&-2\\0&1}$ does have the property that
+\[ A C=I_2=C A.\]
+(You should check this!). So a solution to $AX=B$ where $B=\mat{3&4\\5&6}$ is $X=CB=\mat{\tfrac12&-2\\0&1}\mat{3&4\\5&6} = \mat{-8.5&-10\\5&6}$.
+Notice that having found the matrix $C$, then we can solve $AX=C$ easily for any $2\times 2$ matrix $C$: the answer is $X=CC$. This is quicker than having to solve four new linear equations using our more tedious method above.
+==== Definition: invertible ====
+{{page>invertible}}
+==== Examples ====
+  * $A=\mat{2&4\\0&1}$ is invertible, and the matrix $C=\mat{\tfrac12&-2\\0&1}$ is an inverse of $A$
+  * a $1\times 1$ matrix $A=[a]$ is invertible if and only if $a\ne0$, and if $a\ne0$ then an inverse of $A=[a]$ is $C=[\tfrac1a]$.
+  * $I_n$ is invertible for any $n$, since $I_n\cdot I_n=I_n=I_n\cdot I_n$, so an inverse of $I_n$ is $I_n$.
+  * $0_{n\times n}$ is not invertible for any $n$, since $0_{n\times n}\cdot C=0_{n\times n}$ for any $n\times n$ matrix $C$, so $0_{n\times n}\cdot C\ne I_n$.
+  * $A=\mat{1&0\\0&0}$ is not invertible, since for any $2\times 2$ matrix $C=\mat{a&b\\c&d}$ we have $AC=\mat{a&b\\0&0}$ which is not equal to $I_2=\mat{1&0\\0&1}$ since the $(2,2)$ entries are not equal.
+  * $A=\mat{1&2\\-3&-6}$ is not invertible. We'll see why later!
+==== Proposition: uniqueness of the inverse ====
+If $A$ is an invertible $n\times n$ matrix, then $A$ has a //unique// inverse.
+=== Proof ===
+Suppose $C$ and $C'$ are both inverses of $A$. Then
+$AC=I_n=CA$ and $AC'=I_n=C'A$. So
+\begin{align*} C&=CI_n\quad\text{by the properties of $I_n$}\\&=C(AC')\quad\mbox{because }AC'=I_n\\&=(CA)C'\quad\mbox{because matrix multiplication is associative}\\&=I_nC'\quad\mbox{because }CA=I_n\\&=C'\quad\text{by the properties of $I_n$}.\end{align*}
+So $C=C'$, whenever $C$ and $C'$ are inverses of $A$. So $A$ has a unique inverse. ■
+==== Definition/notation: $A^{-1}$ ====
+{{page>the inverse}}
+If a matrix $A$ is not invertible, then $A^{-1}$ does not exist.
+=== Warning ===
+If $A$ is a matrix then $\frac 1A$ doesn't make sense! You should never write this down. In particular, $A^{-1}$ definitely doesn't mean $\frac 1A$.
+Similarly, you should **never** write down $\frac AB$ where $A$ and $B$ are matrices. This doesn't make sense either!
+==== Examples revisited ====
+  * $A=\mat{2&4\\0&1}$ has $A^{-1}=\mat{\tfrac12&-2\\0&1}$. In other words, $\mat{2&4\\0&1}^{-1}=\mat{\tfrac12&-2\\0&1}$.
+  * a $1\times 1$ matrix $A=[a]$ with $a\ne 0$ has $[a]^{-1}=[\tfrac1a]$.
+  * $I_n^{-1}=I_n$.
+  * $0_{n\times n}^{-1}$ does not exist
+  * $\mat{1&0\\0&0}^{-1}$ does not exist
+  * $\mat{1&2\\-3&-6}^{-1}$ does not exist