Lecture 6: Matrix Inverses and Diagonalization

Instructions

• This section covers the concepts listed below. You can click the button to go directly to that topic.
• After each video, there are notes for the material in the video.
• At the end of the page, there are exercises covering the material in the section.
• When you have finished the material below, you can go to the next section or return to the main page.

Concepts

Matrix Inverses

Video Notes

We can view matrix–vector equations like:
\begin{align*}
M\boldsymbol{u} = \boldsymbol{b}
\end{align*}
in an analogy with scalar–valued equations like $$mx = y$$. For scalar equations like this, we can multiply both side by $$m^{-1}$$ to get:
\begin{align*}
m^{-1}mx = m^{-1}y
&\Longleftrightarrow 1x = m^{-1}y
\\&\Longleftrightarrow x = m^{-1}y.
\end{align*}

​To see the full video page and find related videos, click the following link.

Video Notes

If there was a matrix $$M^{-1}$$ with the property that $$M^{-1}M = I$$ ($$I$$ is the identity), then we can multiply both sides of $$M\boldsymbol{u}=\boldsymbol{b}$$ by $$M^{-1}$$ to get:
\begin{align*}
M\boldsymbol{u} = \boldsymbol{b}
&\Longleftrightarrow
M^{-1}M\boldsymbol{u} = M^{-1}\boldsymbol{b}
\\&\Longleftrightarrow  I \boldsymbol{u} = M^{-1}\boldsymbol{b}
\\&\Longleftrightarrow \boldsymbol{u} = M^{-1}\boldsymbol{b}.
\end{align*}
In this lecture, we will discuss whether such an "inverse" matrix exists and how to find it.

​To see the full video page and find related videos, click the following link.

​To see the full video page and find related videos, click the following link.

Video Notes

To begin with, we are going to change the question and ask: for a matrix $$M$$can we find a matrix $$A$$ such that $$MA = I$$? This doesn't directly answer the question of whether there is a matrix $$B$$ such that $$BM = I$$. However, assume there is such a matrix $$A$$ such that $$MA = I$$ and assume there is a matrix $$C$$ such that $$AC = I$$. Then if we multiply both sides of $$AC = I$$ by $$M$$ we get $$MAC = M$$ and since $$MA = I$$ this becomes $$C=M$$. In other words $$I = MA = AC = AM$$. In particular, if there is a matrix $$A$$ with $$MA = I$$ then $$MA = AM = I$$.

​To see the full video page and find related videos, click the following link.

Video Notes

To find $$MA = I$$ we need to solve a matrix–matrix equation. If we let $$\boldsymbol{a}_1, \ldots, \boldsymbol{a}_n$$ denote the columns of $$A$$ then this equation becomes:
\begin{align*}
M\begin{pmatrix}\boldsymbol{a}_1 \cdots \boldsymbol{a}_n\end{pmatrix}
=\begin{pmatrix}M\boldsymbol{a}_1 \cdots M\boldsymbol{a}_n\end{pmatrix}
=\begin{pmatrix}\boldsymbol{e}_1 \cdots \boldsymbol{e}_n\end{pmatrix},
\end{align*}
where $$\boldsymbol{e}_k$$ is the vector in $$\mathbb{R}^n$$ with a $$1$$ in the $$k^{\textnormal{th}}$$ positionand zeros elsewhere. So this is a system of equations:
\begin{align*}
M\boldsymbol{a}_1 &= \boldsymbol{e}_1\\
\vdots\\
M\boldsymbol{a}_n &= \boldsymbol{e}_n.
\end{align*}
The way we would do this is by row–reducing the augmented matrices:
\begin{align*}
\left(\begin{array}{c|c}M & \boldsymbol{e}_1\end{array}\right)& \sim
\left(\begin{array}{c|c}I & \boldsymbol{a}_1\end{array}\right)\\
&\vdots \\
\left(\begin{array}{c|c}M & \boldsymbol{e}_n\end{array}\right) &\sim
\left(\begin{array}{c|c}I & \boldsymbol{a}_n\end{array}\right).
\end{align*}
But note that in the row reduction of these augmented matrices, since the $$M$$ is the same for all of them, the steps are the exact same each time. So, instead of row–reducing $$n$$ times, we just augment the matrix $$M$$ with the matrix $$I$$ and row reduce that:
\begin{align*}
\left(\begin{array}{c|c}M & I\end{array}\right)\sim
\left(\begin{array}{c|c}I & A\end{array}\right).
\end{align*}
Since $$MA = AM$$ we call $$A$$ the inverse of $$M$$ and denote it by $$M^{-1}$$. Observe that we are assuming that $$M$$ can be row-reduced to the identity. If it can't be row–reduced to the identity, it means the columns of $$M$$ are not a spanning set for $$\mathbb{R}^n$$ and so there are some vectors $$\boldsymbol{b}\in\mathbb{R}^n$$ for which we cannot solve the equation $$M\boldsymbol{u}=\boldsymbol{b}$$.

We'll illustrate the above discussion with an example:

Example

Find the inverse of $$\begin{pmatrix}2&1\\4&6\end{pmatrix}.$$

Video 1

We need to find a matrix $$A = \begin{pmatrix}\boldsymbol{a}_1 & \boldsymbol{a}_2\end{pmatrix}$$ such that $$MA = I$$. So we need to solve:
\begin{align*}
\begin{pmatrix}2&1\\4&6\end{pmatrix}\boldsymbol{a}_1 &= \begin{pmatrix}1\\0\end{pmatrix}\4mm] \begin{pmatrix}2&1\\4&6\end{pmatrix}\boldsymbol{a}_2 &= \begin{pmatrix}0\\1\end{pmatrix}. \end{align*} And we do this by row–reducing: \begin{align*} \left(\begin{array}{cc|c}2&1&1\\4&6&0\end{array}\right) &\sim\left(\begin{array}{cc|c}2&1&1\\0&4&-2\end{array}\right)\\[4mm] &\sim\left(\begin{array}{cc|c}2&1&1\\0&1&-{2}/{4}\end{array}\right)\\[4mm] &\sim\left(\begin{array}{cc|c}2&0&{6}/{4}\\0&1&-{2}/{4}\end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc|c}1&0&{6}/{8}\\0&1&-{2}/{4}\end{array}\right) \end{align*} \begin{align*} \left(\begin{array}{cc|c}2&1&0\\4&6&1\end{array}\right) &\sim\left(\begin{array}{cc|c}2&1&0\\0&4&1\end{array}\right)\\[4mm] &\sim\left(\begin{array}{cc|c}2&1&0\\0&1&{1}/{4}\end{array}\right)\\[4mm] &\sim\left(\begin{array}{cc|c}2&0&-{1}/{4}\\0&1&{1}/{4}\end{array}\right)\\[4mm] &\sim\left(\begin{array}{cc|c}1&0&-{1}/{8}\\0&1&{1}/{4}\end{array}\right) \end{align*} And so: \begin{align*} A = \begin{pmatrix} 6/8 & -1/8 \\ -2/4 & 1/4\end{pmatrix}. \end{align*} Video 2 Instead of row–reducing twice, though, we can just row–reduce: \begin{align*} \left(\begin{array}{cc | cc} 2&1&1&0\\4&6&0&1 \end{array}\right) &\sim \left(\begin{array}{cc | cc} 2&1&1&0\\0&4&-2&1 \end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc | cc} 2&1&1&0\\0&1&-2/4&1/4 \end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc | cc} 2&0&6/4&-1/4\\0&1&-2/4&1/4 \end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc | cc} 1&0&6/8&-1/8\\0&1&-2/4&1/4 \end{array}\right). \end{align*} And so $$A$$ is the matrix to the right of the bar: \begin{align*} A = \begin{pmatrix} 6/8 & -1/8 \\ -2/4 & 1/4\end{pmatrix} = \frac{1}{8}\begin{pmatrix}6 & -1\\-2 & 1\end{pmatrix}. \end{align*} In general, for a $$2\times 2$$ matrix, we have: \begin{align*} \begin{pmatrix}a & b\\c&d\end{pmatrix}^{-1} =\frac{1}{ad-bc}\begin{pmatrix}d & -b\\-c & a\end{pmatrix} \end{align*} There is no such (easy) general formula for $$n\times n$$ matrices. Video 1 Video 2 To see the full video page and find related videos, click the following link. Linear Algebra for Math 308: L6V8 In general, for a $$2\times 2$$ matrix, we have: \begin{align*} \begin{pmatrix}a & b\\c&d\end{pmatrix}^{-1} =\frac{1}{ad-bc}\begin{pmatrix}d & -b\\-c & a\end{pmatrix} \end{align*} There is no such (easy) general formula for $$n\times n$$ matrices. Observe that a matrix has an inverse if and only if it can be row reduced to the identity matrix. That is, there are no free variables in the matrix when put in row echelon form. This happens if and only if the columns of the matrix are linearly independent. Example Solve \begin{align*} \begin{pmatrix} 1 & 1 & 0\\ 1 & 0 & 1\\ 1 & -1 & -1 \end{pmatrix} \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} = \begin{pmatrix}2\\1\\0\end{pmatrix} \end{align*} using inverses. \begin{align*} \left(\begin{array}{ccc|ccc} 1 & 1 & 0 & 1 & 0 & 0\\ 1 & 0 & 1 & 0 & 1 & 0\\ 1 & -1 & -1 & 0 & 0 & 1 \end{array}\right) &\sim \left(\begin{array}{ccc|ccc} 1 & 1 & 0 & 1 & 0 & 0\\ 0 & 1 & -1 & 1 & -1 & 0\\ 0 & 2 & 1 & 1 & 0 & -1 \end{array}\right) \\[4mm]&\sim \left(\begin{array}{ccc|ccc} 1 & 1 & 0 & 1 & 0 & 0\\ 0 & 1 & -1 & 1 & -1 & 0\\ 0 & 0 & 3 & -1 & 2 & -1 \end{array}\right) \\[4mm]&\sim \left(\begin{array}{ccc|ccc} 1 & 1 & 0 & 1 & 0 & 0\\ 0 & 1 & -1 & 1 & -1 & 0\\ 0 & 0 & 1 & -1/3 & 2/3 & -1/3 \end{array}\right) \\[4mm]&\sim \left(\begin{array}{ccc|ccc} 1 & 1 & 0 & 1 & 0 & 0\\ 0 & 1 & 0 & 2/3 & -1/3 & -1/3\\ 0 & 0 & 1 & -1/3 & 2/3 & -1/3 \end{array}\right) \\[4mm]&\sim \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & 1/3 & 1/3 & 1/3\\ 0 & 1 & 0 & 2/3 & -1/3 & -1/3\\ 0 & 0 & 1 & -1/3 & 2/3 & -1/3 \end{array}\right) \end{align*} Therefore, the inverse is: \begin{align*} \frac{1}{3} \begin{pmatrix} 1 & 1 & 1\\ 2 & -1 & -1\\ -1 & 2 & -1 \end{pmatrix} \end{align*} And therefore: \begin{align*} \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} &= \frac{1}{3} \begin{pmatrix} 1 & 1 & 1\\ 2 & -1 & -1\\ -1 & 2 & -1 \end{pmatrix} \begin{pmatrix}2\\1\\0\end{pmatrix}\\[4mm] &=\frac{1}{3}\begin{pmatrix}3\\3\\0\end{pmatrix}\\[4mm] &=\begin{pmatrix}1\\1\\0\end{pmatrix} \end{align*} ​To see the full video page and find related videos, click the following link. Linear Algebra for Math 308: L6V9 Diagonalization ​To see the full video page and find related videos, click the following link. Video Notes Let's say the matrix $$M$$ has $$n$$ linearly independent eigenvectors $$\boldsymbol{v}_1, \ldots, \boldsymbol{v}_n$$ with eigenvalues $$\lambda_1, \ldots, \lambda_n$$. If $$V$$ is the matrix whose columns are the eigenvectors, that is: \begin{align*} V = \begin{pmatrix}\boldsymbol{v}_1 & \cdots & \boldsymbol{v}_n\end{pmatrix} \end{align*} then: \begin{align*} MV = M\begin{pmatrix}\boldsymbol{v}_1 & \cdots & \boldsymbol{v}_n\end{pmatrix} = \begin{pmatrix}M\boldsymbol{v}_1 & \cdots & M\boldsymbol{v}_n\end{pmatrix} = \begin{pmatrix}\lambda_1\boldsymbol{v}_1 & \cdots & \lambda_n\boldsymbol{v}_n\end{pmatrix}. \end{align*} In other words, $$MV$$ is the matrix whose columns are the eigenvectors multiplied by the corresponding eigenvalues. Notice that for any $$n\times n$$ matrix $$A$$, $$A\boldsymbol{e}_k$$ is just the $$k^{\textnormal{th}}$$ column of the matrix $$A$$. In particular, if $$D$$ is the diagonal matrix with diagonal equal to the eigenvalues, that is: \begin{align*} D = \begin{pmatrix}\lambda_1\boldsymbol{e}_1 & \ldots & \lambda_n\boldsymbol{e}_n \end{pmatrix} \end{align*} then: \begin{align*} VD = V\begin{pmatrix}\lambda_1\boldsymbol{e}_1 & \ldots & \lambda_n\boldsymbol{e}_n \end{pmatrix} = \begin{pmatrix}\lambda_1V\boldsymbol{e}_1 & \ldots & \lambda_nV\boldsymbol{e}_n \end{pmatrix} =\begin{pmatrix}\lambda_1\boldsymbol{v}_1 & \ldots & \lambda_n\boldsymbol{v}_n \end{pmatrix}. \end{align*} In other words: \begin{align*} VD = MV. \end{align*} In particular: $$VDV^{-1} = M$$. When we factor $$M$$ this way, we call it diagonalizing the matrix. Example Diagonalize the matrix $$M = \begin{pmatrix}3 & -1\\4&-2\end{pmatrix}$$ We have already seen that the eigenvalues are $$-1$$ and $$2$$ and the corresponding eigenvectors are $$\begin{pmatrix}1\\4\end{pmatrix}$$ and $$\begin{pmatrix}1\\1\end{pmatrix}$$. Then we can compute: \begin{align*} MV = \begin{pmatrix}3 & -1\\4&-2\end{pmatrix} \begin{pmatrix}1&1\\4&1\end{pmatrix} = \begin{pmatrix}-1 & 2\\-4 & 2\end{pmatrix} \end{align*} and \begin{align*} VD = \begin{pmatrix}1&1\\4&1\end{pmatrix} \begin{pmatrix}-1&0\\0&2\end{pmatrix} = \begin{pmatrix}-1 &2\\-4&2\end{pmatrix}. \end{align*} That is, we have illustrated that $$VD = MV$$. Then to factor $$M$$, we need to compute $$V^{-1}$$: \begin{align*} \left(\begin{array}{cc|cc} 1 & 1 & 1 & 0\\ 4 & 1 & 0 & 1 \end{array}\right) &\sim \left(\begin{array}{cc|cc} 1 & 1 & 1 & 0\\ 0 & -3 & -4 & 1 \end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc|cc} 1 & 1 & 1 & 0\\ 0 & 1 & 4/3 & -1/3 \end{array}\right)\\[4mm] &\sim \left(\begin{array}{cc|cc} 1 & 0 & -1/3 & 1/3\\ 0 & 1 & 4/3 & -1/3 \end{array}\right) \end{align*} So $$V^{-1} = \frac{1}{3}\begin{pmatrix}-1&1\\4&-1\end{pmatrix}$$ and $$M=VDV^{-1}.$$ ​To see the full video page and find related videos, click the following link. Linear Algebra for Math 308: L6V11 Note. (see video for previous example) If the matrix $$M$$ can be diagonalized as $$M=VDV^{-1},$$ where $$V$$ is an invertible matrix and $$D$$ is a diagonal matrix $$D = \begin{pmatrix} a_1\boldsymbol{e}_1, \ldots, a_n\boldsymbol{e}_n \end{pmatrix}$$ for some scalars $$a_1,\ldots, a_n,$$ then \[ M^n=VD^nV^{-1}
and $$D^n = \begin{pmatrix}a^n_1\boldsymbol{e}_1, \ldots, a^n_n\boldsymbol{e}_n \end{pmatrix}.$$ Therefore, if we can diagonalize a matrix $$M$$ using $$n$$ linearly independent eigenvectors and the corresponding eigenvalues, then some calculations such as $$M^n$$ will be easier to compute using the diagonalization.
Exercises

1. Solve the system:
\begin{align*}
x_1 + x_2 + 2x_3 &= 1\\
2x_1 + x_2 + x_3 &= 2\\
x_1 + x_2 + x_3  &= 3.
\end{align*}

We are going to solve this by finding the inverse matrix. First, we write this as an augmented matrix:
\begin{align*}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
2 & 1 & 1 & 0 & 1 & 0\\
1 & 1 & 1 & 0 & 0 & 1
\end{array}\right).
\end{align*}
Then perform the Gauss-Jordan elimination algorithm:
\begin{align*}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
2 & 1 & 1 & 0 & 1 & 0\\
1 & 1 & 1 & 0 & 0 & 1
\end{array}\right)
&\xrightarrow{R2 := -(R2 - 2R1)}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
0 & 1 & 3 & 2 & -1 & 0\\
1 & 1 & 1 &  0 & 0 & 1
\end{array}\right)\\[4mm]
&\xrightarrow{R3 := -(R3 - R1)}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
0 & 1 & 3 & 2 & -1 & 0\\
0 & 0 & 1 & 1 & 0 & -1
\end{array}\right)\\
\end{align*}

Now the matrix is in row echelon form. We now work from the right–most pivot to the left–most.
\begin{align*}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
0 & 1 & 3 & 2 & -1 & 0\\
0 & 0 & 1 & 1 & 0 & -1
\end{array}\right)
&\xrightarrow{R2 := R2 - 3R3}
\left(\begin{array}{lll|lll}
1 & 1 & 2 & 1 & 0 & 0\\
0 & 1 & 0 & -1 & -1 & 3\\
0 & 0 & 1 & 1 & 0 & -1
\end{array}\right)
\\[4mm]&\xrightarrow{R1 := R1 - 2R3}
\left(\begin{array}{lll|lll}
1 & 1 & 0 & -1 & 0 & 2\\
0 & 1 & 0 & -1 & -1 & 3\\
0 & 0 & 1 & 1 & 0 & -1
\end{array}\right)
\\[4mm]&\xrightarrow{R1 := R1 - R2}
\left(\begin{array}{lll|lll}
1 & 0 & 0 & 0 & 1 & -1\\
0 & 1 & 0 & -1 & -1 & 3\\
0 & 0 & 1 & 1 & 0 & -1
\end{array}\right)
\end{align*}
So the solution is:
\begin{align*}
\begin{pmatrix}
x_1\\x_2\\x_3
\end{pmatrix}=
\begin{pmatrix}
0 & 1 & -1\\
-1 & -1 & 3\\
1 & 0 & -1
\end{pmatrix}
\begin{pmatrix}
1\\2\\3
\end{pmatrix}
=\begin{pmatrix}
-1\\6\\-2
\end{pmatrix}.
\end{align*}

​To see the full video page and find related videos, click the following link.
Linear Algebra for Math 308: L6E1inv

2. Solve the system:\begin{align*}
x_1 - x_2 + x_3 &=1\\
x_1 + 3x_2 + 5x_3 &=0\\
3x_1 + 2x_2 + x_3 &=1.
\end{align*}

Note: There is no video solution for this exercise.

We are going to find the inverse matrix:
\begin{align*}
\left(\begin{array}{ccc|ccc}
1 & -1 & 1 & 1 & 0 & 0\\
1 & 3 & 5 & 0 & 1 & 0\\
3 & 2 & 1 & 0 & 0 & 1
\end{array}\right)
&\xrightarrow[{R2 := \frac{1}{4}(R2 - R1)}]{R3 := R3 - 3R1}
\left(\begin{array}{ccc|ccc}
1 & -1 & 1 & 1 & 0 & 0\\
0 & 1 & 1 & -1/4 & 1/4 & 0\\
0 & 5 & -2 & -3 & 0 & 1
\end{array}\right)
\\[4mm]&\xrightarrow{R3 := -\frac{1}{7}(R3 - 5R2)}
\left(\begin{array}{ccc|ccc}
1 & -1 & 1 & 1 & 0 & 0\\
0 & 1 & 1 & -1/4 & 1/4 & 0\\
0 & 0 & 1 & 1/4 & 5/28 & -1/7
\end{array}\right)
\end{align*}
This is in REF; we now put it in RREF.
\begin{align*}
\left(\begin{array}{ccc|ccc}
1 & -1 & 1 & 1 & 0 & 0\\
0 & 1 & 1 & -1/4 & 1/4 & 0\\
0 & 0 & 1 & 1/4 & 5/28 & -1/7
\end{array}\right)
&\xrightarrow[{R1 := R1 - R3}]{R2 := R2 - R3}
\left(\begin{array}{ccc|ccc}
1 & -1 & 0 & 3/4 & -5/28 & 1/7\\
0 & 1 & 0 & -1/2 & 1/14 & 1/7\\
0 & 0 & 1 & 1/4 & 5/28 & -1/7
\end{array}\right)
\\&\xrightarrow{R1 := R1 + R2}
\left(\begin{array}{ccc|ccc}
1 & 0 & 0 & 1/4 & -3/28 2/7\\
0 & 1 & 0 & -1/2 & 1/14 & 1/7\\
0 & 0 & 1 & 1/4 & 5/28 & -1/7
\end{array}\right)
\end{align*}
So then:
\begin{align*}
\begin{pmatrix}
x_1\\x_2\\x_3
\end{pmatrix}
&=\begin{pmatrix}
1/4 & -3/28 & 2/7\\
-1/2 & 1/14 & 1/7\\
1/4 & 5/28 & -1/7
\end{pmatrix}
\begin{pmatrix}1\\0\\1\end{pmatrix}
\\[4mm]&=\frac{1}{28}\begin{pmatrix}15\\-10\\3\end{pmatrix}
\end{align*}