Appendix A

Matrices and Linear Algebra A.1

Introductory Remarks

The objective of this appendix is to present a brief overview of the fundamentals of matrices and linear algebra. Further details concerning matrices and linear algebra can be found in standard books on the subject such as [1, 2, 3, 4], to name a few. • Definition A matrix is a rectangular or linear assemblage of numbers or variables arranged in rows and in columns. The numbers or variables comprising a matrix are called elements or entries of the matrix. The elements may be real or they may be complex. A matrix shall be denoted by an uppercase Latin letter such as [A] or A. An element of a matrix will be denoted by a corresponding lowercase Latin letter with two subscripts. The first subscript represents the row number and the second subscript the column number. Thus the elements of A are denoted by aij , where i and j are the row and column numbers, respectively. On occasion it is desirable to show the size of a matrix along side its symbol. Thus to explicitly indicate the dimensions of the m by n matrix A, we write [A](m∗n) or A(m∗n) . • Definition An n-vector is a collection of n real or complex numbers v1 to vn arranged in a column; e.g.,   v1        v2  (A.1) .   ..       vn Vectors shall be denoted by lowercase Latin letters such as {v} or v.

1

2

A.

A.2

Matrices and Linear Algebra

Special Matrices

• Definition Since no distinction is made between (m ∗ 1) matrices and m-vectors, such a matrix is called an m-dimensional column vector. Like general vectors, column vectors will be denoted by lowercase Latin symbols. • Definition A (1 ∗ n) matrix is called an n-dimensional row vector. Like general vectors, row vectors will be denoted by lowercase Latin symbols. • Definition An (m ∗ n) matrix A is square if m = n. • Definition Let A be an (m ∗ n) matrix and let k = min(m, n). The elements aii {i = 1, 2, · · · , k} are said to lie on the diagonal of A. The elements ai i+1 are said to lie on the superdiagonal of A. The elements ai i−1 are said to lie on the subdiagonal of A. . Example 1: Diagonal/Superdiagonal/Subdiagonal Elements in a Matrix Consider the following matrix: 

δ β  A= a31 a41

α δ β a42

a13 α δ β

a14 a24 α δ

 a15 a25   a35  a45

(A.2)

The diagonal elements are denoted by δ; the superdiagonal elements are denoted by α; and, the subdiagonal elements are denoted by β. . • Definition A square matrix is diagonal if its only non-zero elements lie on the diagonal. Remark 1. If all the elements in the diagonal square matrix A are equal to unity, then A=I, where I is the identity matrix. • Definition An (m ∗ n) matrix A is upper trapezoidal if aij = 0 for i > j. If aij = 0 for i < j, then A is lower trapezoidal. . Example 2: Upper Trapezoidal Matrix The following matrix is upper trapezoidal.

A.2. Special Matrices

3  u11 U= 0 0

u12 u22 0

u13 u23 u33

 u14 u24  u34

(A.3) .

• Definition A square upper (lower) trapezoidal matrix is said to be upper (lower) triangular. If a matrix is upper (lower) triangular with zero diagonal elements, then the matrix is said to be strictly upper (lower) triangular. If the diagonal elements of the matrix are unity, then the matrix is said to be unit upper (lower) triangular. . Example 3: Triangular Matrices From the previous discussion, it follows that the matrix   l11 0 0 L = l21 l22 0  l31 l32 l33

(A.4)

is lower triangular. The matrix 

0 t21 T= t31 t41

0 0 t32 t42

is strictly lower triangular. Finally, the matrix  1 u12 0 1 U= 0 0 0 0

0 0 0 t43

u13 u23 1 0

 0 0  0 0

(A.5)

 u14 u24   u34  1

(A.6)

is unit upper triangular. . Remarks 1. From the above discussion it follows that a matrix is diagonal if and only if it is both upper and lower triangular. 2. The identity matrix I is both unit upper and unit lower triangular. • Definition A square matrix H is upper Hessenberg if i > j + 1 i < j − 1 ⇒ hij = 0.

⇒ hij = 0. It is lower Hessenberg if .

Example 4: Hessenberg Matrix In light of the above definition, the following matrix is (5 ∗ 5) lower Hessenberg.

4

A.  h11 h21  H= h31 h41 h51

h12 h22 h32 h42 h52

0 h23 h33 h43 h53

0 0 h34 h44 h54

Matrices and Linear Algebra

 0 0   0   h45  h55

(A.7)

. Remark 1. If it is both upper and lower Hessenberg, a square matrix is tridiagonal. • Definition Consider a square matrix A. If | i − j| > k ⇒ aij = 0, then A is called a band matrix. The bandwidth for a matrix of this type is equal to 2k + 1. The half-bandwidth is equal to k + 1. Remark 1. For k = 0, 1, 2, the matrix A is it diagonal, tridiagonal, and pentadiagonal, respectively. . Example 5: Bandwidth and Half-bandwidth Consider the square matrix  2 1  3 A= 0  0 0

1 4 5 1 0 0

3 3 4 8 2 0

0 1 1 2 1 9

0 0 2 1 4 2

 0 0  0  6  3 6

(A.8)

From row 3 (i.e., i = 3) it is evident that aij = 0 for j > 5 and j < 1, implying that | i − j | = | 3 − 5 | = 2 or | i − j | = | 3 − 1 | = 2. Consequently k = 2, the bandwidth is equal to 2(2) + 1 = 5 and the half-bandwidth is equal to 2 + 1 = 3. A tridiagonal matrix has its non-zero elements arranged in a band along the diagonal, with a bandwidth equal to 3. For example,   4 1 0 0 0 1 4 1 0 0    A= (A.9) 0 1 4 1 0 0 0 1 4 1 0 0 0 1 4 . • Definition A square matrix A of order n is symmetric if aij = aji {i, j = 1, 2, · · · , n} It follows that the tridiagonal matrix given in equation (A.9) is symmetric.

(A.10)

A.3. Algebraic Operations on Matrices

5

• Definition An (n ∗ n) matrix A is said to be strictly diagonally dominant if |aii | >

n X

|aij |

(A.11)

|aij |

(A.12)

j=1 ; j6=i

holds for each i = 1, 2, · · · , n. If |aii | ≥

n X j=1 ; j6=i

then A is said to be diagonally dominant. • Definition A symmetric (n ∗ n) matrix A is called positive definite if the relation xT Ax > 0 holds for every n-dimensional vector x 6= 0. A weaker case is when xT Ax ≥ 0 holds for every n-dimensional vector x 6= 0. In this case A is said to be positive semi-definite.

A.3

Algebraic Operations on Matrices

• Definition Two matrices A and B are said to be equal if and only if: 1. Both A and B are of the same size; and, 2. aij = bij for each i = 1, 2, · · · , m and j = 1, 2, · · · , n. • Definition Two matrices can be added if and only if they have the same dimensions. Thus let A and B be (m ∗ n) matrices. The sum of A and B (i.e., C = A + B) is the matrix C whose elements are given by cij = aij + bij

(A.13)

where i = 1, 2, · · · , m and j = 1, 2, · · · , n. • Theorem Let A, B and C be (m ∗ n) matrices. Then matrix addition is commutative A+B=B+A

(A.14)

(A + B) + C = A + (B + C)

(A.15)

and associative

• Definition Let A be an (m ∗ n) matrix and let λ be a scalar. The product of A and λ is written λA. The elements of λA are given by λaij , where i = 1, 2, · · · , m and j = 1, 2, · · · , n; i.e., each element of A is multiplied by the scalar λ.

6

A.

Matrices and Linear Algebra

• Theorem Let A and B be matrices and let λ and µ be real numbers. Then (λµ)A = λ(µA)

(A.16)

(λ + µ)A = λA + µA

(A.17)

λ(A + B) = λA + λB

(A.18)

(1)A = A

(A.19)

It follows that the operation of matrix subtraction involves the addition of a matrix multiplied by the scalar (-1), viz., A − B = A + (−1)B

(A.20)

• Definition Any matrix whose elements are all zero is called a zero matrix and is denoted by the symbol [0] or 0. In light of this definition, it follows that for any matrix A A+0=A

(A.21)

A + (−1)A = A + (−A) = 0

(A.22)

and

• Definition Let A be an (l ∗ m) matrix and let B be an (m ∗ n) matrix. The product of A and B is the (l ∗ n) matrix P; i.e., P = AB. The elements of P are given by pij =

m X

aik bkj

(A.23)

k=1

where i = 1, 2, · · · , l and j = 1, 2, · · · , n. Remarks 1. The product of two matrices is defined only if the matrices are conformable, that is, the prefactor matrix, or pre-multiplier, has the same number of columns as the postfactor matrix, or post-multiplier, has rows. 2. The product of two matrices generalizes the notion of multiplication of two vectors. More precisely, it is the result of forming a matrix whose elements are scalar (dot) products obtained by multiplying (1 ∗ m) row vectors in the prefactor A with (m ∗ 1) column vectors in the postfactor B. . Example 6: Matrix Multiplication Let A be a (2 ∗ 3) matrix and let B be a (3 ∗ 3) matrix. Their product is then given by

A.3. Algebraic Operations on Matrices

7

· P(2 ∗ 3) = A(2 ∗ 3) B(3 ∗ 3) =

a11 a21

a12 a22

 ¸ b11 a13  b21 a23 b31

b12 b22 b32

 b13 b23  b33

(A.24)

where p11 =

3 X

a1k bk1 = a11 b11 + a12 b21 + a13 b31

(A.25)

a1k bk2 = a11 b12 + a12 b22 + a13 b32

(A.26)

a1k bk3 = a11 b13 + a12 b23 + a13 b33

(A.27)

a2k bk1 = a21 b11 + a22 b21 + a23 b31

(A.28)

a2k bk2 = a21 b12 + a22 b22 + a23 b32

(A.29)

a2k bk3 = a21 b13 + a22 b23 + a23 b33

(A.30)

k=1

p12 =

3 X k=1

p13 =

3 X k=1

p21 =

3 X k=1

p22 =

3 X k=1

p23 =

3 X k=1

.

• Theorem Let A, B and C be matrices with conformable dimensions and let λ be a real number. Then (AB)C = A(BC)

(A.31)

A(B ± C) = AB ± AC

(A.32)

(A ± B)C = AC ± BC

(A.33)

λ(AB) = (λA)B = A(λB)

(A.34)

Remark 1. In general, matrix multiplication is not commutative; i.e., even if A and B have conformable dimensions, AB 6= BA. . Example 7: Further Insight into Matrix Multiplication Let ·

2 1 A= 4 0 It follows that

¸

·

¸

2 5 and B = −1 3

(A.35)

8

A. · 2 AB = 4

and

· BA =

¸·

1 0

2 5 −1 3

Matrices and Linear Algebra

¸ · ¸ 2 5 3 13 = −1 3 8 20

¸·

¸ · 2 1 24 = 4 0 10

(A.36)

¸

2 −1

(A.37)

The above result confirms the fact that matrix multiplication is not commutative. Next consider the following matrix product · ¸· ¸ · ¸ 1 1 2 −2 0 0 = (A.38) 1 1 −2 2 0 0 This shows that it is possible for the product of two non-zero matrices to be zero. From this result it follows that if AX = AY with A 6= 0, we cannot conclude that X = Y. That is, the cancellation law for real numbers does not extend to matrices. . • Theorem Let A be an (m ∗ n) matrix, and let I be the identity matrix. Then I(m∗m) A = AI(n∗n) = A • Theorem The product of a diagonal matrix and a diagonal (trapezoidal, Hessenberg or tridiagonal) matrix is diagonal (trapezoidal, Hessenberg or tridiagonal). The product of two upper (lower) trapezoidal matrices is an upper (lower) trapezoidal matrix. Concerning the powers of matrices, let A be a square matrix and let n be a positive integer. Then An = AA · · · · · · A

(i.e., n

factors)

(A.39)

Having defined the process of matrix multiplication, it follows that the set of linear equations a11 x1 + a12 x2 + a13 x3 = b1

(A.40)

a21 x1 + a22 x2 + a23 x3 = b2

(A.41)

a31 x1 + a32 x2 + a33 x3 = b3

(A.42)

can be written in the more compact form Ax = b, where     a11 a12 a13 x1  A = a21 a22 a23  ; x = x2 ;   a31 a32 a33 x3

  b1  b = x = b2   b3

(A.43)

Furthermore, matrix multiplication, when applied to two systems of linear equations, greatly simplifies manipulations associated with their solution. For example, consider a system of linear equations that expresses x variables in terms of y variables, a11 y1 + a12 y2 = x1 a21 y1 + a22 y2 = x2

(A.44) (A.45)

a31 y1 + a32 y2 = x3

(A.46)

A.3. Algebraic Operations on Matrices

9

Also consider a second set of linear equations that express y variables in terms of z variables y1 = c11 z1 + c12 z2 y2 = c21 z1 + c22 z2

(A.47) (A.48)

The two sets of equations are written in matrix form as A(3 ∗ 2) y(2 ∗ 1) = x(3 ∗ 1)

(A.49)

y(2 ∗ 1) = C(2 ∗ 2) z(2 ∗ 1)

(A.50)

and

To obtain new equations that express the x variables in terms of the z variables, simply substitute equation (A.50) into equation (A.49), to give A(3 ∗ 2) y(2 ∗ 1) = A(3 ∗ 2) C(2 ∗ 2) z(2 ∗ 1) = x(3∗ 1)

(A.51)

or upon expansion  a11 a21 a31

 a12 · c a22  11 c21 a32

  ¸ ½ ¾ x1  c12 z1 = x2 c22 z2   x3

(A.52)

10

A.

A.4

Matrices and Linear Algebra

Matrix Operations

• Definition Let A be an (m ∗ n) matrix. The transpose of A, denoted by AT , is the (n ∗ m) matrix obtained by writing the rows of A as the columns of AT and the columns of A as the rows of AT ; i.e., aTij = aji . . Example 8: Transposition of Column Vectors The transpose of a column vector is a row vector. Thus, the transpose of   5       1  2 b=    0      −6

(A.53)

is © bT = 5

1

2

ª

0

−6

(A.54) .

Remarks 1. A matrix is symmetric if and only if it is equal to its transpose; i.e., if A = AT or aij = aji . 2. If AT = −A, the matrix is said to be skew or anti-symmetric. In a skew-symmetric matrix all the diagonal elements are equal to zero. • Theorem Let A, B and C be matrices and let λ be a scalar. Then ¡ T ¢T A =A

(A.55)

T

(A + B) = AT + BT T

(λA) = λA T

T

T T

(ABC) = C B A

(A.56) (A.57)

T

(A.58)

Equation (A.58) holds for any number of matrices. . Example 9: Transposition of a Specific Matrix Product For any matrix A, the matrices AAT and AT A are defined and are symmetric. To prove this assertion, first note that the matrix sizes are conformable for multiplication, that is, the number of columns in A is, by definition, equal to the number of rows in AT , and the number of columns in AT is equal to the number of rows in A. Finally, consider the following manipulations ¡

AT A

¢T

¡ ¢T = AT A = AT AT

(A.59)

A.4. Matrix Operations

11

which completes this proof. . Example 10: Inner Product of Two Vectors Let x and y be n-dimensional vectors. Then the product xT y is a scalar whose value is xT y =

n X

xi yi

(A.60)

i=1

The resulting scalar is called the inner product of x and y. . • Definition Let D be a square unsymmetric matrix. D may be decomposed into symmetric and skewsymmetric parts in the following manner ¢ 1¡ ¢ 1¡ D + DT + D − DT (A.61) 2 2 The first matrix sum appearing in equation (A.61) can be shown to be symmetric in the following manner D=

µ

¶ ¢ T ¢ 1¡ ¢ 1¡ 1¡ T D + DT = D +D = D + DT 2 2 2

(A.62)

Similarly, to show that the second matrix sum appearing in equation (A.61) is skew-symmetric, note that µ

¢ 1¡ D − DT 2

¶T =

¢ ¢ 1¡ T 1¡ D − D = − D − DT 2 2

(A.63) .

Example 11: Matrix Decomposition In order to better understand the above definition,  2 1 A = −1 2 3 0

decompose the matrix  3 5 1

(A.64)

into a symmetric and a skew-symmetric part. The symmetric part is obtained in the following manner       2 1 3 2 −1 3 2 0 3 ¢ 1¡ 1 (A.65) A + AT = −1 2 5 + 1 2 0 = 0 2 2.5 2 2 3 0 1 3 5 1 3 2.5 1 The skew-symmetric part is then obtained in a similar fashion, viz.,   0 1 0 ¢ 1¡ 0 2.5 A − AT = −1 2 0 −2.5 0

(A.66)

12

A.

Matrices and Linear Algebra .

• Definition Many discussions of matrices center around the so-called elementary row operations. These represent simple operations that can be performed on the rows of a matrix A to yield a new matrix. The elementary row operations are: 1. Multiply a row of A by a scalar m; 2. Interchange two rows in A; and, 3. Add m times one row to a second row of A. Remark 1. Elementary row operations are used extensively in the solution of simultaneous linear equations using Gauss elimination.

A.5. Partitioning of Matrices

A.5

13

Partitioning of Matrices

In certain instances it is advantageous to partition a matrix into submatrices. • Definition A submatrix is a matrix that is obtained from the original matrix by including the elements of certain rows and columns. In particular, let A be an (m ∗ n) matrix and let 1 ≤ i1 < i2 < · · · < ik ≤ m and 1 ≤ j1 < j2 < · · · < jp ≤ n. The (k ∗ p) matrix S whose (u, v)th element is suv = aiu iv is called a submatrix of A. • If k = p and i1 = j1 , i2 = j2 , · · · , ik = jk , then S is a principal submatrix of A. • If i1 = 1, i2 = 2, · · · , ik = k and j1 = 1, j2 = 2, · · · , jp = p, then S is called the leading submatrix of A. • Theorem A principal submatrix of a diagonal (or tridiagonal, triangular, strictly triangular, Hessenberg, symmetric) matrix is diagonal (or tridiagonal, triangular, strictly triangular, Hessenberg, symmetric). . Example 12: Matrix Partitioning Consider the (3 ∗ 6) matrix A  a11 A = a21 a31

a12 a22 a32

a13 a23 a33

a14 a24 a34

 a16 a26  a36

a15 a25 a35

(A.67)

which is partitioned in the following manner: ·

A11 A= A21

A12 A22

A13 A23

¸ (A.68)

where ·

¸

a11 a21

a12 a22

£ A21 = a31

a32

A11 =

¤

;

A12 =

· a13 a23

a14 a24

a15 a25

;

£ A22 = a33

a34

a35

¸

¤

·

a16 a26

¸

;

A13 =

;

£ ¤ A23 = a36

(A.69)

(A.70)

Notes: • A11 is the leading principal submatrix of A. • With the exception of A23 , each of the above submatrices could, in turn, be further partitioned. .

14

A.

A.6

Matrices and Linear Algebra

Determinants of Matrices

The determinant of a matrix is a single number that is evaluated from the elements of the matrix. Determinants are only defined for square matrices. The determinant of the matrix A is written as det ([A]), det(A) or |A|. • Definition Consider the square matrix A. The complementary minor of the matrix element aij , denoted by Mij , is the determinant formed by the elements that remain when the ith row and jth column are deleted from the original determinant. • Definition The cofactor or algebraic complement associated with the element aij , denoted by a ˜ij , is given by a ˜ij = (−1)(i+j) Mij

(A.71)

where Mij is the complementary minor of the matrix element aij as defined above. • Definition The determinant of an (n∗n) matrix A is equal to the sum of the products of the elements of any row or any column and their respective cofactors. Thus, for i = 1, 2, · · · , n (i.e., a row definition) |A| = ai1 a ˜i1 + ai2 a ˜i2 + · · · + ain a ˜in =

n X

aij a ˜ij

(A.72)

j=1

Likewise, for j = 1, 2, · · · , n (i.e., a column definition) |A| = a1j a ˜1j + a2j a ˜2j + · · · + anj a ˜nj =

n X

aij a ˜ij

(A.73)

i=1

• Theorem If all the elements in a row or in a column of a determinant are zero, the value of the determinant is zero. . Example 13: Computing Determinants for (2 ∗ 2) and (3 ∗ 3) Matrices Consider the following (2 ∗ 2) matrix: ·

5 3 A= −1 4

¸ (A.74)

The determinant is first computed using a row definition involving the first row |A| = (5)˜ a11 + (3)˜ a12 = (5)(−1)(1+1) M11 + (3)(−1)(1+2) M12 = (5)(4) + (3)(−1)(−1) = 23

(A.75)

The determinant is next recomputed using a column definition involving the second column

A.6. Determinants of Matrices

15

|A| = (3)˜ a12 + (4)˜ a22 = (3)(−1)(1+2) M12 + (4)(−1)(2+2) M22 = (3)(−1)(−1) + (4)(5) = 23 This leads to the following general result for (2 ∗ 2) matrices: ¯ ¯ ¯a11 a12 ¯ ¯ ¯ ¯a21 a22 ¯ = a11 a22 − a12 a21

(A.76)

(A.77)

Next consider the (3 ∗ 3) matrix  1 2 A = 6 5 7 0

 3 4 9

(A.78)

The determinant is computed using a row definition involving the third row |A| = (7)˜ a31 + 0 + (9)˜ a33 = (7)(−1)(3+1) M31 + (9)(−1)(3+3) M33 ¯ ¯2 = (7) ¯¯ 5

¯ ¯ ¯1 3¯¯ ¯ + (9) ¯ ¯6 4

¯ 2¯¯ = (7) [(2)(4) − (3)(5)] + (9) [(1)(5) − (2)(6)] 5¯ = −49 − 63 = −112

(A.79)

This leads to the following general result for (3 ∗ 3) matrices: ¯ ¯a11 ¯ ¯a21 ¯ ¯a31

a12 a22 a32

¯ a13 ¯¯ a23 ¯¯ = a11 a22 a33 + a12 a23 a31 + a13 a21 a32 a33 ¯

−a13 a22 a31 − a11 a23 a32 − a12 a21 a33

(A.80) .

• Theorem If each element in one row, or in one column, of a determinant is multiplied by a scalar constant c, the value of the determinant is multiplied by c. • Theorem ¯ ¯ If A is any square matrix, then |A| = ¯AT ¯ The determinant of a matrix product is equal to the product of the determinants, that is, |ABC · · · · · · Z| = |A| |B| |C| · · · · · · |Z|

(A.81)

• Theorem An interchange of any two rows or columns of a square matrix will change only the sign of the determinant. • Theorem

16

A.

Matrices and Linear Algebra

If a square matrix has two rows or columns that are equal to each other or are a multiple of each other, then the value of the determinant of the matrix will be zero. • Theorem Let k denote a scalar. If to any row or column of a square matrix is added k times another row or column of the matrix, then the value of the determinant is unchanged. • Theorem A square matrix A is invertible (non-singular) if and only if |A| 6= 0. If |A| = 0, the matrix is said to be singular.

A.7. Matrix Inversion

A.7

17

Matrix Inversion

• Definition If A is a square matrix, and if a ˜ij is the cofactor of the element aij , then the transpose of the matrix of cofactors is called the adjoint of the matrix A, and is written as adj A • Definition The inverse of a non-singular square matrix is equal to the adjoint of the matrix divided by its determinant. Thus, for the square matrix A A−1 =

1 adj A |A|

(A.82)

• Theorem The product of a non-singular matrix A and its inverse, in either order, is an identity matrix I, viz., A−1 A = AA−1 = I

(A.83)

Corollary : If A is a non-singular matrix, then A−1 is also non-singular and

• Theorem If A is non-singular, then

• Theorem If AT is non-singular, then

¯ −1 ¯ ¯A ¯ = 1 |A|

(A.84)

¡ −1 ¢−1 A =A

(A.85)

¡ T ¢−1 ¡ −1 ¢T A = A

(A.86) .

Example 14: Inversion of a (2 ∗ 2) Matrix Consider the matrix · A=

2 4 −1 3

¸ (A.87)

The cofactors of A are a ˜11 = (−1)(1+1) M11 = 3

(A.88)

a ˜12 = (−1)(1+2) M12 = (−1)(−1) = 1

(A.89)

a ˜21 = (−1)(2+1) M21 = (−1)(4) = −4

(A.90)

(2+2)

a ˜22 = (−1)

M22 = (2) = 2

(A.91)

18

A.

Matrices and Linear Algebra

The adjoint of A is thus · adj A =

¸T

3 1 −4 2

· =

3 1

−4 2

¸ (A.92)

Since |A| = (2)(3) − (−1)(4) = 10, it follows that the inverse of A is equal to A−1 =

· adj A 1 3 = |A| 10 1

−4 2

¸ (A.93)

As a check, compute the product of A and its inverse, viz.,

−1

AA ·

· 1 3 = 10 1

¸·

−4 2

¸

2 4 −1 3

¸ 1 (3)(2) + (−4)(−1) (3)(4) + (−4)(3) = =I (1)(4) + (2)(3) 10 (1)(2) + (2)(−1)

(A.94)

This leads to the following general result for (2 ∗ 2) matrices A−1 =

· adj A 1 a22 = |A| a11 a22 − a12 a21 −a21

−a12 a11

¸ (A.95) .

• Theorem −1 Let A and B be non-singular (n ∗ n) matrices. Then (AB) = B−1 A−1 • Definition If A is a non-singular matrix, and if r is a positive integer, then ¡ ¢r A−r = A−1 = A−1 A−1 · · · · · · A−1

(i.e.,

r

factors)

(A.96)

• Theorem If A is a non-singular matrix, then for all integral values of the positive integers r and s, Ar As = A(r+s)

(A.97)

and s

(Ar ) = A(rs) • Definition If RT R = RRT = I (that is, RT = R−1 ), then R is said to be orthogonal. Remark 1. In a linear coordinate transformation, an orthogonal matrix gives a rotation.

(A.98)

A.8. The Trace of a Matrix

A.8

19

The Trace of a Matrix

Like the determinant, a trace operation is defined only for square matrices. • Definition Let A be an (n ∗ n) matrix. The trace of A, which is denoted by tr (A), is equal to tr (A) =

n X

aii = a11 + a22 + · · · + ann

(A.99)

i=1

• Definition Let A and B be (n ∗ n) matrices. The trace of the product AB is equal to tr (AB) = tr (BA)

(A.100)

Also, if C is an (n ∗ n) matrix, then it follows that tr (ABC) = tr (BCA) = tr (CAB)

A.9

(A.101)

The Eigenvalue Problem

• Definition If A is a real (n ∗ n) matrix and I is the identity matrix, then the polynomial defined by p(λ) = det(A − λI)

(A.102)

is called the characteristic polynomial of A. p(λ) is an nth degree polynomial with real coefficients. Consequently, it has at most n distinct roots, some of which may be complex. • Definition If p(λ) is the characteristic polynomial of A, then the zeroes of p are called the eigenvalues or characteristic values of A. • Definition If λ is an eigenvalue of A and x 6= 0 has the property that (A − λI)x = 0, then x is called an eigenvector or characteristic vector of A corresponding to the eigenvalue λ. Put another way, a non-zero vector x(x ∈ Rn ) is called an eigenvector of A if Ax is a scalar multiple of x; i.e. Ax = λx. The matrix A is thus mapped onto a scalar λ. • Definition © ª Let v(1) , v(2) , · · · v(k) be a set of vectors in Rn . The set is said to be linearly independent if scalars α1 , α2 , · · · , αk exist, not all zero, such that a linear combination of v(1) , v(2) , · · · v(k) equals zero; i.e., α1 v(1) + α2 v(2) + · · · + αk v(k) = 0. A set of vectors that is not linearly dependent is called linearly independent. • Theorem

20

A.

Matrices and Linear Algebra

© ª If v(1) , v(2) , · · · v(n) is a set of linearly independent vectors in Rn , then any vector x(x ∈ Rn ) can be written uniquely as x = β1 v(1) + β2 v(2) + · · · + βn v(n)

(A.103)

for some collection of constants β1 , β2 , · · · , βn . Any collection of n linearly independent vectors in Rn is called a basis for Rn . • Theorem If A is a matrix and λ1 , λ2 , · · · , λk are distinct eigenvalues of A with associated eigenvectors x(1) , x(2) , · · · , x(k) , then the eigenvectors form a linearly independent set. • Definition © ª ¡ ¢T A set of vectors v(1) , v(2) , · · · v(n) is called orthogonal if v(i) v(j) = 0 for all i 6= j. If, ¡ ¢T in addition, v(i) v(i) = 1 for all i = 1, 2, · · · , n, then the set is called orthonormal. • Theorem An orthogonal set of vectors that does not contain the zero vector is linearly independent. • Theorem If A is a symmetric (n ∗ n) matrix, then there exist n eigenvectors of A that form an orthonormal set. • Theorem The eigenvalues of a symmetric matrix are all real numbers. • Theorem If an eigenvalue λ of a symmetric (n ∗ n) matrix A is repeated k times as the root of the characteristic polynomial, then the eigenspace corresponding to λ is k–dimensional.

A.10. Exercises

A.10

21

Exercises

All of the following exercises should be analyzed using a finite element program that can perform two-dimensional analyses of linear elastic continua. A.1 Consider the matrix H, where H = I − 2xxT . Here I is the identity matrix and x is a vector such that xT x = 1. Show that, in general (i.e., without using matrices of a specific size) 1. H is symmetric (i.e., HT = H) 2. H2 = I 3. HT H = I

22

A.

Matrices and Linear Algebra

References [1] Anton, H., Elementary Linear Algebra, 3rd Edition. New York: J. Wiley and Sons (1981). [2] Joshi, A. W., Matrices and Tensors in Physics, 2nd Edition. A Halsted Press Book, New York: J. Wiley and Sons (1984). [3] Noble, B., Applied Linear Algebra. Englewood Cliffs, NJ: Prentice Hall (1969). [4] Strang, G., Linear Algebra and Its Applications. New York: Academic Press (1976).

23