Review of Matrix Algebra

1
Review of Matrix Algebra

• Department of Statistics
• Texas AM University
• presented by Curtis Alexander

2
What is a Matrix ?

• A matrix is a rectangular table, or array,
  composed of either
• numbers
• OR
• variables
• A matrix may also contain
• fractions
• AND/OR
• decimals

3
Matrix Notation I

• Matrices (the plural of matrix) are enclosed in
• brackets (as we have seen)
• OR
• parenthesis (less commonly used notation)
• Note We will use the more conventional bracket
  notation.

4
Matrix Notation II

• A matrix is typically denoted by a capital
  letter.
• When a matrix appears inline with text, it is
  often written in bold.
• Note Different textbooks will use different
  notation for matrices so check the notation!

In simple linear regression, we call H the hat
matrix because it transforms the vector of
observed responses into the vector of fitted
responses.
5
Elements of a Matrix

• The individual numbers (or variables) in a matrix
  are called elements.
• Each possible location in a matrix must contain
  an element.
• Is this a matrix ?
• NO!!

6
Rows and Columns

• What other parts of a matrix are named?
• Rows run horizontally
• AND
• Columns run vertically

7
Location of Elements

• We refer to a specific element in a matrix by
  referencing its location using rows and columns.
• When referencing an element within a matrix, we
  use the lower case letter of the matrix.
• We list the row first, followed by the column.
• Thus we say that the element t23 5 .

T
8
Elements of a Matrix Practice

• w12 ?
• w12 5
• 6 ?
• 6 w24
• The element with value 16 is located in which
  row?
• Row 3
• How many columns (total) does this matrix
  contain?
• 4 columns

W
9
Size of a Matrix

• The size of a matrix is expressed in the form r
  rows by p columns.
• Matrix A has 4 rows and 3 columns, so we list its
  size as 4 x 3 which we read four by three and
  which we write as A4x3.
• Note Dimension is another word used for size of
  a matrix.
• What is the size (or dimensions) of matrix X?
• 2 x 2

10
Special Matrices I

• Vector a matrix that has only one row or one
  column
• Column vector matrix with one column
• Row vector matrix with one row
• Note Vectors are denoted by lower case letters.
• Scalar a matrix that has only one row and one
  column, or alternatively a matrix that only has
  one element
• Scalars are usually written without brackets.
• Think of scalars as merely constants.

ß
11
11
Special Matrices II

• Square matrix the number of rows and columns
  are equal
• Diagonal the elements that run from the upper
  left element to the lower right element in a
  square matrix also called the main diagonal
• Diagonal matrix a square matrix with all
  non-diagonal elements equal to zero

12
Special Matrices III

• Zero matrix a matrix with all elements equal to
  zero
• Identity matrix a diagonal matrix with all
  elements along the diagonal equal to one
• The identity matrix is denoted by the capital
  letter I.
• The identity matrix will become important later
  when we discuss the inverse of a matrix.

13
Matrix Addition

• Matrices may only be added if they are the same
  size.
• A and B are both 2x3 so they may be added.
• The result of AB, which we call C, is also size
  2x3.
• Note Matrix addition IS commutative in general
  AB BA
• Take each matching element of A and B and add
  them together, placing the sum in the same
  elemental position of C.

14
Matrix Subtraction

• Like matrix addition, matrices may only be added
  if they are the same size.
• D and E are both 2x3 so they may be subtracted.
• The result of D - E, which we call F, is also
  size 2x3.
• Note Matrix subtraction is NOT commutative in
  general D - E ? E - D
• Take the matching element of D and E and subtract
  the element from matrix E from the element from
  matrix D, placing the difference in the same
  elemental position of F.

15
Matrix Add/Sub Practice

• A B ?
• C A ?
• B D ?
• Sorry trick question! We cannot sum
  because their sizes are different!

A
C
B
D
16
Matrix Multiplication Introduction

• To multiply matrices, the of columns of the
  first matrix MUST equal the of rows of the
  second matrix.
• How do we easily go about determining if M can be
  multiplied by N?
• Write the dimensions of M adjacent to the
  dimensions of N. If the inner dimensions match,
  then they may be multiplied.
• The outer dimensions determine the dimensions of
  the product P.
• Note Matrix multiplication is NOT commutative in
  general --
• MN ? NM

17
Matrix Multiplication Properties

• If A, B, and C are matrices and if the
  multiplicative combinations below are assumed to
  have the correct size then in general
• (AB)C A(BC) associativity
• (AB)C AC BC left distributivity
• C(AB) CA CB right distributivity
• However, commutativity is does NOT hold in the
  general case
• AB ? BA

18
Multiplication Scalar Matrix

• This is the easiest form of matrix multiplication
  a scalar, or constant, multiplied by a matrix.
• Simply multiply each element in the matrix by the
  scalar.

19
Multiplication Vector Vector

• The only vectors that can be multiplied together
  are
• row vector column vector
• column vector row vector
• Check to ensure that the columns (or rows) of
  the vector equals the rows (or columns) of the
  vector.
• To evaluate jk, multiply the first element of j
  by the first element of k. The result is placed
  in l11. Next multiply the first element of j by
  the second element of k and place it in l12.
• This is continued where the element location of j
  becomes the row location of the result and the
  element location of k becomes the column location
  of the result.

20
Multiplication Vector Matrix I

• Only row vectors can be multiplied by a matrix.
  Check to ensure that the columns of the vector
  equals the rows of the matrix.
• To evaluate aB c, first multiply the elements
  of vector a that correspond with the elements of
  the first column of B. Second, sum all three of
  these intermediate product results to get the
  final result. This result becomes the first
  element of c (c11).
• The result of any vector multiplied by a matrix
  is a vector.

21
Multiplication Vector Matrix II

• To further evaluate aB c, multiply the
  elements of vector a that correspond with the
  elements of the second column of B. Sum all
  three of these intermediate product results to
  get the final result. This result becomes the
  second element of c (c12).

a
B
c

22
Multiplication Vector Matrix III

• To finish evaluating aB c, multiply the
  elements of vector a that correspond with the
  elements of the third column of B. Sum all three
  of these intermediate product results to get the
  final result. This result becomes the third
  element of c (c13).

a
B
c

23
Multiplication Matrix Matrix I

• To multiply matrices, the columns of the first
  matrix must equal the rows of the second
  matrix.
• We proceed similar as to when multiplying a
  vectormatrix in order to multiply XY Z.
• First, multiply the first row of matrix X by the
  first column of matrix Y (matching the elements
  as we have done previously). Sum these products
  and the result goes in z11.

24
Multiplication Matrix Matrix II

• To find z21, multiply the second row of matrix X
  by the first column of matrix Y (matching the
  elements as we have done previously). Sum these
  products and the result goes in z21.
• Note For a specific element in Z, say z21, we
  know this means multiply the second row of matrix
  X by the first column of matrix Y.

z21
25
Multiplication Matrix Matrix III

• For all subsequent elements of Z, multiply the
  appropriate row of matrix X by the appropriate
  column of matrix Y (matching the elements as we
  have done previously). Sum these products and
  the result is placed in the appropriate location
  in Z.
• As a final example, to find z13 multiply the
  first row of matrix X by the third row of matrix
  Y and then sum the products of these rows and
  columns.

Y
X
Z
26
Matrix Multiplication Practice

• bA ?
• CE ?
• Which of the following products exist?
• Ab AD bC EC AC DD bb
• AD bC AC DD

A
b
D
C
E
27
Trace

• To find the trace of a square matrix, simply sum
  all the elements that lie along the main diagonal
  of the matrix.

28
Transpose

• To take the transpose of a matrix, either
• write the rows as columns
• OR
• write the columns as rows
• The transpose of a column vector is a row vector
  and vice versa.
• Note Transpose may either be written with a
  capital T or using an apostrophe.

29
Inverse Introduction I

• We have seen addition, subtraction, and
  multiplication of matrices. There does not exist
  division of matrices per se instead we shall
  use the inverse of a matrix.
• For example, in order to solve for x at right,
  you simply divide both sides of the equation by
  three and find that x4.
• Another way we say this is accomplished is by
  multiplying both sides of the equation by 1/3.

30
Inverse Introduction II

• If A is a square matrix and the inverse of A
  exists, then
• AA-1 A-1A I (which is similar to
• the algebraic expression ? 3 1).
• We say that A is invertible if AA-1 I.
• Note Not every square matrix has an inverse. If
  a matrix, say A, does not have an inverse then we
  say that the inverse of A does not exist.

31
Determinant of a 2x2 Matrix

• Before we can get to calculating the inverse of a
  matrix, we need to know how to calculate the
  determinant.
• To calculate the determinant, simply multiply the
  opposite corner elements and subtract the product
  results.

32
Inverse of a 2x2 Matrix I

• First, calculate the determinant of the matrix
  you would like to invert.
• Next, invert the signs of element a12 and a21 and
  then exchange the values of a11 and a22.
• Finally, multiply by this new matrix.

33
Inverse of a 2x2 Matrix II

• For a generic matrix X, the formula for
  calculating the inverse is at right.
• When does the inverse not exist?
• Whenever the determinant is 0 because we would be
  dividing by 0 in the inverse equation.

34
Determinant and Inverse Practice

• det(A) ?
• 45 37-1
• D-1?
• det(B)?
• 12 32-4
• C-1?
• The inverse does not exist because the
  determinant is 0 --
• det(C)45-2100

A
B
D
C
35
Writing a System of Linear Equations in Matrix
Form

• What if we had the following system of linear
  equations.
• How could we write this in matrix notation?
• First, write the coefficients in a 2x2 matrix (or
  an nxn matrix depending upon the number of
  equations).
• Next, create a column vector containing the
  variables and place it to the right of the
  coefficient matrix.
• Finally create another column vector containing
  the values on the right of the equal sign.

36
Solving Simple Matrix Equations

• To solve equations like AXY for X, we do not
  divide Y by A, but multiply the inverse of A
  (which is written as A-1) by Y to find X.

37
Example from Multiple Linear Regression I

• Where do we see matrices in statistics?
• One such place is in multiple linear regression,
  which is seen at right. This is a multiple
  linear regression model for n independent
  observations generated from p predictor
  variables.
• Instead of writing out n equations, this may be
  written using matrices as

38
Example from Multiple Linear Regression II

• As a final example of where matrices occur in
  statistics, we look again at multiple linear
  regression.
• The residual sum of squares of ß is seen at
  right.
• Using RSS(ß), we calculate the least squares
  estimate of ß. In this form, we obtain the least
  squares estimate of each ßi with just a few
  matrix computations.

39
Matrix Calculator

• Web based Matrix Calculator
• Note that there are other matrix calculators
  online, but this is among the easiest to use.
• Now I will demonstrate some of its capabilities.

http//people.hofstra.edu/stefan_waner/Realworld/m
atrixalgebra/fancymatrixalg2.html