Systems of Linear Equations

1
Systems of Linear Equations

• Error Analysis
• and
• System Condition

2
Question

• Suppose we use a computer to solve A1x1 b1 and
  A2x2 b2 and obtain x'1 and x'2.
• Which of these two solutions is more accurate (as
  compare relatively to the true solutions)?

3
Error Analysis

• Suppose we want to find the solution of
• Ax b
• Would there be a x' which gives b Ax' 0?
• i.e., is the system solvable?
• Would a small rounding error in b or in A results
  in large change in the solution x?

4
Error AnalysisRelationship between error and
residual

• Suppose x' is an estimated solution of Ax b.
• Let the residual be r b Ax' ---- (1)
• Ax b gt 0 b Ax ---- (2)
• (1) (2) gt r A(x x')
• gt x x' A-1r
• Thus, if A-1 has elements much larger than 1
  (assuming A has been scaled), a small residual,
  r, may result in a large error x x'. (i.e.,
  ill-condition cases)
• The inverse of A can indicate if A is
  ill-condition.

5
Question

• If a matrix A is scaled (s.t. the largest element
  is 1), then A-1 can indicate whether A is
  ill-condition or not.

Suppose both A1 and A2 are scaled. How can you
tell which of them is more ill-condition? We
need a way to quantify the condition.
6
Condition number

• A formal way to state the condition of a system
  (ill-condition or not)
• A way to check the condition is the matrix
  condition number.
• Cond(A) A A-1
• where is the norm, which is a
  real-valued function that measures the size of
  length of vectors or matrices.
• Cond(A) 1

7

• Various definitions of norm for Vectors
• p-norm
• Euclidean-norm
• 1-norm
• 8-norm

8

• Various definitions of norm for Matrices
• p-norm
• Euclidean-norm
• 1-norm (max column)
• 8-norm (max row)

9
Error AnalysisRelationship between error in x
and (rounding error in A)

• If there is a small change in the value of A,
  what would be the change in x?
• It can also be shown that
• If the condition number is considerably greater
  than unity, it suggest that the system is
  ill-conditioned.

10
Examples
11
Iterative Refinement

• In some cases, we can reduce round-off errors by
  the following procedure (can repeat for several
  times)
• Suppose in solving Ax b, we obtain an
  approximated solution x' s.t. x x' e.
• Substituting x' back into the system yields
• Ax' b' ---- (1)
• Ax b gt A(x' e) b ---- (2)
• (2) (1) gt Ae b b' ---- (3)
• Solving the system of equations in (3) yieldse.
• By adding e (called the correction factor) to x',
  we can improve the solution to Ax b.

12
Summary

• The inverse of a matrix can indicate if the
  matrix is ill-condition or not.
• The matrix condition number can be calculated as
• Cond(A) A A-1
• Large condition number implies matrix is
  ill-condition.