SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:

1
SE301 Numerical MethodsTopic 3
Solution of Systems of Linear Equations Lectures
12-17
KFUPM Read Chapter 9 of the textbook
2
Lecture 12Vector, Matrices, andLinear Equations
3
VECTORS
4
MATRICES
5
MATRICES
6
Determinant of a MATRICES
7
Adding and Multiplying Matrices
8
Systems of Linear Equations
9
Solutions of Linear Equations
10
Solutions of Linear Equations

• A set of equations is inconsistent if there
  exists no solution to the system of equations
  

11
Solutions of Linear Equations

• Some systems of equations may have infinite
  number of solutions

12
Graphical Solution of Systems ofLinear Equations
Solution x11, x22
13
Cramers Rule is Not Practical
14
Lecture 13 Naive Gaussian Elimination

• Naive Gaussian Elimination
• Examples

15
Naive Gaussian Elimination

• The method consists of two steps
• Forward Elimination the system is reduced to
  upper triangular form. A sequence of elementary
  operations is used.
• Backward Substitution Solve the system starting
  from the last variable.

16
Elementary Row Operations

• Adding a multiple of one row to another
• Multiply any row by a non-zero constant

17
ExampleForward Elimination
18
ExampleForward Elimination
19
ExampleForward Elimination
20
ExampleBackward Substitution
21
Forward Elimination
22
Forward Elimination
23
Backward Substitution
24
Lecture 14Naive Gaussian Elimination

• Summary of the Naive Gaussian Elimination
• Example
• Problems with Naive Gaussian Elimination
• Failure due to zero pivot element
• Error
• Pseudo-Code

25
Naive Gaussian Elimination

• The method consists of two steps
• Forward Elimination the system is reduced to
  upper triangular form. A sequence of elementary
  operations is used.
• Backward Substitution Solve the system starting
  from the last variable. Solve for xn ,xn-1,x1.

26
Example 1
27
Example 1
28
Example 1Backward Substitution
29
Determinant
30
How Many Solutions Does a System of Equations
AXB Have?
31
Examples
32
Pseudo-Code Forward Elimination

• Do k 1 to n-1
• Do i k1 to n
• factor ai,k / ak,k
• Do j k1 to n
• ai,j ai,j factor ak,j
• End Do
• bi bi factor bk
• End Do
• End Do

33
Pseudo-Code Back Substitution

• xn bn / an,n
• Do i n-1 downto 1
• sum bi
• Do j i1 to n
• sum sum ai,j xj
• End Do
• xi sum / ai,i
• End Do

34
Lectures 15-16Gaussian Elimination with Scaled
Partial Pivoting

• Problems with Naive Gaussian Elimination
• Definitions and Initial step
• Forward Elimination
• Backward substitution
• Example

35
Problems with Naive Gaussian Elimination

• The Naive Gaussian Elimination may fail for very
  simple cases. (The pivoting element is zero).
• Very small pivoting element may result in serious
  computation errors

36
Example 2
37
Example 2Initialization step
Scale vector disregard sign find largest in
magnitude in each row
38
Why Index Vector?

• Index vectors are used because it is much easier
  to exchange a single index element compared to
  exchanging the values of a complete row.
• In practical problems with very large N,
  exchanging the contents of rows may not be
  practical.

39
Example 2Forward Elimination-- Step 1 eliminate
x1
40
Example 2Forward Elimination-- Step 1 eliminate
x1
First pivot equation
41
Example 2Forward Elimination-- Step 2 eliminate
x2
42
Example 2Forward Elimination-- Step 3 eliminate
x3
Third pivot equation
43
Example 2Backward Substitution
44
Example 3
45
Example 3Initialization step
46
Example 3Forward Elimination-- Step 1 eliminate
x1
47
Example 3Forward Elimination-- Step 1 eliminate
x1
48
Example 3Forward Elimination-- Step 2 eliminate
x2
49
Example 3Forward Elimination-- Step 2 eliminate
x2
50
Example 3Forward Elimination-- Step 3 eliminate
x3
51
Example 3Forward Elimination-- Step 3 eliminate
x3
52
Example 3Backward Substitution
53
How Do We Know If a Solution is Good or Not

• Given AXB
• X is a solution if AX-B0
• Compute the residual vector R AX-B
• Due to rounding error, R may not be zero

54
How Good is the Solution?
55
Remarks

• We use index vector to avoid the need to move the
  rows which may not be practical for large
  problems.
• If we order the equation as in the last value of
  the index vector, we have a triangular form.
• Scale vector is formed by taking maximum in
  magnitude in each row.
• Scale vector does not change.
• The original matrices A and B are used in
  checking the residuals.

56
Lecture 17 Tridiagonal Banded Systems and
Gauss-Jordan Method

• Tridiagonal Systems
• Diagonal Dominance
• Tridiagonal Algorithm
• Examples
• Gauss-Jordan Algorithm

57
Tridiagonal Systems

• Tridiagonal Systems
• The non-zero elements are in the main diagonal,
  super diagonal and subdiagonal.
• aij0 if i-j gt 1

58
Tridiagonal Systems

• Occur in many applications
• Needs less storage (4n-2 compared to n2 n for
  the general cases)
• Selection of pivoting rows is unnecessary
  (under some conditions)
• Efficiently solved by Gaussian elimination

59
Algorithm to Solve Tridiagonal Systems

• Based on Naive Gaussian elimination.
• As in previous Gaussian elimination algorithms
• Forward elimination step
• Backward substitution step
• Elements in the super diagonal are not affected.
• Elements in the main diagonal, and B need
  updating

60
Tridiagonal System
61
Diagonal Dominance
62
Diagonal Dominance
63
Diagonally Dominant Tridiagonal System

• A tridiagonal system is diagonally dominant if
• Forward Elimination preserves diagonal dominance

64
Solving Tridiagonal System
65
Example
66
Example
67
ExampleBackward Substitution

• After the Forward Elimination
• Backward Substitution

68
Gauss-Jordan Method

• The method reduces the general system of
  equations AXB to IXB where I is an identity
  matrix.
• Only Forward elimination is done and no backward
  substitution is needed.
• It has the same problems as Naive Gaussian
  elimination and can be modified to do partial
  scaled pivoting.
• It takes 50 more time than Naive Gaussian method.

69
Gauss-Jordan MethodExample
70
Gauss-Jordan MethodExample
71
Gauss-Jordan MethodExample
72
Gauss-Jordan MethodExample