MATH 685 CSI 700 OR 682 Lecture Notes

1
MATH 685/ CSI 700/ OR 682 Lecture Notes

• Lecture 8.
• Nonlinear equations

2
Nonlinear Equations

• Given a function f, we are looking for a value x,
  s.t.
• f(x)0
• (a root of the equation, or a zero of the
  function f). The problem is called root finding
  or zero finding.

3
Example
4
Existence/uniqueness
5
Examples in 1d
6
Example of a system in 2d
7
Multiplicity
8
Sensitivity and conditioning
9
Sensitivity and conditioning
10
Sensitivity and conditioning
11
Convergence rate
12
Convergence rate
13
Bisection method
14
Example bisection iteration
15
Bisection method
16
Fixed-point iterations
17
Examples
18
Example fixed point problems
19
Examples FPI
20
Example FPI
21
Convergence of FPI
22
Newtons method
23
Newtons method
24
Newtons method
25
Convergence of Newtons method
26
Newtons method
27
Secant method
28
Secant method
29
Example
30
Higher-degree interpolation
31
Inverse interpolation
32
Inverse quadratic interpolation
33
Example
34
Linear fractional interpolation
35
Example
36
Safeguarded methods

• Rapidly convergent methods for solving nonlinear
  equations may not converge unless started close
  to solution, but safe methods are slow
• Hybrid methods combine features of both types of
  methods to achieve both speed and reliability
• Use rapidly convergent method, but maintain
  bracket around solution
• If next approximate solution given by fast method
  falls outside bracketing interval, perform one
  iteration of safe method, such as bisection
• Fast method can then be tried again on smaller
  interval with greater chance of success
• Ultimately, convergence rate of fast method
  should prevail
• Hybrid approach seldom does worse than safe
  method, and usually does much better
• Popular combination is bisection and inverse
  quadratic interpolation, for which no derivatives
  required

37
Zeros of polynomials
38
Systems of nonlinear equations

• Solving systems of nonlinear equations is much
  more
• difficult than scalar case because
• Wider variety of behavior is possible, so
  determining existence and number of solutions or
  good starting guess is much more complex
• There is no simple way, in general, to guarantee
  convergence to desired solution or to bracket
  solution to produce absolutely safe method
• Computational overhead increases rapidly with
  dimension of problem

39
Fixed-point iteration (FPI)
40
Newtons method
41
Example
42
Example
43
Convergence of Newtons method
44
Cost of Newtons method
45
Secant updating methods
46
Broydens method
47
Broydens method
48
Example
49
Example
50
Example
51
Example (cont)
52
Robust Newton-like methods
53
Trust-region methods