Engineering Computation

1
EngineeringComputation

• Part 2

2
Roots of Equations
Objective Solve for x, given that f(x)
0 -or- Equivalently, solve for x such
that g(x) h(x) gt f(x) g(x) h(x) 0
3
Roots of Equations
Chemical Engineering (CC 8.1, p. 187) van der
Waals equation v V/n ( volume/ moles) Find
the molal volume v such that
p pressure, T temperature, R universal
gas constant, a b empirical constants
4
Roots of Equations
Civil Engineering (CC Prob. 8.17, p. 205)
Find horizontal component of tension, H, in a
cable that passes through (0,y0) and (x,y)
w weight per unit length of cable
5
Roots of Equations
Electrical Engineering (CC 8.3, p. 194) Find
the resistance, R, of a circuit such that the
charge reaches q at specified time t
L inductance, C capacitance, q0
initial charge
6
Roots of Equations
Mechanical Engineering (CC 8.4, p. 196) Find
the value of stiffness k of a vibrating
mechanical system such that the displacement x(t)
becomes zero at t 0.5sec. The initial
displacement is x0 and the initial velocity is
zero. The mass m and damping c are known, and ?
c/(2m).
in which
7
Roots of Equations

• Determine real roots of
• Algebraic equations (including polynomials)
• Transcendental equations such as f(x) sin(x)
  e-x
• Combinations thereof

8
Roots of Equations
Engineering Economics Example A municipal bond
has an annual payout of 1,000 for 20 years. It
costs 7,500 to purchase now. What is the
implicit interest rate, i ? Solution
Present-value, PV, is
in which PV present value or purchase price
7,500 A annual payment 1,000/yr n
number of years 20 yrs i interest rate
? (as a fraction, e.g., 0.05 5)
9
Roots of Equations
Engineering Economics Example (cont.) We need to
solve the equation for i
Equivalently, find the root of
10
Roots of Equations
Excel
11
Roots of Equations

• Graphical methods
• Determine the friction coefficient c necessary
  for a parachutist of mass 68.1 kg to have a speed
  of 40 m/seg at 10 seconds.
• Reorganizing.

12
Roots of Equations
Two Fundamental Approaches 1. Bracketing or
Closed Methods - Bisection Method -
False-position Method 2. Open Methods - One
Point Iteration - Newton-Raphson Iteration -
Secant Method
13
Bracketing Methods
14
Bracketing Methods

• Though the cases above are generally valid, there
  are cases in which they do not hold.

15
Bracketing Methods (Bisection method)
Bisection Method
f(x)
f(x1) f(xr) gt 0
x
xr gt x1
16
Bracketing Methods (Bisection method)
Bisection Method Given lower and upper bounds,
xl and xu, which bracket the root f(xl)
f(xu) lt 0 1) Estimate the Root by midpoint 2)
Revise the bracket f(xl) f(xr) lt 0, xr gt xu,
f(xl) f(xr) gt 0, xr gt xl 3) Repeat steps 1-2
until a) f(xr) lt k, b) ?a lt ?s , with ?a
c) xl xu lt ? d) maximum of iterations
is reached. (Always do this in iteration
algorithms.)
17
Bracketing Methods (Bisection method)
Engineering Economics Example We need to solve
the equation for i
Equivalently, find the root of
Make conservative guesses at the upper and lower
bounds 100 interest rate, f(1.0) 6,500
0 interest rate, f(0.0) -12,500
18
Bracketing Methods (Bisection method)
Excel
19
Bracketing Methods (Bisection method)
Engineering Economics Example Score Sheet for
Rootfinding Example   Method Initial Est(s). ?s
2 E-2 ?s 2 E-7   Bisection (0.00,
1.00) 9 26 (0.05, 0.15) 6 22 Convergence
guaranteed
20
Bracketing Methods (Bisection method)
One important advantage of this method is that
one can calculate the number of required
iterations for a given error.
21
Bracketing Methods (Bisection method)
Parachutist Example
22
Bracketing Methods (Bisection method)
Parachutist Example
23
Bracketing Methods (Bisection method)

• Bisection Method
• Advantages
• 1. Simple
• 2. Good estimate of maximum error
• 3. Convergence guaranteed
• Disadvantages
• 1. Slow
• 2. Requires two good initial estimates which
  define an interval around root
• use graph of function,
• incremental search, or
• trial error

24
Bracketing Methods (False-position Method)
25
Bracketing Methods (False-position Method)
Similar to bisection. Uses linear interpolation
to approximate the root xr 1) 2) Revise the
bracket f(x1) f(xr) lt 0, xr gt xu, f(x1)
f(xr) gt 0, xr gt x1 3) Repeat steps 1-2
until a) f(xr) lt k, b) ?a lt ?s , with ?a
c) xu x1 d d) maximum of
iterations is reached. (Always do this in
iteration algorithms.)
26
Bracketing Methods (False-position Method)
Excel
27
Bracketing Methods (False-position Method)
28
Bracketing Methods (False-position Method)
Score Sheet for False-Position Example Method I
nitial Est(s). ?s 2 E-2 ?s 2
E-7   Bisection (0.00, 1.00) 9 26 (0.05,
0.15) 6 22   False-pos. (0.00,
1.00) 11 28 (0.05, 0.15) 3 14
29
Bracketing Methods (False-position Method)
Parachutist Example
30
Bracketing Methods (False-position Method)
Parachutist Example
31
Bracketing Methods (False-position Method)
There are some cases in which the false position
method is very slow, and the bisection method
gives a faster solution.
32
Bracketing Methods (False-position Method)
Summary of False-Position Method Advantages 1.
Simple 2. Brackets the Root Disadvantages 1.
Can be VERY slow 2. Like Bisection, need an
initial interval around the root.
33
Open Methods
Roots of Equations - Open Methods Characteristics
1. Initial estimates need not bracket the
root 2. Generally converge faster 3. NOT
guaranteed to converge Open Methods
Considered - One Point Iteration -
Newton-Raphson Iteration - Secant Method
34
Open Methods

• An alternative method consists of separating the
  function into two parts.

35
Open Methods (Fixed point method)

• Fixed point Method
• predict a value of xi1 as a function of xi.
• Convert f(x) 0 to x g(x)
• iteration steps xi1 g(xi )
• 
  x(new) g(x(old) )

36
Open Methods (Fixed point method)
Example I
37
Open Methods (Fixed point method)
Convergence Does x move closer to real root
(?) Depends on 1. nature of the function 2.
accuracy of the initial estimate Interested
in 1. Will it converge or will it diverge? 2.
How fast will it converge ? (rate of
convergence)
38
Open Methods (Fixed point method)
Convergence of the Fixed point Method Root
satisfies xr g(xr) The Taylor series for
function g is xi1 g(xr) g'(x)(xi - xr) xr
lt x lt xi Subtracting the second equation from the
first yields (xr xi1) g'(x) (xr xi)
or 1. True error for next iteration is smaller
than the true error in the previous iteration if
g'(x) lt 1.0 (it will converge). 2. Because
g'(x) is almost constant, the new error is
directly proportional to the old error (linear
rate of convergence).
39
Open Methods (Fixed point method)
Further Considerations Convergence depends on
how f(x) 0 is converted into x g(x) So .
. . Convergence may be improved by recasting
the problem.
40
Open Methods (Fixed point method)
Convergence Problem For slowly converging
functions
can be small, even though xnew is not close to
root. Remedy Do not completely rely on ea to
ensure that the problem is solved. Check to make
sure f(xnew) lt k .
41
Open Methods (Fixed point method)
42
Open Methods
43
Roots of Equations
Two Fundamental Approaches 1. Bracketing or
Closed Methods - Bisection Method -
False-position Method 2. Open Methods - One
Point Iteration - Newton-Raphson Iteration -
Secant Method
44
Open Methods (Newton-Raphson Method)
Newton-Raphson Method Geometrical Derivation
Slope of tangent at xi is
Solve for xi1 Note that this is the
same form as the generalized one-point iteration,
xi1 g(xi)
45
Open Methods (Newton-Raphson Method)
Newton-Raphson Method
Tangent w/slopef '(xi )
f(x)
f(x)
f(xi)
f(xi)
f(xi1)
f(xi1)
x
xi1
x
(xi)
xi
xi1
xi xi1
46
Open Methods (Newton-Raphson Method)
First order Taylor Series Derivation 0
f(xr) ? f(xi) f '(xi) (xr xi) solve for xr to
yield next guess xi1
This has the form xi1 g(xi) with
47
Open Methods (Newton-Raphson Method)
Newton-Raphson iteration
This iteration process is repeated
until 1. f(xi1) ? 0, i.e., f(xi1) lt
k, with ? small number 2. 3. Maximum number
of iterations is reached.
48
Open Methods
a) Inflection point in the neighboor of a root.
b) Oscilation in the neighboor of a maximum or
minimum.
c) Jumps in functions with several roots.
d) Existence of a null derivative.
49
Open Methods (Newton-Raphson Method)
Bond Example To apply Newton-Raphson method to
We need the derivative of the function
50
Open Methods (Newton-Raphson Method)
Score Sheet for Newton-Raphson Example Method I
nitial Est(s). ?s 2 E-2 ?s 2
E-7   Bisection (0.00, 1.00) 9 26 (0.05,
0.15) 6 22   False-pos. (0.00,
1.00) 11 28 (0.05, 0.15) 3 14   N-R 1.0 diverge
s diverges 0. 5 2, but wrong 48 0.25 5 7 0.1
5 3 5 0.05 4 5
EXCEL
51
Open Methods (Newton-Raphson Method)
Error Analysis for N-R Recall
that Taylor Series gives
where xr ? x ? xi and f(xr) 0
52
Open Methods (Newton-Raphson Method)
Dividing through by f '(xi) yields
OR
Ei1 is proportional to Ei2 gt quadratic rate
of convergence.
53
Open Methods (Newton-Raphson Method)

• Summary of Newton-Raphson Method
• Advantages
• Can be fast
• Disadvantages
• May not converge
• 2. Requires a derivative

54
Open Methods (Secant Method)
Secant Method Approx. f '(x) with backward
FDD Substitute this into the N-R
equation to obtain the iterative expression
55
Open Methods (Secant Method)
Secant Method
f(x)
f(x)
f(xi-1)
f(xi)
f(xi-1)
f(xi)
x
xi1
xi-1
xi1
x
xi-1
xi
xi
xi xi1
56
Open Methods (Secant Method)

• 1) Requires two initial estimates xi-1 and xi
• These do NOT have to bracket root !
• 2) Maintains a strict sequence
• Repeated until
• a. f(xi1) lt k with k small number
• b.
• c. Max. number of iterations is reached.
• 3. If xi and xi1 were to bracket the root, this
  would be the same as the False-Position Method.
  BUT WE DON'T!

57
Open Methods (Secant Method)

• In the secant method, the values are replaced in
  a strict sequence, xi1 to xi, and this to xi-1.
  Thus, the new values can be on the de same sode
  of the root, and sometimes diverge.

58
Open Methods (Secant Method)
Score Sheet for Secant Example Method Initial
Est(s). ?s 2 E-2 ?s 2 E-7 Bisection (0.00,
1.00) 9 26 (0.05, 0.15) 6 22 False-pos.
(0.00, 1.00) 11 28 (0.05, 0.15) 3 14 N-R 1.0 d
iverges diverges 0.5 2, but wrong 48 0.25 5 7
0.15 3 5 0.05 4 5 Secant (0,
1) diverges diverges (0.00, 0.50) 4, but wrong
(chaotic) 27 (0.05, 0.15) 3 6
59
Open Methods

• Why do open methods fail?
• Function may not look linear.
• Remedy recast into a linear form. For example,

Is a poorly constrained problem in that there is
a large, nearly flat zone for which the
derivative is near zero. Recast as i f(i) 0
7,500 i - 1000 1 - (1i)-20
60
Open Methods

• Recast as i f(i) 0 7,500 i - 1000 1 -
  (1i)-20
• The recast function, "i f(i) will have the same
  roots as f(i) plus an additional root at i 0.
• It will not have a large, flat zone, thus
• h(i) i f(i) 7,500 i 1000 1 (1
  i)20
• To apply N-R we also need the first derivative
• h'(i) 7,500 - 20,000 (1 i)-21

61
Open Methods
Score Sheet for Open Methods Method Initial
Est(s). ?s 2 E-2 ?s 2 E-7   N-R 1.0 diverges
diverges 0. 5 2, but wrong 48 0.25 5 7 0.15
3 5 0.05 4 5 Secant (0.00, 0.50) 4, but
wrong 27 (0.05, 0.15) 3 6 N-R 1.00 3 4 as i
f(i) 0.150 2 4 0.050 4 5 0.047 crazy
results 0.03 converges to i0
62
Open Methods

• Cases of Multiple Roots
• Multiple Roots
• f(x) (x 2)2 (x 4)
• x 2 represents two of the
• three roots.

63
Open Methods

• Problems and Approaches
• Cases of Multiple Roots
• 1.Bracketing Methods fail locating x 2.
• Note that f(x?) f(xr) gt 0.
• 2. At x 2, f(x) f '(x) 0.
• Newton-Raphson and Secant methods may experience
  problems.
• Rate of convergence drops to linear.
• Luckily, f(x) ? 0 faster than f '(x) ? 0
• 3. Other remedies, recasting problem
• Find x such that u(x) 0 where
• Note that u(x) and f(x) have same roots.

64
Summary -- Rates of Convergence

• m 1 linear convergence
• m 2 quadratic convergence
• Method m
• Bisection 1
• False Position 1
• Secant, mult. root 1
• NR, mult. root 1
• Secant, single root 1.618 "super linear"
• NR, single root 2
• Accel. NR, mult. root (f(x)/f'(x)0) 2

65
Multivariate (Multidimensional) Equations

• Solve
• fi(x1, ..., xn) 0 for i 1,...,n
• Let X (x1, ..., xn)T
• Given intial guess Xt, try to solve

where
Obtain ?X (Xi1 Xi) from linear
equations
66
Alternative Stopping Criteria

• Always limit number of iterations using an outer
  DO loop. The problem may not converge and could
  try to go on forever.
• Absolute error criteria for "small" differences
  xt - xt-1 lt d
• 3. Relative error criteria for "relatively
  small" changes
• xt xt-1 lt e xt
• 4. Can combined error criteria 2 3 for large
  and small x-values
• xt xt-1 lt d e xt
• 5. Converge on zero residual
• f(xt) lt k

67
Three Performance Criteria

• Stopping Criteria
• xi xi-1 lt ? ? xi
• or f(xi) lt ?
• or Max. iterations
• Convergence Criteria
• xi xi-1 lt ? ? xi
• and f(xi) lt ?
• N-R and Secant Confirmation of Convergent
  Behavior
• x in feasible region
• and f(xi) 0.5 f(xi-1)
• and xi xi-1 0.6 xi-1 xi-2
• otherwise, do Bisection for a while.

68
Three Phase Rootfinding Strategy

• A real rootfinding problem can be viewed
• as having three phases
• 1) Opening moves One needs to find the region
  of the parameter space in which desired root can
  be found.
• Understanding of problem, physical insight, and
  common sense are valuable.
• 2) Middle Game Use robust algorithm to reduce
  initial region of uncertainty.
• 3) End game Generate a highly accurate solution
  in a few iterations.