NUMERICAL DIFFERENTIATION AND INTEGRATION

1
NUMERICAL DIFFERENTIATION AND INTEGRATION

• ENGR 351
• Numerical Methods for Engineers
• Southern Illinois University Carbondale
• College of Engineering
• Dr. L.R. Chevalier
• Dr. B.A. DeVantier

2
Numerical Differentiation and Integration

• Calculus is the mathematics of change.
• Engineers must continuously deal with systems and
  processes that change, making calculus an
  essential tool of our profession.
• At the heart of calculus are the related
  mathematical concepts of differentiation and
  integration.

3
Differentiation

• Dictionary definition of differentiate - to mark
  off by differences, distinguish ..to perceive
  the difference in or between
• Mathematical definition of derivative - rate of
  change of a dependent variable with respect to an
  independent variable

4
f(x)
Dy
Dx
x
5
Integration

• The inverse process of differentiation
• Dictionary definition of integrate - to bring
  together, as parts, into a whole to unite to
  indicate the total amount
• Mathematically, it is the total value or
  summation of f(x)dx over a range of x. In fact
  the integration symbol is actually a stylized
  capital S intended to signify the connection
  between integration and summation.

6
f(x)
x
7
Mathematical Background
8
Mathematical Background
9
Overview

• Newton-Cotes Integration Formulas
• Trapezoidal rule
• Simpsons Rules
• Unequal Segments
• Open Integration
• Integration of Equations
• Romberg Integration
• Gauss Quadrature
• Improper Integrals

10
Overview

• Numerical Differentiation
• High accuracy formulas
• Richardsons extrapolation
• Unequal spaced data
• Uncertain data
• Applied problems

11
Specific Study Objectives

• Understand the derivation of the Newton-Cotes
  formulas
• Recognize that the trapezoidal and Simpsons 1/3
  and 3/8 rules represent the areas of 1st, 2nd,
  and 3rd order polynomials
• Be able to choose the best among these formulas
  for any particular problem

12
Specific Study Objectives

• Recognize the difference between open and closed
  integration formulas
• Understand the theoretical basis of Richardson
  extrapolation and how it is applied in the
  Romberg integration algorithm and for numerical
  differentiation

13
Specific Study Objectives

• Recognize why both Romberg integration and Gauss
  quadrature have utility when integrating
  equations (as opposed to tabular or discrete
  data).
• Understand the application of high-accuracy
  numerical-differentiation.
• Recognize data error on the processes of
  integration and differentiation.

14
Newton-Cotes Integration

• Common numerical integration scheme
• Based on the strategy of replacing a complicated
  function or tabulated data with some
  approximating function that is easy to integrate

15
Newton-Cotes Integration

• Common numerical integration scheme
• Based on the strategy of replacing a complicated
  function or tabulated data with some
  approximating function that is easy to integrate

16
Newton-Cotes Integration

• Common numerical integration scheme
• Based on the strategy of replacing a complicated
  function or tabulated data with some
  approximating function that is easy to integrate

fn(x) is an nth order polynomial
17
The approximation of an integral by the area
under - a first order polynomial - a second
order polynomial
We can also approximated the integral by using a
series of polynomials applied piece wise.
18
An approximation of an integral by the area under
straight line segments.
19
Newton-Cotes Formulas

• Closed form - data is at the beginning and end of
  the limits of integration
• Open form - integration limits extend beyond the
  range of data.

20
Trapezoidal Rule

• First of the Newton-Cotes closed integration
  formulas
• Corresponds to the case where the polynomial is a
  first order

21
A straight line can be represented as
22
Integrate this equation. Results in the
trapezoidal rule.
23
The concept is the same but the trapezoid is on
its side.
base
height
height
height
width
base
24
Error of the Trapezoidal Rule
This indicates that is the function being
integrated is linear, the trapezoidal rule will
be exact. Otherwise, for section with second and
higher order derivatives (that is with
curvature) error can occur. A reasonable
estimate of x is the average value of b and a
25
Multiple Application of the Trapezoidal Rule

• Improve the accuracy by dividing the integration
  interval into a number of smaller segments
• Apply the method to each segment
• Resulting equations are called multiple-applicatio
  n or composite integration formulas

26
Multiple Application of the Trapezoidal Rule
where there are n1 equally spaced base points.
27
We can group terms to express a general form


width
average height
28
The average height represents a weighted
average of the function values Note that the
interior points are given twice the weight of the
two end points
29
EXAMPLE
Evaluate the following integral using the
trapezoidal rule and h 0.1
30
Simpsons 1/3 Rule

• Corresponds to the case where the function is a
  second order polynomial

31
Simpsons 1/3 Rule

• Designate a and b as x0 and x2, and estimate
  f2(x) as a second order Lagrange polynomial

32
Simpsons 1/3 Rule

• After integration and algebraic manipulation, we
  get the following equations

33
Error
Single application of Trapezoidal Rule.
Single application of Simpsons 1/3 Rule
34
Multiple Application of Simpsons 1/3 Rule
35
The odd points represent the middle term for each
application. Hence carry the weight 4 The even
points are common to adjacent applications and
are counted twice.
36
Simpsons 3/8 Rule

• Corresponds to the case where the function is a
  third order polynomial

37
Integration of Unequal Segments

• Experimental and field study data is often
  unevenly spaced
• In previous equations we grouped the term (i.e.
  hi) which represented segment width.

38
Integration of Unequal Segments

• We should also consider alternately using higher
  order equations if we can find data in
  consecutively even segments

trapezoidal rule
1/3 rule
trapezoidal rule
3/8 rule
39
EXAMPLE
Integrate the following using the trapezoidal
rule, Simpsons 1/3 Rule and a multiple
application of the trapezoidal rule with n2.
Compare results with the analytical solution.
40
Simpsons 1/3 Rule f(2) 109.196
41
Multiple Application of the Trapezoidal Rule
We are obviously not doing very well on our
estimates. Lets consider a scheme where we
weight the estimates
....end of example
42
Integration of Equations

• Integration of analytical as opposed to tabular
  functions
• Romberg Integration
• Richardsons Extrapolation
• Romberg Integration Algorithm
• Gauss Quadrature
• Improper Integrals

43
Richardsons Extrapolation

• Use two estimates of an integral to compute a
  third more accurate approximation
• The estimate and error associated with a multiple
  application trapezoidal rule can be represented
  generally as
• I I(h) E(h)
• where I is the exact value of the integral
• I(h) is the approximation from an n-segment
  application
• E(h) is the truncation error
• h is the step size (b-a)/n

44
Make two separate estimates using step sizes of
h1 and h2 . I(h1) E(h1) I(h2) E(h2)
Recall the error of the multiple-application of
the trapezoidal rule
Assume that is constant regardless of the
step size
45
Substitute into previous equation I(h1) E(h1)
I(h2) E(h2)
46
Thus we have developed an estimate of the
truncation error in terms of the integral
estimates and their step sizes. This estimate
can then be substituted into I I(h2)
E(h2) to yield an improved estimate of the
integral
47
What is the equation for the special case where
the interval is halved? i.e. h2 h1 / 2
48
(No Transcript)
49
EXAMPLE
Use Richardsons extrapolation to evaluate
50
ROMBERG INTEGRATION
We can continue to improve the estimate by
successive halving of the step size to yield a
general formula
k 2 j 1
Note the subscripts m and l refer to more and
less accurate estimates
51
Following a similar pattern to Newton divided
differences, Rombergs Table can be produced
Error orders for
j values i 1 i 2
i 3 i 4 j
O(h2) O(h4) O(h6)
O(h8) 1 h I1,1
I1,2 I1,3 I1,4 2
h/2 I2,1 I2,2
I2,3 3 h/4 I3,1
I3,2 4 h/8 I4,1
Trapezoidal Simpsons 1/3 Simpsons 3/8
Booles Rule Rule
Rule Rule
52
Gauss Quadrature
Extend the area under the straight line
f(x)
x
53
Method of Undetermined Coefficients
Recall the trapezoidal rule
Before analyzing this method, answer this
question. What are two functions that should be
evaluated exactly by the trapezoidal rule?
This can also be expressed as
where the cs are constant
54
The two cases that should be evaluated exactly by
the trapezoidal rule 1) y constant
2) a straight line
f(x)
y 1
f(x)
y x
x
-(b-a)/2
(b-a)/2
-(b-a)/2
x
(b-a)/2
55
Thus, the following equalities should hold.
FOR y1 since f(a) f(b) 1
FOR y x since f(a) x -(b-a)/2 and f(b) x
(b-a)/2
56
Evaluating both integrals
For y 1 For y x
Now we have two equations and two unknowns, c0
and c1. Solving simultaneously, we get c0
c1 (b-a)/2 Substitute this back into
57
We get the equivalent of the trapezoidal rule.
DERIVATION OF THE TWO-POINT GAUSS-LEGENDRE
FORMULA
Lets raise the level of sophistication by -
considering two points between -1 and 1 - i.e.
open integration
58
f(x)
-1 x0 x1 1
x
Previously ,we assumed that the equation fit the
integrals of a constant and linear
function. Extend the reasoning by assuming that
it also fits the integral of a parabolic and a
cubic function.
59
f(xi) is either 1, xi, xi2 or xi3
Solve these equations simultaneously
60
This results in the following
The interesting result is that the simple
addition of the function values at
61
However, we have set the limit of integration at
-1 and 1. This was done to simplify the
mathematics. A simple change in variables can be
use to translate other limits. Assume that the
new variable xd is related to the original
variable x in a linear fashion. x a0
a1xd Let the lower limit x a correspond to xd
-1 and the upper limit xb correspond to
xd1 a a0 a1(-1) b a0
a1(1)
62
a a0 a1(-1) b a0 a1(1)
SOLVE THESE EQUATIONS SIMULTANEOUSLY
substitute
63
These equations are substituted for x and dx
respectively. Lets do an example to appreciate
the theory behind this numerical method.
64
EXAMPLE
Estimate the following using two-point Gauss
Legendre
65
Higher-Point Formulas
For two point, we determined that c0 c1
1 For three point c0 0.556 x0-0.775 c1
0.889 x10.0 c2 0.556 x20.775
66
Higher-Point Formulas
Your text goes on to provide additional
weighting factors (cis) and function arguments
(xis) in Table 22.1 p. 623.
67
Improper Integrals
How do we deal with integrals that do not have
finite limits and bounded integrands? Our answer
will focus on improper integrals where one limit
(upper or lower) is infinity.
68
Choose -A so that is is sufficiently large
to approach zero asymptotically. Once chosen,
evaluate the second part using a Newton-Cotes
formula. Evaluate the first part with the
following identity.
69
Numerical Differentiation

• Forward finite divided difference
• Backward finite divided difference
• Center finite divided difference
• All based on the Taylor Series

70
Forward Finite Difference
71
Forward Divided Difference
f(x)
(x i1,y i1)
actual
estimate
(xi, yi)
x
72
Error is proportional to the step size
first forward divided difference
O(h2) error is proportional to the square of the
step size O(h3) error is proportional to the
cube of the step size
73
f(x)
actual
(xi,yi)
estimate
(xi-1,yi-1)
x
74
Backward Difference Approximation of the First
Derivative Expand the Taylor series backwards
The error is still O(h)
75
Centered Difference Approximation of the First
Derivative Subtract backward difference
approximation from forward Taylor series expansion
76
f(x)
actual
(xi1,yi1)
(xi,yi)
estimate
(xi-1,yi-1)
x
77
f(x)
f(x)
forward finite divided difference approx.
true derivative
x
x
f(x)
f(x)
centered finite divided difference approx.
backward finite divided difference approx.
x
x
78
Numerical Differentiation

• Forward finite divided differences Fig. 23.1
• Backward finite divided differences Fig. 23.2
• Centered finite divided differences Fig. 23.3
• First - Fourth derivative

79
Richardson Extrapolation

• Two ways to improve derivative estimates
• decrease step size
• use a higher order formula that employs more
  points
• Third approach, based on Richardson
  extrapolation, uses two derivatives estimates to
  compute a third, more accurate approximation

80
Richardson Extrapolation
For a centered difference approximation
with O(h2) the application of this formula will
yield a new derivative estimate of O(h4)
81
EXAMPLE
Given the following function, use Richardsons
extrapolation to determine the derivative at
0.5. f(x) -0.1x4 - 0.15x3 - 0.5x2 - 0.25x
1.2 Note f(0) 1.2 f(0.25) 1.1035 f(0.75)
0.636 f(1) 0.2
82
Derivatives of Unequally Spaced Data

• Common in data from experiments or field studies
• Fit a second order Lagrange interpolating
  polynomial to each set of three adjacent points,
  since this polynomial does not require that the
  points be equispaced
• Differentiate analytically

83
Derivative and Integral Estimates for Data with
Errors

• In addition to unequal spacing, the other problem
  related to differentiating empirical data is
  measurement error
• Differentiation amplifies error
• Integration tends to be more forgiving
• Primary approach for determining derivatives of
  imprecise data is to use least squares regression
  to fit a smooth, differentiable function to the
  data
• In absence of other information, a lower order
  polynomial regression is a good first choice

84
(No Transcript)