CIS 541

1
CIS 541 Numerical Methods

• Mathematical Preliminaries

2
Derivatives

• Recall the limit definition of the first
  derivative.

3
Partial Derivatives

• Same as derivatives, keep each other dimension
  constant.

4
Tangents and Gradients

• Recall that the slope of a curve (defined as a 1D
  function) at any point x, is the first derivative
  of the function.
• That is, the linear approximation to the curve in
  the neighborhood of t is l(x) b f(t)x

y
x
5
Tangents and Gradients

• Since we also want this linear approximation to
  intersect the curve at the point t.
• l(t) f(t) b f(t)t
• Or, b f(t) - f(t)t
• We say that the line l(x) interpolates the curve
  f(x) at the point t.

6
Functions as curves

• We can think of the curve shown in the previous
  slide as the set of all points (x,f(x)).
• Then, the tangent vector at any point along the
  curve is

7
Side note on Curves

• There are other ways to represent curves, rather
  than explicitly.
• Functions are a subset of curves (x,y(x)).
• Parametric equations represent the curve by the
  distance walked along the curve (x(t),y(t)).
• Circle (cos?, sin?)
• Implicit representations define a contour or
  level-set of the function f(x,y) c.

8
Tangent Planes and Gradients

• In higher-dimensions, we have the same thing
• A surface is a 2D function in 3D
• Surface (x, y, f(x,y) )
• A volume or hyper-surface is a 3D function in 4D
• Volume (x, y, z, f(x,y,z) )

9
Tangent Planes and Gradients

• The linear approximation to the
  higher-dimensional function at a point (s,t), has
  the form axbyczd0, or z(x,y)
• What is this plane?

10
Construction of Tangent Planes
Images courtesy of TJ Murphy http//www.math.ou.e
du/tjmurphy/Teaching/2443/TangentPlane/TangentPla
ne.html
11
Construction of Tangent Planes
12
Construction of Tangent Planes
13
Tangent Planes and Gradients

• The formula for the plane is rather simple
• z(s,t) f(s,t) - interpolates
• z(sdx,t) f(s,t) fx(s,t)dx b adx
• Linear in dx
• Of course, the plane does not stay close to the
  surface as you move away from the point (s,t).

14
Tangent Planes and Gradients

• The normal to the plane is thus
• The 2D vectoris called the gradient of the
  function.
• It represents the direction of maximal change.

15
Gradients

• The gradient thus indicates the direction to walk
  to get down the hill the fastest.
• Also used in graphics to determine illumination.

16
Review of Functions

• Extrema of a function occur where f(x)0.
• The second derivative determines whether the
  point is a minimum or maximum.
• The second derivative also gives us an indication
  of the curvature of the curve. That is, how fast
  it is oscillating or turning.

17
The Class of Polynomials

• Specific functions of the form

18
The Class of Polynomials

• For many polynomials, the latter coefficients are
  zero. For example
• p(x) 3x25x3

19
Taylors Series

• For a function, f(x), about a point c.
• I.E. A polynomial

20
Taylors Theorem

• Taylors Theorem allows us to truncate this
  infinite series

21
Taylors Theorem

• Some things to note
• (x-c)(n1) quickly approaches zero if x-cltlt1
• (x-c)(n1) increases quickly if x-cgtgt1
• Higher-order derivatives may get smaller (for
  smooth functions).

22
Higher Derivatives

• What is the 100th derivative of sin(x)?
• What is the 100th derivative of sin(3x)?
• Compare 3100 to 100!
• What is the 100th derivative of sin(1000x)?

23
Taylors Theorem

• Hence, for points near c we can just drop the
  error term and we have a good polynomial
  approximation to the function (again, for points
  near c).
• Consider the case where (x-c)0.5
• For n4, this leads to an error term around
  2.610-4 f?????(?)
• Do this for other values of n.
• Do this for the case (x-c) 0.1

24
Some Common Derivatives
25
Some Resulting Series

• About c0

26
Some Resulting Series

• About c0

27
Books Introduction Example

• Eight terms for first series not even yielding a
  single significant digit.
• Only four for second serieswith
  foursignificantdigits.

28
Mean-Value Theorem

• Special case of Taylors Theorem, where n0, xb.
• Assumes f(x) is continuous and its first
  derivative exists everywhere within (a,b).

29
Mean-Value Theorem

• So what!?!?! What does this mean?
• Function can not jump away from current value
  faster than the derivative will allow.

f(x)
a
secant
b
?
30
Rolles Theorem

• If a and b are roots (f(a)f(b)0) of a
  continuous function f(x), which is not everywhere
  equal to zero, then f(t)0 for some point t in
  (a,b).
• I.e., What goes up, must come down.

f(x)
f(t)0
a
t
b
31
Caveat

• For Taylors Series and Taylors Theorem to hold,
  the function and its derivatives must exist
  within the range you are trying to use it.
• That is, the function does not go to infinity, or
  have a discontinuity (implies f(x) does not
  exist),

32
Implementing a Fast sin()

• const int Max_Iters 100,000,000
• float x -0.1
• float delta 0.2 / Max_Iters
• float Reimann_sum 0.0
• for (int i0 iltMax_Iters i)
• Reimann_Sum sinf(x)
• xdelta
• Printf(Integral of sin(x) from 0.1-gt0.1 equals
  f\n, Reimann_Sumdelta )

33
Implementing a Fast sin()

• const int Max_Iters 100,000,000
• float x -0.1
• float delta 0.2 / Max_Iters
• float Reimann_sum 0.0
• for (int i0 iltMax_Iters i)
• Reimann_Sum my_sin(x) //my own sine func
• xdelta
• Printf(Integral of sin(x) from 0.1-gt0.1 equals
  f\n, Reimann_Sumdelta )

34
Version 1.0

• my_sin( const float x )
• float x2 xx
• float x3 xx2
• return (x x3/6.0 x2x3/120.0 )

35
Version 2.0 Horners Rule

• Static const float fac3inv 1.0 / 6.0f
• Static const float fac5inv 1.0 / 120.0f
• my_sin( const float x )
• float x2 xx
• return x(1.0 x2(fac3inv - x2fac5inv))

36
Version 3.0 Inline code

• const int Max_Iters 100,000,000
• float x -0.1
• float delta 0.2 / Max_Iters
• float Reimann_sum 0.0
• for (i0 iltMax_Iters i)
• x2 xx
• Reimann_Sum x(1.0x2(fac3inv-x2fac5inv)
• xdelta
• Printf(Integral of sin(x) from 0.1-gt0.1 equals
  f\n, Reimann_Sumdelta )

37
Timings

• Pentium III, 600MHz machine

Time in seconds Result Max(sin(x)-my_sin(x) )
Using sinf 27 -0.0041943
Version 1.0 20 -0.0041943 1.9376510-11
Version 2.0 13 -0.0041943 8.049510-12
Version 3.0 2 -0.0041943 8.049510-12
38
Observations

• Is the result correct?
• Why did we gain some accuracy with version 2.0?
• Is (0.1,0.1) a fair range to consider?
• Is the original sinf() function optimized?
• How did we achieve our speed-ups?
• We will re-examine this after Lab1.

Ask these question for Lab1 !!!
39
Homework

• Read Chapters 1 and 2 for next class.
• Start working on Lab 1 and Homework 1.