INTRODUCTION TO SCIENTIFIC COMPUTING

1
INTRODUCTION TO SCIENTIFIC COMPUTING
2
SCIENTIFIC COMPUTING

• Design and analysis of algorithms for solving
  mathematical problems in science and engineering
  numerically
• Traditionally called Numerical Analysis or
  Numerical Methods
• Distinguishing features
• Continuous quantities
• Effects of approximations
• Why scientific computing?
• Simulation of physical phenomena
• Virtual prototyping of products

3
Well Posed Problem

• Problem is well posed if the solution
• --Exists
• --Is unique
• --Depends continuously on problem data
• --Solution may still be sensitive to input
  
• data
• --Algorithm should not make sensitivity
  worse

4
General Strategy

• Replace difficult problem by easier one having
• same or closely related solution
• --infinite ? finite
• --differential ? algebraic
• --nonlinear ?linear
• --complicated ? simple
• --Solution obtained may only approximate that
• of original problem

5
Sources of Approximation

• Before computation
• ?modeling
• ?empirical measurements
• ?previous computations
• During computation
• ?truncation or discretization
• ?rounding
• Accuracy of final result reflects all these
• Uncertainty in input may be amplified by problem
• Perturbations during computation may be amplified
  by the algorithm

6
Example

• Computing surface area of Earth using formula
• A 4 p r2
• involves several approximations
• Earth modeled as sphere, idealizing its true
  shape
• Value for radius based on empirical measurement
  and previous computations
• Value for p requires truncating an infinite
  process
• Values for input data and results of arithmetic
  operations rounded in computer

7
Absolute and Relative Error

• Absolute error approx value - true value
• Relative error absolute error/true value
• Equivalently,
• Approx value (true value).(1relative
  error)
• True value is usually unknown, so estimate or
  bound error rather than compute it exactly
• Relative error is often taken relative to
  approximate
• value, rather than (unknown) true value

8
Data Error Computational Error

• Typical problem
• Compute value of function
  ,for given argument
• x true value of input, f(x) desired
  result
• x approximate (inexact) input
• f approximate function computed
• Total error
• f (x )-f (x) (f (x) f (x) ) (f
  (x) - f(x))
• Computational error Propagated data error
• Algorithm has no effect on propagated data error

9
Truncation Error Rounding Error

• Truncation error Deference between the true
  result (for actual input) and result produced by
  a given algorithm using exact arithmetic
• ?Due to approximations such as truncating
  infinite series or terminating iterative
  sequence before convergence
• Rounding error Deference between result
  produced by given algorithm using exact
  arithmetic and result produced by same algorithm
  using limited precision arithmetic
• ?Due to inexact representation of real numbers
  and arithmetic operations upon them
• Computational error is sum of truncation error
  and rounding error, but one of these usually
  dominates

10
Floating Point Number Line
11
Floating point numbers in Matlab

• IEEE Standard for double precision numbers
• Round-off eps 2-52
• Underflow realmin 2-1022
• Overflow realmax (2-eps) 21023

exponent
Sign bit
Fraction of mantissa
0ltflt1
normalized
More info http//www.mathworks.com/company/newsle
tters/news_notes/pdf/Fall96Cleve.pdf
12
Computational Algorithms-1
Assume that a set of instruction are to be given
to a high school student to solve the following
pair of equations for x and y.
Thus if we compute
and replace the
value of by it and compute
and replace the value of by it
Now we can substitute in the first equation
and find
13
Computational Algorithms - 2
This method of solving equations may be expressed
as the following series of instructions to be
given to the high school student

• Read six numbers and assign them respectively to
• Compute .Replace the
  value of by this value.
• Compute .Replace the
  value of by it.
• If , then write No Solution and
  stop. Else continue
• Set
• Set
• Write the values of x and y as answers.
• Stop

14
Computational Algorithms - 3

• If and are respectively
  2, 3, 4, 5, 6 and 7 then the following
  computations would be performed when the
  instructions are executed.
• Instruction 1 a b c d e f
• 2 3 4 5
  6 7
• Instruction 2 q ? 6 (15/2) 1.5
• Instruction 3 r ? 7 (20/2) 3
• Instruction 4 as q ? 0 continue
• Instruction 5 y ( 3/ 1.5 ) 2
• Instruction 6 x (4 6)/2 1
• Instruction 7 x 1, y 2 answers
• Instruction 8 Stop

The values of the variables at the beginning at
the end of the executions are given below
a b c p
q r x
y 2 3 4
5 6 7 --
-- 2 3
4 5 -1.5 -3
-1 2
Observe that when instruction 2 is executed the
previous value of q namely 6 is replaced by a
new value, namely, 1.5 Similarly when
instruction 3 is executed the value of r
namely 7 is replaced by a new value 3 , and so
on..
15
Computational Algorithms - 4

• A digital computer is analogous to the high
  school student in the above example.
• It is necessary to give detailed set of
  instructions to the computer for solving a
  problem.
• Such a set of instructions which leads to a step
  by step procedure for solving a problem is
  called an algorithm.
• It is necessary to express the algorithm in a
  language understood by the computer.
• An algorithm coded in the computer language is
  called a program and the language of coding is
  called a programming language.

16
Computational Algorithms - 5

• A code for the earlier given set of instruction
  would be
• 1 Read A, B, C, P, Q, R.
• 2 Let Q Q (P B/A)
• 3 Let R R (P C/A)
• 4 If Q 0 THEN 9
• 5 LET Y R / Q
• 6 LET X (C B Y)/a
• 7 PRINT X, Y, ANSWERS
• 8 STOP
• 9 PRINT NO SOULUTION
• 10 STOP
• 11 DATA 2, 3, 4, 5, 6,7
• 12 END

THIS PROGRAM IS WRITTEN IN BASIC
17
Some examples of algorithms

• A Fibonacci sequence of integers is given below
• 0, 1, 1, 2, 3, 5, 8, 13, 21,
  34, 55
• Observe that a term in the sequence is
  obtained as a sum of the immediately preceding
  two terms. Thus we have the recurrence relation.
• 1. Previous term 0
• 2. Current term 1
• 3. New term current term previous term
• 4. Replace previous term by current term
• 5. Replace current term by new term.
• Now by repeating steps 3, 4 and
  5 we generate successive terms in the Fibonacci
  sequence. This is illustrated in this table.

Previous Term Current
Term New
Term 0
1
1 1
1
2 1
2

3 2
3
5 3
5
8
5 8

13 8









Initial values
18
GENERATION OF THE FIBONNACI SEQUENCE a pseudo
code

• previous term ? 0
• current term ? 1
• new term ? current term previous term
• write previous term, current term, new term
• While new term 100 do
• begin
• previous term ?
  current term
• current term ? new
  term
• new term ? current
  term previous term
• write new term
• end
• stop

The underlined terms are not MATLAB commands
19
Solution of simultaneous equations a pseudo
code
Read a, b, c, p, q, r q ? q (pb/a) r ? r
(pc/a) if q 0 then write No
Solution else begin
y ? r /q x ? (c
by)/a write x, y
end stop
We will be using these pseudo codes a lot
20
Program Flow

• Straight Line Code

One line of code after another just in sequence.
Also called sequential code.
21
Program Flow cont.

• Conditional Code

Based on a test, perform one alternative set of
code and not another
YES
NO
test
22
Program Flow cont.

• Iterative code

Execute the same block of code again and again
repeat
23
Iterative Methods

• In trial and error method of solution a sequence
  of steps is repeated over and over again (that is
  why it is disliked by the human mind). However
  such a procedure is concisely expressible as a
  computational algorithm. It is thus the natural
  method to use with Computers.
• It is possible to formulate, using trial and
  error, algorithms which tackle a class of similar
  problems. For instance a general computational
  algorithm to solve polynomial equations of order
  n ( where n is any integer) may be written
  in a general form.
• Rounding errors are negligible in trial and error
  procedures compared to procedures based on closed
  form solutions. The reason for this will become
  clear as we proceed with iterative methods.

• Given a quadratic equation
• The closed form solution is given by
• Similar formulae are available for cubic and
  quadratic polynomial equations but we rarely
  remember them.
• For higher order polynomial equations and
  non-polynomial equations it is difficult and in
  many cases impossible, to get closed form
  solutions.
• Besides this, when numbers are substituted
  in (available) closed form solutions, rounding
  errors often reduce their accuracy.
• Another general approach to solving equations is
  by trial and error.
• It is, however disliked by a human problem solver
  (human mind). On the other hand,
• for a Computer, it is the most natural technique
  to use due to the following reasons.

24
Beginning an iterative method

• We are ready to discuss a number of trial and
  error or iterative methods of finding solutions
  of the equation f (x) 0. These solutions are
  also known as the roots of the equation f (x)
  0
• There are five basic questions one has to ask
  which are relevant to all iterative methods.
  These are
• Where does one start the iterative cycle? In
  other words, how does one choose the initial
  guess value of the root.
• How does one proceed from the initial guess to
  the subsequent approximations?
• How fast do these successive approximations
  converge to the root?
• How much computational effort is required in each
  iteration?
• When does one stop the iteration cycle?

25
Finding Roots ?Basic Ideas

• Problem-dependent decisions
• Approximation Since we cannot have
  exactness, we specify our tolerance to error
• Convergence We also specify how long we are
  willing to wait for a solution. Caution We might
  imagine a reasonable procedure for finding
  solutions, but can we guarantee it terminates?
• Method We choose a method easy to implement
  and yet powerful enough and general
• Put a human in the loop Since no general
  procedure can find roots of complex equations, we
  let a human specify a neighborhood of a solution

26
Finding Roots ?Basic Ideas-cont.

• The Practical Approach hand calculations
• Choose method and initial guess wisely.
• Good idea to start with a crude graph. If you
  are looking for a single root you only need one
  positive and one negative value
• If even a crude graph is difficult, generate a
  table of values and plot graph from that.

27
Practical Approach - example

• Example
• e-x sin(?x/2)
• Solve for x.
• Graph functions - the crossover points are
  solutions
• This is equivalent to finding the roots of the
  difference function
• f (x) e-x - sin(?x/2)

28
Graph Functions
29
Solution

• One crossover point at about 0.5, another at
  about 2.0 (there are very many of them )
• Compute values of both functions at 0.5
• Decrement/increment slightly watch the sign of
  the difference-function!
• If there is an improvement continue, until you
  get closer.
• Stop when you are close enough

30
Tabulating the function
31
Bisection Method
Do we have the 30 students for todays lab? Was
there any trouble in downloading and opening the
lectures?

• The Bisection Method slightly modifies this
  educated guess approach of hand calculation
  method.
• We will discuss this method and many more such
  methods in the next lecture.
• We will learn to write our codes in MATLAB