Round-off Errors

1

• Round-off Errors

2
Key Concepts

• Round-off / Chopping Errors
• Recognize how floating point arithmetic
  operations can introduce and amplify round-off
  errors
• What can be done to reduce the effect of
  round-off errors

3
There are discrete points on the number lines
that can be represented by our computer. How
about the space between ?
4
Implication of FP representations

• Only limited range of quantities may be
  represented.
• Overflow and underflow
• Only a finite number of quantities within the
  range may be represented.
• round-off errors or chopping errors

5
Round-off / Chopping Errors(Error Bounds
Analysis)

• Let
• z be a real number we want to represent in a
  computer, and
• fl(z) be the representation of z in the
  computer.
• What is the largest possible value of
  ?
• i.e., in the worst case, how much data are we
  losing due to round-off or chopping errors?

6
Chopping Errors (Error Bounds Analysis)
Suppose the mantissa can only support n digits.
Thus the absolute and relative chopping errors are
Suppose ß 10 (base 10), what are the values of
ai such that the errors are the largest?
7
Chopping Errors (Error Bounds Analysis)
8
Round-off Errors (Error Bounds Analysis)
Round down
Round up
fl(z) is the rounded value of z
9
Round-off Errors (Error Bounds Analysis)
Absolute error of fl(z)
When rounding down
Similarly, when rounding up
i.e., when
10
Round-off Errors (Error Bounds Analysis)
Relative error of fl(z)
11
Summary of Error Bounds Analysis
Chopping Errors Round-off errors
Absolute
Relative
ß base n of significant digits or of digits
in the mantissa
Regardless of chopping or round-off is used to
round the numbers, the absolute errors may
increase as the numbers grow in magnitude but the
relative errors are bounded by the same magnitude.
12
Machine Epsilon
Relative chopping error
Relative round-off error
eps is known as the machine epsilon the
smallest number such that 1 eps gt 1
epsilon 1 while (1 epsilon gt 1) epsilon
epsilon / 2 epsilon epsilon 2
Algorithm to compute machine epsilon
13
Propagation of Errors

• Each number or value of a variable is represented
  with error
• These errors (Ex and Ey) are carried over to the
  result of every arithmetic operation (, -, x, ?)
• How much error is propagated to the result of
  each arithmetic operation?

14
Example 1
Assume 4 decimal mantissa with rounding are used
(Final value after round-off)
How many types of errors and how much errors are
introduced to the final value?
15
Example 1
Propagated Error (xT yT) - (xA yA) ExEy
Propagated Error -0.4000x10-1 0.3000x10-3
16
Example 1
Rounding Error
17
Example 1
Finally, the total error is
The total error is the sum of the propagated
error and rounding error
18
Propagation of Errors (In General)

• Let ? be the operation between xT and yT
• ? can be any of , -, x, ?
• Let ? be the corresponding operation carried out
  by the computer
• Note xA ? yA ? xA ? yA

19
Propagation of Errors (In General)

• Error between the true result and the computed
  result is
• (xT ? yT ) (xA ? yA)
• (xT ? yT xA ? yA) (xA ? yA xA ? yA)

Errors in x and y propagated by the operation
Rounding error of the result
xA ? yA xA ? yA fl(xA ? yA) xA ? yA x
eps
20
Analysis of Propagated ErrorsAddition and
Subtraction

• Addition

Subtraction
21
Propagated Errors Multiplication
Very small and can be neglected
22
Propagated Errors Division
if ey is small and negligible
23
Example 2Effects of rounding errors in
arithmetic manipulations

• Assuming 4-digit decimal mantissa
• Round-off in simple multiplication or division

Results by the computer
24
Danger of adding/subtracting a small number
to/from a large number
Possible workarounds 1) Sort the numbers by
magnitude (if they have the same signs) and add
the numbers in increasing order 2) Reformulate
the formula algebraically
25
Associativity not necessarily hold for floating
point addition (or multiplication)
The two answers are NOT the same!
Note In this example, if we simply sort the
numbers by magnitude and add the number in
increasing order, we actually get worse answer!
Better approach is analyze the problem
algebraically.
26
Subtraction of two close numbers
The result will be normalized into 0.9550 x
101 However, note that the zero added to the end
of the mantissa is not significant. Note 0.9550
x 101 implies the error is about 0.00005 x 101
but the actual error could be as big as 0.00005
x 102
27
Subtractive Cancellation Subtraction of two
very close numbers
The error bound is just as large as the
estimation of the result! Subtraction of nearly
equal numbers are major cause of errors! Avoid
subtractive cancellation whenever possible.
28
Avoiding Subtractive Cancellations
Example 1 When x is large, compute
Is there a way to reduce the errors assuming that
we are using the same number of bits to represent
numbers?
Answer One possible solution is via
rationalization
29
Subtraction of nearly equal numbers
Example 2 Compute the roots of ax2 bx c 0
using
Solve x2 26x 1 0
30
Example 2 (continue)
Assume 5 decimal mantissa,
implies that one solution is more accurate than
the other one.
31
Example 2 (continue)
Alternatively, a better solution is
with
i.e., instead of computing
we use
as the solution for the second root
32

• Note This formula does NOT give more accurate
  result in ALL cases.
• We have to be careful when writing numerical
  programs.
• A prior estimation of the answer, and the
  corresponding error, is needed first. If the
  error is large, we have to use alternative
  methods to compute the solution.

33
Assignment 1 (Problem 1)

• Assume 3 decimal mantissa with rounding
• Evaluate f(1000) directly.
• Evaluate f(1000) as accurate as possible using an
  alternative approach.
• Find the relative error of f(1000) in part (a)
  and (b).

34
Propagation of Errors in a Series

• Let the series be

Is there any difference between adding (((x1
x2) x3) x4) xm and (((xm xm-1) xm-2)
xm-3) x1
35
Example
for (i 0 i lt 100000 i) sumx sumx
x sumy sumy y sumz sumz z
printf("sumx f\n", sumx)
printf("sumy f\n", sumy) printf("sumz
f\n", sumz) return 0
include ltstdio.hgt int main() float sumx, x
float sumy, y double sumz, z int i sumx
0.0 sumy 0.0 sumz 0.0 x 1.0 y
0.00001 z 0.00001
Output sumx 100000.000000 sumy 1.000990 sumz
0.99999999999808375506
36
Exercise

• Discuss to what extent
• (a b)c ac bc
• is violated in machine arithmetic.

37
Example Evaluate ex as
include ltstdio.hgt include ltmath.hgt int main()
float x 10, sum 1, term 1, temp 0
int i 0 while (temp ! sum) i
term term x / i temp sum sum
sum term printf("2d -12f -14f\n", i,
term, sum) printf("exact value f\n",
exp((double)x)) return 0
38
Output (when x 10)
17 281.145752 21711.982422 18 156.192078
21868.173828 19 82.206360 21950.380859 20
41.103180 21991.484375 21 19.572943
22011.056641 22 8.896792 22019.953125 23
3.868171 22023.822266 24 1.611738
22025.433594 25 0.644695 22026.078125 26
0.247960 22026.326172 27 0.091837
22026.417969 28 0.032799 22026.451172 29
0.011310 22026.462891 30 0.003770
22026.466797 31 0.001216 22026.468750 32
0.000380 22026.468750 exact value
22026.465795

• term

sum
1 10.000000 11.000000 2 50.000000
61.000000 3 166.666672 227.666672 4
416.666687 644.333374 5 833.333374
1477.666748 6 1388.889038 2866.555664 7
1984.127197 4850.682617 8 2480.158936
7330.841797 9 2755.732178 10086.574219 10
2755.732178 12842.306641 11 2505.211182
15347.517578 12 2087.676025 17435.193359 13
1605.904541 19041.097656 14 1147.074585
20188.171875 15 764.716431 20952.888672 16
477.947754 21430.835938
39
Example Evaluate ex as
include ltstdio.hgt include ltmath.hgt int main()
float x 10, sum 1, term 1, temp 0
int i 0 while (temp ! sum) i
term term x / i temp sum sum
sum term printf("2d -12f -14f\n", i,
term, sum) printf("exact value f\n",
exp((double)x)) return 0

• Arithmetic operations that introduce errors

40
Output (when x -10)
29 -0.011310 -0.002908 30 0.003770
0.000862 31 -0.001216 -0.000354 32
0.000380 0.000026 33 -0.000115
-0.000089 34 0.000034 -0.000055 35
-0.000010 -0.000065 36 0.000003
-0.000062 37 -0.000001 -0.000063 38
0.000000 -0.000063 39 -0.000000
-0.000063 40 0.000000 -0.000063 41
-0.000000 -0.000063 42 0.000000
-0.000063 43 -0.000000 -0.000063 44
0.000000 -0.000063 45 -0.000000
-0.000063 46 0.000000 -0.000063
exact value 0.000045

• term

sum
1 -10.000000 -9.000000 2 50.000000
41.000000 3 -166.666672 -125.666672 4
416.666687 291.000000 5 -833.333374
-542.333374 6 1388.889038 846.555664 7
-1984.127197 -1137.571533 8 2480.158936
1342.587402 9 -2755.732178 -1413.144775 10
2755.732178 1342.587402 11 -2505.211182
-1162.623779 12 2087.676025 925.052246 13
-1605.904541 -680.852295
Not just incorrect answer! We get negative value!
41
Errors vs. Number of Arithmetic Operations

• Assume 3-digit mantissa with rounding
• (a) Evaluate y x3 3x2 4x 0.21 for x
  2.73
• (b) Evaluate y (x 3)x 4 x 0.21 for x
  2.73
• Compare and discuss the errors obtained in part
  (a) and (b).

42
Summary

• Round-off/chopping errors
• Analysis
• Propagation of errors in arithmetic operations
• Analysis and Calculation
• How to minimize propagation of errors
• Avoid adding huge number to small number
• Avoid subtracting numbers that are close
• Minimize the number of arithmetic operations
  involved