Data Structures

1
Data Structures

• 1. Basic Concepts
• Chih-Hung Wang
• Fall 2008
• Textbook Ellis Horowitz, Sartaj Sahni and Dinesh
  P. Mehta. Fundamentals of Data Structures in C
  (Second Edition). Silicon Press, 2007.

2
Algorithm Specification

• An algorithm is a finite set of instructions
  that, if followed, accomplishes a particular
  task. All algorithms must satisfy
• Input. Zero or more quantities are externally
  supplied.
• Output. At least one quantity is produced.
• Definiteness. Each instruction must be clear and
  unambiguous.
• Finiteness. An algorithm terminates in a finite
  number of steps.
• Effectiveness. Every instruction must be basic
  enough to be carried out.

3
Selection Sort (1)

• From those integers that are currently unsorted,
  find the smallest and place it next in the sorted
  list.
• Algorithm

for ( int i 0 i lt n i) examine ai to
an-1 and suppose the smallest integer is at
aj interchange ai and aj
4
Selection Sort (2)
void SelectionSort ( int a, const int n)
//Sort the n integers a0 to an-1 into
nondecreasing order. for (int i 0 i lt n i
) int j i //find smallest integer in
ai to an-1 for (int k i 1 k lt n
k) if (ak lt aj) j k swap(ai,
aj)

• 5 1 8 3
• 5 7 8 3
• 1 3 7 8 5
• 1 3 5 8 7
• 1 3 5 7 8

5
Binary Search (1)

• Assume that we have n1 distinct integers that
  are already sorted and stored in the array a0,
  a1, , an-1.
• Our task
• determine if the integer x is present and If so
  to return j such that x aj otherwise return
  -1.

6
Binary Search (2)

• Let left and right denote the left and right ends
  of the list to be searched. Initially, left0,
  rightn-1.
• middle (leftright) / 2.
• One of three results in comparing amiddle with
  x
• x lt amiddle x (if present) must be in the
  positions between 0 and middle -1. Set right to
  middle -1.
• x amiddle return middle.
• x gt amiddle x (if present) must be in the
  positions between middle 1 and n-1. Set left to
  middle 1.
• x cannot be found if left gt right

7
Binary Search (3)
int BinarySearch (int a, const int x, const int
n) // Search the sorted array a0, , an-1
for x Initialize left and right while (there
are more elements) Let middle be the middle
element if (x lt amiddle) set right to be
middle-1 else if (x gt amiddle) set left to
be middle1 else return middle Not
found
8
Binary Search (4)

• C function for binary search

int BinarySearch (int a, const int x, const int
n) // Search the sorted array a0, , an-1
for x int left 0, right n-1 while (left
lt right) // there are more elements int
middle (left right)/2 if (x lt amiddle)
rightmiddle-1 else if (x gt amiddle) left
middle1 else return middle // end of
while return -1 // not found
9
Recursive Algorithms

• Recursive implementation of binary search

int BinarySearch(int a, const int x, const int
left, const int right) // search the sorted
array aleft, , aright for x if (left lt
right) int middle (left right)/2 if (x
lt amiddle) return BinarySearch(a, x, left,
middle-1) else if (x gt amiddle) return
BinarySearch(a, x, middle1, right) else
return middle // end of if return -1
//not found
10
Performance Analysis and Measurement

• Space complexity
• The amount of memory needs to run to completion.
• Time complexity
• The amount of computer time needs to run to
  completion.
• Priori estimates and posteriori testing

11
Time Complexity

• The time, T(P), taken by a program P is the sum
  of the compile time and the run time.
• The run time is denoted by tp(n), where n is the
  instance characteristic.
• Statement types
• Comments (0)
• Declarative statements (0)
• Expressions and assignment statements (1)
• Iteration statements (sum of the step counts)
• Switch statements
• If-else statement
• Function invocation
• Memory management statements (1)
• Function statements
• Jump statements (1)
• Asymptotic complexity
• Complexity in terms of O, O, and T.

12
Ex. Program 1.19
float Sum (float a, const int n) float s
0 count // count is global for (int
i 0 i ltn i) count // for
for s ai count // for
assignment count // for last time of
for count // for return return s
2n3 steps
13
Example Recursive Ver. Of P1.18
float Rsum (float a , const int n) count
// for if consitional if (n lt 0) count //
for return return 0 else count // for
return return (Rsum (a, n-1) a n - 1)
14
Time Complexity
15
Example 1.14Fibonnaci numbers (1)

• 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55,
• Total steps 4n1

16
Example 1.14Fibonnaci numbers (2)
void Fibonacci (int n) // Compute the Fibonacci
number Fn if (n lt 1) cout ltlt n ltlt endl //
F0 0 ? F1 1 else // compute Fn int
fn int fnm2 0 int fnm1 1 for (int i
2 i lt n i) fn fnm1 fnm2
fnm2 fnm1 fnm1 fn // end of
for cout ltlt fn ltlt endl // end of
else
17
Asymptotic Notation

• 3n2 and 100n10
• As n is very large?
• 100n10 and 3n22
• As n is sufficiently large, 3n22 is larger (or
  slower) than 100n10.

18
Definition Big oh

• Big oh f(n) O(g(n)) iff there exist
  positive constants c and n0 such that f(n)?cg(n)
  for all n where n?n0.
• Example 1.15
• 3n2 O(n)

19
Theorem 1.2
c
n0
20
Definition Omega

• Omega f(n) O(g(n)) iff there exist positive
  constants c and n0 such that f(n)?cg(n) for all n
  where n? n0.

21
Theorem 1.3
22
Definition Theta

• Theta f(n) T(g(n)) iff there exist positive
  constants c1, c2 and n0 such that c1g(n)?
  f(n)?c2g(n) for all n where n? n0.

23
Theorem 1.4
24
Asymptotic notation (again)

• ? notation

g(n) is an asymptotically tight bound for
f(n) The definition of ?(g(n)) requires that
every member f(n)? ?(g(n)) be asymptotically
nonnegative.
25
? Notation

• Example

Dividing by n2

• For any value of n?1 by choosing c2 ?1/2.
• For any value of n?7 by choosing c1 ?1/14.
• c1 ?1/14, c2 ?1/2, n07
• Why? 6n3? ?(n2)

26
Why 6n3 ??(n2)

• Find c2 and n0 such that
• 6n3 c2n2 for all nn0
• But then n c2/6
• Cannot possibly hold for arbitrarily large n,
  since c2 is constant.

27
? Notation
General polynomial
Note constant function as ?(n0) or ?(1)
28
O Notation

• Asymptotic upper bound

implies
Since ? notation is a strong notation than
O-notation
Example 2n2 O(n3), with c 1 and n0 2.
29
Example of O function

• Example of O(n2)
• n2
• n2 n
• n2 1000n
• 1000n2 1000n
• Also,
• n
• n/1000
• n1.99999
• n2/ lg lg lg n

30
? Notation

• Asymptotic lower bound
• Example with c 1 and n0
  16.

31
Example of ? Function

• Example of ?(n2)
• n2
• n2 n
• n2 -n
• 1000n2 1000n
• 1000n2 - 1000n
• Also,
• n3
• n2.00001
• n2 lg lg lg n
• 22n

32
Theorem

• Theorem 3.1
• For any two function f(n) and g(n), we have f(n)
  ?(g(n)) if and only if f(n)O(g(n)) and f(n)
  ?(g(n)).

33
Exponential

• Useful Formula
• a-1 1/a
• (am)n amn
• aman amn

gt
34
Logarithms

• Notations

when xlt1
when xgt-1
35
Comparison Example

• Rank the following functions by order of growth
• 4 lg n n2 (3)
• n3 (2)
• lg2n (4)
• (lgn)lgn nlglgn (1)

36
Asymptotically Comparable

• Not all functions are asymptotically comparable.
• Example
• n1sin n and n cannot be compared using
  asymptotic notation, since 1 sin n oscillates
  between 0 and 2.

37
Asymptotic Complexity of Sum (1.17)
38
Asymptotic Complexity of Rsum (1.18)
2n2 ?(n)
39
Binary Search

• The overall worst-case complexity of binary
  search is .

n/16
n/4
n/8
n/2
40
Magic Square (1)

• Simple rule
• Start with 1 in the middle of the top row then
  go up and left, assigning numbers in increasing
  order to empty squares if you fall off the
  square imagine the same square as tiling the
  plane and continue if a square is occupied, move
  down instead and continue.

The computing time O(n2)
41
Magic Square (2)
void Magic (const int n) //Create a magic square
of size n, n is odd const int MaxSize 51
// maximum square size int square
MaxSizeMaxSize, k, l // check
correctness of n if ((n gt MaxSize) (n lt
1)) throw "Error!..n out of range "
else if (!(n2)) throw "Error!..n is even \n"
// n is odd, Coxeters rule can be used
for (int i 0 i lt n i) // initialize square
to 0 fill(squarei, squarei n, 0)
// STL algorithm square0(n-1)/2 1 //
middle of first row // i and j are current
position int key 2 i 0 int j
(n-1)/2
42
Magic Square (3)
while (key lt nn) // move up and left
if (i-1 lt 0) k n-1 else k i-1 if
(j-1 lt 0) l n-1 else l j-1 if
(squarekl) i (i1)n // square occupied,
move down else // squarekl is
unoccupied i k j l
squareij key key // end
of while // output the magic square cout
ltlt "magic square of size " ltlt n ltlt endl for
( i 0 i lt n i) for ( j 0 j lt
n j) copy(squarei, squarei n,
ostream_iteratorltintgt(cout, " ")) cout
ltlt endl
43
Practical Complexities

• Function values

44
Plot of Function Values
45
Growth Rate
46
Execution Time
Times on a 1-billion-steps-per-second computer