Data Structures with C Using STL

1
Data Structures with CUsing STL

• Chapter Three
• Introduction to
• Algorithms

2
Analysis of Algorithms

• Programs judged on correctness, ease of use and
  efficiency.
• Efficiency influenced by computer system, amount
  of available memory, and complexity of
  algorithms.

3
Performance criteria

• System efficiency - run different algorithms to
  perform the same task on the same machine and
  compare execution times.
• Space efficiency - measuring the relative amount
  of internal memory used by different algorithms
  to perform the same task.
• Computational efficiency - measures the
  computational complexity of an algorithm relative
  to the number of data elements processed.

4
Big O Notation

• Attempts to measure the computational complexity
  of an algorithm by the key operations performed.
• Computational complexity varies based on
• collection type
• amount of data
• initial order of data
• best case, worse case, average case

5
Formal definition

• Function f(n) is O(g(n)) if there is a constant K
  and a count n0 such that f(n)ltKg(n) for n ? n0.

Kg(n)
f(n)
6
Big O notation

• Traditionally Big O value selected from a small
  set of polynomial, logarithmic, and exponential
  values.
• O(c)
• O(log2N)
• O(n)
• O(Nlog2N)
• O(n2)
• O(n3)
• O(2n)

7
Selecting a Big O

• Use only the dominant term.
• If an algorithm performs comparison for every
  element of the collection, it is probably O(N)
• If it performs N(N-1)/2 or (N2 - N)/ 2
  comparisons
• disregard the constant 1/2.
• N2 is dominant over N, so O(N2) is the complexity

8
Basic Premise of Selection Sort

• Find smallest value, place it in the first
  position.
• Find next smallest value, place it in the second
  position.
• Continue doing this until you have placed the
  second to the largest value in the second to the
  last position.

9
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

75
28
52
16
43
31
0
0
1
10
An Example of Selection Sort

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

75
28
52
16
43
31
0 1
0
1
11
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

75
28
52
16
43
31
3
0
3
12
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

75
28
52
16
43
31
3
0
6
13
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
52
75
43
31
1
1
2
14
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
52
75
43
31
1
1
6
15
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
52
75
43
31
2
2
3
16
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
52
75
43
31
4
2
4
17
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
52
75
43
31
5
2
6
18
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
31
75
43
52
3 4
3
4
19
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
31
75
43
52
4
3
6
20
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
31
43
75
52
4 5
4
6
21
An Example of Selection Sort

• // For every spot in list except last
• for (I0 I lt N-1 I)
• // Find current smallest value
• Small I
• for (JI1 Jlt N J)
• if (ItemJ lt ItemSmall)
• Small J
• // Place it in its correct spot
• Swap (ItemI,ItemSmall)

• Item0
• Item1
• Item2
• Item3
• Item4
• Item5
• Small
• I
• J

16
28
31
43
52
75
5
4
6
22
Analysis of Selection Sort

• Locate critical region
• Inside of nested loop (i.e. comparisons)
• Determine how many times critical region is
  executed.
• N-1 N-2 1
• N(N-1)/2 or (N2 - N)/2
• O(N2)

23
Searching an Unordered List

• Sequential or Linear Search
• Basic Premise
• Look at each element until either the desired
  element is found or the end of the list is
  detected.

24
An Example

• Search For 16

Not Found
43
28
52
16
75
31
25
An Example

• Search For 16

43
Not Found
28
52
16
75
31
26
An Example

• Search For 16

43
28
52
Not Found
16
75
31
27
An Example

• Search For 16

43
28
52
16
Found
75
31
28
Linear Search Of Unordered List

• int LinearSearch (const int Arr, int N, int
  Key)
• int I, Pos
• bool Done
• Pos -1
• Done false
• for (I0 Ilt N (! Done) I)
• if (ArrI Key)
• Pos I
• Done true
• return (Pos)

29
Analysis of Linear Search

• If there are N numbers in the list, the algorithm
  looks at all N numbers to determine that it is
  not there.
• Therefore it is O(N).

30
Searching Ordered Lists

• Modified Linear Search - Basic Premise
• Similar to Linear Search except stop as soon as
  you pass where the element should occur.
• Binary Search - Basic Premise
• Look at middle element determine if that is the
  desired element or which half of the list is
  where the desired element would be located.
  Continue this process with the part of the list
  where the desired element would be if it is in
  the list. When there are no elements in the list
  or the element is found stop search.

31
Example of Modified Linear Search

• Search For 25

Not Found
16
28
31
43
52
75
32
Example of Modified Linear Search

• Search For 25

16
Not Found, and Too Far
28
31
43
52
75
33
Linear Search Of Ordered List
int LinearSearch2(const int Arr, int N, int
Key) int I, Pos bool Done
// Initialization Pos -1 Done
false for (I0 (IltN !Done) I)
if (Arr I Key) // if found
Pos I Done true
else if (ArrI gt Key) //
else if too far Done true
else I //
else look at next value return
(Pos)
34
Analysis of Modified Linear Search

• To determine if an element is in the list the
  algorithm looks at N/2 element on average.
• To determine if an element is not in the list the
  algorithm looks at N/2 elements on average.
• Therefore the algorithm is O(N)

35
Example of Binary Search

• Search for 25

16
Top 0
28
31
Middle 2
43
52
75
Bottom 5
36
Example of Binary Search
Search for 25
Top 0 Middle 0
16
Bottom 1
28
31
43
52
75
37
Example of Binary Search
Search for 25
16
28
Top 1 Bottom 1 Middle 1
31
43
52
75
38
Example of Binary Search
Search for 25
16
Bottom 0
Top 1
28
31
43
52
75
39
Binary Search Function

• int BinSearch (const int Arr, int N,int Key)
• int Top, Bottom, Middle, Pos
• bool Done
• // Initialization
• Top 1 Bottom N Pos -1
  Done false
• while ((Top lt Bottom) (!Done)) //
  while more data and not found
• Middle (Top Bottom)/
  2 // calculate Middle
• if (ArrMiddle
  Key) // if found
• Pos Middle
• Done true
• else if (ArrMiddle gt
  Key) // else is in top part of list
• Bottom Middle-1
• else Top Middle 1
  // else is in bottom part of list
• return (Pos)

40
Analysis of the Binary Search

• Every time the algorithm examines an element the
  list of elements to be searched is reduced in
  size by 1/2.
• For example consider a list of 30 elements

41
Analysis of Binary Search

• Note 32 25, and it took 5 compares to examine
  a list of 30 numbers. The size of the list could
  go to 63 elements and still only take one more
  comparison.
• The binary search is O(log2N)

42
Generic Type Definition

• When specifying the data type using a typedef in
  the client program we tie the data type to the
  class for the entire program.
• We can not have objects of the same class that
  use different data types, such as a list of
  students and a list of courses in the same client
  program.

43
Templates

• The template directive allows us to specify the
  data type as a variable which will be supplied at
  the time the object is declared.
• Therefore the specified data type is tied to the
  object not the entire client, allowing objects of
  different data types of the same class.

44
Template functions

• We specify a parameter for each template type
  used by the function in the template function
  declaration.
• These types will be used throughout the function
  to define parameters and local variables.
• The arguments for these template parameters are
  specified based on the arguments passed to the
  function.

45
Syntax for template functions

• Template function declaration and definitions
  begin with a template parameter list of the form
• template ltclass T1, class T2, . . . , class TNgt
• The function definition then uses the template
  parameters T1 through TN when declaring parameter
  and/or variables for the function.

46
The compiler's job

• The compiler uses the arguments to the function
  to specify the values of the template parameters.
• The compiler will build a separate instance of
  the function for each unique set of runtime
  parameter types.

47
The programmer's job

• It is the programmers responsibility to make sure
  that all operations needed by the generic
  template function are defined for the particular
  data types of the arguments used.

48
An Example of a Template Function

• template lt class Tgt
• void Swap(T item1, T item2)
• T TempItem
• TempItem item1
• item1 item2
• item2 TempItem

49
Using the Swap template function

• int main()
• int Num150, Num239
• char Letter1'A', Letter2'a'
• Swap (Num1, Num2)
• cout ltlt Num1 ltlt " and " ltlt Num2 ltlt endl
• Swap (Letter1, Letter2)
• cout ltlt Letter1 ltlt " and " ltlt Letter2 ltlt endl

50
Recursive Function Call

• A recursive call is a function call in which the
  called function is the same as the one making the
  call.
• In other words, recursion occurs when a function
  calls itself!
• We must avoid making an infinite sequence of
  function calls (infinite recursion).

51
Finding a Recursive Solution

• Each successive recursive call should bring you
  closer to a situation in which the answer is
  known.
• A case for which the answer is known (and can be
  expressed without recursion) is called a base
  case or terminal case.
• Each recursive algorithm must have at least one
  base case, as well as the general (recursive) case

52
General format formany recursive functions

• if (some condition for which answer is known)
  
• // base case
• solution statement
• else // general case
• recursive function call

SOME EXAMPLES . . .
53
Writing a recursive function for n factorial

• DISCUSSION
• The function call Factorial(4) should have
  value 24, because that is 4 3 2 1 .
• For a situation in which the answer is known,
  the value of 0! is 1.
• So our base case could be along the lines of
• if ( number 0 )
• return 1

54
Writing a recursive function to find Factorial(n)

• Now for the general case . . .
• The value of Factorial(n) can be written as
• n the product of the numbers from (n - 1)
  to 1,
• that is,
• n (n - 1) . . . 1
• or, n Factorial(n - 1)
• And notice that the recursive call
  Factorial(n - 1) gets us closer to the base
  case of Factorial(0).

55
Recursive Solution

• int Factorial ( int number )
• // Pre number is assigned and number gt 0.
• if ( number 0) // base case
• return 1
• else // general case
• return number Factorial ( number - 1 )

56
Three-Question Method of verifying recursive
functions

• Base-Case Question Is there a non-recursive way
  out of the function?
• Smaller-Caller Question Does each recursive
  function call involve a smaller case of the
  original problem leading to the base case?
• General-Case Question Assuming each recursive
  call works correctly, does the whole function
  work correctly?

57
Another example where recursion comes naturally

• From mathematics, we know that
• 20 1 and 25 2 24
• In general,
• x0 1 and xn x xn-1
• for integer x, and
  integer n gt 0.
• Here we are defining xn recursively, in terms
  of xn-1

58

• // Recursive definition of power function
• int Power ( int x, int n )
• // Pre n gt 0. x, n are not both zero
• // Post Function value x raised to the
  power n.
• if ( n 0 )
• return 1 // base case
• else
  // general case
• return ( x Power ( x , n-1 ) )

Of course, an alternative would have been to use
looping instead of a recursive call in the
function body.
59
class ListType

• class ListType
• private
• int length / number of elements in the
  list
• int info MAX_ITEMS
• public
• bool ValueInList( ListType list ,
  int value , int startIndex )

60
Recursive function to determine if value is in
list

• PROTOTYPE
• bool ValueInList( ListType list , int value ,
  int startIndex )
• Already searched Needs to be searched

index of current element to
examine
61

• bool ValueInList ( ListType list , int
  value , int startIndex )
• // Searches list for value between positions
  startIndex
• // and list.length-1
• // Pre list.info startIndex . .
  list.info list.length - 1
• // contain values to be searched
• // Post Function value
• // ( value exists in list.info startIndex
  . . list.info list.length - 1 )
• if ( list.infostartIndex value ) //
  one base case return true
• else if (startIndex list.length -1 ) //
  another base case
• return false
• else // general case
• return ValueInList( list, value, startIndex 1
  )

62
Why use recursion?
Those examples could have been written without
recursion, using iteration instead. The
iterative solution uses a loop, and the recursive
solution uses an if statement. However, for
certain problems the recursive solution is the
most natural solution.
63
When a function is called...

• A transfer of control occurs from the calling
  block to the code of the function. It is
  necessary that a return be made to the correct
  place in the calling block after the function
  code is executed. This correct place is called
  the return address.
• When any function is called, the run-time stack
  is used. On this stack is placed an activation
  record (stack frame) for the function call.

64
Stack Activation Frames

• The activation record
• Stores
• the return address for this function call,
• the parameters,
• the local variables,
• the functions return value, if non-void.
• The activation record for a function call is
  popped off the run-time stack when the final
  closing brace in the function code is reached, or
  when a return statement is executed.
• Control is passed back to the return address and
  the functions return value, if non-void,
  replaces the function call.

65

• // Another recursive function
• int Func ( int a, int b )
• // Pre a and b have been assigned values
• // Post Function value ??
• int result
• if ( b 0 ) // base
  case
• result 0
• else if ( b gt 0 ) //
  first general case
• result a Func ( a , b - 1 ) ) //
  instruction 50

66
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
FCTVAL
? result ?
b 2
a 5 Return Address
100
original call at instruction 100 pushes on this
record for Func(5,2)
67
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
FCTVAL ?
result ?
b 1
a 5 Return Address
50 FCTVAL ?
result 5Func(5,1) ?
b 2
a 5 Return Address 100
call in Func(5,2) code at instruction 50 pushes
on this record for Func(5,1)
68
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
call in Func(5,1) code at instruction 50 pushes
on this record for Func(5,0)
FCTVAL ?
result ?
b 0 a
5 Return Address 50
FCTVAL ?
result 5Func(5,0) ?
b 1 a
5 Return Address 50
FCTVAL ?
result 5Func(5,1) ?
b 2 a
5 Return Address 100
69
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
FCTVAL 0
result 0
b 0 a
5 Return Address 50
FCTVAL ?
result 5Func(5,0) ?
b 1 a
5 Return Address 50
FCTVAL ?
result 5Func(5,1) ?
b 2 a
5 Return Address 100
record for Func(5,0) is popped first with its
FCTVAL
70
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
FCTVAL 5
result 5Func(5,0) 5 0
b 1
a 5 Return Address
50 FCTVAL
? result 5Func(5,1) ?
b 2
a 5 Return Address
100
record for Func(5,1) is popped next with its
FCTVAL
71
Run-Time Stack Activation Records x Func(5,
2) // original call is instruction 100
FCTVAL
10 result 5Func(5,1)
55 b 2
a 5 Return
Address 100
record for Func(5,2) is popped last with its
FCTVAL
72
Show Activation Records for these calls

• x Func( - 5, - 3 )
• x Func( 5, - 3 )
• What operation does Func(a, b) simulate?

73
Tail Recursion

• The case in which a function contains only a
  single recursive call and it is the last
  statement to be executed in the function.
• Tail recursion can be replaced by iteration to
  remove recursion from the solution as in the next
  example.

74
Removal of Tail Recursion

• Create conditional loop with opposite of the base
  condition as its condition.
• (i.e. while ( ! base condition )
• Perform actions before recursive call
• Change value of parameter to value in recursive
  call

75

• // USES TAIL RECURSION
• bool ValueInList ( ListType list , int value
  , int startIndex )
• // Searches list for value between positions
  startIndex
• // and list.length-1
• // Pre list.info startIndex . .
  list.info list.length - 1
• // contain values to be searched
• // Post Function value
• // value exists in list.info startIndex . .
  list.info list.length - 1
• if ( list.infostartIndex value ) //
  one base case return true
• else if (startIndex list.length -1 ) //
  another base case
• return false
• else // general case
• return ValueInList( list, value, startIndex 1
  )

76

• // ITERATIVE SOLUTION
• bool ValueInList ( ListType list , int
  value , int startIndex )
• // Searches list for value between positions
  startIndex
• // and list.length-1
• // Pre list.info startIndex . .
  list.info list.length - 1
• // contain values to be searched
• // Post Function value
• // ( value exists in list.info startIndex
  . . list.info list.length - 1 )
• bool found false
• while ( !found startIndex lt
  list.length )
• if ( value list.info startIndex )
• found true
• else startIndex
• return found

77
Use a recursive solution when

• The depth of recursive calls is relatively
  shallow compared to the size of the problem.
• The recursive version does about the same amount
  of work as the nonrecursive version.
• The recursive version is shorter and simpler than
  the nonrecursive solution.

SHALLOW DEPTH EFFICIENCY
CLARITY
78
Using quick sort algorithm

• A . . Z
• A . . L
  M . . Z
• A . . F G . . L M .
  . R S . . Z

79
Before call to function Split

• splitVal 9
• GOAL place splitVal in its proper
  position with
• all values less than or equal to
  splitVal on its left and all larger
  values on its right
• 9 20 6 10 14 3
  60 11

valuesfirst
last
80
After call to function Split

• splitVal 9
• smaller values larger
  values
• 6 3 9 10 14 20
  60 11

values first splitPoint
last
81
How does Split work?

• Search from top of list down until you find a
  value larger than the pivot.
• Search from bottom up until you find a value
  smaller than the pivot.
• If there are still items in the middle, swap the
  two values and repeat process.
• When no more elements in the middle, swap the
  pivot and the smaller value.

82
How does Split work?

• Original list
• 9 20 6 10 14 3 60 11
• Find value larger than valuefirst starting at
  valuefirst1
• Element 20 or valuefirst1
• Find value smaller than valuefirst starting at
  valueend
• Element 3 or valueend-2
• Swap these two values
• 9 3 6 10 14 20 60 11
• Repeat the process until the subscript of the
  larger value is larger than the subscript of the
  smaller value.

83
How does Split work?

• After first swap list
• 9 3 6 10 14 20 60 11
• Find value larger than valuefirst starting at
  valuefirst1
• Element 10 or value3
• Find value smaller than valuefirst starting at
  valueend
• Element 6 or value2
• When the subscript of the larger value is larger
  than the subscript of the smaller value, swap
  valuefirst with the value that is larger than
  valuefirst.
• 6 3 9 10 14 20 60 11

84

• // Recursive quick sort algorithm
• template ltclass ItemType gt
• void QuickSort ( ItemType values , int
  first , int last )
• // Pre first lt last
• // Post Sorts array values first. .last into
  ascending order
• if ( first lt last ) //
  general case
• int splitPoint
• Split ( values, first, last, splitPoint )
• // values first . . valuessplitPoint - 1
  lt splitVal
• // values splitPoint splitVal
• // values splitPoint 1 . . values last
  gt splitVal