4190.204 Data Structures Lecture 10

1
4190.204 Data StructuresLecture 10

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University
• Partly based on
• Carrano Prichard, Data Abstraction and Problem
  Solving with Java, Chapter 9
• Lecture Notes of F M Carrano, Prof Moon Byung Ro

2
Sorting and Efficiency

• Efficiency
• Big O Notation
• Worst, best and average case performance
• Simple sorting algorithms
• Selection sort
• Bubblesort
• Insertion Sort
• Partitioning Algorithms
• Merge Sort
• Quicksort
• Radix Sort

3
Ch. 9 Algorithm Efficiency Sorting

• O( ) Big-Oh
• An algorithm is said to take O(f (n))
• if
• its running time is upper-bounded by cf(n)
• e.g., O(n), O(n log n), O(n2), O(2n),
• Formal definition
• O(f(n)) g(n) ?c gt 0, n0 0 s.t.?n n0,
  cf(n) g(n)
• g(n) ? O(f(n)? ??? ????? g(n) O(f(n))??? ??.
• ??? ??
• g(n) O(f(n)) ? g grows no faster than f

4
Figure 9.1 Time requirements as a function of the
problem size n
5
Figure 9.3a A comparison of growth-rate
functions a) in tabular form
6
Figure 9.3b A comparison of growth-rate
functions b) in graphical form
7
Types of Running-Time Analysis

• Worst-case analysis
• Analysis for the worst-case input(s)
• Average-case analysis
• Analysis for all inputs
• More difficult to analyze
• Best-case analysis
• Analysis for the best-case input(s)
• Often not very meaningful
• (because we dont get to choose the cases)
• Can be important if we can expect to make good
  guesses

8
Running Time for Search in an Array

• Sequential search
• Worst case O(n)
• Average case O(n)
• Best case O(1)
• Binary search
• Worst case O(log n)
• Average case O(log n) ? ?? ?? ??!
• Best case O(1)

9
Sorting Algorithms

• ??? O(n2)? O(nlogn) ??
• Input? ??? ??? ???? ???? O(n) sorting? ??
• E.g., input? O(n)? O(n) ??? ??

10
Selection Sort

• An iteration
• Find the largest item
• Swap it to the rightmost place
• Exclude the rightmost item
• Repeat the iteration until only one item remains

11
(No Transcript)
12
selectionSort(theArray , n) for (last
n-1 last gt1 last--) largest
indexOfLargest(theArray, last1) Swap
theArraylargest theArraylast indexOfL
argest(theArray, size) largest 0 for (i
1 i lt size i) if (theArrayi gt
theArraylargest) largest i
The loop in selectionSort calls indexOfLargest n
times Each call to IndexOfLargest creates a loop
of 1 less time than the previous

• (n-1)(n-2)21 O(n2)

13
Bubble Sort
Worst case Average case

• Running time (n-1)(n-2)21 O(n2)

14
Insertion Sort
Worst case 12(n-2)(n-1) Average case ½
(12(n-2)(n-1))

• Running time O(n2)

15
Figure 9.6 An insertion sort partitions the array
into two regions
16
Merge Sort

• A recursive sorting algorithm
• Gives the same performance, regardless of the
  initial order of the array items
• Strategy
• Divide an array into halves
• Sort each half
• Merge the sorted halves into one sorted array

17
Mergesort

• Algorithm mergeSort(S)
• // Input sequence S with n elements
• // Output sorted sequence S
• if (S.size( ) gt 1)
• Let S1, S2 be the 1st half and 2nd half of S,
  respectively
• mergeSort(S1)
• mergeSort(S2)
• S ? merge(S1, S2)
• Algorithm merge(S1, S2)
• sorting? ? sequence S1, S2 ? ??
• sorting ? ??? sequence S? ???

18
Animation (Mergesort)
7 2 9 4 ? 3 8 6 1
7 2 9 4
7 2
7
19
Animation (Mergesort)
7 2 9 4 ? 3 8 6 1
1 3 6 8
7 2 9 4
2 4 7 9
7 2
2 7
9 4
4 9
7
2
9
4

• Running time O(nlogn)

20
Figure 9.8 A mergesort with an auxiliary
temporary array
21
Figure 9.9 A mergesort of an array of six integers
22
Figure 9.10 A worst-case instance of the merge
step in mergesort
23
Quicksort

• Algorithm quickSort(S)
• // Input sequence S with n elements
• // Output sorted sequence S
• if (S.size( ) gt 1)
• x ? pivot of S
• (L, R) ? partition(S, x) // L left partition,
  R right partition
• quickSort(L)
• quickSort(R)
• return L x R // concatenation
• Algorithm partition(S, x)
• sequence S?? x?? ?? item? partition L?,
• x?? ??? ?? item? partition R? ??.

24
Animation (Quicksort)
5 1 9 4 2 6 8 3
3 1 4 2

• Average-case running time O(nlogn)
• Worst-case running time O(n2)

25
Figure 9.12 A partition with a pivot

• Partitioning ??? ????.
• ???? ? ? ? ?? ??? ???? ??.

26
Stable and Deterministic Sorting

• A sort algorithm must sort the elements into
  order
• But what happens to elements which are equal?
• Stable sort
• Elements are in the same order as the original
  sequence
• Eg merge-sort
• Deterministic Sort
• Elements are always in the same order
• Non-deterministic example
• Quicksort with random pivot
• Stable ? deterministic
• But not vice versa

27
Radix Sort

• radixSort(theArray, d)
• // Sort n d-digit integers in the array
  theArray
• for (j d downto 1)
• Do a stable sort on theArray by jth digit
• Stable sort
• ?? ?? ?? item?? sorting ??? ??? ??? ???? ??? ??
  sort? ????.

28

• Running time O(n) d a constant (?)

29
Some Remarks on Efficiency

• How efficient is radix sort?
• The text says O(n) d (a constant)
• Mathematically, this is highly misleading
• How many digits does it take to represent n
  numbers?
• Just coincidentally, d log n
• So radix sort is really an O(n log n) algorithm
• But practically, the ability to use standard
  integer representation can make it highly
  efficient for even reasonably large n
• Why are simple sorts so slow?
• Its provable that alorithms which only exchange
  elements must take O(n2)

30
Comparison of Sorting Efficiency

• - highly debatable

31
Summary

• Efficiency
• Big O Notation
• Worst, best and average case performance
• Simple sorting algorithms
• Selection sort
• Bubblesort
• Insertion Sort
• Partitioning Algorithms
• Merge Sort
• Quicksort
• Radix Sort