Sorting And Searching

1
Sorting And Searching

• CSE116A,B

2
Introduction

• The problem of sorting a collection of keys to
  produce an ordered collection is one of the
  richest in computer science.
• The richness derives from the fact that there are
  a number of ways of solving the sorting problem.
• Sorting is a fascinating operation that provides
  a model for investigating many problems in
  Computer Science. Ex analysis and comparison of
  algorithms, divide and conquer, space and time
  tradeoff, recursion etc.

3
Sorting

• Comparison based selection, insertion sort (with
  online animations)
• Divide and conquer merge sort, quick sort (with
  animations from supplements of your text book)
• Priority Queue /Heap based heap sort
• Assume Ascending order for all our discussion

4
Selection Sort

• Select the first smallest element, place it in
  first location (by exchanging contents of
  locations), find the next smallest, place it in
  the next location, and so on.
• We will look at an example, an algorithm, analyze
  the algorithm, and look at code implementation.

5
Selection Sort Example
2
8
16
10
6
4
2
4
16
10
6
8
2
4
16
10
6
8
Sorted array
2
4
16
8
6
10
2
4
10
8
6
16
6
Selection Sort Pseudo Code

• Let cursor be pointer to first element of an n
  element list to be sorted.
• Repeat until cursor points to last but one
  element.
• 2.1 Search for the smallest element in the list
  starting from the cursor. Let it be at location
  target.
• 2.2 Exchange elements at cursor and target.
• 2.3 Update cursor to point to next element.

7
Sort Analysis

• Let el be the list n be the number of elements
  cursor 0 target 0
• while (cursor lt n-1)
• 2.1.1 target cursor
• 2.1.2 for (i cursor1 i lt n i)
• if (eli lt eltarget) target
  i
• 2.2 exchange (eltarget, elcursor) //
  takes 3 assignments
• 2.3 cursor
• (n-1) 4 2 (n (n-1) (n-2) ..1)
• 4n 4 2n(n1)/2 4n 4 n2 n n2 5n -
  4
• An2 B n C where A 1, B 5, C -4 for
  large n, drop the constant, lower order term in
  n, and the multiplicative constant to get big-O
  notation
• O(n2) quadratic sort

8
Insertion Sort
Unsorted array
10
Trivially sorted
8
10
Insert 8
6
8
10
Insert 6
2
6
10
8
Insert 2
2
6
16
10
8
Insert 4
Insert 16
9
Insertion Sort Pseudo Code

• Single element is trivially sorted start with
  first element
• Repeat for second to nth element of the list
• 2.1 cursor next location
• 2.2 Find a location to insert for listcursor
  by comparing
• and shifting
• let the location be target
• 2.3 Insert listtarget listcursor

10
Insertion Sort Analysis

• cursor 0
• while (cursor lt n)
• 2.1 cursor cursor 1
• 2.2 temp listcursor // save element to
  inserted
• 2.2.2 j cursor //find location
• 2.2.3 while (j gt 0 listj-1 gt temp )
• listj listj-1 //shift
  right
• j j 1
• // assert location found or hit left
  end(j0)
• 2.3 listj temp
• Worst case O(n2) quadratic
• Best case linear (when the list is already
  sorted)

11
Merge Sort

• Divide and Conquer
• Divide the list into two subsets s1, and s2
• (recurse) Sort s1 and s2 by divide and conquer
• (conquer) Merge the sorted s1 and s2.
• O(n log n) algorithm

12
Merge Sort (s)

• mergeSort(s)
• If S.size() gt 1
• 1. S1, S2 ? partition (S, n/2)
• 2. mergeSort(s1)
• 3. mergeSort(s2)
• 4. S ? merge(s1,s2)
• Lets look at examples.

13
Example
partition
2
S1
S2
8
6
10
2
16
4
S21
S22
S11
S12
8
6
16
4
S121
S122
S221
S222
14
Example
merge
2
S1
S2
6
8
10
2
4
16
S21
S22
S11
S12
8
6
16
4
S121
S122
S221
S222
15
Algorithm list merge(s1, s2)

• s ? empty list
• while (!s1.empty() !s2.empty())
• // Assert both lists non empty
• 2.1 if s1.firstElem() lt s2.firstElem()
• s.insertLast(s1.removeFirst())
• else
• s.insertLast(s2.removeFirst())
• 3. //Assert s1 is empty or s2 is empty
• 3.1 while (!s1.empty())
• s.insertLast(s1.removeFirst())
• 3.2 while (!s2.empty())
• s.insertLast(s2.removeFirst())
• 4. return s

2k
n-k n k n cn (c1)n O(n)
16
Analysis of merge sort

• Running time is time spent each level merging the
  nodes
• Number of levels 1 ceiling(log n)
• Since the time spent at each of the is O(n), we
  have the following result
• Algorithm mergesort sorts a list of size n in O(n
  log n) time in the worst case.

17
Quick Sort

• Recursive sort divide and conquer
• Divide select an element called the pivot
  typically last or first element is chosen to be
  the pivot partition the list into three lists
• L elements in S less than pivot
• E elements in S equal to pivot (single element
  for list of distinct elements)
• G elements in S greater than pivot
• Recurse Recursively quick sort the lists L and
  G.
• Conquer Form the sorted list by concatenating L,
  E and G.

18
Quicksort Example
pivot
16
4
2
16
null
Partition around pivot
19
Quicksort Example
pivot
16
4
2
4
16
2
null
null
Concatenate LpivotG
10
null
20
Quicksort

• Worst case when the list is already sorted. Let
  si be the sum of all sizes of the nodes to be
  sorted at level i. In the worst case the number
  of levels is n.
• S0 n
• S1 n 1 since every element skews to one side
• S2 n 2 and so on.
• worst case running time is O(n (n-1) (n-2)
  ..1) O(n2)
• best case O(n log n)

21
Heap Definition

• Heap is a loosely ordered complete binary tree.
• A heap is a complete binary tree with values
  stored in its nodes such that no child has a
  value greater than the value of its parent.
• A heap is a complete binary tree
• 1. That is empty or
• 2a. Whose root contains a search key greater
  than or equal to both its children node.
• 2b. Whose left subtree and right subtree are
  heaps.

22
Types of heaps

• Heaps can be max heaps or min heaps. Above
  definition was for a max heap.
• In a max-heap the root is higher than or equal to
  the values in its left and right child.
• In a min-heap the root is smaller than or equal
  to the values in its left and right child.

23
ADT Heap

• createHeap ( )
• destroyHeap ( )
• empty ( )
• heapInsert (Object newItem)
• Object heapDelete ( ) // always the root

24
Example
Consider data set
25
Example
1
26
Delete Root Item

• In a max heap the root item is the largest and is
  the chosen one for deletion.
• 1. After deletion of root, two disjoint heaps
  result.
• 2. Place last node as a root, to form a
  semi-heap.
• 3. Use trickle-down to form a heap.
• Running time 3log N 1 O(log N)
• Consider a heap example discussed above.
• Delete root item.

27
Insert An Item

• 1. Insert a node as a last node.
• 2. Trickle up (repeat for various levels) to form
  a heap.
• Consider inserting 15 into the heap of the
  delete example.
• Insert is also a O(log N) operation.

28
Insert Node Example
15
5
9
3
6
2
29
Priority Queue

• Priority Queue implemented as a heap. Priority
  queue inserts are done according to some criteria
  known as priority
• Insert location are according to priority and
  delete are to the head of queue.
• PQueue constructor
• PQueueInsert
• PQueueDelete
• Data
• Heap pQ

30
Heap Sort

• Heapsort
• 1. Represent elements as a min-heap.
• 2. Delete item at root and add to result, as it
  is the smallest.
• 3. Heap, repeat step 2, until one item is left.
• 4. Delete the last item and add to result.
• O(Nlog N) in the worst case!

31
Heap sort Example
2
Min-heap
4
8
6
16
10
Lets do that rest by hand.
32
Heap Sort in place

• Since heap is a complete binary tree, the nodes
  can be stored in contiguous storage such as an
  array.
• Assuming root is at 0
• Parent of a node(j) is node ((j-1)/2)
• Left child of node(j) is node(2j1) if 2j lt n
• Right child of node(j) is node(2(j1)) if
  2(j1)ltn
• Adding last leaf is equivalent to adding a
  element as last element of the contiguous storage.

33
Priority Queue(PQ)

• Queue that maintains a list of elements according
  to some priority.
• Queue front is the element removed.
• Element added as last.
• For linear list PQ insert is O(n) since we have
  to search through the list to find the right
  place. Strict ordering.
• For heap PQ insert is O(log n) since we insert
  it a leaf and siftUp() which in O(log n) loose
  ordering.

34
Heap and Priority Queue
35
Heap and Priority Queue (code)

• Heap will have other support methods root(),
  parent(), rightChild(), leftChild() etc.
• Homework To reinforce the concepts, implement
  the heap and implement the PQ using the class
  diagram given.
• Adapt the code in your textbook p.312-313

36
Quicksort in place

• quickSort(T, left, right)
• 1. if left lt right
• 1.1 pivot partition(T, left, right)
• // assert T is partially sorted during
  partition
• 1.2 quickSort(T, left, pivot 1)
• 1.3 quickSort(T, pivot1, right)
• 2. return

37
Partition(T,first,last)

• Initialize i first j last-1
• while ( i lt j)
• 2.1 Decrease j while (iltj) if T(j) gt
  T(last), j j 1 else break
• 2.2 Increase i while(jgti) if T(i) gt T(last),
  i i1 else break
• 2.3 Exchange/Done
• 2.4 If j gt i,
• exchange(T(i), T(j))
• 2.5 else
• 2.5.1. exchange(T(j),T(last))
• 2.5.2 return j // done j is the pivot
  between
• // partitions

38
Example

• 85 24 63 45 17 31 96 50

39
Comparison

• Small data set selection sort
• Insert into already sorted list insertion sort
• Medium size data set to be sorted in place quick
  sort
• Very large data set on disk/tape merge sort

40
Summary of Sorting Algorithms
Algorithm Time Notes
selection-sort O(n2) slow in-place for small data sets (lt 1K)
insertion-sort O(n2) slow in-place for small data sets (lt 1K)
heap-sort O(n log n) fast in-place for large data sets (1K 1M)
merge-sort O(n log n) fast sequential data access for huge data sets (gt 1M)