Insertion Sort

1
Insertion Sort

• Algorithm Design Analysis
• 5

2
In the last class

• Recurrence Algorithm and Recursion Equations
• Solution of the Recurrence Equations
• Guess and proving
• Recursion tree
• Master theorem
• Divide-and-conquer

3
Insertion Sort

• Comparison-based sorting
• Insertion sort
• Analysis of insertion sorting algorithm
• Lower bound of local comparison based sorting
  algorithm
• Shell sort

4
Why Sorting

• Practical use
• Enough different algorithms for comparing
• Easy to derive the strong lower bounds for worst
  case and average behavior

5
Ordering Is Out of Order

• According to an English dictionary, sorting is
  defined as a process of separating or arranging
  things according to class or kind.
• Here, sorting is ordering
• A sentence from Knuth
• Since only two of our tape drives were in working
  order, I was ordered to order more tape units in
  short order, in order to order the data several
  orders of magnitude faster.

6
Comparison-Based Algorithm

• The class of algorithms that sort by comparison
  of keys
• comparing (and, perhaps, copying) the key
• no other operations are allowed
• The measure of work used for analysis is the
  number of comparison.

7
As Simple as Inserting

8
Shifting Vacancy the Specification

• int shiftVac(Element E, int vacant, Key x)
• Precondition vacant is nonnegative
• Postconditions Let xLoc be the value returned to
  the caller, then
• Elements in E at indexes less than xLoc are in
  their original positions and have keys less than
  or equal to x.
• Elements in E at positions (xLoc1,, vacant) are
  greater than x and were shifted up by one
  position from their positions when shiftVac was
  invoked.

9
Shifting Vacancy Recursion

• int shiftVacRec(Element E, int vacant, Key x)
• int xLoc
• 1. if (vacant0)
• 2. xLocvacant
• 3. else if (Evacant-1.key?x)
• 4. xLocvacant
• 5. else
• 6. EvacantEvacant-1
• 7. xLocshiftVacRec(E,vacant-1,x)
• 8. Return xLoc

The recursive call is working on a smaller range,
so terminating The second argument is
non-negative, so precondition holding
Worst case frame stack size is ?(n)
10
Shifting Vacancy Iteration

• int shiftVac(Element E, int xindex, Key x)
• int vacant, xLoc
• vacantxindex
• xLoc0 //Assume failure
• while (vacantgt0)
• if (Evacant-1.key?x)
• xLocvacant //Succeed
• break
• EvacantEvacant-1
• vacant-- //Keep Looking
• return xLoc

11
Insertion Sorting the Algorithm

• Input E(array), n?0(size of E)
• Output E, ordered nondecreasingly by keys
• Procedure
• void insertSort(Element E, int n)
• int xindex
• for (xindex1 xindexltn xindex)
• Element currentExindex
• Key xcurrent.key
• int xLocshiftVac(E,xindex,x)
• ExLoccurrent
• return

12
Worst-Case Analysis

• At the beginning, there are n-1 entries in the
  unsorted segment, so

The input for which the upper bound is reached
does exist, so W(n)??(n2)
13
Average Behavior
Sorted (i entries)

• Assumptions
• All permutations of the keys are equally likely
  as input.
• There are not different entries with the same
  keys.
• Note For the (i1)th interval(leftmost), only i
  comparisons are needed.

x
x may be located in any one of the i1
intervals(inclusive), assumingly, with the same
probability
14
Average Complexity

• The average number of comparisons in shiftVac to
  find the location for the ith element
• For all n-1 insertions

for the leftmost interval
15
Inversion and Sorting

• An unsorted sequence E
• x1, x2, x3, , xn-1, xn
• If there are no same keys, for the purpose of
  sorting, it is a reasonable assumption that x1,
  x2, x3, , xn-1, xn1,2,3,,n-1,n
• ltxi, xjgt is an inversion if xigtxj, but iltj
• All the inversions must be eliminated during the
  process of sorting

16
Eliminating Inverses Worst Case

• Local comparison is done between two adjacent
  elements.
• At most one inversion is removed by a local
  comparison.
• There do exist inputs with n(n-1)/2 inversions,
  such as (n,n-1,,3,2,1)
• The worst-case behavior of any sorting algorithm
  that remove at most one inversion per key
  comparison must in ?(n2)

17
Eliminating Inverses Average

• Computing the average number of inversions in
  inputs of size n (ngt1)
• Transpose x1, x2, x3, , xn-1, xn
• xn, xn-1, , x3, x2, x1
• For any i, j, (1?j?i?n), the inversion (i,j ) is
  in exactly one sequence in a transpose pair.
• The number of inversions (i,j ) is n(n-1)/2.
• So, the average number of inversions in all
  possible inputs is n(n-1)/4.
• The average behavior of any sorting algorithm
  that remove at most one inversion per key
  comparison must in ?(n2)

18
Traveling a Long Way

• Problem
• If a1, a2, an is a random permutation of
  1,2,n, what is the average value of
• a1-1 a2-2 a1-n
• The answer is the average net distance traveled
  by all records during a sorting process.

19
Shellsort the Strategy
h6

h4
h3
5
8
24
13
7
18
31
19
44
63
82
29
5
7
18
13
8
24
31
19
29
63
82
44
h2
20
Shellsort the Strategy (cont.)

• For each subsequence, insertion sorting is use,
  since, generally, in most cycle, there are not
  many element out of the order.

h1
The solution
21
Shellsort the Algorithm

• Input E, an unsorted array of elements, n?0, the
  number of elements, a sequence of diminishing
  increments ht, ht-1,, h1, where h11, and t, the
  number of increments.
• Output E, with the elements in nondecreasing
  order of their keys.
• Procedure
• void shellSort(Element E, int n, int h,
  int t)
• int xindex, s
• for (st s?1 s--)
• for (xindexhs xindexltn xindex)
• element currentExindex
• key x current.key
• int xLocshiftVacH(E, hs,
  xindex,x)
• ExLoccurrent
• return

Is there any possibility that Shellsort is an
improvement on Insertion Sort, since only the
last pass has to do the same thing as Insertion
Sort, let along the passes done before that?
???
A special version for Shellsort
22
Complexity of Shellsort

• The efficiency of Shellsort should be higher than
  insertSort, since
• More than 1 inverse may be removed by one
  comparison, when hgt1
• Sorting with one increment will not undo any of
  the work done previously when a different
  increment was used
• The behavior of Shellsort is mostly unknown
• Function of the sequence of increments used
• Influences of pre-processing(when hgt1) is
  unknown

23
Invariable Ordering Pairs under Sorting
At least r xs no less than r different ys
Unordered
No greater than
Ordered independently
No greater than hold
In non-decreasing order
24
Previous k-Ordering Kept
r is the largest integer making the subscript
maximal no greater than n
If a k-ordered file is h-ordered, it remains
k-ordered. That is assume that a k-ordered
sequence a1,a2, ,an has been h-sorted, then for
each i1,2,n-k, we have to prove that ai?aik
Non-existing if vrhgtn
Let i?u, ik?v (mod h), note
Last r items
No greater than
First r items
25
Some Results about Shellsort

• If an h-ordered array is k-sorted, the array will
  still be h-ordered.
• For a two-pass case of Shellsort(i.e. the
  increments are 1,h)
• The average number of inversions in an h-ordered
  permutation of 1,2,n is
• Running time is approximately proportional to
  the best choice of h is
  about ,which gives
  O(n5/3)

26
Home Assignment

• pp.208-
• 4.6
• 4.8
• 4.9
• 4.11
• 4.45