Iterative Algorithms

1
Iterative Algorithms Loop Invariants
Steps in an Iterative Algorithm Typical Loop
Invariants Lower Bound for Sort
(same as in 3101)

• Jeff Edmonds
• York University

COSC 3101
Lecture 1
2
A Sequence of Actions
vsA Sequence of Assertions
Max( a,b,c )
preCond Input has 3 numbers.
m a
assert m is max in a
if( bgtm ) m bendif
assert m is max in a,b
if( cgtm ) m cendif
It is help to have different ways of looking at
it.
assert m is max in a,b,c
return(m)
postCond return max in a,b,c
3
Designing an Algorithm
Is this sufficient?
4
Explaining Insertion Sort

• We maintain a subset of elements sorted within
  a list. The remaining elements are off to the
  side somewhere. Initially, think of the first
  element in the array as a sorted list of length
  one. One at a time, we take one of the elements
  that is off to the side and we insert it into the
  sorted list where it belongs. This gives a sorted
  list that is one element longer than it was
  before. When the last element has been inserted,
  the array is completely sorted.

5
Partial Correctness

• Proves that IF the program terminates then it
  works ltPreCondgt ltcodegt Þ ltPostCondgt

6
Consider an Algorithm
7
Loop Invariant Maintained
8
Computation can always proceed
9
Computation always makes progress
10
Computation Terminates
11
Computation Terminates
12
Consider an Algorithm
This is sufficient!
13
Typical Types of Loop Invariants

• More of the input
• More of the output
• Narrowed the search space
• Case Analysis
• Work Done

14
More of the Input Loop Invariant
The input consists of an array of objects
We have read in the first i objects. We will
pretend that this prefix is the entire
input. We have a solution for this prefix plus
some additional information
15
Loop Invariants andDeterministic Finite Automaton

• L a Î 0,1 a has length at most three and
  the number of 1's is odd .

16
More of the Input Loop Invariant
The input consists of an array of objects
We read in the i1st object. We will pretend that
this larger prefix is the entire input. We
extend the solution we have to one for this
larger for prefix. (plus some additional
information)
17
More of the Input Loop Invariant
The input consists of an array of objects
i to i1
18
More of the Input Loop Invariant
The input consists of an array of objects
End the end, we have read in the entire
input. The LI gives us that we have a solution
for this entire input.
19
Insertion Sort
The input consists of an array of integers
Solution
We have read in the first i objects. We will
pretend that this prefix is the entire
input. We have a solution for this prefix
20
Insertion Sort
The input consists of an array of integers
We read in the i1st object. We will pretend that
this larger prefix is the entire input. We
extend the solution we have to one for this
larger for prefix.
21
Typical Types of Loop Invariants

• More of the input
• More of the output
• Narrowed the search space
• Case Analysis
• Work Done

22
More of the OutputLoop Invariant
The output consists of an array of objects
I have produced the first i objects.
Produce the i1st output object.
Done when output n objects.
i to i1
23
Selection Sort
Input
The output consists of an array of integers
14,23,25,30,31,52,62,79,88,98
I have produced the first i objects.
Produce the i1st output object.
Done when output n objects.
24
Typical Types of Loop Invariants

• More of the input
• More of the output
• Narrowed the search space
• Case Analysis
• Work Done

Three Binary Search like examples
25
Define Loop Invariant

• Maintain a sub-list.
• If the key is contained in the original list,
  then the key is contained in the sub-list.

key 25
26
Two Graph Problems

• Tournament
• Euclidian Cycle

27
A Lower Bound on Sorting
Merge, Quick, and Heap Sort can sort N numbers
using O(N log N) comparisons between the values.
Theorem No algorithm can sort faster.
28
A Lower Bound on Sorting
I have an algorithm A that I claim works and is
fast.
Oh yeah, I have an input I for which it does not .

• I win if A on input I gives
• the wrong output or
• runs slow.

29
A Lower Bound on Sorting

• Sorting
• Input
• Output
• Lower Bound
• Input An Algorithm for Sorting A
• Output An instance I on which alg A either
• Takes too much time
• Or give the wrong answer

30
Algorithm Definition Completed
31
A Lower Bound on Sorting
I give you algorithm AI claim it sorts.
32
A Lower Bound on Sorting
I give you algorithm AI claim it sorts.
Need to know what the algorithm does before we
can know what input to give it.
Break this cycle, one iteration at a time.
Need to give the algorithm an input before we
can know what the algorithm does input.
33
A Lower Bound on Sorting
Restrict the search space
34
A Lower Bound on Sorting
I maintain a set of instances(permutations of
1..n)
Oh dear!The first t time steps of my algorithm A
are the same given any of these instances!!!
35
A Lower Bound on Sorting
Initially, I consider all N! permutations of 1..N
Oh dear!The first 0 time steps of my algorithm A
are the same given any of these instances!!!
36
A Lower Bound on Sorting
Initially, I consider all N! permutations of 1..N
The measure of progressis the number of
instances.Initially, there are N! of them.
37
A Lower Bound on Sorting

• My t1st time step
• a7 lt a2
• 5th bit of a7 a2
• Any yes/no question

Because the first t time steps are the same,
what the alg knows is the same.Hence, its t1st
step is the same.
38
A Lower Bound on Sorting

• My t1st time step
• a7 lt a2
• 5th bit of a7 a2
• Any yes/no question

I partition my instances based on what they do on
this t1st step.
Because the first t time steps are the same,
what the alg knows is the same.Hence, its t1st
step is the same.
39
A Lower Bound on Sorting
I keep the larger half.
40
A Lower Bound on Sorting
I keep the larger half.
Oh dear!The first t1 time steps of my algorithm
A are the same given any of these instances!!!
41
A Lower Bound on Sorting
I keep the larger half.
Initially, I have n! Permutations.After t time
steps I have N!/2t
42
A Lower Bound on Sorting
I keep the larger half.
Initially, I have n! Permutations.After t time
steps I have N!/2t
We exit when there are two instances.
T log(N!)
43
A Lower Bound on Sorting
N! ? NN
We exit when there are two instances.
T log(N!)
Q(N log(N)).
44
A Lower Bound on Sorting
I give you algorithm AI claim it sorts.
45
A Lower Bound on Sorting
Oops!
Case 1 The algorithm does not stop at time T on
these two instances.
46
A Lower Bound on Sorting
Oops!I must give the wrong answer on one of
these!
Case 2 The algorithm stops at time T and gives
an answer.
and these need different answers.
47
A Lower Bound on Sorting
Theorem For every sorting algorithm A,on the
worst case input instance I, Q(N log2 N)
comparisons (or other bit operations) need to be
executed to sort N objects.
48
End Iterative Algorithms
Math Review

• Recursive Algorithms