Chap' 5 Decrease and Conquer

1
Chap. 5 Decrease and Conquer

• Reduce problem instance to smaller instance of
  the same problem and extend solution
• Solve smaller instance
• Extend solution of smaller instance to obtain
  solution to original problem
• Also referred to as inductive or incremental
  approach

2
Examples of Decrease and Conquer

• Decrease by one
• Insertion sort
• Graph search algorithms
• DFS
• BFS
• Topological sorting
• Algorithms for generating permutations, subsets
• Decrease by a constant factor
• Binary search
• Fake-coin problems
• multiplication à la russe
• Josephus problem
• Variable-size decrease
• Euclids algorithm
• Selection by partition

3
Whats the difference?

• Consider the problem of exponentiation Compute
  an
• Brute Force an a a a
  a (n-1 multiplications)
• Divide and conquer an a n/2 a
  n/2 if n gt 1
• 
  a if n 1
• Decrease by one f(n) f(n-1) a
  if n gt 1
• 
  a if n 1
• Decrease by constant factor (a n/2
  )2 if n is even positive
• 
  an (a (n-1)/2 )2 a if is odd
  n gt 1
• 
  a if n 1

4
Insertion Sort

• Assume the left subarray has already been sorted
• A0 Aj lt Aj1 Ai-1 Ai
  An-1
• already sorted
• Three alternatives
• Scan the sorted subarray L?R until the 1st
  element Ai. Then insert Ai before that
  element.
• Scan the sorted subarray R?L until the 1st
  element Ai. Then insert Ai after that
  element.
• Use binary search to find the appropriate
  position for Ai

5
Insertion Sort - Algorithm

• InsertionSort (A0..n-1)
• for i ? 1 to n-1 do
• v ? Ai
• j ? i - 1
• while (j gt 0 and Aj gt v) do
• Aj1 ? Aj
• j ? j 1
• Aj1 ? v

6
Insertion Sort - Example

• i
• 89 45 68 90
  29 34 17
• 45 89 68 90
  29 34 17
• 45 68 89 90
  29 34 17
• 45 68 89 90
  29 34 17
• 29 45 68 89
  90 34 17
• 29 34 45 68
  89 90 17
• 17 29 34 45
  68 89 90

7
Insertion Sort - Analysis

• Based on the of key comparisons of (Aj gt v)
• Best case C(n) ? 1 n - 1 ?
  (n)
• Worst case C(n) ? ? 1
  n (n-1 ) / 2
• 
  
  ? (n²)
• Avg. case C(n) n² / 4 ? (n²)
• for
  randomly ordered array

n - 1
i 1
n - 1
i - 1
i 1
j 0
8
Graph Traversal

• Many problems require processing all graph
  vertices in systematic fashion
• Graph traversal algorithms
• Depth-first search
• Breadth-first search

9
Depth-first search

• Explore graph always moving away from last
  visited vertex
• Similar to preorder tree traversals
• Pseudocode for Depth-first-search of graph
  G(V,E)
• DFS(G)
• count 0
• mark each vertex with 0 (unvisited)
• for each vertex v? V do
• if v is marked with 0
• dfs(v)
• dfs(v)
• count count 1
• mark v with count
• for each vertex w adjacent to v do
• if w is marked with 0
• dfs(w)

10
Example undirected graph

• Depth-first traversal

11
Types of edges

• Tree edges edges comprising forest
• Back edges edges to ancestor nodes
• Forward edges edges to descendants (digraphs
  only)
• Cross edges none of the above

12
Example directed graph
a
b
c
d
e
f
g
h

• Depth-first traversal

13
Depth-first search Notes

• DFS can be implemented with graphs represented
  as
• Adjacency matrices T(V2)
• Adjacency linked lists T(VE)
• Yields two distinct ordering of vertices
• preorder as vertices are first encountered
  (pushed onto stack)
• postorder as vertices become dead-ends (popped
  off stack)
• Applications
• checking connectivity, finding connected
  components
• checking acyclicity
• searching state-space of problems for solution
  (AI)

14
Breadth-first search

• Explore graph moving across to all the neighbors
  of last visited vertex
• Similar to level-by-level tree traversals
• Instead of a stack, breadth-first uses queue
• Applications same as DFS, but can also find
  paths from a vertex to all other vertices with
  the smallest number of edges

15
Breadth-first search algorithm
BFS(G) count 0 mark each vertex with 0 for each
vertex v? V do bfs(v)

• bfs(v)
• count count 1
• mark v with count
• initialize queue with v
• while queue is not empty do
• a front of queue
• for each vertex w adjacent to a do
• if w is marked with 0
• count count 1
• mark w with count
• add w to the end of the queue
• remove a from the front of the queue

16
Example undirected graph

• Breadth-first traversal

17
Example directed graph
a
b
c
d
e
f
g
h

• Breadth-first traversal

18
Breadth-first search Notes

• BFS has same efficiency as DFS and can be
  implemented with graphs represented as
• Adjacency matrices T(V2)
• Adjacency linked lists T(VE)
• Yields single ordering of vertices (order
  added/deleted from queue is the same)

19
Directed acyclic graph (dag)

• A directed graph with no cycles
• Arise in modeling many problems, eg
• prerequisite structure
• food chains
• Imply partial ordering on the domain

20
Topological sorting

• Problem find a total order consistent with a
  partial order
• Example

tiger
Order them so that they dont have to wait for
any of their food (i.e., from lower to higher,
consistent with food chain)
human
fish
sheep
shrimp
NB problem is solvable iff graph is dag
wheat
plankton
21
Topological sorting Algorithms

• DFS-based algorithm
• DFS traversal noting order vertices are popped
  off stack
• Reverse order solves topological sorting
• Back edges encountered?? NOT a dag!
• Source removal algorithm
• Repeatedly identify and remove a source vertex,
  ie, a vertex that has no incoming edges
• Both T(VE) using adjacency linked lists

22
Variable-size-decrease Binary search trees

• Arrange keys in a binary tree with the binary
  search tree property

Example 1 5, 10, 3, 1, 7, 12, 9 Example 2 4,
5, 7, 2, 1, 3, 6
k
ltk
gtk

• What about repeated keys?

23
Searching and insertion in binary search trees

• Searching straightforward
• Insertion search for key, insert at leaf where
  search terminated
• All operations worst case key comparisons h
  1
• lg n h n 1 with average (random files)
  1.41 lg n
• Thus all operations have
• worst case T(n)
• average case T(lgn)
• Bonus inorder traversal produces sorted list
  (treesort)