Lower bound: Decision tree and adversary argument

1
Lower bound Decision tree and adversary argument

• Lower bound ?(f(n)) in the worst case.
• Decision tree model
• Example for comparison sorting.
• Adversary argument
• Example find the maximum

2
Comparison sort

• Comparison sort
• Insertion sort, O(n2), upper bound in worst case
• Merge sort, O(nlg n), upper bound in worst case
• Heapsort, O(nlg n), upper bound in worst case
• Quicksort, O(nlg n), in average case
• Question
• what is the lower bounds for any comparison
  sorting i.e., at least how many comparisons
  needed in worst case?
• It turns out Lower bound in worst case ?(nlg
  n), how to prove?
• Merge and Heapsort are asymptotically optimal
  comparison sorts.

3
Decision Tree Model

• Assumptions
• All numbers are distinct (so no use for ai aj )
• All comparisons have form ai ? aj (since ai ? aj,
  ai ? aj, ai lt aj, ai gt aj are equivalent).
• Decision tree model
• Full binary tree
• Internal node represents a comparison.
• Ignore control, movement, and all other
  operations, just see comparison
• Each leaf represents one possible result.
• The height (i.e., longest path) is the lower
  bound.

4
Decision tree model
12
?
gt
23
13
gt
?
?
gt
23
13
lt1,2,3gt
lt2,1,3gt
?
gt
gt
?
lt1,3,2gt
lt3,1,2gt
lt2,3,1gt
lt3,2,1gt
Internal node ij indicates comparison between ai
and aj. suppose three elements lt a1, a2, a3gt with
instance lt6,8,5gt Leaf node lt?(1), ?(2), ?(3)gt
indicates ordering a?(1)? a?(2)? a?(3). Path of
bold lines indicates sorting path for
lt6,8,5gt. There are total 3!6 possible
permutations (paths).
5
Lower bound for comparison sort

• The Longest path is the worst case number of
  comparisons. The length of the longest path is
  the height of the decision tree.
• Theorem 8.1 Any comparison sort algorithm
  requires ?(nlg n) comparisons in the worst case.
• Proof
• Suppose height of a decision tree is h, and
  number of paths (i,e,, permutations) is n!.
• Since a binary tree of height h has at most 2h
  leaves,
• n! ? 2h , so h ? lg (n!) ? ?(nlg n) (By equation
  3.18).
• That is to say any comparison sort in the worst
  case needs at least nlg n comparisons.

6
Maximum O(n)

• MAXIMUM(A)
• max?A1
• for i ?2 to lengthA
• do if maxltAi
• then max ?Ai
• return max

Running time O(n), n-1 comparisons are
sufficient.
Similar for minimum.
7
Lower bound for maximum

• n-1 is the upper bound of maximum,
• How about the lower bound (of worst case)?
• Suppose n elements are distinct.
• So n-1 elements are not maximum.
• Every comparison has only one loser.
• Therefore at least n-1 comparisons are needed.
• So lower bound is n-1.
• This method is called the tournament method.
• Can the decision tree is used for determining the
  lower bound for selection?
• Any one could be the maximum, so there are n
  leaves (output)
• Thus, the height will be ?lg n?,
• If think ?lg n? be the lower bound, it will be
  wrong.
• So decision tree does not apply here.
• Why? ((really n leaves? Duplicate leaves!)

8
Adversary argument

• Playing a guessing game between you and your
  friend.
• You are to pick up a date, and the friend will
  try to guess the date by asking YES/NO questions.
• Your purpose is forcing your friend to ask as
  many questions as possible.
• To question is it in winter, your answer should
  be NO.
• To question is the first letter of the months
  name in the first half of the alphabet? Your
  answer should be YES.
• Idea
• You did not pick up a date in advance at all, but
• Construct a date according to the questions.
• The requirement is that the finally constructed
  date should be consistent to all your answers to
  the questions.
• Looks like cheating, but it is a good way to find
  the lower bound.

9
Adversary argument

• Suppose we have an algorithm we think efficient.
• Image an adversary tries to prove otherwise.
• At each point in the algorithm, whenever an
  decision (i.e., key comparison) is made, the
  adversary tells us the result of the decision.
• The adversary chooses the answer which tries to
  force the algorithm work hard (i.e., do a lot of
  decision, or to say, the answer releases as less
  new information as possible).
• You can think the adversary is constructing a
  bad input while it is answering the questions.
  The only requirement on the answers is that they
  must be internally consistent.
• If the adversary can force the algorithm to
  perform f(n) steps, then f(n) is the lower bound,
  i.e, at least how many steps in the worst case.

10
Adversary argument for maximum

• The adversary answers the questions as treating
  aii, for each i .
• To the query is ailtaj, adversary answers
  YES if (and only if) iltj.
• If the algorithm halts in less than n-1
  comparisons and claims that the maximum is
  located in index k, then there will be one more
  non-loser (non-smaller element). Suppose the
  non-smaller element is located in the index j?k
• Thus, the adversary can thus demonstrate this
  algorithm to be incorrect by claiming that the
  array holds the values ai i for all i ? j and
  aj n1, thus making ak lt aj. A
  contradiction!
• So the algorithm must have at least n-1
  comparisons to determine the maximum.
• So the previous linear scan algorithm for maximum
  is optimal.
• If you design another algorithm, such as first
  divide into n/2 pairs and make comparisons to get
  winners, then divide the winners into pairs and
  make comparisons, , the total number of
  comparisons is still n-1.

11
Adversary argument for finding both maximum and
minimum.

• n distinct elements,
• Count each win and each lose as one unit of
  information.
• One maximum and one minimum, so n-1 loses for
  maximum and n-1 wins for minimum, thus, at least
  2n-2 units of information is needed.
• The adversary gives the answer in the way that
  will give away as few as possible units of new
  information with each comparison.

12
Adversary argumentfour status

• Denote the key status in any moment as
• W has won at least one comparison and never lost
• L has lost at least one comparison and never
  won
• B has both won and lost at least one comparison
• N has not yet participated in a comparison

13
Adversary argument --strategy
Status of x and y in comparison Adversary response New status Units of new information
N, N xgty W,L 2
W,N xgty W,L 1
B, N xgty B,L 1
L,N xlty L,W 1
W,W xgty W,B 1
B,W xlty B,W 0
L,W xlty L,W 0
B,B Original values B,B 0
L,B xlty L,B 0
L,L xlty L,B 1
Each W or L is one unit of info., B 2 units of
info.
Except B,B, either key chosen as winner has not
lost any comparison or as loser has not won any
comparison.
Suppose the comparison is x and y, adversary
chooses x as winner and x never loses. Even x
value assigned previously is less than y,
adversary can increases x to beat y without
conflicting all previous answers.,
14
Adversary argumentproof

• Except N,N case, all other cases release at most
  1 unit of information.
• So the algorithm compares two keys not involved
  in any comparison as much as possible
• Suppose n is even, then divide to n/2 pairs. thus
  obtain n units of information. There needs at
  least n-2 comparison, each for at most 1 unit of
  information, thus, n/2n-23n/2-2.
• If n is odd, (n-1)/2n-1 (3n-3)/2.
• in one formula ?3n/2? -2.

15
Minimum and Maximum 3 ?n/2? comparisons

• MIN-MAX(A) //assume n is odd
• min ?max?A1
• for i ?2 to lengthA step 2
• do if AiltAi1
• then if mingtAi
• then min ? Ai
• if maxltAi1
• then max ? Ai1
• else if mingt Ai1
• then min ? Ai1
• if maxltAi
• then max ? Ai
• return min, max

pairs ?n/2? comparisons/per pair3 total
comparisons 3 ?n/2?, i.e., ?3n/2? -2.
Similarly, write the code for n being even. The
total comparisons is 3n/2-2, i.e., ?3n/2? -2.
This algorithm achieves optimal.