Amortized%20Algorithm%20Analysis

1
Amortized Algorithm Analysis

• COP3503
• July 25, 2007
• Andy Schwartz

2
Contents

• Indirect Solutions
• Amortized Analysis Introduction
• Extendable Array Example
• Binomial Queue
• Binomial Queue Example

3
Indirect Solution
10 yards / minute 100 yards / minute
(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
4
Indirect Solution
10 yards / minute 100 yards / minute
Geometric Series?
The easy solution is indirect. It takes a kitten
5 minutes togo 50 yards, how far can the mother
go in 5 minutes?.... 500 yards!
(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
5
Amortized Analysis Introduction

• The worst-case running time is not always the
  same as the worst possible average running time.
• Example
• Worst case-time is O(n)
• Amortized worst-case is O(1)
• This could be from a series of table inserts and
  clears

(Source Arup Guha. CS2 Notes Summer 2007.
6
Amortization Techniques

• Two Techniques
• Potential Function
• Accounting Method
• Start with X dollars for n operations.
• Each simple operation costs 1, but larger
  operations are more expensive.
• If X dollars is enough for n operations, then
  the amortized worst-case is x / n.
• Example School Semester costs (worst-case most
  expensive in Fall, but amortized worst-case
  including spring and summer is cheaper).
• Compute the amortized worst-case running time of
  n consecutive variable-popping stack operations.
  (details on board)

(Source Arup Guha. CS2 Notes Summer 2007.
7
Extendable Array Example

• ArrayList (or Vector) in Java
• Can we maintain an extendable array and keep the
  amortized worst-case running time down to O(1)?
• Yes! (or this would be a bad example)
• How?....
• Simple case add an element and space is left
  over.
• Worst case ?

(Source Arup Guha. CS2 Notes Summer 2007.
8
Extendable Array Example

• Simple Case Add an element and space is left
  over.
• Worst Case Add an element, and there is no more
  space after ? we need to make space ? allocate
  new array and copy elements
• The time complexity of this worst-case depends on
  n, the size of the original array. How big should
  we make the new array?
• There were n-1 simple operations in order to fill
  the array to this point, so lets double it. That
  way a series of n-1 simple insertions, followed
  by one worst case will result in about n-1 n
  2n total operations.
• 2n operations / n add operations 2 O(2)
  O(1)
• There were really more than n-1 operations if
  this was not the first extension, but we know a
  series of those operations were O(1) and O(1)
  O(2) O(1).

O(1)
O(k)
(Source Arup Guha. CS2 Notes Summer 2007.
9
Binomial Queue

• Binomial Trees
• B0 B1 B2
• B3
• B4

Each tree doubles the previous.
10
Binomial Queue

• Binomial Queue
• A queue of binomial trees. A Forest.
• Each tree is essentially a heap constrained to
  the format of a binary tree.
• Example B0, B2, B3
• Insertion Create a B0 and merge
• Deletion remove minimum( the root) from tree Bk.
  This leaves a queue of trees B0, B1, , Bk-1.
  Merge this queue with the rest of the original
  queue.

(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
11
Binomial Queue Example

• The Most important step is Merge
• Merge Rules (for two binomial queues)
• 0 or 1 Bk trees ? just leave merged
• 2 Bk trees ? meld into 1 Bk1 tree.
• 3 Bk trees ? meld two into 1 Bk1, and leave the
  third.
• Insertion Runtime
• M1 steps, where M is smallest tree not present.
  Worst-case is k2, when Bk1 is the smallest tree
  not present. How does k relate to the total
  number of nodes in the tree?
• k lg n, thus (nonamortized) worst-case time is
  O(lg n).

(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
12
Binomial Queue Example

• MakeBinQ Problem Build a binomial queue of N
  elements. (Like makeHeap). How long should this
  take?
• Insertion worst-case runtime
• Worst-case O(lg n) for 1 insert? O(n lg n) n
  inserts, but we want O(n) like makeHeap
• Try amortized analysis directly
• Considering each linking step of the merge. The
  1st, 3rd, 5th, ettc odd steps require no linking
  step because there will be no B0. So ½ of all
  insertions require no linking, similarly ¼
  require 1 linking steps.
• We could continue down this path, but the itll
  be come especially difficult for deletion (we
  should learn an indirect analysis).

(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
13
Binomial Queue Example

• Indirect Analysis (time M 1)
• If no B0 ? cost is 1 (M is 0)
• Results in 1 B0 tree added to the forest
• If B0 but no B1 ? cost is 2 (M is 1)
• Results in same of trees (new B1 but B0 is
  gone)
• When cost is 3 (M is 2)
• Results in 1 less tree (new B2 but remove B0 and
  B1)
• .etc
• When cost is c (M is c 1)
• Results in increase of 2 c trees

(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
14
Binomial Queue Example

• increase of 2 c trees
• How can we use this?
• Ti of trees after ith iteration
• T0 0 of trees initially
• Ci Cost of ith iteration
• Then, Ti Ti-1 (2 Ci) ?
• Ci (Ti Ti-1) 2
• This is only the ith iteration

(Source Mark Allen Weiss. Data Structures and
Algorithm Analysis in Java.
15
Binomial Queue Example

• Ci (Ti Ti-1) 2
• To get all iterations
• C1 (T1 T0) 2
• C2 (T2 T1) 2
• Cn-1 (Tn-1 Tn-2) 2
• Cn (Tn Tn-1) 2
• n
• S Ci (Tn T0) 2n (T1 .. Tn-1 cancel out)
• i1

16
Binomial Queue Example

• n
• S Ci (Tn T0) 2n
• i1
• T0 0 and Tn is definitely not negative, so Tn
  T0 is not negative.
• n
• ?S Ci lt 2n
• i1

Thus, the total cost lt 2n ? makeBinQ
O(n) Since, makeBinQ consists of O(n) inserts,
then the amortized worst-case of each insert is
O(1).