Amortized Analysis

1
Amortized Analysis

• The average cost of a sequence of n operations on
  a given Data Structure. Aggregate Analysis
• Accounting Method
• Potential Method

2
Amortized Analysis

• Amortized analysis computes the average time
  required to perform a sequence of n operations on
  a data structure
• Often worst case analysis is not tight and the
  amortized cost of an operation is less than its
  worst case.
• Analogy (making coffee)

3
Applications of amortized analysis

• Vectors/ tables
• Disjoint sets
• Priority queues
• Heaps, Binomial heaps, Fibonacci heaps
• Hashing
• Reducing the load factor by doubling

4
Difference between amortized and average cost

• To do averages we need to use probability
• For amortized analysis no such assumptions are
  needed
• We compute the average cost per operation for any
  mix of n operations

5
The Data Structure

• The data structure has a set of operations.
• A stack with
• push(), pop() and MultiPop(k).
• Often some operations may be slow while others
  are fast.
• push() and pop() are fast.
• MultiPop(k) may be slow.
• Sometime the time of a single operation can vary

6
Methods

• Aggregate analysis- the total amount of time
  needed for the n operations is computed and
  divided by n
• Accounting - operations are assigned an amortized
  cost. Objects of the data structure are assigned
  a credit
• Potential The prepaid work (money in the
  bank) is represented as potential energy that
  can be released to pay for future operations

7
Aggregate analysis

• n operations take T(n) time
• Amortized cost of an operation is T(n)/n

8
Stack - aggregate analysis

• A stack with operations
• Push, Pop and Multipop.
• Multipop(S, k)
• while not empty(S) and kgt0 do
• Pop(S)
• kk-1
• end while

9
Stack - aggregate analysis

• Push and Pop are O(1) (to move 1 data element)
• Multipop is O(min(s, k)) where
• s is the size of the stack and
• k the number of elements to pop.
• Assume a sequence of n Push, Pop and Multipop
  operations

10
Stack - aggregate analysis

• Each object can be popped only once for each time
  it is pushed
• So the total number of times Pop can be called (
  directly or from Multipop) is bound by the
  number of Pushes ltn.

As comparison using the worst case
analysis Assume the stack size is at most n,
the worst-case time of any stack operation is
O(n), and a sequence of n operation costs O(n2)
11
Stack - aggregate analysis

• A sequence of n Push and Pop operations is
  therefore O(n) and the amortized cost of each is
• O(n)/nO(1)

12
Stack Example
Operation
Stack
Stack
Start
a
push a
a
b
b
push b
a
a
c
c
b
b
push c
a
a
b
Multipop(3)
pop c
a
pop b
a
pop a
6 Operation 6 Moves
4 Operation 6 Moves
13
Accounting Method

• Charge each operation an (invented) amortized
  cost.
• Often different from actual run time cost. Some
  operations may have an amortized cost larger than
  runtime, others less.
• Unlike businesses we do not want to make a profit
• We want to cover the actual cost
• Amount charged but not used in performing an
  operation is stored with objects of the data
  structure

14
Accounting method

• Later operations can use stored amount to pay for
  their actual cost
• Credit balance must not go negative (always
  enough to pay for performance of future
  operations)

15
Stack - amortized analysis

• We assign the amortized costs
• 2 for Push
• 0 for both Pop and Multipop
• For a sequence of n Push and Pop operations the
  total amortized cost is at most 2n or O(n)

16
Stack - amortized analysis

• Each time we do a Push we pay 1 for the actual
  cost of the Push and the element has a credit of
  1.
• Each time an element is popped we take the 1
  credit to pay for it
• Thus the balance is always nonnegative

17
k bit binary counter
// Add 1 (modulo 2k) to the value in the counter

• Initially the counter contains 0
• Eventually it becomes 2k -1
• Next it is reset to 0

• Increment (A)
• i 0
• while i lt lengthA and Ai 1
• Ai 0
• i
• if i lt lengthA
• Ai 1

A
k -1
1 0
18
Aggregate analysis
3 bit counterk3

• Bit 2 1 0
• D No.
• 0 0 0 0
• 0 0 1
• 2 0 1 0
• 3 0 1 1
• 4 1 0 0
• 5 1 0 1
• 6 1 1 0
• 1 1 1
• Flips 2 4 8

• Count number of times a bit is flipped.
• Let number increments n 2k (if n lt 2k analysis
  similar)
• A0 flipped n times
• A1 flipped n/21 times
• Ak - 1 flipped n/2 k-1 times

19

• As comparison using the worst case analysis
• A single execution of INCREMENT takes time O(k)
  in the worst case, in which array A contains all
  1s. Thus a sequence of n INCREMENT operations on
  an initially zero counter takes time O(nk) in the
  worst case.

20
Accounting method

• Charge amortized cost of 2 to set a bit to 1
• When a bit is set to 1, pay 1 for actual cost
  and store 1 with bit
• Note at all times a bit with value 1 has 1
• When a bit is reset to 0 use 1 to pay for actual
  cost
• 2 per Increment operation

21
Accounting method

• Let the value stored in the counter be
• After increment and a payment of
• 2 1 1

1 1 0 0 1 0 0 1 1 1
1 1 0 0 1 0 0 1 1 1
1 1 0 0 1 0 1 0 0 0
1 1 0 0 1 0 1 0 0 0
22

• Amortized Analysis of Disjoint set

23
Review Running Time ofKruskals Algorithm

• Expensive operations
• Sort edges O(E lg E)
• O(V) MakeSet()s
• O(E) FindSet()s
• O(V) Union()s
• Upshot
• Comes down to efficiency of disjoint-set
  operations, particularly Union()

24
Review Disjoint Set Union

• So how do we represent disjoint sets?
• Naïve implementation use a linked list to
  represent elements, with pointers back to set
• MakeSet() O(1)
• FindSet() O(1)
• Union(A,B) Copy elements of A into set B by
  adjusting elements of A to point to B O(A)
• How long could n Union()s take?

25
Review Disjoint Set Union Analysis

• Worst-case analysis O(n2) time for n Unions
• Union(S1, S2) copy 1 element
• Union(S2, S3) copy 2 elements
• Union(Sn-1, Sn) copy n-1 elements
• O(n2)
• Amortized analysis Improvement - always copy
  smaller into larger
• How long would above sequence of Unions take?
• Worst case n Unions take O(n lg n) time
• Proof uses amortized analysis

26
Amortized Analysis of Disjoint Sets

• If elements are copied from the smaller set into
  the larger set, an element can be copied at most
  lg n times
• Worst case Each time copied, element in smaller
  set
• 1st time resulting set size ? 2
• 2nd time ? 4
• (lg n)th time ? n

27
Amortized Analysis of Disjoint Sets

• Since we have n elements each copied at most lg n
  times, n Union()s takes O(n lg n) time
• Therefore we say the amortized cost of a Union()
  operation is O(lg n)
• This is the aggregate method of amortized
  analysis
• n operations take time T(n)
• Average cost of an operation T(n)/n

28

• Amortized Analysis of Dynamic table

29
Dynamic table (object table, hash table, vector,
etc)

• The table is dynamic
• We cant predict its maximum size
• We would like to avoid allocating a lot of unused
  space (reasonable load balance)
• May not be able to avoid table overflow.
• Overflow should not cause run time failure

30
java.util.Vector

• A built in class which is a growable array.
• The user can set
• initialCapacity - the initial capacity of the
  vector.
• capacityIncrement - the amount by which the
  capacity is increased when the vector overflows.
• The default for capacityIncrement is to double
  the size

31
Dynamic table

• Idea Allocate more memory as needed.
• Duplicate the size of the table after each
  overflow.
• After each duplication must copy elements from
  old to new table
• We assume for now the only operation is insert
  and calculate amortized cost
• Table sizes 1, 2, 4, 8, , 2k

32
Aggregate method
Op. Size Cost The table
before after
1 0 1 1 2 1 2 11 3
2 4 21 4 4 4
1 5 4 8 41 6 8
8 1 7 8 8 1
24/9 8 8 8 1 9 8
16 81
1 2 3 4
1 2 3 4 5 6 7 8
1 2
1
1 2 3 4 5 6 7 8 9 . .
Copy
33
Aggregate Analysis

• Let the number of inserts be 2k lt n ? 2k1
• (Note 22k 2k1 lt 2n)
• At this point the size of the table is 2k1
• After n inserts
• The total cost for copy operations only is 1 2
  4 . . . 2k 2k1 - 1 lt 2n
• The total cost for n inserts (without copy) is
  n.
• Total lt 3n
• Therefore the amortized time is O(1).

2k
n
2k
34
Accounting analysis

• Charge each insert 3.
• When the table is not full, use 1 for the cost
  of insert, and store 2 with element
• When the table doubles from m to 2m
• m/2 elements that never moved before have 2
  credit,
• m/2 elements which already moved have 0
• After copy all m elements have 0 credit

35
Accounting analysis
Size 8 4 copied elements with 0
Size 4 2 copied elements with 0 2 new
elements with 2
0 0 0 0 2 2 2 2
0 0 0 0
Size 8 4 copied elements with 0 4 new elements
with 2
0 0 2 2
36
java.util.Vector fixed increment

• Let the number of inserts n satisfy
• c0 (m-1)c lt n ? c0 mc
• So (n- c0 )/c ? m lt1 (n- c0 )/c and m ?(n)
• At this point the size of the array is c0 mc
• After the nth insert
• The total time for copy operations is ?
• The total time for n inserts (without copy) is n.
  

37
java.util.Vector fixed increment
Initial capacity c0
0
Capacity increment c
1
2
m-1
???
Cost for m vector copies mc0 c? (1 2 3
(m-1)) mc0 c?m(m-1)/2 ?(m2) ?(n2)
38
java.util.Vector fixed increment

• After the nth insert
• The total time for copy operations is ?(n2)
• The total time for n inserts (without copy) is n.
  
• Average insert time is ?(n)