Abstract Data Types Stack, Queue Amortized analysis

1
Abstract Data TypesStack, QueueAmortized
analysis
Cormen Ch 10, 17 (11, 18)
2
ADT is an interface

• It defines
• the type of the data stored
• operations, what each operation does (not how)
• parameters of each operation

3
ADT
4
ADT how to work with

• Advantage
• Application programming can be done INDEPENDENTLY
  of implementation
• If care about efficiency (complexity)
• BE CAREFUL!!
• Using the ADT the wrong way might become quite
  inefficient!!!

5
Example Stacks

• Push(x,S) Insert element x into S
• Pop(S) Delete the last (time) element inserted
  into S
• Empty?(S) Return yes if S is empty
• Top(S) Return the last element inserted into S
• Size(S)
• Make-stack()

6
The Stack Data Abstraction
7
The Stack Data Abstraction
push
push
Last in, First out.
push
pop
8
The Stack Applications??

1. REAL-LIFE
2. Rifle
3. Some document handling?

Last in, First out.

1. Computers/ Communications
2. In some applications LIFO preferred on FIFO
3. Program control algorithm a KEY

9
A stack application
Infix Postfix (2 3) 5 2 3 5 ( (5 (7 / 3)
) (2 7) ) 5 7 3 / 2 7 -

• Evaluate an expression in postfix or Reverse
  Polish Notation

10
A stack application
2 3 5
3
2
11
A stack application
2 3 5
5
5
12
A stack application
2 3 5
25
13
A Word on Pseudo-code

• Pseudo ?????
• Combination of
• Programming (like) commands
• free style English
• Allows to conveniently express and algorithm.
• See Cormen (one page) for his language
  constructs.
• No need for FORMAL syntax. (can deviate Cormen)

14
Pseudo-code
S ? make-stack() while ( not eof ) do B ?
read the next data if B is an operand then
push(B,S) else X ? pop(S) Y
? pop(S) Z ? Apply the operation B
on X and Y push(Z,S) return(top(S)
)
15
Implementation

• We will be interested in algorithms to implement
  the ADT..
• And their efficiency..

16
Using an array
t
12
1
3
A
A1
A2
AN-1
A0
The stack is represented by the array A and
variable t
3
1
12
17
Using an array
t
12
1
3
A
A1
A2
AN-1
A0
The stack is represented by the array A and
variable t
make-stack() Allocates the array A, which is of
some fixed size N, sets t ? -1
18
Operations
t
12
1
3
A
AN-1
A1
A2
A0
size(S) return (t1)
empty?(S) return (t lt 0)
top(S) if empty?(S) then error
else return At
19
Pop
t
12
1
3
A
AN-1
A1
A2
A0
pop(S) if empty?(S) then error
else e ?At t ? t 1
return (e)
pop(S)
20
Pop
t
12
1
3
A
AN-1
A1
A2
A0
pop(S) if empty?(S) then error
else e ?At t ? t 1
return (e)
pop(S)
21
Push
t
12
1
3
A
AN-1
A1
A2
A0
push(x,S) if size(S) N then error
else t ?t1 At ?
x
push(5,S)
22
Push
t
12
1
5
A
AN-1
A1
A2
A0
push(x,S) if size(S) N then error
else t ?t1 At ?
x
push(5,S)
23
Implementation with lists
top
size3
5
12
1
24
Implementation with lists
top
size3
5
12
1
make-stack() top ? null
size ? 0
25
Operations
top
size3
5
12
1
size(S) return (size)
empty?(S) return (top null)
top(S) if empty?(S) then error
else return top.element
26
Pop
top
size3
5
12
1
pop(S) if empty?(S) then error
else e ?top.element top ?
top.next size ? size-1
return (e)
pop(S)
27
Pop
top
size2
5
12
1
pop(S) if empty?(S) then error
else e ?top.element top ?
top.next size ? size-1
return (e)
pop(S)
28
Garbage collection
top
size2
5
1
pop(S) if empty?(S) then error
else e ?top.element top ?
top.next size ? size-1
return (e)
pop(S)
29
Push
top
size2
5
1
push(x,S) n new node
n.element ?x n.next ? top
top ? n size ?
size 1
push(5,S)
30
Push
top
size2
5
1
5
push(x,S) n new node
n.element ?x n.next ? top
top ? n size ?
size 1
push(5,S)
31
Push
top
size2
5
1
5
push(x,S) n new node
n.element ?x n.next ? top
top ? n size ?
size 1
push(5,S)
32
Push
top
size3
5
1
5
push(x,S) n new node
n.element ?x n.next ? top
top ? n size ?
size 1
push(5,S)
33
Analysis

• Bound the running time of an operation on the
  worst-case
• As a function of the size, n, of the data
  structure
• Example T(n) lt 4n7
• Too detailed, most cases we are just interested
  in the order of growth

34
Why order of growth

• Precise cost can change from computer to computer
  
• From Programmer to programmer
• Anyhow MOORE ? may go down by factor of 1.5 -2
  next year

35
Big-O
- ???? c ?- ?? ?
?????
36
Big-O
37
More examples

• 4n O(n2)
• 4n2 O(n2)
• 2n O(n10)
• 10 O(1)
• 100n310n O(n3)
• log2(n) O(log10(n))

38
The running time of our stack and queue operations

• Each operation takes O(1) time

39
Stacks via extendable arrays

• Array implementation difficulties
• Pick large N wasted space ?
• Pick small N stack is limited ?
• Solution
• Pick moderate N, and
• When the array is full we will double its size
  (N) (DOUBLING)

40
Push
t
12
1
A
AN-1
A0
A1
push(x,S) / N is size of array/
if size(S) N then
allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x
41
Push
t
12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
push(x,S) if size(S) N
then allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x

push(5,S)
42
Push



t
12
1
3
3
4
5
7
3
2
8
1
A
A0
A1
push(x,S) if size(S) N
then allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x

push(5,S)
43
Push



12
1
3
3
4
5
7
3
2
8
1
t
12
1
3
3
4
5
7
3
2
8
1
A
A0
A1
push(x,S) if size(S) N
then allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x
push(5,S)
44
Push
t
12
1
3
3
4
5
7
3
2
8
1



A
A2N-1
A0
A1
push(x,S) if size(S) N
then allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x
push(5,S)
45
Push
t
12
1
3
3
4
5
7
3
2
8
1
5


A
A2N-1
A0
A1
push(x,S) / N is array size /
if size(S) N then
allocate a new array of size 2N
copy the old array to the new one
N ? 2N t ?t1
At ? x
push(5,S)
46
Analysis

• An operation may take O(n) worst case time !
• But that cannot happen often..

47
Amortized Analysis

• How long it takes to do m operations in the worst
  case ?
• Well, O(nm)
• Yes, but can it really take that long ?

48
(No Transcript)
49
x
50
x
x
51
x
x
x
x
x
52
x
x
x
x
x
x
53
x
x
x
x
x
x
x
x
x
x
x
54
x
x
x
x
x
x
x
x
x
x
x
x
55
x
x
x
x
x
x
x
x
x
x
x
x
x
56
x
x
x
x
x
x
x
x
x
x
x
x
x
x
57
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
58
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
59
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
60
x
x
x
x
N
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Let N be the size of the array we just copied
(half the size of the new array)
After N-1 pushes that cost 1 we will have a push
that costs 2N1
Total time 3N for N pushes, 3 per push
61
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Start from the second row z2
3(241)34
1(221)32
7(281) 38
z8
z4
Total cost O(m) !!
62
We proved
Theorem A sequence of m operations on a stack
takes O(m) time
63
Amortized Analysis (The banks view)

• You come to do an operation with a bunch of
  tokens (amortized cost of the operation)

64
Operation
65
You have to pay for each unit of work by a token
(from your POCKET)
Operation
66
Operation
67
If you have more tokens than work you put the
remaining tokens in the bank
Operation
68
Operation
69
If we have more work than tokens we pay with
token from the bank
Operation
70
If we have more work than tokens we pay with
tokens from the bank
Operation
71
Amortized Analysis (The banks view)

• Suppose we prove that the bank is never in a
  deficit ?

• The total work is no larger than the total number
  of tokens

• 3m in our example if m is the number of
  operations

72
For a proof

• Define exactly how each operation pays for its
  work, and how many tokens it puts in the bank
• Prove that the balance in the bank is always
  non-negative
• Count the total of tokens thats your bound

73
Easy push

• Each push comes with 3 tokens (pocket)
• If its an easy push then we pay with one token
  for the single unit of work, and put two in the
  bank.

t


12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
74
Easy push

• Imagine that the tokens in the bank are placed on
  the items of the stack
• We put one token on the item just inserted and
  another token on an arbitrary item without a
  token

t
5


12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
75
t
5
6

12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
76
t
5
6
6
7
1
10
4
67
2
5
7
12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
77
hard push

• Tokens from the bank pay for copying the array
  into the larger array

t
5
6
6
7
1
10
4
67
2
5
7
12
1
3
3
4
5
7
3
2
8
1
A
AN-1
A0
A1
78
hard push

• Then you pay a token and put two in the bank as
  an easy push

t
5
6
6
7
1
10
4
67
2
5
7
12
1
3
3
4
5
7
3
2
8
1

A
79
Need to prove
Starting with empty array

• The balance is never negative
• When we get to an expensive push there are enough
  tokens to pay for copying
• By induction prove that after the i-th push
  following a copying there are 2i tokens in the
  bank
• N/2 easy pushes before the hard push ? N tokens
  are at bank at the hard push.

80
How many tokens we spent ?

• Each operation spent 3 tokens

81
Summary

• So the total of tokens is 3m
• THM A sequence of m operations takes O(m) time
  !! (we finished the proof)
• Each operation takes O(1) amortized time

82
Theorem A sequence of m operations on a stack
takes O(m) time
proof (3) .

• We formalize the bank as a potential function

83
Let ? be a potential function (can represent the
amount in the bank)
Define
Amortized(op) actual(op) ??
Out of pocket on Operation on ? (bank)
84
Out of pocket on Operation on ? (bank)
Amortized(op1) actual(op1) ?1- ?0
Amortized(op2) actual(op2) ?2- ?1

Amortized(opm) actual(opm) ?m- ?(m-1)
?iAmortized(opi) ?iactual(opi) ?m- ?0
?iAmortized(opi) ? ?iactual(opi) if ?m- ?0 ? 0
We need to find ? that gives a nontrivial
statement
85
Example Our extendable arrays
Define the potential of the stack
86
Without copying (easy)
Amortized(push) actual(push) ??
1 2(t1) N - (2(t1) N)
12(t-t) 3
With copying (hard)
Amortized(push) N 1 ??
N 1 (2 - N) 3
N 1 (?after - ?before)
Amortized(pop) 1 ??
1 2(t-1) N - (2t N) -1
Can we gain (make money)? No! pop push ?
amortzgt 2
87
Amortized(op1) actual(op1) ?1- ?0
Amortized(op2) actual(op2) ?2- ?1

Amortized(opn) actual(opn) ?n- ?(n-1)
?iAmortized(opi) ?iactual(opi) ?n- ?0
3m
?iAmortized(opi) ? ?iactual(opi) if ?n- ?0 ? 0
88
Summary

• So the total of tokens is 3m
• ? THM A sequence of m operations starting from
  an empty stack takes O(m) time !!
• Each operations take O(1) amortized time

89
Queue

• Inject(x,Q) Insert last element x into Q
• Pop(Q) Delete the first element in Q
• Empty?(Q) Return yes if Q is empty
• Front(Q) Return the first element in Q
• Size(Q)
• Make-queue()

90
The Queue Data Abstraction
91
The Queue Data Abstraction
inject
inject
First in, First out (FIFO).
pop
92
Using an array
t
12
1
4
2
5
A
A1
A2
AN-1
A0
pop(Q)
93
Using an array
t
1
4
2
5
A
A1
A2
AN-1
A0
pop(Q)
94
Using an array
t
1
4
2
5
A
A1
A2
AN-1
A0
This would be inefficient if we insist that
elements span a prefix of the array USE a MOVING
ARRAY
95
Using an array
r
f
12
1
4
2
5
A
A1
A2
AN-1
A0
r
f
A
A1
A2
A0
Empty queue fr
96
Using an array
r
f
12
1
4
2
5
A
A1
A2
AN-1
A0
pop(Q)
97
Using an array
r
f
1
4
2
5
A
A1
A2
AN-1
A0
pop(Q)
inject(5,Q)
98
Using an array
r
f
1
4
2
5
5
A
A1
A2
AN-1
A0
pop(Q)
inject(5,Q)
inject(5,Q)
99
Using an array
r
f
1
4
2
5
5
5
A
A1
A2
AN-1
A0
pop(Q)
inject(5,Q)
inject(5,Q)
pop(Q)
pop(Q)
100
Using an array
r
f
2
5
5
5
A
A1
A2
AN-1
A0
pop(Q)
inject(5,Q)
inject(5,Q)
pop(Q)
pop(Q)
pop(Q), inject(5,Q), pop(Q), inject(5,Q),.
101
Using an array
r
f




5
5
5
5

A
A1
A2
AN-1
A0
pop(Q)
inject(5,Q)
inject(5,Q)
pop(Q)
pop(Q)
pop(Q), inject(5,Q), pop(Q), inject(5,Q),.
Fall off the edge? ?
102
Make the array circular
r
f
5
5



5
5
A
A1
A2
AN-1
A0
Pop(Q), inject(5,Q), pop(Q), inject(5,Q),.
103
Operations
r
f
1
4
2
5
A
A1
A2
AN-1
A0
empty?(Q) return (f r)
top(Q) if empty?(Q) then error
else return Af
104
Operations
r
f
1
4
2
5
A
A1
A2
AN-1
A0
size(Q) if (r gt f) then return (r-f)
else return N-(f-r)
105
Operations
r
f
5
5



5
5
A
A1
A2
AN-1
A0
size(Q) if (r gt f) then return (r-f)
else return N-(f-r)
106
Pop
r
f
1
4
2
5
A
A1
A2
A0
pop(Q) if empty?(Q) then error
else e ?Af f ? (f 1) mod
N return (e)
pop(Q)
107
Pop
r
f
1
4
2
5
A
A1
A2
A0
pop(Q) if empty?(Q) then error
else e ?Af f ? (f 1) mod
N return (e)
pop(Q)
108
Inject
r
f

4
2
5
A
A1
A2
A0
Why?
inject(x,Q) if size(Q) N-1 then error
else Ar ? x
r ? (r1) mod N
fr? empty
inject(5,Q)
109
Inject
r
f

4
2
5
5
A
A1
A2
A0
inject(x,Q) if size(Q) N-1 then error
else Ar ? x
r ? (r1) mod N
Or could do doubling
inject(5,Q)
110
Implementation with lists
head
size3
5
12
1
tail
inject(4,Q)
111
Implementation with lists
head
size3
4
5
12
1
tail
inject(4,Q)
112
Implementation with lists
head
size3
4
5
12
1
tail
inject(4,Q)
Complete the details by yourself
113
Continue in DS2.Slide 26
114
Double ended queue (deque)

• Push(x,D) Insert x as the first in D
• Pop(D) Delete the first element of D
• Inject(x,D) Insert x as the last in D
• Eject(D) Delete the last element of D
• Size(D)
• Empty?(D)
• Make-deque()

115
Implementation with doubly linked lists
head
tail
size2
13
5
116
Empty list
head
tail
size0
We use two sentinels here to make the code simpler
117
Push
head
tail
size1
5
push(x,D) n new node
n.element ?x n.next ?
head.next (head.next).prev ? n
head.next ? n
n.prev? head size ? size
1
118
head
tail
size1
5
push(x,D) n new node
n.element ?x n.next ?
head.next (head.next).prev ? n
head.next ? n
n.prev? head size ? size
1
push(4,D)
119
head
tail
size1
5
push(x,D) n new node
n.element ?x n.next ?
head.next (head.next).prev ? n
head.next ? n
n.prev? head size ? size
1
push(4,D)
120
head
tail
size1
5
push(x,D) n new node
n.element ?x n.next ?
head.next (head.next).prev ? n
head.next ? n
n.prev? head size ? size
1
push(4,D)
121
head
tail
size2
5
push(x,D) n new node
n.element ?x n.next ?
head.next (head.next).prev ? n
head.next ? n
n.prev? head size ? size
1
push(4,D)
122
Implementations of the other operations are
similar

• Try by yourself