A study of waitfree queue algorithms for interthread communication for Java

1
A study of wait-free queue algorithms for
inter-thread communication for Java

• by
• Daniel Cederman and Tord Dellsén
• Supervisor
• Philippas Tsigas

2
Plan

• Real-Time and synchronization
• Real-Time Java
• Implementations
• Test Results
• Conclusion

3
Real-Time
4
Real-Time

• Predictable and Consistent
• Speed is second priority
• Tasks have cost, deadline...

5
Threads

• Different parts have different criticality
• Threads more appropriate than sequential
  programming
• Given different priorities
• Need to be synchronized

6
Enabled-Late-Write
Shared Variables Counter 0
Thread As Variables i undefined
Thread Bs Variables i undefined
void incCounter() int i counter i i
1 counter i
7
Enabled-Late-Write
Shared Variables Counter 0
Thread As Variables i 0
Thread Bs Variables i undefined
void incCounter() int i counter i i
1 counter i
8
Enabled-Late-Write
Shared Variables Counter 0
Thread As Variables i 1
Thread Bs Variables i undefined
void incCounter() int i counter i i
1 counter i
9
Enabled-Late-Write
Shared Variables Counter 0
Thread As Variables i 1
Thread Bs Variables i 0
void incCounter() int i counter i i
1 counter i
10
Enabled-Late-Write
Shared Variables Counter 0
Thread As Variables i 1
Thread Bs Variables i 1
void incCounter() int i counter i i
1 counter i
11
Enabled-Late-Write
Shared Variables Counter 1
Thread As Variables i 1
Thread Bs Variables i 1
void incCounter() int i counter i i
1 counter i
12
Enabled-Late-Write
Shared Variables Counter 1
Thread As Variables i 1
Thread Bs Variables i 1
void incCounter() int i counter i i
1 counter i
13
Enabled-Late-Write
Shared Variables Counter 1
Thread As Variables i 1
Thread Bs Variables i 1
void incCounter() int i counter i i
1 counter i
14
Locking

• Use mutual exclusion allow only one thread
  access at a time
• Easy just add the keyword synchronized in Java
• Disadvantages
• Deadlock
• Priority Inversion

15
Deadlock
R1
T1
R2
T2
16
Priority Inversion
17
Priority Inversion
Locks R
18
Priority Inversion
Waits for R
Locks R
19
Priority Inversion
Waits for R
Runs
Locks R
Unlocks R
20
Priority Ceiling Emulation Protocol

• Priority Inversion can be solved using PCEP
• Whenever a thread acquires a lock it increases
  its priority to the maximum priority, the ceiling
• No other thread can now interrupt the thread with
  the lock
• When the thread releases the lock it lowers its
  priority to its original value

21
Lock-free

• Most of the time there is no conflict
• and when there is, we can just try again
• but we need to know when a conflict has occured

22
Compare-And-Swap (CAS)

• Atomic CPU instruction
• CAS takes three arguments value (shared
  variable), oldvalue (local) and newvalue (local)
• It compares oldvalue with value and if they
  match, updates value to newvalue

23
Compare-And-Swap
while(!written) oldValue value newValue
doStuff(oldValue) written CAS(value,
oldValue, newValue)
24
Compare-And-Swap
while(!written) oldValue value newValue
doStuff(oldValue) written CAS(value,
oldValue, newValue)
25
Problem

• Unbounded
• A thread might have to retry an indefinite amount
  of times
• Real-time systems must be predictable

26
Wait-Free

• Like lock-free, but more suitable for real-time
  because of predictability
• Bounded
• More complex
• Uses helping More on this later
• Increase priority every time to convert lock-free
  to wait-free

27
RTSJ
28
RTSJ

• Real-Time Specification for Java
• The Real-time for Java Expert Group
• Implementation created by TimeSys

29
RTSJ

• Real-Time threads
• Schedulers
• High resolution timing
• Special memory types
• Wait-free queues for inter-thread communication

30
Wait-Free Queues

• Used for one-way communication between threads
• Java classes WaitFreeWriteQueue and
  WaitFreeReadQueue
• It has a read and a write function
• One thread writes to the queue and the other one
  reads from the queue
• Only one side is wait-free either the read or
  the write function the other side is blocking

31
Example using the queue
32
Thread B writes to the queue
33
Thread C writes to the queue
34
Thread C writes to the queue
35
Thread Z reads from the queue
36
Implementations
37
The Reference Implementation

• Made by TimeSys
• Works as a blocking queue
• Uses PCEP
• This is allowed by the specification

38
The TimeSys algorithm
39
The Anderson algorithm
40
James Anderson et al.'s algorithm

• Uses a linked list with two sentinels

Head
Tail
A
B
41
Writing to the queue

• First the object to be written, C, is stored in a
  node

Head
Tail
A
B
C
42
Writing to the queue

• The new node is made to point at the same node
  that the head node is pointing at using CAS

Head
Tail
A
B
C
43
Writing to the queue

• Finally the head nodes pointer is changed to
  point to the new node using CAS

Head
Tail
A
B
C
44
Reading from the queue

• First the third last node is found by traversing
  the queue from the head

Head
Tail
A
B
45
Reading from the queue

• Its pointer is changed so that it points to the
  tail node using CAS

Head
Tail
A
B
46
Reading from the queue

• Node B is no longer in the linked list and can be
  read

Head
Tail
A
47
Compare-and-swap

• CAS is not yet available in Java.
• We emulate CAS using an algorithm described by
  Rammamurthy et. al.
• This emulation is unfortunately very slow and
  memory intensive

48
Wait-free?

• Using CAS could perhaps be enough to make the
  queue lock-free, but how to make it wait-free?
• We make use of helping using a method described
  by Anderson et. al. called incremental helping

49
Incremental Helping

• Have we interrupted another write or read?
• If so, perform the other threads read or write
  first
• Announce that we are perfoming a read or a write
• Perform the read or write
• Remove the announcement

50
Incremental Helping

• If we should be interrupted our operation will be
  done by the interrupting thread.
• This means that our operation is guaranteed to
  succeed whenever we call the write or read
  function
• Either we do it ourselves or get helped by
  another thread

51
Incremental Helping

• Task C announces its read operation but is
  interrupted before it could complete it

52
Incremental Helping

• Task B sees that Cs operation isnt completed so
  it begins helping it, but is interrupted by A

53
Incremental Helping

• A helps C and does its own operation. Since B
  hasnt announced its operation it is not helped

54
Incremental Helping

• When A is finished B notices that C has already
  been helped so it performs its own operation

55
Incremental Helping

• When B finishes task C sees that it has already
  been helped so its done

56
The ptyz algorithm
57
Philippas Tsigas and Yi Zhangs Algorithm

• We call it ptyz
• It also uses linked lists
• Only one side is wait-free as in the
  specification
• We only describe the wait-free write queue, the
  wait-free read queue works in a similar way
• Helps all, not just one (not incremental helping
  like anderson)

58
Wait-Free Write Queue

• Instead of only one tail it has a tail for each
  priority, but only one is the actual tail of the
  linked list

Head
C
1
B
A
D
2
3
59
Reading from the queue

• Reading is simple since its blocking, just make
  the head variable point to the next node in the
  list

Head
C
1
B
A
D
2
3
60
Reading from the queue

• Like this!

A
C
1
B
D
2
Head
3
61
Writing to the queue

• First we need to find the actual tail.

Head
C
1
B
A
D
2
3
62
Writing to the queue

• We go through the tail array looking for the node
  that doesnt point to another node

Head
C
1
B
A
D
2
3
63
Writing to the queue

• We make it point to the new node

Head
C
1
B
A
D
2
E
3
64
Writing to the queue

• Then we store it in the tail array at the threads
  priority, in this case 1

Head
E
1
B
A
D
2
C
3
65
Writing to the queue

• By doing this we avoid the enabled-late-write
  that could have occured when changing the tail

Head
E
1
B
A
D
2
C
3
66
Test Results
67
Test Setup
68
Test Results

• Software emulated CAS
• Has to help reads as well as writes
• Read has to go through the whole list

69
Ptyz vs TimeSys

• Timesys uses pcep

• TimeSys uses PCEP
• Cant interrupt lower priority threads

70
Memory usage

• Timesys uses an array
• Anderson
• Ptyz

71
Conclusions

• Anderson is slow (software emulated CAS, etc)
• Timesys has better memory performance than ptyz
• Ptyz is faster on high priorities
• Ptyz is wait-free

72
Finally

• The Real-Time Java Specification is available on
  www.rtj.org (no s)
• The thesis (and some other material including
  this presentation) www.dtek.chalmers.se/d00ceder
  /exjobb/
• Questions?