Challenges in multithread synchronization Winter Meeting 2004

1
Challenges in multi-thread synchronization-
Winter Meeting 2004 -

• Håkan Sundell
• PhD-Student for Philippas Tsigas since 1999-
  Licentiate in 2002- PhD planned for 2004

2
Why Synchronization

• Multi-thread programs work with shared data
• Alt 1. Message Passing
• Alt 2. Shared Memory

T2
T1
T3
T2
? ? ?? ? ?
T1
T3
3
Shared Memory
CPU
CPU
CPU
. . .
Cache
Cache
Cache
Memory
- Uniform Memory Access (UMA)
...
...
...
CPU
CPU
CPU
CPU
CPU
CPU
. . .
Cache bus
Cache bus
Cache bus
Memory
Memory
Memory
- Non-Uniform Memory Access (NUMA)
4
Memory Consistency

• Example scenario
• Models Relaxed Memory Order, Sequential
  Consistency, etc.

Ti
W(x,1)
R(y)0
W(x,0)
R(y)1
Tj
W(y,0)
W(y,1)
R(x)1
Tk
R(x)1
R(y)1
R(x)0
t
5
Linearizability

• All concurrent executions can be transformed into
  an equivalent serial sequence of atomic
  operations preserving the partial order

Ti
Write
Tj
Read
Tk
Write
t
Ser
6
Hardware Synchronization Primitives

• Consensus 1
• Atomic Read/Write
• Consensus 2
• Atomic Test-And-Set (TAS), Fetch-And-Add (FAA)
• Consensus Infinite
• Atomic Compare-And-Swap (CAS)
• Atomic Load-Linked/Store-Conditionally

Read
Read
Write
Mf(M,)
7
Mutual Exclusion

• Access to shared data will be atomic because of
  lock
• Reduced Parallelism by definition
• Blocking, Danger of priority inversion and
  deadlocks.
• Solutions exists, but with high overhead,
  especially for multi-processor systems

P1
P2
P3
8
Non-blocking Synchronization

• Perform operation/changes using atomic primitives
• Lock-Free Synchronization
• Optimistic approach
• Retries until succeeding
• Wait-Free Synchronization
• Always finishes in a finite number of its own
  steps
• Coordination with all participants

9
Non-Blocking Dynamic Memory Management

• i.e. Malloc/Free
• Fixed block size, Fixed purpose
• Lock-Free
• Wait-Free (to be published)
• Any block size, Any/Fixed purpose
• Open problem

10
Reference Counting

• Question
• When to free objects?
• How to de-reference pointers safely?
• Solutions (in cooperation with dynamic memory
  manager)
• Lock-Free
• Wait-Free (to be published)

P2
P1
P3
11
Software Synchronization Primitives

• Atomic Read/Write (WF with RT-requirements
  published).
• Multi-variable Read/Write, Snapshot (WF with
  RT-requirements published).
• LL/SC (LF to be published)
• Multi-word Compare-And-Swap (CASN) i.e.
  transactions
• Lock-Free
• Wait-Free (to be published)
• LL/SCN

12
Shared Data Structures

• General LF/WF schemes exist to implement all data
  structures. However, not practical
• NOBLE library (published)
• Lock-Free
• Stack
• Queue
• Priority Queue (published)
• Dictionary (published)
• Deque (to be published)
• Singly Linked Lists
• Doubly Linked Lists (to be published)

13
Evaluation of Non-Blocking Synchronization

• Experiments on parallel machines, 2,4,64 CPUS
  with UMA/NUMA architectures
• Stress tests of data structures (published)
• Benchmark applications
• Real applications
• Open subject

14
Clusters and Grids
...
...
...
CPU
CPU
CPU
CPU
CPU
CPU
. . .
Cache bus
Cache bus
Cache bus
Memory
I/O
Memory
I/O
Memory
I/O

• Distributed Shared Memory
• Atomic synchronization primitives?
• Relaxed Memory Models?
• Modification of non-blocking algorithms?