xCalls: Safe I/O in Memory Transactions

1
xCalls Safe I/O in Memory Transactions

• Haris Volos,
• Andres Jaan Tack, Neelam Goyal,
• Michael Swift, Adam Welc

University of Wisconsin - Madison


2
Transactional Memory (TM)

• CMP leads to more concurrency within programs
• Synchronizing access to shared data via locks is
  hard
• atomic construct simplifies synchronizing access
  to shared data

Thread 1
Thread 2
Isolation OBSERVE both red transactions updates
to A and B or none
atomic B B 10 A A 10
atomic x x - c y y c
LOCK(L) x x - c y y c UNLOCK(L)
store(A) store(B) store(A)
atomic A A 20 B B 20
Atomicity PERFORM both updates to A and B or none
Conflict
Abort blue
ABORT conflicting transactions
3
A challenging world

• Real world programs frequently take actions
  outside of their own memory
• Firefox 1 critical sections do system call
  Baugh TRANSACT 07
• Most TM systems apply only to user-level memory

Thread 1
memory
file
Thread 2
Memory updates dropped
Interleaved writes
1a
atomic item procItem(queue) write (file,
principal) write (file, item-gtheader)
atomic item procItem(queue) write (file,
principal) write (file, item-gtheader)
2b
2b
1a
2a
1a
File writes not dropped
1b
2b
Abort
4
State of the art

• Defer
• Undo
• Global Lock

Ignore failures
NAS
Stop the world
atomic item procItem(queue) write (file,
principal) write (file, item-gtheader) send
(socket, item-gtbody)
LAN
Internet
Defer
COMMIT Perform send
5
Contribution

• xCall programming interface
• Exposes transactional semantics to programmer
• Enables I/O within transactions w/o stopping the
  world
• Exposes all failures to the program

Transactional Program
xCalls
Legacy calls
xCall Library
Runs in user mode
System call interface
Transaction-unaware kernel
6
Outline

• Motivation
• xCall Design Implementation
• Evaluation
• Conclusion

7
Design overview

• Principles
• As early as possible but not earlier
• Expose all failures
• Components
• Atomic execution
• Isolation
• Error handling

8
Atomic execution

• Provide abort semantics for kernel data and I/O
• Expose to programmer when action is performed

file
buffers
atomic item procItem(queue) x_write
(file, principal) x_write (file,
item-gtheader)
Abort
9
Isolation

• Prevent conflicting changes to kernel data made
  within a transaction
• Sentinels
• Revocable user-level locks
• Lock logical kernel state visible through system
  calls

Thread 1
memory
file
Thread 2
atomic item procItem(queue) x_write
(file, principal) x_write (file,
item-gtheader)
atomic item procItem(queue) x_write
(file, principal) x_write (file,
item-gtheader)
Conflict
10
Error handling

• Some errors might not happen until transaction
  commits or aborts
• Inform programmer when failures happen
• Handle errors after transaction completes

Deferred send FAILED err3 error
atomic item procItem(queue) x_write
(file, principal, err1) x_write (file,
item-gtheader, err2) x_send (socket,
item-gtbody, err3) if (err1 err2 err3)
/ CLEANUP /
LAN
Internet
Defer
COMMIT Perform send
Handle error here
11
Example file write
Error handling

• ssize_t x_write (int fd, void buf,
• ssize_t nbytes )
• void localbuf
• ssize_t bytes
• get_sentinel(fd)
• localbuf alloc_local_buf(nbytes)
• read(fd, localbuf, nbytes)
• bytes write(fd, buf, nbytes)
• if (bytes ! -1)
• compensate(x_undo_write, fd, localbuf,
• bytes, result)
• int x_undo_write (int fd, void buf,
• ssize_t nbytes, int result)
• off_t ret1, ret2
• lseek(fd, -nbytes, SEEK_CUR)
• if (ret1)
• pwrite(fd, buf, nbytes, ret1)
• if (ret1 -1 ret2 -1)

, int result
Isolation
Atomic execution
Atomic execution
Atomic execution
ret1
ret2
Error handling
12
Summary

• xCall API exposes transactional semantics
• Atomicity
• Isolation
• Error handling
• Prototype implementation
• Executes as user-mode library
• Relies on Intel STM for transactional memory
• Provides 27 xCalls including file handling,
  communication, threading

13
Outline

• Motivation
• xCall Design Implementation
• Evaluation
• Benefit of xCalls over global lock
• Benefit of TM over locks
• Conclusion

14
Evaluation platform

• Transactified three large multithreaded apps
• Berkeley DB
• BIND
• XMMS
• Configurations
• Native locks system calls
• STM transactions system calls global lock
• xCalls transactions xCalls
• Run on 4 quad-core 2 GHz AMD Barcelona

15
Performance Berkeley DB (1/2)

• Workload TPC-C

• xCalls scales better than STM with global lock
• TM worse than locks due to STM overhead

16
Performance Berkeley DB (2/2)

• Workload Lockscale

• xCalls improve concurrency over coarse grain lock
• Global lock kills optimistic concurrency

17
Performance BIND

• Workload QueryPerf

• Transactions scale better than coarse grain locks
• xCalls enable additional concurrency

18
Performance summary

• xCalls benefit programs with I/O concurrency
• TM benefits programs with contended but not
  conflicting critical sections

19
Conclusion

• xCall programming interface
• Brings common OS services to TM programs
• Requires no kernel modifications
• Improves scalability over state of the art

Questions?