Taking Transactions Mainstream: Social Failure Modes and Recovery

1
Taking Transactions Mainstream Social Failure
Modes and Recovery

• Don Box, Microsoft Corp

2
Disclaimers

• This talk is mostly looking backwards
• Future is way less clear than the past
• Lots of interesting data to be harvested
• STM may look more like the past than we think
• Microsoft stack used in examples
• Its the one I know best
• Ill qualify where things are different
• When in doubt, assume we have it worse

3
Microsoft Transactions To Date
4
Transactional Mechanics 101

• Three Party System
• Application establishes transaction boundaries
  and initiates work
• Resource Manager (RM) performs transactional
  work on behalf of application
• Transaction Manager (TM) coordinates outcome
  with application and resource managers
• Resource manager may be durable (survives
  crashes) or volatile (doesnt)

5
Isolation and the Three Parties

• The Application specifies the desired isolation
  level when it asks the TM for new transaction
• RMs discover isolation level from transaction at
  enlistment-time
• RMs implement isolation however they see fit
• May use two-phase locking
• May use multi-versioning
• May provide RM-specific overrides

6
Three Parties and Performance

• In the limit, every party is on a distinct box
• Lots of marshaling and x-host communication
• If theres only one RM, no 2PC needed
• TM delegates commit to RM
• If all resources are volatile, TM neednt log
• If all resources are volatile and in-proc, 2PC
  reduces to virtcalls

7
Transactional Mechanics 102

• Transactions protect operations on a managed
  resource
• Some resource managers dynamically enlist with
  the transaction at the time of invocation
• Op1(tx, args)
• Op2(tx, args)
• Some resource managers statically enlist a
  session with the transaction
• Bind(conn, tx)
• Op1(conn, args)
• Op2(conn, args)

8
Static vs. Dynamic Enlistment

• Dynamic enlistment is what people expect
• Transactions are temporal phenomena
• Transactions flow with thread of control
• Static enlistment was a performance optimization
• Most early RMs were x-proc/x-host
• Marshaling transactions non-trivial due to
  logging
• Matters get worse when transactions are passed
  implicitly
• Does op use cached tx, current tx, or no tx?
• State of the practice is RTFM

9
The App Server Era

• 1996-2006 App Server era
• OLTP Distributed Objects the Web
• Microsoft MTS DCOM IIS/ASP
• Sun et al EJB RMI Servlets/JSP
• Emblematic features
• Declarative Transactions
• Managed execution/deployment environment
• N-tier design style (transaction composition)

10
Declarative Transactions

• Captured transactional requirements as data
• The system guaranteed that your code ran with
  an implicit transaction (typically in TLS)
• Your implicit transaction propagated to other
  chunks of code you called by default
• Your code gets to influence transaction outcome
  via implicit context

11
Declarative Transactions in MTS
int MyMethod(string name) GetObjectContext().S
etAbort() int id CreateUser(name)
AuditChange(id, name " was added")
GetObjectContext().SetComplete() return id
Class MyClass TransactionRequired
12
Trouble in Paradise?

• Previous example shows the ideal
• Declaration of intent visible to system
• Work composes without manual transaction
  enlistment or propagation
• Most important Code knows its transactional
• Potential Problem 1 Transaction Extraction
• Potential Problem 2 Transaction Injection

13
Problem Transaction Extraction

• Developer writes code with expectations of
  atomicity
• Declaration captures this expectation
• At least one system allows admin to change
  declaration without developer consent
• It took us years to fix this
• ASP got it right from day one ?

14
Avoiding Transaction Extraction

• Obvious Solution Embed it with (or in) code

int MyMethod(string name) int id 0 using
(TransactionScope scope new TransationScope())
id CreateUser(name) AuditChange(id,
name " was added") scope.Complete()
return id
OperationBehavior(TransactionScopeRequiredtrue)
int MyMethod(string name) int id
CreateUser(name) AuditChange(id, name " was
added") return id
15
Problem Transaction Injection

• Transaction Injection problem way more insidious
  (and harder to fix)
• Transaction-ignorant code tends to
• Interact with non-transactional resources
• Interact with transactional resources
• Cache connections to static-binding RMs
• Interact with user

16
Avoiding Transaction Injection

• If you know about transactions, you can turn them
  off (or at least suppress propagation)

int MyMethod(string name) int id 0 using
(TransactionScope scope new
TransationScope(TransactionScopeOption.RequiresNew
)) id CreateUser(name)
AuditChange(id, name " was added")
scope.Complete() return id
17
The Dangers of Suppressing Propagation

• Previous example forced itself into a transaction
  that was distinct from its callers
• Two independent outcomes (by design)
• These are not nested transactions (by design)
• If caller and callee access a common resource, we
  have an isolation problem
• Will likely tank one of the transactions due to
  timeout (deadlock) or validation error

18
Transactions and Trust

• Previous example demonstrated one side of
  composition problem (callee distrusts caller)
• Problem also applies in reverse direction
• Passing transaction to callee gives ability to
  rollback (feature and bug)
• Passing transaction to callee gives ability to
  significantly increase transaction time
• The latter flies in the face of TX dogma

19
Transactions and Time

• Transaction isolation discourages long-running
  transactions
• Two-phase locking leads to blocking/starvation
• Multi-versioning leads to rollbacks from
  validation failures
• Neither of these an issue with private resources
• Unfortunately, were still screwed due to
  heterogeneity
• No central lock manager means deadlock avoidance
  is done using timeouts

20
Core Problem Composition

• Most problems shown are a result of composing
  code (including RMs) in a single transaction
• Here are a few other problems to address
• Mixing isolation levels
• Mixing 2PL and multiversion RMs
• Mixing capabilities (nesting, chaining)
• Shared data and quiescence
• Mixing enlisted and non-enlisted work
• Non-Idempotency
• Declarative notations for all of the above

21
Life After A Transaction

• Getting things right within a single transaction
  is the simple problem
• Things get much dicier after the transaction is
  complete
• What if the transaction returned a result?
• What if the transactions success implies more
  work needs to be done?

22
Transactions and Results

• Care is needed when RM state leaks out of a
  transaction
• All locks have been released
• Results are potentially stale
• MTS/COM took draconian measures to make it hard
  to retain results across transactions
• Enter the stateless object
• Common practice is to reassert assumptions in
  subsequent transactions
• Most data access stacks provide the basics for
  free

23
Transactions and Future Work

• Ensuring future work requires the ability to
  trigger work on successful TX outcome
• E.g., do next step IFF this tx commits
• Classically done with transacted message queues
  (JMS, MSMQ, MQ)
• Control flow state implicit in queue state
• Increasingly done using process engines on top
  of TP system
• Control flow state explicit in persisted process
• Huge growth area likely successor to App Server

24
So, Where are we?

• Implicit transactions are a hit despite the
  difficulties
• SQL 2K5 System.Transactions nailed sweet spot
• STM is a logical progression in this trend
• Composition and non-TX operations still hard
• As an industry, were way less baked in the
  cross-transaction problem space
• Lots of ad hoc machinery being built
• This is where the enterprie is hurting
• The web is hurting too