Extending DIDUCE

1
Extending DIDUCE

• Kostadis Roussos
• Kostadis_at_netapp.com

2
Goals

• Extend DIDUCE to handle large C applications
• Extend DIDUCE to handle embedded applications
• Extend DIDUCE to report dynamic information when
  invariant violation occurs

3
What problem are we interested in?

• Debugging non-critical errors
• Critical errors for the most part are tractable
  given existing tools (assert core file)
• Non-critical errors affect 1 of user base
• Non-critical errors that are debugged using
  asserts turn 1 downtime into 100 downtime.

4
Example NetCache (1)

• Streaming media proxy cache
• Consists of a two-tiered software architecture,
  protocol engine and data flow engine.
• Protocol engine responds to user requests
• Data flow engine delivers data

5
Example NetCache (2)

• When a stream-switch occurs protocol engine must
  first receive request, validate request,
  determine if data is on disk, stop the current
  data flow, and restart
• The data flow engine consists of three
  asynchronous threads that process data from disk
  to the network and handle requests from the
  protocol engine

6
Example NetCache (3)

• If an error occurs because of a bug, the only
  user observable behavior is a glitch or a
  disconnected client
• If a panic is induced, at the time error state is
  detected, the relevant data structures and state
  at time of error have disappeared
• Many such errors
• Difficult to reproduce because they are races

7
Extending DIDUCE

• Add support for large C programs
• Be able to collect arbitrary dynamic state
  information when an error occurs
• Enhancements above and beyond.

8
Extending DIDUCE Overview
9
Issues

• Data structure
• Use a hash table (500k 8bytes 4 not that
  much memory)
• Instrumentation
• Inlined function calls
• A portable stack trace
• Under thought development
• How to support dynamic information
• Under thought development

10
How to validate

• Use to debug already closed problems.
• Use to aid debugging existing customer
  escalations
• Compare time to fix.

11
Example NetCache (4)

• If our tool existed we would be able to capture
  at the point where a state transition is made
  information about the total state of the protocol
  engine and the data flow engine.
• Manually collecting that state, and manually
  verifying it via asserts is not tractable or
  practical.
• An automated tool would let us detect such errors
  immediately.

12
Enhancements

• More precise information
• Tracking related state, not only explicitly
  examined state
• How to selectively log information
• Real-time modification of behavior like purify

13
Related Work

• DIDUCE
• Purify
• Daikon
• Prefix
• Metal
• SLAM