Xtrace: a Crosslayer network trace tool

1
Xtracea Cross-layer network trace tool

• Rodrigo Fonseca and George Porter
• R. Katz, S. Shenker, I. Stoica

2
XTrace Introduction

• Understanding the behavior of large scale,
  distributed systems is very hard today
• Why?
• Layering
• Increased composition
• Middleboxes, proxies, overlays, web services
• Common network tools focus on a single layer
• E.g., traceroute, ping, http tools
• XTrace a cross-layer network tracing tool

3
XTrace in DADO

• How does XTrace fit into the RadLab?
• Assess
• Determine the behavior of a large-scale
  distributed Internet application while it is
  running
• Operate
• On-line network diagnostic tool for fault
  isolation
• Develop
• A new protocol can support XTrace with very
  simple requirements on its design
• Deploy

4
Outline

• Two motivating examples
• Our approach XTrace
• Research problems
• Supporting cross-layer metadata
• A secure notification architecture
• Revisiting the example
• Current status and future work

5
Example 1 HTTP Proxy
HTTP
TCP
IP
Client
Proxy
Server

• Routers cant map individual packets to HTTP
  sessions
• HTTP clients cant detect IP packet losses

6
Example 2 OCALA
IP ISP 2
BRO IDS
IP ISP 3

• Network infrastructure services like I3 are out
  there and are going to be used if theyre useful
  to people
• Stovepipe tracing and debugging not sufficient

7
Examples summary

• Layering
• Devices at one layer not aware of other layers
• Even per-layer traceroute not enough
• Need a network architecture that lets layers
  communicate state in a clean way
• Applications rely on components and network
  services

8
Alternative approaches

• In-band tracing
• Shares fate with faults in the connection
• Limited space receiver must send trace data back
  to sender

• Network layer
• Traceroute
• Per-protocol
• I3 traceroute in development
• Cross-layer as a service (Rexford, et al)
• Focus on control-path information

9
XTrace Approach

• Put metadata into packets that reaches the
  endpoint
• Accessible to all layers
• Spans multiple connections

• Have devices use that metadata to notify the
  sender of what theyve seen
• Out of band
• Only for layers they support

Client
Proxy
Server
10
Research problem 1Supporting cross-layer
metadataResearch problem 2 Designing a
notification architecture
11
Research problem 1Supporting cross-layer
metadata
12
Cross-Layer Metadata

• Vision annotation layer
• Cross-layer scratchpad that every layer can
  read from
• Sufficient for XTrace, but not necessary
• XTrace solution replicated metadata in each
  layer
• Read-only, sender-specified id destination of
  reports
• Uses each protocols extension/optional fields

13
Client
Proxy
Server
GET / HTTP/1.1 Host berkeley.edu X-Layer
123456
HTTP
dport 8080 sport 17658 seq_num
6253 tcp_option XTR123456
TCP
dst_ip proxy src_ip client ttl 58 ip_option
XTR123456
IP
14
Cross-layer metadata

• Key problem
• Propagating metadata to lower layers on new
  connections
• As new connections are opened (i.e., by the
  proxy), copy the data into the new connection
• via libxtrpropagate

• Protocols this approach can support
• Has an option field or extension header
• This header propagated to the destination
• Protocol support
• HTTP, TCP, IP, I3, NAT, Firewall
• Soon SMTP, DNS
• Desired Ethernet/wireless

15
Libxtrpropagate Example
libxtrpropagate
Squid Proxy
xtr_connect(d, anno)
Web Server
Kernel
Annotated request

• Squid reads the XTR annotation out of the web
  request
• xtr_connect() sets up state in kernel so that
  when the SYN packet goes out, it is properly
  annotated

16
Research problem 2Designing a notification
architecture
17
Notification example
report.net
r4.router.net
r1.router.net
r3.router.net
r5.router.net
me.client.org
p1.proxy.net
Annotation (type I3 data i3_trigger)
www.berkeley.edu
18
Secure Notification Architecture

• Requirements
• Metadata dictates how and where notifications are
  sent
• Completely stateless for each device
• Incrementally deployable
• The device/AS controls who gets notified
• Fine-grained access control of internal
  information

• Challenges
• Visibility
• Preventing abuse
• Against the device itself
• Against the receiver of notifications

19
Notification Approach

• Flexibility of transports
• UDP over IP, I3, Locally kept logs
• Visibility
• Shared secret / capability
• Trust relationships between ISPs (capability
  switching)
• Multiple layers
• E.g., unauthenticated just the border routers
  authenticated detailed topology information
• Preventing abuse
• I3s DoS protections

20
Network diagnosis w/ XTrace
Bro Process stopped
IP ISP 3
I3 Overlay

• Notification from BRO server
• IP saddr server2.i3.com daddr
  server.bro.com
• TCP sport 3212 dport 80 (no listening
  socket)

21
Current Status

• Initial implementation
• HTTP, TCP, IP, I3, Nat, Firewall (soon DNS,
  SMTP, SIP?)
• Built on Apache runtime (APR)
• Linux, Mac, Windows
• Libxtrreport
• I3, UDP
• Local logging (soon)
• Libxtrpropagate
• Complex design using IPC message queues

• To be done
• Controlled visibility solution
• DoS prevention
• Tools

22
Thank you

• Feedback/suggestions/discussion encouraged