Deploying Safe User-Level Network Services with icTCP

1
Deploying Safe User-Level Network Services
withicTCP

• Haryadi S. Gunawi
• Andrea C. Arpaci-Dusseau
• Remzi H. Arpaci-Dusseau

The ADvanced Systems Laboratory (ADSL) Univ. of
Wisconsin - Madison
2
Motivation

• Vast number of proposed modifications to TCP/IP
• Some adopted, others not deployed
• TCP Vegas SIGCOMM 94
• TCP Nice OSDI 02
• Congestion Manager (CM) SIGCOMM 99
• Heart of deployment problem OS kernel
• OS tend to be substantial and complex
• Vendors dislike changing them when benefit is not
  imminent
• A range of OS-es must implement the modifications
• Transition of good research ideas into
  wide-spread use is slow
• Emerging different network environments
• Wireless (lossy network), Load-balancing (packet
  reordering)
• Different extensions for different network
  environment
• TCP only support some

3
Current TCP implementation
TCP Applications
TCP
Reno
4
Inside to outside
TCP Applications

• Why in-kernel extensions?
• Information and control only available within the
  kernel
• Ex TCP Vegas
• Information per-packet RTT and TCP States
• Control Congestion Window
• Questions?
• Can extensions be built in application layer?
• What information and control need to be exposed?

TCP
f ( )
5
Proposed Framework

• icTCP (information and control TCP)
• Address the problem of deployment
• Slightly modified in-kernel TCP stack
• Exports key pieces of information
• Provides safe control to user-level libraries
• Given information and control, extensions can be
  built in application layer
• Evaluation
• Converting TCP to icTCP requires a small amount
  of additional code
• The resulting flow is TCP friendly
• Minimal overhead
• Implement 6 user-level extensions
• Little complexity in implementing extensions at
  user-level

6
Benefits

• icTCP facilitates the development
• of user-level extensions

7
Outline

• Motivation and Overview
• icTCP Design and Implementation
• Information
• Control
• 5 Axes of Evaluation
• Conclusion

8
icTCP Design

• Different TCP connections, different libraries
• User-Libraries on top of icTCP
• icTCP exposes Information and Control

9
Information

• Which information should be exposed?
• Too little ? limited extensions
• Too much ? expanded interface
• 2 types of Information
• Variables part of TCP specification (RFC 793)
• cwnd, ssthresh
• Message list and Ack list
• More detailed information
• History of recent packets and acknowledgements
• Enabled by user-level services to save memory

TCP Clients
User Libraries
10
How and when to obtain information?

• Polling
• BSD socket interfaces
• Unnecessary run-time overhead
• Kernel-user space copy
• When to obtain accurately?
• TCP variables are updated
• upon receipt of an ACK
• end of a round
• Interrupt mechanism
• icTCP notifies application when these events
  happen

11
Control

• Allow internal TCP variables to be externally set
  in a safe manner
• What variables and what values?
• TCP-Friendly dont harm other competing standard
  flows
• Philosophy icTCP must be conservative
• Control is allowed if not aggressive
• Control 10 (virtual) variables that can be safely
  set

TCP Clients
User Libraries
12
Implementation of Safe-Control

• Idea Virtual variables
• congestion window (cwnd) ? virtual congestion
  window (vcwnd)
• Manipulate policies through virtual variables
• TCP Reno keeps running using the original
  variables
• Safe Ranges
• Enforce friendliness
• 0 vcwnd cwnd
• Without safe ranges, resulting flows not
  TCP-friendly
• Choosing 10 variables, defining safe-ranges
• Check safe-setting theoretically and empirically
• Theoretically Reno equation
• congestion window control how many packets in
  the network
• vcwnd gt cwnd more packets in the network, thus
  more aggressive
• Empirical Prove
• Set virtual congestion window outside the safe
  ranges

13
Outline

• Motivation and Overview
• icTCP Design and Implementation
• Framework Evaluation
• Conclusion

14
5 Axes of Evaluation
How easy to convert TCP to icTCP?
1
How difficult to develop TCP extensions in this
way?
2
5
Is icTCP friendly?
4
3
What are the overheads? Does it scale?
What types of extension can be built and deployed
with icTCP?
15
Converting TCP to icTCP
TCP to icTCP
1

• icTCP in Linux 2.4.18
• Add 316 lines of code
• Non-intrusive modifications

16
Network Safety Unconstrained icTCP
Friendly?
2
TCP-Friendly 1
TCP-Unfriendly
17
Network Safety Constrained icTCP
Friendly?
2
TCP-Friendly
TCP-Unfriendly

• Safety ? Safe Ranges
• are required!

18
Scalability and Overhead
3
Scale? Overhead?

• What is the overhead? Does it scale?
• Rate of getting info and setting variables
• Different extensions, get/set at different rate
• Two factors
• Per-ack or per-round interrupts
• Need the message list and ack list
• 3 synthetic libraries
• per-ack interrupt
• per-round interrupt
• per-round interrupt gets message list

19
Overhead
3
Scale? Overhead?
(96 conns)
R1
R2
per-round 12
S
per-ack 30
per-round msgList (8 conns)
12 MB/s
per-ack (4 conns)
R3
R4

• Noticeable overhead,
• but not prohibitive

20
TCP Extensions
4
Range of Extensions?

• Range of TCP extensions can be built on top of
  icTCP
• Implement 3 sets case studies (6 extensions)
• Congestion window
• TCP Vegas Brakmo, SIGCOMM 94
• TCP Nice Venkataramani, OSDI 02
• Congestion Manager (CM) Balakhrisnan, SIGCOMM
  99
• Duplicate threshold
• Reordering-Robust Ext Zhang, ICNP 03
• Efficient Fast Retransmit Tamura, LCN 98
• Retransmission Timeout
• Eifel RTO CCR 00

21
User-level TCP Vegas Implementation
4
Range of Extensions?

• Using information and control is simple
• Complexity is within the algo itself

TCP Clients
lib-Vegas
icTCP
TCP Vegas Brakmo et.al., SIGCOMM 94
22
Does it work?
4
Range of Extensions?

• TCP Reno Send more packets until drop.
• TCP Vegas Maintain the right amount of extra
  data in the network
• e.g. keep only 3 packets in the bottleneck queue

Reno
User-Level Lib. TCP Vegas
In-kernel TCP Vegas

• Same Behavior

23
icTCP Strengths
4
Range of Extensions?

• Implement less aggressive TCP variants at
  user-level
• No need to push changes into the kernel
• Ideally suited for tuning parameters whose
  optimal values depend upon the environment
• Lossy Network ? retransmit faster ? Use Efficient
  Fast Retransmit
• Packet reordering ? postpone retransmission ?
  Reordering Robust (RR) Extensions
• Different environment, opposing solutions
• TCP favors one solution
• Correcting errors in parameter values
• Eifel RTO CCR 00 RTO prediction flaw when RTT
  drops
• In newer Linux version, this prediction flaw is
  corrected with adding 2 new lines of code
• Must wait vendors to correct this flaw

24
Other approaches
4
Range of Extensions?

• Cant implement all extensions
• Only allow 10 existing variables to be set
• Conservative Safety
• STP (Self-Spreading Transport Protocols) SOSP
  03
• Remotely upgrade others protocol stack
• More TCP extensions can be deployed
• Use separate module to enforce friendliness
• Why we dont use enforcer
• Drawback terminate non-conforming flows
• icTCP modulates aggressive flows

vs.
25
Ease of Development
Difficult Develop?
5

• How difficult to develop TCP extensions in this
  way?
• Complexity at user-level vs. in-kernel

1200()
438
26
Composability
Difficult Develop?
5

• Composing multiple libraries
• Vegas good for small bottleneck queues
• RR good for reordering
• Environment where both situations exist?
• VegasRR Better throughput

TCP Clients
icTCP
27
Conclusion

• icTCP
• Philosophy Information and Control
• Non-intrusive, simple, and TCP-friendly
• Systems with icTCP reap benefits of user-level
  TCP extensions
• Change TCP once now, no need to change it later

28
Questions?

• The ADvanced Systems Laboratory
• www.cs.wisc.edu/adsl

29
Related Work

• Mogul et.al. HotNets-2003
• Arbitrary state setting
• Web100 and Net100
• Export a variety of per-connection statistics
• Does not ensure network safety
• STP SOSP-2003
• Remotely upgrade others protocol stack
• Invasive changes to the kernel
• Additional machinery to ensure friendliness
• InfoTCP/infokernel SOSP-2003
• icTCP exposes more information
• icTCP allows services to directly set cwnd inside
  of TCP
• icTCP allows more variables to be controlled