VROOM: Virtual ROuters On the Move

1
VROOM Virtual ROuters On the Move
Jennifer Rexford Joint work with Yi Wang, Eric
Keller, Brian Biskeborn, and Kobus van der Merwe
(ATT)
http//www.cs.princeton.edu/jrex/papers/vroom08.p
df
2
Virtual ROuters On the Move

• Key idea
• Routers should be free to roam around
• Useful for many different applications
• Simplify network maintenance
• Simplify service deployment and evolution
• Reduce power consumption
• Feasible in practice
• No performance impact on data traffic
• No visible impact on routing protocols

3
The Two Notions of Router

• IP-layer logical functionality, and physical
  equipment

Logical (IP layer)
Physical
4
Tight Coupling of Physical Logical

• Root of many network-management challenges (and
  point solutions)

Logical (IP layer)
Physical
5
VROOM Breaking the Coupling

• Re-mapping logical node to another physical node

VROOM enables this re-mapping of logical to
physical through virtual router migration.
Logical (IP layer)
Physical
6
Case 1 Planned Maintenance

• NO reconfiguration of VRs, NO reconvergence

A
B
7
Case 1 Planned Maintenance

• NO reconfiguration of VRs, NO reconvergence

A
B
8
Case 1 Planned Maintenance

• NO reconfiguration of VRs, NO reconvergence

A
B
9
Case 2 Service Deployment/Evolution

• Move (logical) router to more powerful hardware

10
Case 2 Service Deployment/Evolution

• VROOM guarantees seamless service to existing
  customers during the migration

11
Case 3 Power Savings

• Hundreds of millions/year of electricity bills

12
Case 3 Power Savings

• Contract and expand the physical network
  according to the traffic volume

13
Case 3 Power Savings

• Contract and expand the physical network
  according to the traffic volume

14
Case 3 Power Savings

• Contract and expand the physical network
  according to the traffic volume

15
Virtual Router Migration Challenges

• Migrate an entire virtual router instance
• All control plane data plane processes / states

16
Virtual Router Migration Challenges

• Migrate an entire virtual router instance
• Minimize disruption
• Data plane millions of packets/sec on a 10Gbps
  link
• Control plane less strict (with routing message
  retrans.)

17
Virtual Router Migration Challenges

1. Migrating an entire virtual router instance
2. Minimize disruption
3. Link migration

18
Virtual Router Migration Challenges

1. Migrating an entire virtual router instance
2. Minimize disruption
3. Link migration

19
VROOM Architecture
Data-Plane Hypervisor
Dynamic Interface Binding
20
VROOMs Migration Process

• Key idea separate the migration of control and
  data planes
• Migrate the control plane
• Clone the data plane
• Migrate the links

21
Control-Plane Migration

• Leverage virtual server migration techniques
• Router image
• Binaries, configuration files, etc.

22
Control-Plane Migration

• Leverage virtual server migration techniques
• Router image
• Memory
• 1st stage iterative pre-copy
• 2nd stage stall-and-copy (when the control plane
  is frozen)

23
Control-Plane Migration

• Leverage virtual server migration techniques
• Router image
• Memory

CP
Physical router A
DP
Physical router B
24
Data-Plane Cloning

• Clone the data plane by repopulation
• Enable migration across different data planes
• Avoid copying duplicate information

Physical router A
DP-old
CP
Physical router B
DP-new
DP-new
25
Remote Control Plane

• Data-plane cloning takes time
• Installing 250k routes may take several seconds
• Control old data planes need to be kept
  online
• Solution redirect routing messages through
  tunnels

Physical router A
DP-old
CP
Physical router B
DP-new
26
Remote Control Plane

• Data-plane cloning takes time
• Installing 250k routes takes over 20 seconds
• Control old data planes need to be kept
  online
• Solution redirect routing messages through
  tunnels

Physical router A
DP-old
CP
Physical router B
DP-new
27
Remote Control Plane

• Data-plane cloning takes time
• Installing 250k routes takes over 20 seconds
• Control old data planes need to be kept
  online
• Solution redirect routing messages through
  tunnels

Physical router A
DP-old
CP
Physical router B
DP-new
28
Double Data Planes

• At the end of data-plane cloning, both data
  planes are ready to forward traffic

DP-old
CP
DP-new
29
Asynchronous Link Migration

• With the double data planes, links can be
  migrated independently

DP-old
A
B
CP
DP-new
30
Prototype Implementation

• Virtualized operating system
• OpenVZ, supports VM migration
• Routing protocols
• Quagga software suite
• Packet forwarding
• Linux kernel (software), NetFPGA (hardware)
• Router hypervisor
• Our extensions for repopulating data plane,
  remote control plane, double data planes,

31
Experimental Evaluation

• Experiments in Emulab
• On realistic Abilene Internet2 topology

31
32
Experimental Results

• Data traffic
• Linux modest packet delay due to CPU load
• NetFPGA no packet loss or extra delay
• Routing-protocol messages
• Core router migration (intradomain only)
• Inject an unplanned link failure at another
  router
• At most one retransmission of an OSPF message
• Edge router migration (intra and interdomain)
• Control-plane downtime 3.56 seconds
• Within reasonable keep-alive timer intervals
• All routing-protocol adjacencies stay up

32
33
Where To Migrate

• Physical constraints
• Latency
• E.g, NYC to Washington D.C. 2 msec
• Link capacity
• Enough remaining capacity for extra traffic
• Platform compatibility
• Routers from different vendors
• Router capability
• E.g., number of access control lists (ACLs)
  supported
• Constraints simplify the placement problem
• By limiting the size of the search space

34
Conclusions on VROOM

• VROOM useful network-management primitive
• Separate tight coupling between physical and
  logical
• Simplify network management, enable new
  applications
• Evaluation of prototype
• No disruption in packet forwarding
• No noticeable disruption in routing protocols
• Future work
• Migration scheduling as an optimization problem
• Extensions to router hypervisor for other
  applications

35
Other Projects Related to Router Virtualization
36
Bug-Tolerant Routers

• Seriousness of routing software bugs
• Cause serious outages, misbehavior, vulnerability
• Violate protocol semantics, so not handled by
  traditional failure detection and recovery
• Handling bugs at run time
• Run multiple routing instances in parallel
• Use different execution environments, message
  timings/orderings, or code bases
• Vote on answers forwarding-table entries and
  messages to neighboring routers

Collaboration with Matt Caesar and Yuanyuan Zhou
at UIUC
37
Virtual Network Infrastructure (VINI)

• Experimental platform for network research
• Evaluating prototypes of network architectures
• Supporting multiple experiments in parallel
• Carrying real user traffic connecting to
  Internet
• VINI platform (www.vini-veritas.net)
• Virtual nodes, links, and network stack in Linux
• Instantiation of virtual topology for
  experimenters
• VINI nodes deployed in NLR and Internet2

Collaboration with Nick Feamster (GA Tech) and
Andy Bavier and Larry Peterson (Princeton)
38
Concurrent Architectures are Better than One
(CABO)

• Overcome limitations of todays Internet
• Applications with diverse requirements
• Too many (sometimes conflicting) goals
• Difficulty of coordinating across domains
• New architecture based on virtualization
• Infrastructure providers own and manage
  equipment, and host virtual nodes and links
• Service providers run virtual networks
  customized to their end-to-end services

Collaboration with Nick Feamster (GA Tech) and
Lixin Gao (UMass)
39
Dynamically Adaptive Virtual Networks for a
Customized Internet (DaVinci)

• Multiple applications with different goals
• E.g., throughput-sensitive and delay-sensitive
• Want to operate customized network protocols
• How to allocate bandwidth between classes?
• Static is inefficient, but dynamic may be risky
• Theoretical foundation based on optimization
• Customized protocol for each traffic class
• Dynamic bandwidth allocation rule for each link
• Provably maximizes aggregate performance

Collaboration with Mung Chiang (Princeton)
40
Conclusions

• Router virtualization is exciting
• Enables wide variety of new networking techniques
• for network management service deployment
• and even rethinking the Internet architecture
• Fascinating space of open questions
• What is the right interface to router hardware?
• What is the right programming environment for
  customized protocols on virtual networks?
• Looking forward to discussing more!