Multicast Forwarding and Application State Scalability in the Internet

1
Multicast Forwarding and Application State
Scalability in the Internet

• Tina Wong
• Dissertation Seminar
• Computer Science Division
• University of California, Berkeley
• October 16, 2000

2
Challenge

• in the long run, the biggest issue facing
  multicast deployment is likely to be the
  scalability of multicast forwarding state as the
  number of multicast groups increases.
• --Thaler and Handley 2000
• The memory required to store multicast forwarding
  entries at a router with 32 interfaces is 1024 TB
  for IPv6, assuming 50 address space utilization
• --Radoslavov, Govindan and Estrin 1999

3
Outline

• Introduction, background, motivation
• Multicast state scaling trends in Internet
• Preference clustering protocol
• Application-driven tunable reliability
• Conclusions and future work

4
IP Multicast

• Efficient point-to-multipoint delivery mechanism
• Packets travel on common parts of the network
  only once

5
Multicast Routing

• DVMRP
• Per-source reverse shortest path tree
• Broadcast-and-prune
• MBone

Broadcast
6
Multicast Routing
S

• DVMRP
• Per-source reverse shortest path tree
• Broadcast-and-prune
• MBone

R
R
R
Prune
7
Multicast Routing
S

• DVMRP
• Per-source reverse shortest path tree
• Broadcast-and-prune
• MBone

R
R
R
Forward Data
8
Multicast Routing

• PIM-Dense Mode / Sparse Mode
• Unidirectional shared tree
• Explicit joins
• Core location a problem
• Core Based Trees (CBT)
• Bi-directional shared tree
• More optimal data paths
• Few routing vendors support

9
Multicast Forwarding State

• Router maintains membership state to achieve
  forwarding
• State scales linearly with number of concurrent
  groups
• No natural aggregation
• Number of concurrent multicast groups limited by
  router memory
• Heartbeat messages to maintain state incur
  processing costs

10
Motivation

• Lots of simultaneously active multicast groups on
  the Internet?
• Many small, group-based applications
• Few participants form a single multicast group
• E.g. internet video conferencing, games, events
  notifications, etc
• Few large-scale applications
• Lots of users form many multicast groups
• E.g. Content delivery, stock quotes, DIS, etc

11
Related Work

• Multicast state reduction
• Leaky and non-leaky state aggregation
• Tunneling in backbone (MPLS, DCM)
• Non-branching state elim (DTM, REUNITE)
• Application-level multicast
• End Sytsem Multicast, YOID, Scattercast

12
Contributions

• Comprehensive analysis on multicast state
• Understand scaling trends in the Internet
• Predict future growth
• Estimate potentials for reduction
• Apply to network provisioning, protocol and
  application design
• Mechanisms for network and end-host state
  scalability in large-scale applications
• Interest-based content delivery
• Application-driven loss recovery

13
Outline

• Introduction, background, motivation
• Multicast state scaling trends in Internet
• Preference clustering protocol
• Application-driven tunable reliability
• Conclusions and future work

14
Questions Scaling Trends

• Much research and engineering effort into making
  IP multicast widely deployed...
• How do multiplying peering agreements among
  parallel backbone networks affect multicast state
  scalability?
• How do rising subscriptions to individual
  applications increase multicast state?
• What are the state scaling properties when more
  and more applications use multicast?

15
Questions State Concentration

• An intuition multicast state scalability is most
  critical at core routers

• How concentrated is multicast state at core
  routers?
• How much benefit from tunneling?

16
Questions State Reduction

• An intuition delivery trees of sparse multicast
  groups tend to have large number of non-branching
  routers...

S

• How prominent are non-branching routers?
• Are these routers stateful?

R
R
R
R
17
Basic Model

• Local state
• Fraction of concurrent multicast groups
• True local state
• Local state with only multicast forwarding
• Independent of address space size and number of
  concurrent groups

iif0
oif0 oif1 oif2
5 concurrent groups Local state 2/5 True local
state 1/5
18
Methodology

• Simulations
• Extends upon SGB package
• Parameters
• Topology
• Session density
• Membership model

19
Topology

• 4 AS graphs from Nov97 to Jan00
• Connectivity among Internet autonomous systems
• Study multicast state at inter-domain level
• Over 3 year timespan
• Mbone graph from Feb99
• Study multicast state at intra-domain level
• Generated graphs
• TIERS
• Transit-stub

20
Session Density

• Graphs have different number of nodes, from
  1000 to 6474
• Session density instead of absolute size
• 0.1 to 0.9, 1 to 9, 10 to 90
• E.g., session with 0.1 density in AS-Jan00 with
  6500 nodes involves 7 domains
• E.g., session with 10 density in Mbone with 4200
  nodes involves 420 routers

21
Membership Taxonomy
Topological Correlation within one group
NO
YES
NO
Subscription correlation across multiple groups
YES
22
Experiments

• For each experiment, fix topology, session
  density and membership model
• (1) Pick a set of nodes with these parameters
• (2) Build shortest path tree rooted at a random
  node from this set
• Repeat (1) (2) 1000 times
• Calculate local state and true local state on
  each node in topology
• All combinations of parameters used, yielding 945
  experiments and results!

23
Caveats

• Graphs are symmetric, but Internet routes might
  be asymmetric
• Shortest path trees constructed based on hop
  counts, instead of BGP policy-based edge costs
• A member is chosen as core, which is not always
  true for shared tree construction
• Sensitivity of results continuing work

24
Answers Scaling Trends 1

• How do multiplying peering agreements among
  parallel backbone networks affect multicast state
  scalability?
• More state at a handful of core routers
• Offset by reduced state in majority of routers

25
Topological Properties
26
Hypothesis

• In a more connected network
• Trees have larger fanouts and shorter heights
• Only a few highly peered routers involved in most
  concurrent multicast trees

27
Hypothesis

• In a less connected network
• Trees have smaller fanouts and taller heights
• Backbone routers share responsibility of
  multicast forwarding -- load balancing?

28
Path Lengths
29
Node Degrees
AS-Nov97
MBone
30
Past and Future ScalingTrends

• Implication
• If Internet continues to evolve as it has been,
  multicast memory requirements at most of border
  routers actually decline, all things remain equal
  
• Evidence
• Peering increases for past 3 years
• Maximum domain degree from 605 to 1459, roughly
  50 expansion each year
• Slight decrease in state for majority of nodes
• Slight increase for rest of nodes

31
Local State Variations
32
Answers Scaling Trends 2

• How do rising subscriptions to individual
  applications affect multicast state?
• Follows power law
• fraction of stateful routers grows proportional
  to some constant power of multicast group size
• Exponents within each membership for the Internet
  similar over past 3 years
• Predictive of future state growth

33
Power Law
34
Exponents

• AS-Nov97 to AS-Jan00
• Random 1.12, 1.16, 1.09, 1.10
• Distr Clusters 1.16, 1.18, 1.20, 1.25
• Affinity 1.36, 1.36, 1.37, 1.42
• Disaffinity 0.76, 0.79, 0.78, 0.80
• Can estimate increase in stateful routers as
  subscriptions of certain application classes
  expand

35
Answers State Concentration

• How concentrated is multicast state at the core
  routers?
• State concentration does not follow 10/90 rule
  even when session density is 0.1
• Application-driven membership significantly
  impact state distribution and concentration
• Tunneling useful for multicast applications with
  very sparse and spread-out membership

36
Answers State Reduction

• How prominent are non-branching routers? Are
  these routers stateful?
• Very prominent
• Up to 2 orders of magnitude reduction is possible
  even at top 10 most stateful nodes
• Substantial even at 90 session density
• Promising approach

37
Outline

• Introduction, background, motivation
• Multicast state scaling trends in Internet
• Preference clustering protocol
• Application-driven tunable reliability
• Conclusions and future work

38
Large-scale Applications

• Large-scale applications many receivers, many
  sources, rich data types, UI
• Multicast uses one data stream to satisfy
  potentially heterogeneous receivers
• Lead to Preference Heterogeneity
• Users differ in interest on application data
• E.g. Content delivery, news dissemination, stock
  quotes, network games, DIS, etc

39
Example Stock Quotes Service
www.StockCentral.com
INTC DELL CSCO MSFT
AAPL AMZN EWEB MSFT GABC QCOM
PWBC SISI YHOO
...
Cathy
Amy
Bob
40
Example Network Games
A player's position in virtual environment drives
its preferences on entity updates
41
Preference Heterogeneity

• Assign each logical data stream a unique
  multicast address ?
• No superfluous data
• Multicast routing state scalability
• Multicast address allocation and scarcity
• End-host connection maintenance
• 100 reliability not necessary
• Different levels of reliability desired
• Help to reduce NACK implosion

42
The Clustering Concept
complete heterogeneity
complete similarity
UNICAST
MULTICAST
43
Preference Clustering Protocol

• Clustering algorithm
• On-line and adaptive to changes in preferences
• Customizable to different application and data
  types
• Signaling protocol
• Coordinate clustering within an application
• Scalable, fault tolerant and reliable through
  decentralization, soft state and sampling
• API
• Detailed evaluation

44
App-Level Tunable Reliability

• Consider application semantics in loss recovery
  decisions
• Meta-data to describe data content
• Temporal statistics on update frequency
• Semantic magnitude or importance of change
• Policy-driven by individual receivers

45
Outline

• Introduction, background, motivation
• Multicast state scaling trends in Internet
• Preference clustering protocol
• Application-driven tunable reliability
• Conclusions and future work

46
Conclusions

• Comprehensive study on multicast state
  scalability
• Scaling trends confirmed with past 3 years
• State distribution and concentration
• Potentials for reduction
• Mechanisms to accommodate problem for large-scale
  applications
• Customizable and adaptive preference clustering
  protocol
• Tunable reliable multicast protocol

47
Future Directions

• Compare and contrast methodologically IP
  multicast and application-level multicast
• Params Topology, session density, membership
• Apps Few-to-few, one-to-many
• Metric Bandwidth, latency, complexity, etc
• Placement of service agents in Internet
• Spawning of new agents
• Coalescing based on topology, user population,
  network measurements