CDNPubSub in the net

1
CDN/Pub/Sub in the net

• Jon Crowcroft
• R02 L5
• http//www.cl.cam.ac.uk/jac22/acs-proj-ideas.txt

2
Essay feedback

• NIRA - User choice of part of AS Path
• See HLP - valley free assumption
• http//portal.acm.org/citation.cfm?id1080095
• See Mechanism Design
• http//conferences.sigcomm.org/sigcomm/2004/pins.h
  tml
• IPNL
• Problem is failure (detect/repair)
• equiv of routing - I.e. why not just do IPv6!
• I3
• Route stretch - why not use PIM-SM trick
• To shortcut, after Rendezvous?
• ROFL - nice example of naïve good!

3
CDNs aboundwhats next

• What are the requirements from users?
• What is range of tech. solutions
• Tibco Rendezvous(PGM) - share trading
• PSIRP - pub-sub net- bloom filter/zfilter
• DONA - (haggle like) rendezvous
• CCN - name based
• What are Business impacts?

4
Information Networking - Deep Dive
5
Outline

• A user view of our proposition
• Some basic messages
• Technical foundations
• Information concepts Labels, scopes, and more
• Architecture Rendezvous Forwarding
• Socio-economic work
• Market evaluation, design choice evaluation
• Engagement and exploitation
• Important
• Attempt to give an overview!
• More details can be found at http//www.psirp.org

6
A User View The Information Cloud
7
(No Transcript)
8
The internet requires an explicit address for
every web page.
9
Information Cloud decouples information from
infrastructure and location.
10
Information is organised, accessed and routed
according to its meaning.
11
Information can be tagged in multiple ways,
providing rich meta-data.
To John With love from Sue
12
Information can be tagged in multiple ways,
providing rich meta-data.
Do not open before October 8th.
13
Information relations can change rapidly - so
does the Information Cloud
Flexibility
14
One Information Cloud provides benefit at many
levels within an organisation.
CRM
Example Retail
Supply Chain
Operations
15
Our Work

• We start from the hypothesis that
  information-centric routing makes sense

16
Hypothesis Increased Dynamics Require
Information-centric Network Approaches

• Observation Application developers care about
  information concepts
• Creation of various information topologies at
  increasing rate
• Middleware layers provide application developers
  the programming model they want
• "The Network needs to get out of our way"
• Endpoint-centric networking structures are
  inadequate
• Topological network changes too slow in timescale
• Topological network boundaries (connecting
  machines) often not aligned with information
  topologies (connecting information)
• Overlaying possible but restricted in (developer)
  scalability
• -gt If it is all about information, why not
  routing on information?
• -gt Move from interconnecting machines to
  interconnecting information!

17
A Post-Modern World The Information Internet
Architectural Implementation
18
A New Provisioning Plane High-Level Architecture
RP Rendezvous point ITF Inter-domain topology
formation TM Topology management FN
Forwarding node
Network Architecture
19
Main Design Principles

• Everything is information
• Interconnection information not wires!
• Ranges from simple labelled data to complex
  ontology-based structures
• Information is scoped
• There is an inherent meaning in the structures of
  information that can help us defining a better
  system
• Scoped information neutrality
• Within a single scope, simple (information)
  identifiers are the sole basis of operation
• The architecture ensures a balance of power
• Sending availability of information (publisher)
• Sending interest in information (subscriber)
• -gt Reception comes ONLY after successful match

20
Information-Centrism is Key

• Provide concepts for information structures that
  allow to bootstrap more complex concepts like
  identity, policy, context,
• Allow for minimal information items
• Scopes build information networks
• Policy is metadata
• So is scope!
• Producers and consumers need no
  internetwork-level addressing!

Data Mail
Data Picture
Governance policy
Governance policy
Scope Company A
Scope Family
Scope Friends
Governance policy
Father
Friend
Spouse
Colleague
21
Information Concepts

• Information
• Smallest something
• Information collections
• Set of semantically similar information
• Information networks
• Set of information under some common governance
• Information producer
• Entity publishing information to a particular
  network
• Information consumer
• Entity subscribing to information in a particular
  network

22
Information Structures
23
Grouping of Information Questions

• Can it be beneficial to expose grouping of
  information to the network, apart from scopes?
• Examples of groups identity, channels,
  documents,
• What could be the usage of such (exposed)
  grouping within the network?
• Examples fragmentation, caching, authentication,
  
• How could we do such grouping?

24
Algorithmic RIds
alg(seed)

• Idea
• Use an algorithm to tie together a set of data
  items
• Allow for data items to be addressed individually
  through algorithmically generated RIds
• Allow for addressing collection through algorithm

• Access channel via seed RId, go to segment via
  alg(seed)
• Publish alg as metadata to seed
• -gt Channel implicitly visible to network,
  together with individual segment RIds, by virtue
  of alg as implicit channel Id, alg being
  app-specific

Segment determined via RId alg(seed), e.g.,
alg seqNo
25
Algorithmic RIds (2)

• Algorithm can tie seemingly individual IDs
  together as a collection
• More dynamic than scopes, does not involve
  rendezvous system!
• Alg can be simple algorithm being signalled
  out-of-band
• Publish as metadata to seed
• Make known through other means to pubs/subs,
  similar to seed (e.g., directory)
• Can be signalled in-band, e.g., use header
  information, always assuming alg IDs
• Leave bits to standardization

26
Algorithmic RIds Examples of Usage

• Use (implicit) knowledge of alg(seed) relation of
  the seemingly individual RId set for network
  functions
• E.g., fragmentation, caching, forwarding
  optimization, compensation, authorization,
• Allow for grouping data items according to
  application concepts, e.g.,
• (media) channels, documents, identity (collection
  of identity attributes), context,

27
RECAP Information Information Networks
Data Mail
Data Picture
Governance policy
Governance policy
Scope Company A
Scope Family
Scope Friends
Governance policy
Father
Friend
Spouse
Colleague
28
Scopes in Abundance
IN information network
UK Corporate
Facebook
UK Communities
BT scope
BT IN
Colchester IN
Team scope
Dirk IN
29
Information Hiding in PSIRP
30
Information and Information Networks

• Scopes allow for building information networks
• Specific subclass of data items
• -gt scopes are information
• Scopes are likely to be access controlled
• BUT scopes themselves reveal information
  (structure)
• Can some scopes be private (not known to the
  global rendezvous system) but still be reachable?

31
Information (Structure) Hiding the Role of
Rendezvous
UK Corporate
Facebook
UK Communities
public
BT IN
private
Colchester IN
Dirk IN
32
NAT in an Information-centric World

• The problem of reaching private scopes is akin to
  reaching private IP addresses from outside the
  private network
• The RP serving the parent-node public scope is
  the NAT gateway, in a sense
• HOW DO I DO NAT?

33
Algorithmic SIds

• RECAP Alg(seed) ties set of Ids together
• SIds a subclass of RIds
• -gt what could it mean when applied to SIds, i.e.,
  what could alg(SId) be?
• Seems to enable our vision of mapping social
  structures
• Components using this information
• Rendezvous, compensation, authorization,
  forwarding

34
Examples

• Inclusiveness
• Organizational structures in corporations (teams,
  lines of business, )
• Friend-of-a-friend (FoF)
• Alg is directed graph scopes
• Equivalence
• Special case of above
• Others?

35
Rendezvous
36
A New Provisioning Plane High-Level Architecture
RP Rendezvous point ITF Inter-domain topology
formation TM Topology management FN
Forwarding node
Network Architecture
37
Different Levels of Rendezvous

• Rendezvous is used on every level of information
• Each scope has assigned a rendezvous point (RP)
• Different techniques are used for the different
  scopes, e.g., link announcements

38
The Role of Global Rendezvous

• Rendezvous implements the mapping from
  information topologies onto delivery topologies
  (in runtime)
• A scope is published to the global rendezvous
  system by a serving rendezvous point (RP)
• Create an information network
• Data items are assigned to a particular scope
  through publishing RId to the particular SId
• Global rendezvous system routes request (slow
  path)
• Fast path is established after rendezvous
• Create sequence of FIds (delivery topology)

39
Global Rendezvous Solution

• Problems here
• Incentive-compatibility
• Scalability

Use DHT-like approach, i.e., virtual overlay
J
I
H
E
G
F
Comply with valley-free model
D
B
A
C

• Problems here
• Policy-compliancy

Data
UserB
UserC
Data route
Data path
Peering link
Sibling link
Transit link
40
Global Rendezvous Solution

• Solution developed
• Based on CHORD for overlay
• Solution currently integrated
• Rendezvous node development
• Rendezvous client for end node
• Solution currently evaluated
• Performance evaluation done (left out), i.e.,
  delay, stretch,
• Simulations, based on available topology models
• Socio-economic evaluation ongoing (see later)

41
Forwarding
42
A New Provisioning Plane High-Level Architecture
RP Rendezvous point ITF Inter-domain topology
formation TM Topology management FN
Forwarding node
Network Architecture
43
Link IDs and Bloom filters (zFilters)

• No names for nodes
• Each link identified with a unidirectional Link
  ID
• Link IDs
• Statistically unique
• Periodically changing
• Size e.g. 256 bits
• Local or centrally controlled
• Source routing
• Encode Link IDs into a Bloom filter (zFilter)
• Naturally multicast
• Stateless

B?C
B
C
A?B
A
D
0 1 0 0 0 1 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 1
1 0 1
A?B B?C zF A?B?C
44
Forwarding Decision

• Forwarding decision based on binary AND and CMP
• zFilter in the packet matched with all outgoing
  Link IDs
• Multicasting zFilter contains more than one
  outgoing links

Link ID


Yes/No
zFilter
zFilter
45
Using Link Identity Tags (LIT)

• Better forwarding efficiency with a simple trick
• Define d different LITs instead of a single LID
• LIT has the same size as LID, and also k bits set
  to 1
• Power of choices
• Route creation and packet forwarding
• Calculate d different candidate zFilters
• Select the best performing zFilter, based on some
  policy

Host 1 Iface out
Host 2 Iface out
Link ID
Link ID
Candidate zFilter
LIT 1
LIT 1
zFilter 1
LIT 2
LIT 2
zFilter 2
zFilter d
LIT d
LIT d
46
Forwarding efficiency
Wrongly sent packets

• Simulations with
• Rocketfuel
• SNDlib
• Forwarding efficiency
• 20 receivers
• Basic LID 80
• Optimised 88
• with 8 LITs
• (Large multicast groups considered on a later
  slide)

receivers
47
Scalability beyond 20 Virtual links

• Popular paths/large trees represented as virtual
  links
• A single Link ID for the tree
• Additional state in the forwarding nodes
• Increases scalability

C
D
B
A
E
F
0 0 1 0 1 0 0 0 1
Virtual B?C?D?E
48
Socio-Economic Work

• Motivation

49
Architecture and Business DesignTwo Sides of
the Same Coin

• Where to place control points?
• and where not?
• How flexible is my architecture solution?
• What business does it enable?
• and which ones it does not (and should not)?
• What to place on what layer?
• How to enable generality?
• How to maximize utility?

• How survivable is my business?
• What strategy will sustain my business?
• Where can I extract value in my offering?
• What implements (architecturally) my strategy
  best?
• What makes my strategy worth trying?
• Who to partner with?
• How to be better than my competition?

50
Desired A Framework that Tightly Combines
Architectural Design and Business Modeling

• Assume we had a framework that would combine
  architectural design and business modeling
• Assume that we had a tool that would allow for
  evaluating success and failure of business models
  and architectural designs
• RESULT Design solutions as a duality of
  strategic business planning and architectural
  design with measures for success and failure of
  propositions!

51
Three Usages

• Evaluate the markets created
• Technical solutions create markets
• Markets need to be understood since they create
  forces that impact the viability of the technical
  solutions
• -gt extend the pure technical evaluation
• Evaluate possible design choices
• Crucial functions have various design choices for
  realization
• While technical ability to implement might
  restrict the set of possible choices, other
  socio-economic factors will further impact their
  viability
• Impacts strategies for, e.g., alliances, standard
  activities, impl. efforts
• -gt limit set of possible choices to be
  implemented
• Evaluate opportunities and threats
• Solutions create opportunities and threads for
  existing and new players
• Want to understand them to
• advise stakeholders
• facilitate adoption
• -gt understand deployment, migration and value
  proposition

52
Introduction into the Toolkit

• Concepts and Mechanics

53
Concepts of the Toolkit
Use Case
System Dynamics
Actors
Evaluation
Components
Triggers
Services
Control Point Constellations
Control Points
54
Mechanics of the Toolkit

• Intention is to provide a set of tools in which
  the steps can be executed

55
Overview of Toolkit in XMind

• Different steps for a number of concepts
• Each step is implemented as a separate sheet
• Part 2 implemented via Vensim

56
Usage Market Evaluation

• Example for a Future Internet Proposition

57
Matching Information Availability and Interest in
Large-Scale A Strawman Proposal
Interconnection Overlay
RP
RP
REndezvous NEtwork
RENE
RP
RENE
RP
RP
RP
RP
pub
58
Market Questions

• How many of these overlays will exist?
• How many RENEs will exist?
• To how many overlays is each RENE connected?
• How fragmented is the interconnection?
• -gt use the toolkit to answer these questions

59
No. of Interconnection Overlays

Ubiquitous interconnect
105
Commoditization
104
De-valuation
103
Commercialization
102
Consolidation
10
Dominant Search Engines
1
One Google
Market interest
Initial deployment
t
60
Number of RENE networks

Direct interconnect
The number of RENE networks is indicative for the
'regional' character of resolving SId queries
since it is assumed that RENE networks are formed
under some 'regional' notion, such as geography,
local peering relations, ... (more under the
notion of 'region' following Sollins, not
restricted to geography).
105
Commoditization of RENE regions
104
De-valuation of RENE regions
Formation of stable regions
103
102
Regionalization
De-regionalization
10
1
Extreme regionalization
Initial regionalization
Initial deployment
t
61
Incentive to Interconnect
1
Fully interconnected Internet
Death of fragmentation
Accelerated interconnection
Manifestation of fragmentation
Fragmented Interconnection markets
Growing inter-provider deployment
Adoption as intra solution only
Initial (intra-provider) deployment
Failure of Adoption
Heavy fragmentation
t
62
Causal Loops Number of Overlay Providers
63
Usage Design Choices

• Topology Formation in the Future Internet

64
Recap Roles in this Future Internet
RP Rendezvous point ITF Inter-domain topology
formation TM Topology management FN
Forwarding node
Network Architecture
65
No. of ITF Providers
ITF Providers
103
Heavy fragmentation of peering infrastructure
It is assumed fragments are formed under some
'regional' notion, such as geography, local
peering relations, ... (more as defined by
Sollins, not restricted solely to
geography) Multiple providers might cooperate to
provide coverage over a (geographic) region
Resilience requirements for e.g. finance might
imply fragmentation by market
Commoditization
102
Devaluation
Commercialization
Consolidation
Regional fragmentation
10
Failure, insufficient take-up
BGP-like ubiquitous interconnect
No fragmentation
1
Take-off of peering market
Initial deployment
t
66
Importance of Pub Choice
Importance
1
Publisher choices dominate
Increasing importance Pub often ignores other
views
Publisher Incentives differentiation e.g. QoS,
regulatory policy compliance (w.r.t.
access) Possible publisher mistrust of ISP
behaviour/motivation
Maximum importance Heavy access regulation gt
pub liability Strong pub-ISP mistrust
Compromise Hype/coolness fades ISP
resistance limits growth
Stabilization Pub partially recognises other views
Compromise between publisher and other choices
Low importance Special cases only
Take-off in importance Hype/coolness/price
Perceived usefulness
Minimal initial importance Low technology
deployment
Little/no importance ISP views dominate
Publisher choices relatively unimportant
t
67
Importance of Local ISP Choice
Importance
1
Local ISP choices dominate
Increasing importance ISP often ignores other
views
ISP Incentives link differentiation, attraction
to local access customers, regulatory policy
compliance (w.r.t. access peering), resource
optimization Possible ISP mistrust of publisher
behaviour/motivation ISP tendency to bias
decisions towards own interests e.g. (1)
protecting network vs. publisher needs (2)
topology hiding vs. competitors (3) favouring
cached content nearer to itself
Maximum importance Heavy peering regulation
gt ISP liability ISP selects cache Strong
pub-ISP mistrust ISP very protective
Compromise Hype/coolness fades ISP
price consciousness limits growth
Stabilization ISP partially recognises other views
Compromise between local ISP and other choices
Take-off in importance Hype/coolness/price
Perceived usefulness
Low importance Special cases only
Minimal initial importance Low technology
deployment
Little/no importance Pub views dominate
Local ISP choices relatively unimportant
t
68
Importance of RP Choice
Importance
1
RP choices dominate
Increasing importance RP, pub ISP often disagree
RP Incentives Attraction to publisher/subscriber
via differentiation e.g. QoS, regulatory policy
compliance (w.r.t. information) RP focused on
information (matching distribution) tends to
favour cached content nearer to itself seeks
best long-term compromise to tussle between
RP, publisher and ISP views

• Maximum importance
• Heavy information regulation
• RP liability
• RP selects cache
• Pub ISP views less important

Compromise RP resolves residual tussle
Stabilization Satisfactory long-term solution
available
Compromise between RP and other choices
Take-off in importance Hype/coolness
Perceived usefulness
Low importance Special cases only
Minimal initial importance Low technology
deployment
Little/no importance RP, pub ISP general
agreement
RP topology choices relatively unimportant
t
69
Causal Loops Number of ITF Providers
70
Engagement and Exploitation
71
Potential ImpactsOnly a Few

• More flexible services
• Individual information items allow for faster
  mash-ups
• Opportunity of real-time collaboration
• Increased policy compliance
• Visibility of 'items' on routing level
• Opportunity of flexible policy enforcement on
  routing level
• Increased low-level search capability
• Move from crawling approach to information
  routing (death of common search engines)
• Opportunity to eliminate broken links (increase
  relevance)
• Increased caching
• Could lead to increased locality
• Price transit traffic to decline (death of
  Tier-1)
• Could lead to decline of managed memory (death of
  CDN)
• Opportunity to operate networks more efficiently
  (locally)

72
Which Impact Is It?

• In order to approach stakeholders and identify
  relevant opportunities, we need to understand
  likelihood of scenarios and opportunities arising
  from it
• Socio-economic understanding and business models
  are key
• Need to engage with stakeholders to verify
  understanding
• Must not be disconnected from architectural work
• Work ongoing on socio-economic scenarios,
  exploitation and migration

73
Currently

• Developing engagement opportunities
• Government Creating and information space among
  agencies for the citizens
• Retail A flexible information space for
  increased business dynamics
• Media From core production to edge
  creation/production
• -gt develop opportunities, scenarios, material
• Developing end user benefit scenarios
• Stories
• Videos
• Mock-ups
• Just started this effort

74
Demonstrator
Publisher
Subscriber
Forwarder

• End nodes based on new node architecture (left
  out here)
• Forwarding Node based on zFilter forwarding
• Simple publication/subscription to content
• Ongoing work
• Integration of global rendezvous solution
• Networked test bed at Essex University

75
A Test Bed Under Development at Essex University
Cambridge
Essex
Network/Service Manager
Cambridge
HD Video Conferencing
E-Health
Storage
PlanetLab (Global Network)
IPTV/VoD/VoIP
HD Video Conferencing
Traffic Generators
Storage
Storage
BT
Public IP address
Access D
Access B
Multi-layer Core Backbone
VoD
Internet
Quays
TG
Computer Network (gt50 users)
Differentiated Optical Services
Differentiated Optical Services
FixedAccess A
HD Video Conferencing
WirelessAccess A

• Focus
• demonstration
• development of a pub/sub control plane for
  optical networks

South Courts
TG
iSPACE
76
Working With Partners An Open Innovation Approach
International Consultation and Engagement EIFFEL
support action, FIRE expert group, GENI/FIRE
group, ITU Telecom Forum EC, NSF, ITU
Socio-economics
Cambridge Wireless SIG on "Wireless Sensing" SIG
Champion
Who? What? Why?
Concerns
Capturing concerns
Policy Plane
Privacy Security WG Karen Sollins, David
Clark, David Reed
Hodgkins award (UCL) Data fusion in sensor
networks John Mitchell
Representation, fusion, mining, mediation,
reasoning(Toolbox for information)
Information Plane
Generic data plane for providing any piece of
information
Provisioning Plane
77
Conclusions

• IN is a not (yet) a blueprint for a new network
• Proposition for change with more questions than
  answers at this moment - but we find more and
  more answers
• IN is long-term research
• If I knew what I was doing, it wouldn't be called
  research (A. Einstein)
• IN is working, however
• Demonstrator available
• Growing efforts towards networked environment
  (Essex)
• Question to be answered Is this the type of
  research BT wants (in-house)?

78
For next week (Tuesday 10th nov)

• I want each of you to read about existing CDNs
  such as Akamai
• And come up with
• What do they solve
• What do they not solve
• And email me 1 slide with that on!
• Which YOU will present!
• And we will discuss how the desiderata
  (requirements) changed!

79
Second Essay (due Nov 27)

• Take one of
• PGM/TIBCO Rendezvous, CCN, DONA, PSIRP
• Compare with baseline (e.g Web, even Akamai)
• Whats new/better
• E.g. scale, flexibility
• What are appropriate perf eval
• E.g. security (what threats?)
• Where do they fall down
• 2-4 pages of notes
• Remember to cite any sources you use!