Protocol and Network Design Challenges for a Consumer-Grade Internet

1
Protocol and Network Design Challenges for a
Consumer-Grade Internet

• Henning Schulzrinne
• (with members of the IRT group)
• Dept. of Computer Science
• Columbia University
• ICNP (Berlin)
• October 6, 2004

2
Overview

• Life cycle of network technology
• Challenges for network research
• no longer a young discipline
• from performance to usability
• Anywhere, any time, any media ? leave me
  alone
• Permission-based networking
• spam, DDOS and phishing
• Lessons from designing IETF protocols
• protocol design engineering trade-offs
• Skype, Asterisk and SIP

3
Network evolution
4
Lifecycle of technologies
traditional technology propagation
military
corporate
consumer
opex/capex doesnt matter expert support
capex/opex sensitive, but amortized expert
support
capex sensitive amateur
Can it be done?
Can I afford it?
Can my mother use it?
5
Lifecycle of technologies

• Examples
• content creation
• video cameras, non-linear editors,
• 35mm blue slides ? PowerPoint in elementary
  school
• LANs corporate ? home Ethernet wireless
• But
• now often faster technology evolution for
  consumers
• initial expense discourages replacement
• viz. airline 3270 terminals
• military GPS units

6
Internet and networks timeline
university prototypes
production use in research
commercial early residential
broadband home
theory
1960
1970
1980
1990
2000
2010
port speeds
100 kb/s
1 Mb/s
10 Mb/s
100 Mb/s
1 Gb/s
email ftp
DNS RIP UDP TCP SMTP SNMP finger
ATM BGP, OSPF Mbone IPsec HTTP HTML RTP
XML OWL SIP Jabber
Internet protocols
queuing architecture
routing cong. control
DQDB, ATM QoS VoD
p2p ad-hoc sensor
7
Networking is getting into middle years
idea current
IP 1969, 1980? 1981
TCP 1974 1981
telnet 1969 1983
ftp 1980 1985
8
Technologies at 30 years

• Other technologies at similar maturity level
• air planes 1903 1938 (Stratoliner)
• cars 1876 1908 (Model T)
• analog telephones 1876 1915 (transcontinental
  telephone)
• railroad 1800s -- ?
• (Communications) technologies rarely disappear
  (as long as operational cost is low)
• exceptions
• telex, telegram, semaphores ? fax, email
• X.25 OSI, X.400 ? IP, SMTP
• analog cell phones
• ? thus, NGN (post-IP) discussions likely academic

9
History of networking

• History of networking non-network applications
  migrate
• postal intracompany mail, fax ? email, IM
• broadcast TV, radio
• interactive voice/video communication ? VoIP
• information access ? web, P2P
• disk access ? iSCSI, Fiberchannel-over-IP
• Initially, mainstream applications (email)
• followed later (now) by niches
• high reliability requirements specialized
  features
• air traffic control
• EDI ? XML
• 911 system CAMA ? IP

10
Network evolution

• Only three L2/L3 modes, now thoroughly explored
• packet/cell-based
• message-based (application data units)
• session-based (circuits)
• Applications
• pull
• web, ftp, RPC
• push asynchronous
• email
• push synchronous
• continuous media
• IM

• Replace specialized networks
• left to do embedded systems
• need cost(CPU network) lt 10
• cars
• industrial (manufacturing) control
• commercial buildings (lighting, HVAC, security
  now LONworks)
• remote controls, light switches
• keys replaced by biometrics

11
Transit cost (OC-3, NY London)
12
Commercial access cost (T1)
13
Disk storage cost (IDE)
14
Network Research
15
Networking research is fashion-driven
workshop white paper
DARPA, NSF ?
EU Nth framework
trailing-edge research
Sigcomm Infocom Mobicom ICNP
networking courses First (European) workshop on X
-- YAP on X
secondary conferences
ATM DQDB QoS
mobile networks wireless ad-hoc, sensor
active networks
16
Impact of network research

• Whats promising/interesting two different
  axes
• Intellectual merit ? interesting analysis,
  broadly applicable,
• Satisfies practical needs ? may not be a
  scientific breakthrough
• Field has few grand challenges and metrics
• cf., speech understanding or face recognition
• Depends largely on external technology inputs
• faster CPUs, better optical gear, compression
• typical performance improvements in queueing
  20-50

• Networking research impact
• on deployed systems and protocols?
• on understanding network behavior?
• on other papers?
• Which of the 10,000 QoS papers had real impact?
• What papers were responsible for most important
  networking advances?
• TCP ?, web?, email?

17
Whats fashionable (and not)

• Judging from Infocom submissions and NSF panels
• Security of any sort
• Peer-to-peer networks
• Sensor networks
• Overlay networks
• Network measurements
• Whats not
• QoS scheduling, admission control,
• Active networks
• Multicast

18
Maturing network research

• Old questions
• Can we make X work over packet networks?
• All major dedicated network applications (flight
  reservations, embedded systems, radio, TV,
  telephone, fax, messaging, ) are now available
  on IP
• Can we get M/G/T bits to the end user?
• Raw bits everywhere any media, anytime,
  anywhere
• New questions
• Dependency on communications ? Can we make the
  network reliable?
• Can non-technical users use networks without
  becoming amateur sys-admins? ? auto/zeroconfigurat
  ion, autonomous computing, self-healing networks,
  
• Can we prevent social and financial damage
  inflicted through networks (viruses, spam, DOS,
  identity theft, privacy violations, )?

19
Observations on network research

• Frustration with inability to change network
  infrastructure in less than 10 -- 20 year
  horizons
• IPv6
• Layer-3 multicast
• QoS
• Security
• Network research community has dismal track
  record for new applications
• web, IM, P2P (Gnutella, BitTorrent), vs.
  video-on-demand
• Niche applications get disproportionate attention
• active networks, ad-hoc networks, (structured)
  P2P
• successful applications dont care if they dont
  scale
• centralized IM search, unstructured P2P,
• Disconnect from standardization
• Few attempts to bring research work into
  standards bodies
• Standards bodies slow to catch up (e.g., P2P)

20
Why do good ideas fail?

• Research O(.), CPU overhead
• per-flow reservation (RSVP) doesnt scale ? not
  the problem
• at least now -- routinely handle O(50,000)
  routing states
• Reality
• deployment costs of any new L3 technology is
  probably billions of
• Cost of failure
• conservative estimate (1 grad student year 2
  papers)
• 10,000 QoS papers _at_ 20,000/paper ? 200 million

21
Cause of death for the next big thing
QoS multi- cast mobile IP active networks IPsec IPv6
not manageable across competing domains ? ? ? ?
not configurable by normal users (or apps writers) ? ? ?
no business model for ISPs ? ? ? ? ? ?
no initial gain ? ? ? ? ?
80 solution in existing system ? ? ? ? ? ? (NAT)
increase system vulnerability ? ? ? ?
22
QoS

• QoS is meaningless to users
• difficult to engineer service that is
  consistently poor, but usable
• common QoS models now
• scavenger service (worse-than-best-effort) ?
  self-protection
• DiffServ on access routers and NAT boxes
• care about service availability ? reliability
• but most commercial service is good enough for
  VoIP/video/ most of the time
• charging model problem ? users will arbitrage and
  buy basic quality except during congestion
  periods
• see multi-homing vs. high-end providers
• as more and more value depends on network
  services, can't afford random downtimes

23
Why did QoS fail?

• hypothesis The success of a technology is
  inversely proportional to the number of papers
  published before its popularity.
• ACM 10,158 papers with QoS or quality of
  service in abstract
• IEEE 7,297 papers
• real-time streaming video-on-demand ? DVD via
  Netflix or TCP onto 200 GB hard disk

• bandwidth too cheap to meter
• undemocratic some traffic is more equal than
  others
• reminds you of your mom no, you cant have that
  10 Mb/s now
• socialist administer scarcity - we like SUVs (or
  to drive 100 mph)!
• risky scheme security
• only displacement applications (such as
  telephony) need QoS
• requires cooperation edge-ISP, transit ISPs, end
  systems
• snake oil add QoS, lose half your bandwidth

24
Why did QoS fail? (contd)

• dishonesty we only talk about the beneficiaries
• network has become harder to evolve
• network address translation
• firewalls
• high packetization overhead (VPNs, IPv6)
• to be useful, has to be nearly universally
  supported (no, you cant make calls to AS 123)
• network QoS vs. business class model coach is
  empty, please refund fare

• currently, the ISP interface is IP and BGP
  adding a third one is a big deal
• new Internet service model TCP client (inside)
  server (outside)
• exception peer-to-peer on college campuses
• network to host you first, no, you first
• failure of IP QoS ? success of MPLS
• more TE than QoS

25
Standardization

• Oscillate convergence ? divergence
• continued convergence clearly at physical layer
• connectivity trumps functionality
• niches larger ? support separate networks
• Really two facets of standardization
• public, interoperable description of protocol,
  but possibly many (Tanenbaum)
• reduction to 1-3 common technologies
• LAN Arcnet, tokenring, ATM, FDDI, DQDB, ?
  Ethernet
• WAN IP, X.25, OSI ? IP
• OS dozens ? Windows Linux

26
Standardization

• Have reached phase 2 in most cases, with RPC
  (SOAP) and presentation layer (XML) most recent
  'conversions
• Often, non-standardized technologies can be
  deployed faster
• single (dominant) vendor
• Skype vs. SIP and H.323
• AOL IM and Jabber vs. SIMPLE
• SMB vs. NFS
• ? Standardization after success
• IETF one-protocol-for-application vs.
  everything-is-RPC
• not enough network experts ? standardization
  scales better

27
Network research

• Old goal "new universal network"
• IP, OSI, ATM
• New goal "niche networks"
• may claim universality
• peer-to-peer, ad hoc, sensor, mesh,
• New research community realizations
• difficulty in rolling out new network ?
  incrementalism
• spectrum issues (open spectrum, SDR, )

28
Infrastructure research questions Scaling,
Maintainability, Security,

• Scaling
• no major changes for 20 years (link-state, DV,
  etc.)
• two-layer (intra/inter) ? other routing paradigms
• Maintainability
• protocols and systems are not designed with fault
  diagnosis capabilities
• e.g., transparent proxies, routing, DNS, hacked
  traceroute
• Security
• secure routing protocols
• DOS prevention (pushback, source discovery)

29
and Reliability

• we dont know precisely why network applications
  fails
• components and backbones appear to pretty
  reliable
• but we measured at 99.5 of usable time ? far
  below 99.999 in telecom networks
• lots of possible culprits, including DNS and
  carrier interconnects
• temporary overloads
• reduce operator errors
• e.g., XCONF effort in IETF
• inherently safe or fail-safe protocols?
• faster convergence in routing protocols
• BGP ? up to 20-30 minutes!

30
Transport layer

• After XTP flop, flurry of new protocols SCTP,
  DDCP, UDP-lite
• fill in gaps in TCP/UDP
• flow control without reliability
• multiple logical streams with one flow control
  stream (cf. HTTP/1.1)
• Issues of very fat pipes
• single packet loss can wipe out effective
  throughput

31
Applications

• Transition of custom protocols to XML, SOAP?
• but this is the not the first try (DCE, SunRPC,
  COM, Java RMI, Corba, )
• Scalable event systems (research)
• Presence (SIP/SIMPLE, Jabber, )
• P2P systems
• Application-layer routing and overlays,
  multicast, discovery,
• The categorical imperative for network design
  act only in accordance with that maxim through
  which you can at the same time will that it
  become a universal law.
• satisfied by overlay networks for packet routing?
  flooding P2P systems?

32
New applications

• New bandwidth-intensive applications
• Reality-based networking
• (security) cameras
• Distributed games often require only
  low-bandwidth control information
• current game traffic VoIP
• Computation vs. storage vs. communications
• communications cost has decreased less rapidly
  than storage costs
• Emphasis on user control of communications
• from anywhere, anytime, any media to where
  appropriate, my time, my media
• Guess 1 user-selected research problem fix
  spam
• 2 keep cell phone from ringing in the movie
  theater

33
Fundamental re-thinking

• "Hour glass" with single address space ? multiple
  gatewayed networks with separate address spaces
• diversity vs. uniformity
• Application-neutral connectivity ? filtered
  restricted (? tunneling over port 80)

34
New network architectures for security
35
Security challenges

• DOS, security attacks ? permissions-based
  communications
• only allow modest rates without asking
• effectively, back to circuit-switched
• Higher-level security services ? more
  application-layer access via gateways, proxies,
• User identity
• problem is not availability, but rather
  over-abundance

36
Trustability Internet decay

• Decay of inner cities small number of bad
  elements lack of social controls and law
  enforcement
• Small number of miscreants
• The bulk of U.S. spam is coming from a very
  limited set of IPs with high-bandwidth
  connections," said Alperovitch, who estimated
  that the high-volume spamming addresses number
  fewer than 10,000 and the number of spammers at
  less than 200. (Informationweek, Aug. 2004)
• Naïve users
• with increasing firepower

37
Trustability problems

• Traditional security didnt solve user interface
  problem
• is citi-bank.com my bank or phishing?
• traditional firewall (crunchy outside, squishy
  inside)
• fails with any content even JPEGs arent safe
• email usability rapidly decreasing
• most spam proposals unlikely to work
• notion of global village is an oxymoron
• in a village, you know your neighbors
• on-going approaches useful, but limited
• conversion of protocols to secured versions
  (e.g., via TLS)
• prevent source address spoofing
• OS and application robustness against buffer
  overflow attacks
• IETF MARID (SenderID, SPF, ) for email sender
  identification
• DOS traceback
• thus, may need to rethink network architecture

38
Trustability A more polite Internet

• introduce yourself first
• shoot first, ask later (Bush)
• ask first, shoot later (Kerry)

yes, up to 10 kb/s
may I send?

• limits large-scale DDOS
• more circuit-oriented
• may get permission slip for future use

39
Restoring the village part of the global village

• Its not what you know, its who you know
• Authentication works only if addresses can be
  recognized by policy or human
• Doesnt work well for first-time contacts ? much
  of communications
• wont be fixed by SPF and SenderID
• Need to leverage indirect knowledge
• our approach social networks for recognizing
  users in SIP systems
• leverage knowledge across media visiting web
  page enables receipt of email from related
  address ? make phishing more difficult

40
Reflections on protocol design
41
Internet services the missing entry
Service/delivery synchronous asynchronous
push instant messaging presence event notification session setup media-on-demand messaging
pull data retrieval file download remote procedure call peer-to-peer file sharing
42
Filling in the protocol gap
Service/delivery synchronous asynchronous
push SIP RTSP, RTP SMTP
pull HTTP ftp SunRPC, Corba, SOAP (not yet standardized)
43
SIP trapezoid
destination proxy (identified by SIP URI domain)
outbound proxy
1st request
SIP trapezoid
2nd, 3rd, request
a_at_foo.com 128.59.16.1
registrar
voice traffic RTP
44
Reflections on protocol development SIP

• Decisions that worked well
• generality over efficiency
• proxies as first-class protocol entities
• but would remove distinction between proxies and
  B2BUAs
• text-based protocol ( compression)
• XML would be fashionable now, but would not allow
  additional capabilities
• separation of header and session description/body
• allowed expanding scope to presence IM
• allowed S/MIME
• allow simple implementation
• but many simple implementations are lazy

45
and those Id like to re-think

• too many options in text encoding
• lower/upper case, spaces, line folding,
  escaping,
• support of unreliable, non-secure transport
• but UDP ensured early implementations and lower
  latency
• special-casing INVITE transaction
• optimized in messages, but complicates
  implementation
• should have had generic state transition
  indication
• should have allowed for more regular insertion
  mechanism for protocol elements
• but would probably be fairly expensive

46
Protocols are now ecosystems
includes draft-ietf--00 and draft-personal--00
47
Combining client-server and P2P P2P-SIP

• Unlike server-based SIP architecture
• Unlike proprietary Skype architecture
• Robust and efficient lookup using DHT
• Interoperability
• DHT algorithm uses SIP messages
• Hybrid architecture
• Lookup in SIPP2P, end-to-end communications
• Unlike file-sharing applications
• Data storage, caching, delay, reliability
• Disadvantages
• Lookup delay and security

48
Architecture
Signup, Find buddies
IM, call
On reset
Signout, transfer
On startup
Leave
Find
Join
REG, INVITE, MESSAGE
Multicast REG
Peer found/ Detect NAT
REG

• DHT communication using SIP REGISTER
• Known node sip15_at_192.2.1.3
• Unknown node sip17_at_sippeer.net
• User sipalice_at_example.com

49
Dialing Out

• Call, instant message, etc.
• INVITE siphgs10_at_columbia.edu
• MESSAGE sipalice_at_yahoo.com
• If existing buddy, use cache first
• If not found
• SIP-based lookup (DNS NAPTR, SRV,)
• P2P lookup
• Send to super-nodes proxy
• Use DHT to locate proxy or redirect to next hop

INVITE key42
Last seen
302
INVITE
DHT
42
50
Conclusion

• Is networking research becoming like civil
  engineering large, important infrastructure, but
  resistant to fundamental change?
• Challenges are in reliability and
  maintainability, rather than performance or
  packet-loss jitter QoS
• As a community, need to learn more from our
  collective and individual mistakes
• Need series The design mistakes in popular
  system or protocol