Reliable, Scalable and Interoperable Internet Telephony

1
Reliable, Scalable and Interoperable Internet
Telephony

• PhD thesis presentation
• by Kundan Singh
• Advisor Henning Schulzrinne
• Computer Science Department, Columbia University,
  New York
• June 21, 2006

2
My research background/timeline
Conference evaluation
P2P VoIP using SIP
SIP Failover/load sharing
Multimedia collaboration
CINEMA user interface
Interactive voice response
Enterprise VoIP infrastructure
Mobile NAT
Libsip (SIP library)
SIP conferencing
SIP-RTSP voice mail
SIP-H.323 translator
Reliability scalability
PhD_at_cs.columbia
H.323 client gateway
VoIP infrastructure
MS_at_cs.columbia
Work_at_ Motorola
Undergrad _at_India
1997 1999 2000 2001 2002
2003 2004 2005 2006
3
Outline of the presentation

• Introduction
• What is the problem? Why important?
• My contributions
• Server redundancy
• Load sharing and failover in SIP telephony
• Comparison of thread models for SIP server
• Peer-to-peer (P2P)
• SIP servers using external P2P network
• Additionally, P2P maintenance using SIP
• Enterprise IP telephony
• Multi-platform collaboration using SIP
• Scalable centralized conferencing architecture
• Interworking between SIP/SDP and H.323
• Conclusion

36 slides
4
Telephone reliability scalability (PSTN
Public Switched Telephone Network)
bearer network
telephone switch(SSP)
Switches strive for 99.999 availability. Lucents
5E-XC supports 4 million BHCA.
5
Internet telephony(SIP Session Initiation
Protocol)
alice_at_yahoo.com
bob_at_example.com
yahoo.com
example.com
192.1.2.4
129.1.2.3
DB
Can SIP server provide carrier grade reliability
and scalability using commodity hardware
6
What are the problems?

• Can SIP server provide carrier-grade reliability
  and scalability using commodity hardware?
• What affects the server performance?
• How can we build a server-less self-organizing
  peer-to-peer VoIP network?
• Can this be done in standards compliant way?
• Can communication be extended to multi-platform
  collaboration using existing protocols?
• How well multi-party conferencing scales?
• How to interoperate between SIP and H.323?

7
My contributions

• Server redundancy
• Implemented failover using database replication
• Two-stage architecture for SIP load sharing
• Comparison of thread models for SIP server
• Peer-to-peer (P2P)
• SIP servers using external P2P network
• Additionally, P2P maintenance using SIP
• Enterprise IP telephony
• Multi-platform collaboration using SIP
• Scalable centralized conferencing architecture
• Interworking between SIP/SDP and H.323

New architecture, New algorithm or approach,
Implementation, Evaluation
8
Outline of the presentation

• Introduction
• What is the problem? Why important?
• My contributions
• Server redundancy
• Load sharing and failover in SIP telephony
• Comparison of thread models for SIP server
• Peer-to-peer (P2P)
• SIP servers using external P2P network
• Additionally, P2P maintenance using SIP
• Enterprise IP telephony
• Multi-platform collaboration using SIP
• Scalable centralized conferencing architecture
• Interworking between SIP/SDP and H.323
• Conclusion

9
Server redundancyThe problem failure or overload
10
High availabilityImplementation and analysis
Slave/ master
Master/ slave
D2
D1

• Implementation
• Using MySQL replication
• System reliability
• individual component reliability
• Call setup latency
• DNS TTL, time-to-repair, retry timeout
• User unavailability
• Most registers are refreshes
• Retry timeout, replication interval, register
  refresh interval

P2
P1
11
ScalabilityLoad sharing redundant proxies and
databases

• REGISTER
• Write to D1 D2
• INVITE
• Read from D1 or D2
• Database write/ synchronization traffic becomes
  bottleneck

P1
D1
P2
D2
P3
12
ScalabilityLoad sharing divide the user space

• Proxy and database on the same host
• Stateless proxy can become overloaded
• Use many

P1
D1
a-h
P2
D2
i-q
P3
D3
r-z
13
ScalabilityComparison of the two designs
Low Scalability High Reliability
High Scalability Low Reliability
14
Scalability (and reliability)Two stage
architecture
I stage
II stage
a_at_example.com
a.example.com _sip._udp SRV 0 0 a1.example.com
SRV 1 0 a2.example.com
a1
s1
a2
sipbob_at_example.com
s2
sipbob_at_b.example.com
b_at_example.com
b.example.com _sip._udp SRV 0 0 b1.example.com
SRV 1 0 b2.example.com
s3
b1
b2
ex
example.com _sip._udp SRV 0 40 s1.example.com
SRV 0 40 s2.example.com SRV 0 20
s3.example.com SRV 1 0 ex.backup.com
Capacity f(stateless, groups)
15
Load SharingPerformance result calls/second

• Using 33 servers gives carrier-grade performance
  (10 million busy hour call attempts)
• Registration test supports 10 million subscribers

On commodity hardware 3 GHz, Pentium 4, 1 GB
memory Test with UDP, stateless, no DNS and no
mempool
16
Server performanceWhat happens inside a server?
What thread/event models possible?
Web server
SIP server

1. Pure event-based (one thread)
2. Thread-per-msg or transaction
3. Pool-thread per msg (sipd)
4. Two stage thread pool

17
Server performanceResults of my measurement

• Event-based performs 30 better than existing
  thread-pool architecture on single-CPU
• Two stage thread-pool architecture gives better
  performance on multi-CPU
• 60 more on 4xPentium
• Both Pentium and Sparc took 2 MHz of CPU cycles
  per call/s on single-CPU

18
Problem with servers

• Server-based
• Cost maintenance, configuration
• Central points of failures, catastrophic failures
• Controlled infrastructure (e.g., DNS)
• Peer-to-peer
• Robust no central dependency
• Self organizing, no configuration
• Inherent scalability

19
We built P2P-SIP

• Unlike proprietary Skype architecture
• Robust and efficient lookup using DHT
• Interoperability
• DHT algorithm uses SIP communication
• Hybrid architecture
• Lookup in SIPP2P
• Inter-domain P2P-SIP
• Unlike file-sharing applications
• Data storage, caching, delay, reliability
• Data model and service model
• Disadvantages
• Lookup delay and security

20
How to combine SIP P2P?

• P2P-over-SIP
• Additionally, implement P2P using SIP messaging

• SIP-using-P2P
• Replace SIP location service by a P2P protocol

SIP-using-P2P P2P SIP proxies P2P-over-SIP
Maintenance P2P P2P SIP
Lookup P2P SIP SIP
P2P network
REGISTER
INVITE alice
P2P-SIP overlay
FIND
INSERT
Alice 128.59.19.194
INVITE sipalice_at_128.59.19.194
Alice 128.59.19.194
21
SIP-using-P2PLogical Operations

• Contact management
• put (user id, signed contact)
• Key storage
• User certificates and private configurations
• Presence
• put (subscribee id, signed encrypted subscriber
  id)
• Composition needs service model
• Offline message
• put (recipient, signed encrypted message)
• NAT and firewall traversal
• STUN and TURN server discovery needs service model

XML-based data format
22
SIP-using-P2PImplementation in SIPc with the
help of Xiaotao Wu

• OpenDHT
• Using Data model
• Identity protection
• Certificate-based
• SIP id email
• P2P for
• Calls, IM, presence, offline message and name
  lookup

23
P2P-over-SIPArchitecture and implementation

• DHT (Chord) algorithm using SIP messages with
  query and update semantics of REGISTER
• Has SIP registrar, proxy, user agent
• Other discovery, NAT traversal, failover
• Adaptor allows existing SIP devices to become
  P2P

24
P2P-over-SIPAnalysis scalability

• Computed message load as function of
• Refresh rate (keep-alive, finger table, user
  registration), call arrival rate, churn (join,
  leave, failure), scale (number of peer nodes and
  users)
• Number of nodes f(individual node capacity)
• Measured performance 800 register/s.
• Assuming a conservative 10 reqister/s capacity,
  and aggressive refresh and call rate of 1/min, it
  gives more than 16 million peers (super nodes) in
  the network.

25
P2P-over-SIPAnalysis availability and call
setup latency

• To increase user availability
• Fast failure detection increase keep-alive rate
• Reduce unavailability frequent registration
  refresh
• Replicate user and node registrations
• Call setup latency
• Same as DHT lookup latency O(log(N))
• Calls to known locations (buddies) is direct
• Chord 10000 nodes gt 6 hops
• At most a few seconds
• User availability and retransmission timers

26
SIP-using-P2P vs P2P-over-SIP

• Not SIP-specific, hence no implementation
  overhead for non-VoIP but P2P applications
• Low transport and transaction overhead
• No P2P security burden on SIP
• No dependency on single DHT implementation

• Reuse SIP naming, routing, security, NAT/firewall
  traversal
• Easily reuse existing SIP components without
  change
• voicemail, conference
• Single DHT implementation
• Readily supports service model

27
Server-based vs peer-to-peer
Reliability, failover latency DNS-based. Depends on client retry timeout, DB replication latency, registration refresh interval DHT self organization and periodic registration refresh. Depends on client timeout, registration refresh interval.
Scalability, number of users Depends on number of servers in the two stages. Depends on refresh rate, join/leave rate, uptime
Call setup latency One or two steps. O(log(N)) steps.
Security TLS, digest authentication, S/MIME Additionally needs a reputation system, working around spy nodes
Maintenance, configuration Administrator DNS, database, middle-box Automatic one time bootstrap node addresses
PSTN interoperability Gateways, TRIP, ENUM Interact with server-based infrastructure or co-locate peer node with the gateway
28
Outline of the presentation

• Introduction
• What is the problem? Why important?
• My contributions
• Server redundancy
• Load sharing and failover in SIP telephony
• Comparison of thread models for SIP server
• Peer-to-peer (P2P)
• SIP servers using external P2P network
• Additionally, P2P maintenance using SIP
• Enterprise IP telephony
• Multi-platform collaboration using SIP
• Scalable centralized conferencing architecture
• Interworking between SIP/SDP and H.323
• Conclusion

29
Internet telephony infrastructureCINEMA
Columbia InterNet Extensible Multimedia
Architecture
CINEMA servers
Telephone switch
rtspd media server
Local/long distance 1-212-5551212
sipconf Conference server
Quicktime
RTSP
PSTN
RTSP clients
Department PBX
sipum Unified messaging
sipd Proxy, redirect, Registrar server
Internal Telephone Extn 7040
713x
SQL database
cgi
SIP/PSTN Gateway
vxml
Web based configuration
H.323
Built many components in a complete system for
enterprise IP telephony and multimedia
collaboration
siph323 SIP-H.323 translator
NetMeeting
30
My other work

• Communication to collaboration
• Comprehensive, multi-platform collaboration using
  SIP
• Unified messaging The gaps among different media
  (audio, video, text), devices (PC, phone) and
  means of communications (Email, SIP, IM)
  disappear for messaging
• Novel SIP/RTSP based voicemail and answering
  machine
• SIP interface to VoiceXML browser
• Centralized conferencing
• Audio mixing, video forwarding, IM, shared web
  browsing, screen sharing, web-based configuration
  and control, floor control
• Performance evaluation cascaded server
  architecture
• SIP-H.323 translation

31
Conference serverPerformance evaluation of audio
mixer

• On commodity PC
• About 480 participants in a single conference
  with one active speaker
• About 80 four-party conferences, with one speaker
  each
• Both Pentium and Sparc took 6 MHz per participant

32
Conference serverCascaded architecture
SIP REFER message is used to create cascading
?
?
?
?
?
?

• N.(N-1) participants
• Higher delay

• N2/4 participants
• Lower delay

I measured the CPU usage for two cascaded
servers supports about 1000 participants
33
Outline of the presentation

• Introduction
• What is the problem? Why important?
• My contributions
• Server redundancy
• Load sharing and failover in SIP telephony
• Comparison of thread models for SIP server
• Peer-to-peer (P2P)
• SIP servers using external P2P network
• Additionally, P2P maintenance using SIP
• Enterprise IP telephony
• Multi-platform collaboration using SIP
• Scalable centralized conferencing architecture
• Interworking between SIP/SDP and H.323
• Conclusion

34
Revisiting the problems
Developed a two stage scalable and reliable SIP
server architecture linear scaling. Use
event-based.

• Can SIP server provide carrier-grade reliability
  and scalability using commodity hardware?
• What affects the server performance?
• How can we build a server-less self-organizing
  peer-to-peer VoIP network?
• Can this be done in standards compliant way?
• Can communication be extended to multi-platform
  collaboration using existing protocols?
• How well multi-party conferencing scales?
• How to interoperate between SIP and H.323?

Developed P2P-SIP architecture SIP-using-P2P and
P2P-over-SIP
Multi-platform collaboration using existing
protocols and tools, unified messaging,
centralized conferencing (cascaded), SIP-H.323
interworking.
35
Conclusions

• Impact
• Commercialized by SIPquest (now FirstHand) and
  sold to many customers.
• CINEMA was deployed in our department for a brief
  period of time.
• Used in various other projects at IRT NG911,
  firewall controller, presence scalability,
  TCP/TLS measurements,
• P2P-SIP is a hot topic in industry and IETF now
  client desktop, hardware phone as well as
  server vendors are pursuing this.
• SIP-H.323 requirements eventually became an RFC
• Plan to open source SIPc for large scale
  deployment experience of P2P-SIP
• Started working on a P2P-based self organizing
  servers for 3GPP at Bell Labs
• So what (Implications)
• Replacing PSTN better features, quality and
  performance at lower cost and maintenance zero
  cost VoIP using P2P-SIP
• Distributed, multi-provider, component
  architecture for telephony and collaboration

36
My publicationsConference, workshop, technical
report, magazine/journal

• K. Singh and H. Schulzrinne, Using an external
  DHT as a SIP location service", Columbia
  University Technical Report CUCS-007-06, NY,
  Feb06.
• K. Singh and H. Schulzrinne, Peer-to-peer
  Internet telephony using SIP", NOSSDAV, Skamania,
  Washington, Jun 2005..
• K. Singh and H. Schulzrinne, "Peer-to-peer
  Internet Telephony using SIP", New York Metro
  Area Networking Workshop, CUNY, NY, Sep 2004.
• K. Singh and H. Schulzrinne, "Peer-to-peer
  Internet Telephony using SIP", Columbia
  University Technical Report CUCS-044-04, NY, Oct
  2004.
• K. Singh and H. Schulzrinne, Failover, load
  sharing and server architecture in SIP
  telephony, Elsevier Computer Communication
  Journal. To appear. Aug 2006.
• K. Singh and H. Schulzrinne, Failover and load
  sharing in SIP telephony", SPECTS (Symposium on
  performance evaluation of computer and
  telecommunication systems). Philadelphia, PA, Jul
  2005.
• K. Singh and H. Schulzrinne, "Failover and Load
  Sharing in SIP Telephony", Columbia University
  Technical Report CUCS-011-04, NY, May 2004.
• H. Schulzrinne, K. Singh and X. Wu, "Programmable
  Conference Server", Columbia University Technical
  Report CUCS-040-04, NY, Oct 2004.
• K. Singh, Xiaotao Wu, J. Lennox and H.
  Schulzrinne, "Comprehensive Multi-platform
  Collaboration", MMCN 2004 - SPIE Conference on
  Multimedia Computing and Networking, Santa Clara,
  CA, Jan 2004.
• K. Singh, Xiaotao Wu, J. Lennox and H.
  Schulzrinne, "Comprehensive Multi-platform
  Collaboration", Columbia University Technical
  Report CUCS-027-03, NY, Nov 2003.
• M. Buddhikot, A. Hari, K. Singh and S. Miller,
  "MobileNAT A new Technique for Mobility across
  Heterogeneous Address Spaces", ACM MONET journal,
  March 2005.
• M. Buddhikot, A. Hari, K. Singh and S. Miller,
  "MobileNAT A new Technique for Mobility across
  Heterogeneous Address Spaces", WMASH 2003 - ACM
  International Workshop on Wireless Mobile
  Applications and Services on WLAN Hotspots, San
  Diego, CA, Sep 2003.
• K. Singh, A. Nambi and H. Schulzrinne,
  "Integrating VoiceXML with SIP services", ICC
  2003 - Global Services and Infrastructure for
  Next Generation Networks, Anchorage, Alaska, May
  2003.
• K. Singh, A. Nambi and H. Schulzrinne,
  "Integrating VoiceXML with SIP services", Second
  New York Metro Area Networking Workshop, Columbia
  University, NY, Sep 2002.
• K. Singh, W. Jiang, J. Lennox, S. Narayanan and
  H. Schulzrinne, "CINEMA Columbia InterNet
  Extensible Multimedia Architecture", Columbia
  University Technical Report CUCS-011-02, NY, May
  2002.
• W. Jiang, J. Lennox, H. Schulzrinne and K. Singh,
  "Towards Junking the PBX Deploying IP
  Telephony", NOSSDAV 2001.
• W. Jiang, J. Lennox, S. Narayanan, H.
  Schulzrinne, K. Singh and X. Wu, "Integrating
  Internet Telephony Services", IEEE Internet
  Computing (magazine), May/June 2002 (Vol. 6, No.
  3).
• K. Singh, Gautam Nair and H. Schulzrinne,
  "Centralized Conferencing using SIP", 2nd
  IP-Telephony Workshop (IPTel'2001), April 2001.
• K. Singh and H. Schulzrinne, "Unified Messaging
  using SIP and RTSP", IP Telecom Services Workshop
  2000, Atlanta, Georgia, U.S.A, Sept 2000.