Reliable and Scalable Internet Telephony

1
Reliable and Scalable Internet Telephony

• Kundan Singh and Henning Schulzrinne
• Internet Real Time Lab Internal Talk
• Sept 24, 2004

2
Telephone reliability(PSTN Public Switched
Telephone Network)
bearer network
telephone switch(SSP)
3
Internet telephony(SIP Session Initiation
Protocol)
alice_at_yahoo.com
bob_at_example.com
yahoo.com
example.com
192.1.2.4
129.1.2.3
DB
4
SIP network architectureScalability requirement
depends on role
Cybercafe
ISP
IP network
IP phones
GW
ISP
MG
MG
SIP/MGC
GW
SIP/PSTN
SIP/MGC
Carrier network
MG
GW
PBX
T1 PRI/BRI
PSTN phones
PSTN
5
Reliability and scalabilityfor call routing,
registration, conferencing, voicemails

• Requirements
• Reliable
• Mean Time Between Failures (MTBF), Mean Time To
  Recover (MTTR)
• Scalable
• Registration rate, call rate, requests/s
• Proposed solutions
• Server redundancy
• Apply existing web-redundancy designs
• Evaluate quantitatively (future work)
• Peer-to-peer
• Novel P2P-SIP architecture
• Evaluate quantitatively (future work)

6
Server redundancyThe problem failure or overload
REGISTER
INVITE
7
Server redundancyReplicate registration or
search on call
REGISTER
INVITE
REGISTER
INVITE
8
Server redundancyKnown techniques

• Client-based
• Cisco phones primary and backup proxy
• DNS
• NAPTR, SRV
• IP address takeover
• Database redundancy
• . . .

9
High availabilityFailover in CINEMA
Web scripts
Web scripts
D2
D1
Slave/ master
Master/ slave
replication
P2
P1
phone.cs.columbia.edu
sip2.cs.columbia.edu
REGISTER
_sip._udp SRV 0 0 5060 phone.cs.columbia.edu
SRV 1 0 5060 sip2.cs.columbia.edu
proxy1 phone.cs backup sip2.cs
10
High availabilityTime to recover

• Client re-sends INVITE to P2
• Immediately on ICMP error
• Or after 10s otherwise
• sipd has in-memory cache
• Refresh registration much before expiry
• Registrations are additive
• Measurement of recovery time
• Optimal servers

11
ScalabilityLoad sharing redundant proxies and
databases

• REGISTER
• Write to D1 D2
• INVITE
• Read from D1 or D2
• Database write/ synchronization traffic becomes
  bottleneck

P1
D1
P2
D2
P3
12
ScalabilityLoad sharing divide the user space

• Proxy and database on the same host
• Stateless proxy can become overloaded
• Hashing
• Static vs dynamic

P1
D1
a-h
P2
D2
i-q
P3
D3
r-z
13
ScalabilityComparison of the two designs
P1
P1
a-h
D1
D1
P2
P2
i-q
D2
D2
P3
P3
D2
r-z
Total time per DB

• ((tr/D)1)TN
• (A/D) B

• ((tr1)/D)TN
• (A/D) (B/D)

D number of database servers N number of
writes (REGISTER) r reads/writes
(INVREG)/REG 2 T write latency t read
latency/write latency
14
Reliability and scalabilityTwo stage
architecture for CINEMA
a_at_example.com
a.example.com _sip._udp SRV 0 0 a1.example.com
SRV 1 0 a2.example.com
a1
s1
a2
sipbob_at_example.com
s2
sipbob_at_b.example.com
b_at_example.com
b.example.com _sip._udp SRV 0 0 b1.example.com
SRV 1 0 b2.example.com
s3
b1
b2
example.com _sip._udp SRV 0 0 s1.example.com
SRV 0 0 s2.example.com SRV 0 0 s3.example.com
SRV 1 0 ex.backup.com
Request-rate f(stateless, groups) Bottleneck
CPU, memory, bandwidth? Failover latency ?
15
Server-based vs peer-to-peer

• Server-based
• Cost maintenance, configuration
• Central points of failures
• Controlled infrastructure (e.g., DNS)
• Peer-to-peer
• Robust no central dependency
• Self organizing, no configuration
• Scalability ?

16
Related work Skype From the KaZaA community

• Host cache of some super nodes
• Bootstrap IP addresses
• Auto-detect NAT/firewall settings
• STUN and TURN
• Protocol among super nodes ??
• Allows searching a user (e.g., kun)
• History of known buddies
• All communication is encrypted
• Promote to super node
• Based on availability, capacity
• Conferencing

17
We propose P2P-SIP

• Unlike server-based SIP architecture
• Unlike proprietary Skype architecture
• Robust and efficient lookup using DHT
• Interoperability
• DHT algorithm uses SIP communication
• Hybrid architecture
• Lookup in SIPP2P
• Unlike file-sharing applications
• Data storage, caching, delay, reliability
• Disadvantages
• Lookup delay and security

18
P2P-SIPBackground DHT (Chord)
Key node
81 9 14
82 10 14
84 12 14
88 16 21
81624 32
83240 42
1
54
8
58
10
14
47
21

• Identifier circle
• Keys assigned to successor
• Evenly distributed keys and nodes
• Finger table logN
• ith finger points to first node that succeeds n
  by at least 2i-1
• Stabilization for join/leave

42
38
32
38
24
30
19
P2P-SIPDesign Alternatives
servers
1
54
10
38
24
30
clients
Use DHT in server farm
Use DHT for all clients But some are resource
limited

• Use DHT among super-nodes
• Hierarchy
• Dynamically adapt

20
P2P-SIPNode architecture registrar, proxy, user
agent
Signup, Find buddies
IM, call
On reset
Signout, transfer
On startup
Leave
Find
Join
REG, INVITE, MESSAGE
Multicast REG
Peer found/ Detect NAT
REG

• DHT communication using SIP REGISTER
• Known node sip15_at_192.2.1.3
• Unknown node sip17_at_sippeer.net
• User sipalice_at_example.com

21
P2P-SIPNode Startup
columbia.edu

• SIP
• REGISTER with SIP registrar
• DHT
• Discover peers multicast REGISTER
• Join DHT using node-keyHash(ip)
• REGISTER with DHT using user-keyHash(alice_at_columb
  ia.edu)
• Dialing out
• Call, instant message, etc.
• INVITE siphgs10_at_columbia.edu
• MESSAGE sipalice_at_example.com
• Last seen, SIP NAPTR/SRV, DHT

REGISTER
alice_at_columbia.edu
Detect peers
REGISTER alice42
58
42
12
14
REGISTER bob12
32
22
P2P-SIPNode Leaves

• Graceful leave
• Un-REGISTER
• Transfer registrations
• Failure
• Attached nodes detect and re-REGISTER
• New REGISTER goes to new super-nodes
• Super-nodes adjust DHT accordingly

REGISTER key42
REGISTER
OPTIONS
DHT
42
42
23
P2P-SIPImplementation
31

• sippeer C, Unix (Linux), Chord
• Node join and form the DHT
• Node failure is detected and DHT updated
• Registrations transferred on node shutdown
• Co-located sipc can use sippeer service

29
31
25
26
15
24
P2P-SIPEvaluation

• super-nodes needed depends on
• Registration refresh rate, replication
• Join/leave rate, uptime
• Call arrival rate
• CPU, memory, bandwidth limits
• Other metrics
• Call setup latency
• Recovery time after super-node failure

25
P2P-SIPAdvanced services and open issues

• Offline messages
• INVITE or MESSAGE fails gt Responsible node
  stores voicemail, instant message.
• Conferencing
• Mixer, full mesh, multicast
• Open issues
• P2P reputation system
• Motivation to become super node
• Security (SPAM, DOS, spy, )
• . . .

26
Server-based vs peer-to-peer
Reliability, failover latency DNS-based. Depends on client retry timeout, DB replication latency, registration refresh interval DHT self organization and periodic registration refresh. Depends on client timeout, registration refresh interval.
Scalability, number of users Depends on number of servers in the two stages. Depends on refresh rate, join/leave rate, uptime
Call setup latency One or two steps. O(log(N)) steps.
Security TLS, digest authentication, S/MIME Additionally needs a reputation system, working around spy nodes
Maintenance, configuration Administrator DNS, database, middle-box Automatic one time bootstrap node addresses
PSTN interoperability Gateways, TRIP, ENUM Interact with server-based infrastructure or co-locate peer node with the gateway
27
Summary

• Motivation
• PSTN is reliable and scalable
• Can IP telephony do better?
• Server-based
• DNS, stateless, DB replication, two stage
• Peer-to-peer
• SIP, DHT, soft state, self organizing

28
Beyond proxy/registrarCINEMA Columbia InterNet
Extensible Multimedia Architecture
CINEMA servers
Telephone switch
rtspd media server
Local/long distance 1-212-5551212
sipconf Conference server
Quicktime
RTSP
PSTN
RTSP clients
Department PBX
sipum Unified messaging
sipd Proxy, redirect, Registrar server
Internal Telephone Extn 7040
713x
SQL database
cgi
SIP/PSTN Gateway
vxml
Web based configuration
H.323
siph323 SIP-H.323 translator
NetMeeting
29
Communication to collaboration

• Synchronous (tightly coupled)
• Video conference, IM, screen sharing,
• Asynchronous (loosely coupled)
• File sharing, message board,
• Messaging and notifications
• Personalized view
• Per-user calendar, access control, address book

Goal provide personalized access, alternate
between synchronous and asynchronous
communication, and access from different devices
and clients.