Interactive Communications voice, video, IM and presencebased services

1
Interactive Communications voice, video, IM and
presence-based services

• Henning Schulzrinne
• (with Jonathan Lennox, Mathew Mintz-Habib,
  Anshuman Rawat, Ron Shacham, Xiaotao Wu)
• Columbia University
• Lucent Bell Labs
• November 15, 2004

2
Overview

• The state of VoIP
• Context location-based services
• Service creation in end systems
• Service and session mobility
• Emergency calling
• Spam, spit and other unsavory things
• What is identity?
• Whats left to do?

3
(Early) Adulthood

• fully developed and mature
• Not quite yet, but no longer a teenager
• probably need another 6 years to be grown up
• Responsibilities
• Dealing with elderly relatives ? POTS
• Financial issues ? payments, RADIUS
• Family emergencies ? 911

4
Evolution of VoIP
how can I make it stop ringing?
does it do call transfer?
long-distance calling, ca. 1930
going beyond the black phone
amazing the phone rings
catching up with the digital PBX
1996-2000
2000-2003
2004-
5
The state of SIP
6
SIP is PBX/Centrex ready
boss/admin features
centrex-style features
attendant features
from Rohan Mahys VON Fall 2003 talk
7
A constellation of SIP RFCs
Non-adjacent (3327) Symmetric resp.
(3581) Service route (3608) User agent caps
(3840) Caller prefs (3841)
Request routing
Resource mgt. (3312) Reliable prov. (3262) INFO
(2976) UPDATE (3311) Reason (3326)
SIP (3261) DNS for SIP (3263) Events (3265) REFER
(3515)
ISUP (3204) sipfrag (3240)
Mostly PSTN
Core
Content types
Digest AKA (3310) Privacy (3323) P-Asserted
(3325) Agreement (3329) Media auth. (3313) AES
(3853)
DHCP (3361) DHCPv6 (3319)
Configuration
Security privacy
8
SIP, SIPPING SIMPLE 00 drafts
includes draft-ietf--00 and draft-personal--00
9
Protocol interoperability problems

• Three core interoperability problems
• syntactic robustness
• You mean you could have a space there?
• often occurs when testing only against common
  reference implementations
• need stress test (also for buffer overflows)
• implementation by protocol example
• limiting assumptions (e.g., user name format)
• see SIP Robustness Testing for Large-Scale Use,
  First International Workshop on Software Quality
  (SOQA)
• semantic assumptions
• I didnt expect this error
• mutually incompatible extensions
• expect extension to make something work

10
Protocol interoperability

• Proprietary protocol
• Example Skype
• Can only reverse-engineer ? only
  backwards-compatibility problems
• incentive to force upgrades (see Microsoft Word)
• quicker evolution, but limited product selection
• less Metcalfes law value
• Open standard, but dominant vendor
• Example H.323, Asterix
• doesnt matter what the standard says
• NetMeeting and H.323 ? test with Microsoft
  implementation
• limits feature evolution to dominant vendor speed
• Open standard, multiple large and many small
  vendors
• Example SIP
• hardware vs. software, UA vs. proxy, phones vs.
  gateways
• interoperability problems until product maturity
• harder to test internally against all (competing)
  products
• better long-term outcome, but slower

11
Context-aware and location-based services
12
Context

• context the interrelated conditions in which
  something exists or occurs
• anything known about the participants in the
  (potential) communication relationship
• both at caller and callee

13
Legacy IM presence systems ? SIP-based systems

• centralized systems (single name space)
• federated systems, similar to email
• mostly instant text messages
• media-agnostic transmit any media object
• separate from session-based services (VoIP, video
  conferencing)
• integrated
• use IM as part of media sessions
• use presence to facilitate session setup
• limited presence status, mostly manually set
• rich presence, with time information
• imported from sensors, calendars, backend
  systems,
• proprietary systems (AOL, Yahoo!, MSN, ICQ, )
• standards-based systems

14
Presence and event notification

• Presence special case of event notification
• user Alice is available for communication
• Human users
• multiple contacts per presentity
• device (cell, PDA, phone, )
• service (audio)
• activities, current and planned
• surroundings (noise, privacy, vehicle, )
• contact information
• composing (typing, recording audio/video IM, )

• Multimedia systems
• REFER (call transfer)
• message waiting indication
• conference floor control
• conference membership
• push-to-talk
• system configuration
• General events
• emergency alert (reverse 911)
• industrial sensors (boiler pressure too high)
• business events (more than 20 people waiting for
  service)

15
IETF efforts

• SIP, SIPPING and SIMPLE working groups
• but also XCON (conferencing)
• Define SIP methods PUBLISH, SUBSCRIBE, NOTIFY
• GEOPRIV
• geospatial privacy
• location determination via DHCP
• information delivery via SIP, HTTP,
• privacy policies
• SIMPLE
• architecture for events and presence
• configuration (XCAP)
• session-oriented IM (? page mode)
• filtering, rate limiting and authorization

16
Presence data model
calendar
cell
manual
person (views)
alice_at_example.com audio, video, text
r42_at_example.com video
services
devices
17
GEOPRIV and SIMPLE architectures
rule maker
DHCP
XCAP (rules)
target
location server
location recipient
notification interface
publication interface
GEOPRIV
SUBSCRIBE
presentity
presence agent
watcher
SIP presence
PUBLISH
NOTIFY
caller
callee
SIP call
INVITE
INVITE
18
RPID rich presence

• Provide watchers with better information about
  the what, where, how of presentities
• facilitate appropriate communications
• wait until end of meeting
• use text messaging instead of phone call
• make quick call before flight takes off
• designed to be derivable from calendar
  information
• or provided by sensors in the environment
• allow filtering by sphere the parts of our
  life
• dont show recreation details to colleagues

19
RPID rich presence

• Classification
• service class
• device, in-person, service, presentity
• class
• for labeling
• sphere
• work, home,
• relationship
• family, associate, assistant, supervisor

• User activities
• activities
• on-the-phone, away, appointment,
• user-input
• last usage of device
• Mood
• Surroundings
• time offset
• place-type
• home, office, industrial,
• privacy
• public, private

20
CIPID Contact Information

• More long-term identification of contacts
• Elements
• card contact Information
• home page
• icon to represent user
• map pointer to map for user
• sound presentity is available

21
Timed Status

• Presence is about here now
• but often only have (recent) past e.g.,
  calendar
• or future
• will be traveling in two hours
• will be back shortly
• allows watcher to plan communication
• loose synchronization of calendars

• lttuple id"7c8dqui"gt
• ltcontactgt
• sipbob_at_example.com
• lt/contactgt
• ltstatusgt
• ltbasicgtopenlt/basicgt
• lt/statusgt
• ltfstimed-status
• from"2003-08-15T102000.000-0500
• until"2003-08-22T193000.000-0500"gt
• ltbasicgtclosedlt/basicgt
• lt/fstimed-statusgt
• lt/tuplegt
• ltnotegtI'll be in Tokyo next weeklt/notegt

22
Location-based services

• Finding services based on location
• physical services (stores, restaurants, ATMs, )
• electronic services (media I/O, printer, display,
  )
• not covered here
• Using location to improve (network) services
• communication
• incoming communications changes based on where I
  am
• proximity triggers communications
• configuration
• devices in room adapt to their current users
• awareness
• others are (selectively) made aware of my
  location
• security
• proximity grants temporary access to local
  resources

23
Location-based SIP services

• Location-aware inbound routing
• do not forward call if time at callee location is
  11 pm, 8 am
• only forward time-for-lunch if destination is on
  campus
• do not ring phone if Im in a theater
• outbound call routing
• contact nearest emergency call center
• send delivery_at_pizza.com to nearest branch
• location-based events
• subscribe to locations, not people
• Alice has entered the meeting room
• subscriber may be device in room ? our lab stereo
  changes CDs for each person that enters the room

24
Location detection
25
DHCP for locations

• modified dhcpd (ISC) to generate location
  information
• use MAC address backtracing to get location
  information

8020abd5d
DHCP server
CDP SNMP 8020abd5d ? 458/17
DHCP answer staDC locRm815 lat38.89868
long77.03723
458/17 ? Rm. 815 458/18 ? Rm. 816
26
SIP URIs for locations
location beacon

• Identify confined locations by a SIP URI, e.g.,
  siprm815_at_cs.columbia.edu
• Register all users or devices in room
• Allows geographic anycast reach any party in the
  room

siprm815
Contact bob
Contact alice
Room 815
27
Describe a location

• Geospatial coordinates
• e.g., measured by GPS
• expressed in PIDF-LO as GML
• ltgmlcoordinatesgt404834N 735739Wlt/gmlcoordina
  tesgt
• Civic address
• National Emergency Number Association (NENA)
• ltlocation country"USA" A1"NY" A3"New York"
  A6"West 120th Street" HNO"500" LOC"450"gt
• Location attributes
• place type, noise, media privacy, number of
  people, moving,

28
Privacy

• All presence data, particularly location, is
  highly sensitive
• Basic location object (PIDF-LO) describes
• distribution (binary)
• retention duration
• Policy rules for more detailed access control
• who can subscribe to my presence
• who can see what when

lttuple id"sg89ae"gt ltstatusgt ltgpgeoprivgt
ltgplocation-infogt ltgmllocationgt
ltgmlPoint gmlid"point1 srsName"ep
sg4326"gt ltgmlcoordinatesgt374630N
1222510W lt/gmlcoordinatesgt
lt/gmlPointgt lt/gmllocationgt
lt/gplocation-infogt ltgpusage-rulesgt
ltgpretransmission-allowedgtno lt/gpretransmissi
on-allowedgt ltgpretention-expirygt2003-06-2
3T045729Z lt/gpretention-expirygt
lt/gpusage-rulesgt lt/gpgeoprivgt lt/statusgt
lttimestampgt2003-06-22T205729Zlt/timestampgt lt/tupl
egt
29
Privacy policy relationships
common policy
geopriv-specific
presence-specific
future
RPID
CIPID
30
Privacy rules

• Conditions
• identity, sphere, validity
• time of day
• current location
• identity as lturigt or ltdomaingt ltexceptgt
• Actions
• watcher confirmation
• Transformations
• include information
• reduced accuracy

• User gets maximum of permissions across all
  matching rules
• Extendable to new presence data
• rich presence
• biological sensors
• mood sensors

31
Location-based service language
NOTIFY
true
false
action
alert
IM
alert
incoming
proximity
message
outgoing
log
conditions
occupancy
actions
events
notify
call
message
time
transfer
subscription
join
32
Program location-based services
33
(No Transcript)
34
Tracking
35
Overview of SIPc functions
emergency
configuration
handling
multimedia
session setup
presence
network
multicasted session
information
appliance control
Web browsers
Email clients
Real time streaming
Network appliance control
Instant message
Third party call control
audio
Emergency handling
video
SIP Multimedia call control
SIP CGI engine
SAP
white board
location sensors
Location sensing
Floor control
SIP for presence
LESS/CPL engine
Service Location Detection (SLP)
desktop sharing
Some IETF drafts CPL, SIP 3PCC, SIP Device
Control GEOPRIV location format, SIP for IM
SAP RFC 2974
SIP RFC 3261
SDP RFC 2327
RTSP RFC 2326
RTP RFC 1889
SLP RFC 2608
SIP Event Notification RFC 3265
36
Evolving
Conferencing floor control
Device control
Presence notification
ir/x10
xcon
Location tracking
SIP event notification
Service detection
Message waiting indication
Location sensing
SIP
SLP
Voicemail handling
Call
Emergency handling
SDP
RTP
MapLynx
RTSP
Instant messaging
Session broadcasting
SAP
37
Internet2 WG PIC trial
SUBSCRIBE to my location
NOTIFY myself and others locations
PUBLISH presence status
38
Ubiquitous computing
SA
DA
SA
turn on
projector
Service Location
Query
send
audio
Resource
to SIP
Network
Discovery
UA2
Appliance
(SLP UA)
Control
audio
and
Resource
Script
video
Control
engine
streams
(3pcc)
SIP UA1
SIP
UA2
call
39
Service scenario
Media streams
Bluetooth
Call
Authenticate
Use
Location
Resources
Visitor
Call
Info
Resource
Info
Home domain
Register
SIP and AAA
server
40
Example user-adaptive device configuration
all devices that are in the building RFC 3082?
802.11 signal strength ? location
SLP
device controller
HTTP
PA
tftp
REGISTER To 815cepsr Contact alice_at_cs
SUBSCRIBE to each room

• discover room URI
• REGISTER as contact for room URI

SIP
room 815
SUBSCRIBE to configuration for users currently in
rooms
41
Example call flow
42
Service creation
43
Service creation

• Tailor a shared infrastructure to individual
  users
• traditionally, only vendors (and sometimes
  carriers)
• learn from web models

44
Automating media interaction service examples

• If call from my boss, turn off the stereo ? call
  handling with device control
• As soon as Tom is online, call him ? call
  handling with presence information
• Vibrate instead of ring when I am in movie
  theatre ? call handling with location information
• At 900AM on 09/01/2005, find the multicast
  session titled ABC keynote and invite all the
  group members to watch ? call handling with
  session information
• When incoming call is rejected, send email to the
  callee ? call handling with email

45
LESS simplicity

• Generality (few and simple concepts)
• Uniformity (few and simple rules)
• Trigger rule
• Switch rule
• Action rule
• Modifier rule
• Familiarity (easy for user to understand)
• Analyzability (simple to analyze)

modifiers
switches
trigger
actions
46
LESS Decision tree

• No loops
• Limited variables
• Not necessarily
• Turing-complete

47
LESS Safety

• Type safety
• Strong typing in XML schema
• Static type checking
• Control flow safety
• No loop and recursion
• One trigger appear only once, no feature
  interaction for a defined script
• Memory access
• No direct memory access
• LESS engine safety
• Ensure safe resource usage
• Easy safety checking
• Any valid LESS scripts can be converted into
  graphical representation of decision trees.

48
LESS snapshot
incoming call
ltlessgt ltincominggt ltaddress-switchgt
ltaddress issipmyboss_at_abc.com"gt
ltdeviceturnoff devicesipstereo_room
1_at_abc.com/gt ltmedia mediaaudiogt
ltaccept/gt lt/mediagt lt/addressgt
lt/address-switchgt lt/incominggt lt/lessgt
If the call from my boss
Turn off the stereo
Accept the call with only audio
trigger, switch, modifier, action
49
LESS packages

• Use packages to group elements

email
web
im
conference
calendar
location
session
50
When Tom is online,

• ltlessgt
• ltEVENTnotificationgt
• ltaddress-switchgt
• ltaddress is"siptom_at_example.com"gt
• ltEVENTevent-switchgt
• ltEVENTevent is"open"gt
• ltlocation url"siptom_at_example.com"gt
• ltIMim message"Hi, Tom"/gt
• lt/locationgt
• lt/EVENTeventgt
• lt/EVENTevent-switchgt
• lt/lessgt

51
When I am in a movie theatre,

• ltlessgt
• ltincominggt
• ltlocation-switchgt
• ltlocation placetypequietgt
• ltalert soundnone vibrateyes/gt
• lt/locationgt
• lt/location-switchgt
• lt/incominggt
• lt/lessgt

52
(No Transcript)
53
Emergency Services
54
Traditional Emergency Calling

• Basic 911 just route to local PSAP
• based on local switch
• no location delivery
• Enhanced 911 route location delivery (90?)
• multiple PSAPs per PSTN switch
• multiple switches per PSAP
• location delivered out-of-band via caller number
• Phase I wireless (70)
• call delivery based on cell tower and face
• no location delivery
• Phase II wireless (30)
• call delivery based on geo address
• geo location delivery to PSAP

55
Core problems

• PSTN approximate routing often works
• same switch
• based on cell tower
• based on caller number
• PSTN relatively few, regionally-limited telecom
  providers (carriers)
• IP carrier bobs-bakery.com
• IP no such approximations (usually)
• application layer (e.g., SIP) has no clue as to
  location
• L1L3 may know about location (at least
  approximately), but dont know about emergency
  calls

56
How does VoIP differ from landline and wireless
PSTN?

• Telephone companies are no longer needed
• there are still carriers for DSL and cable IP
  dial tone
• but unaware of type of data carried
• VSP may be in another state or country
• Corporations and universities dont have email
  carriers, either

voice service provider (RTP, SIP)
Yahoo
ISP (IP, DHCP, DNS)
MCI
dark fiber provider
NYSERNET
57
Three stages to VoIP 911
58
I3 Location-based call routing UA knows its
location
GPS
INVITE sipssos_at_
48 49' N 2 29' E
outbound proxy server
DHCP
48 49' N 2 29' E ? Paris fire department
59
I2 architecture (draft)
Routing Proxy
Redirect
server(s
)
v6
v4
v5
v4
E9
-
1
-
1
PSAP
Selective
Router
v1
IP Domain
v2
User
v2
Agent
v
-
e2
v0
v8
v3
v7
location information service
VoIP positioning center
validation database
routing database
60
I2 example (embedded LO)
61
Components
No endorsement implied other components likely
will work as well
62
I3 prototype
gray features in progress.
63
Detail I3 - DNS-based resolution
DHCP INFORM
psap.state.vt.gov
SIP w/location
MAC ? loc
Perl sip-cgi script
psap.state.vt.gov
DNS NAPTR addison.vt.us algonquin-dr.addison.vt.u
s
proprietary TCP-based protocol
151.algonquin-dr.addison.vt.us.sos-arpa.net
64
GeoLynx displays location
GeoLynx listens for commands from SIPc
65
Spam and spit
66
SIP unsolicited calls and messages

• Possibly at least as large a problem
• more annoying (ring, pop-up)
• Bayesian content filtering unlikely to work
• ? identity-based filtering
• PKI for every user unrealistic
• Use two-stage authentication
• SIP identity work

mutual PK authentication (TLS)
home.com
Digest
67
Domain Classification

• Classification of domains based on their identity
  instantiation and maintenance procedures plus
  other domain policies.
• Admission controlled domains
• Strict identity instantiation with long term
  relationships
• Example Employees, students, bank customers
• Bonded domains
• Membership possible only through posting of bonds
  tied to a expected behavior
• Membership domains
• No personal verification of new members but
  verifiable identification required such as a
  valid credit card and/or payment
• Example E-bay, phone and data carriers
• Open domains
• No limit or background check on identity creation
  and usage
• Example Hotmail
• Open, rate limited domains
• Open but limits the number of messages per time
  unit and prevents account creation by bots
• Example Yahoo

68
Reputation service
David
Carol
has sent IM to
has sent email to
Frank
Emily
is this a spammer?
Bob
Alice
69
What else is left?

• A random selection
• Higher-level service creation in end systems
• The role of intermediaries
• session-border controllers
• end-to-middle security
• session policies
• Conferencing
• IETF XCON WG struggling with model and complexity
• Application sharing ( remote access)
• pixel-based
• semantically-based

70
Conclusion

• VoIP is