LocationAware Computing

1
Location-Aware Computing

• John KrummMicrosoft ResearchRedmond,
  Washington, USA

2
Thank You

• Toshio Hori
• Takeo Kanade
• Chie Nakamura

Digital Human Research Center
3
Location
4
Why Bother to Sense Location?

• Find conference room near me
• Who is in this meeting with me?
• Where are the people who are supposed to be
  here?
• If Im in conference room, dont allow cell
  phone, alerts, or IM
• How long will it take to get from here to next
  appointment?
• Route planning in strange buildings

5
No Really, Why Bother?

• Remind me when Im near a certain customer
• Where are my kids, buddies, colleagues?
• Index my documents (email, photos) by location
• Change default printer and network settings
  based on location
• Electronic graffiti, e.g. Theres a better Thai
  restaurant one block north.

6
Why Not Use GPS?

• Does not work indoors
• Needs view of satellites

7
Location Sensing
Rosum, promising but where is it going?
Indoor/outdoor, coverage spreading
Still a hard problem, can give much more than
location (good bad)
Hazas, Scott, Krumm, Location-Aware Computing,
IEEE Computer Magazine, February 2004.
8
Beyond Location
sensors ? location ? context
Patterson, Liao, Fox, Kautz, Inferring
High-Level Behavior from Low-Level Sensors, 2003
Use GPS tracking to infer users mode of
transportation as (car, bus, walk)
Sparacino, Sto(ry)chastics A Bayesian Network
Architecture for User Modeling and Computational
Storytelling for Interactive Spaces, 2003
Use indoor location sensing in MIT museum to
classify visitor into (greedy, busy, selective)
9
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

10
Video Tracking
EasyLiving Project Steve Shafer Barry
Brumitt Steve Harris Brian Meyers Greg Smith Mike
Hale John Krumm
11
Video Tracking Is Very

• accurate (centimeters)
• easy for user (no devices to carry)
• hard to set up (camera calibration)
• hard to get right (live demos still rare)
• CPU intensive (one PC per camera)
• intrusive

Research can solve
12
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

13
SmartMoveX
Microsoft Researchs entry into the active badge
space (along with Xerox PARC, UW, Intel, ATT
Cambridge, MIT, etc.)
Receiver
Multiple receivers for position triangulation
from signal strengths
Transmitter
Hardware Lyndsay Williams (Microsoft Research
Cambridge UK) Software John Krumm Greg Smith
(Microsoft Research Redmond)
14
Receiver Network
15
Graph Algorithm

• Compute path instead of single locations
• Constrain path to allowable routes
• Process with Hidden Markov Model (HMM same as
  used for speech recognition)
• Average error 3.05 meters

Constraints make things easier
16
SmartMoveX Evaluation

• Good
• Cheap hardware
• Uses existing network infrastructure
• Graph algorithm imposes natural constraints on
  paths

• Needs Improvement
• Privacy depends on central server
• Convenience
• Extra device to wear
• No display on device
• Infrastructure requires special receivers

17
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

18
Here I Am Coarse Location
With Steve Shafer (Microsoft Research)
19
Here I Am
802.11 Access Point Data
Room Numbers Hand-Entered from Maps
Here I Am returns position of strongest access
point
20
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

21
Location from 802.11 with LOCADIO
John Krumm Eric Horvitz
Wi-Fi (802.11) access point

• Mobile device measures signal strengths from
  Wi-Fi access points
• Computes its own location

Location from Radio
22
LOCADIO - Fine Location
Radio survey to get signal strength as a function
of position
23
LOCADIO Setup

• Map for radio survey and feasible paths
• Radio Survey signal strengths taken at various
  locations
• Feasible Paths Constrain paths where a person
  could walk

• No new access points installed
• Access point positions unnecessary
• Access points need no network connection

24
Hidden Markov Model (HMM)
states are discrete locations
transition probability between each pair of states
observation likelihood for each state (802.11
signal strengths)
25
Radio Survey
63 locations

• Walk around with Wi-Fi device and record signal
  strengths at known locations
• Above calibration had one point for every 10
  square meters
• 1 minute at each point, spinning around
• Saw 25 access points in all
• Average 3.6 access points seen from any given
  location

26
Radio Survey Results
Probability of seeing access point i from
location j
Probability of signal strength at point j if
access point seen from location i
Distributions used later to infer location, but
first some constraints
27
Path Constraints
Constraints make things easier
Draw feasible paths on map
Fill in at 1 meter spacing

• Prevents computed paths from penetrating walls
• Serves as graph of nodes for Hidden Markov Model
  (HMM)
• Need transition probabilities between all nodes

28
Internode Distances

• dij shortest path distance between i and j
• Computed with Dijkstras shortest path algorithm

29
Internode Speeds - Walking
distribution of free-flow walking speeds
obtained in surveys of about 1000
non-baggage-carrying pedestrians inside the Port
Authority Bus Terminal and Pennsylvania Station
in New York City.
John J. Fruin, Pedestrian Planning and Design,
1971, New York Metropolitan Association of Urban
Designers and Environmental Planners.
p(walking speed moving)
30
Internode Speeds - Other
p(speed other)
speed
10.22 meters/second
p(speed moving) p(speed walking) p(walking
moving)
p(speed other) (1- p(walking moving))
American Tim Montgomery, world record 100 meters
in 9.78 seconds, September 14, 2002 in Paris
(average 10.22 meters/second)
Guess p(walking moving) 0.9
31
Internode Still vs. Moving
What is p(moving)?
Variance of still vs. moving rssi
Rssi seems noisier when device is moving
Bayes
32
Smooth Still vs. Moving
14 transitions
84.5 correct classification 172 transitions
87.4 correct classification 24 transitions
Transition probabilities for still vs.
moving s number of seconds (28,800 in 8
hours) m number of moves (10) r rssi
sampling rate (3.7 Hz) aSM aMS mr/s
0.00132
Markov Model
33
Location Transition Probabilities
aij probability of transition between nodes i
and j
Distance dij shortest path distance between
nodes i and j
Time ?t rssi sampling interval
Speed sij dij/?t
d(0)
p(s) p(s moving) p(moving) p(s still) (1
- p(moving)) probability of speed
aij p(sij)

• Transition probabilities account for
• Building floorplan (walls doors)
• Expected pedestrian speeds
• Still vs. moving inference

34
HMM for Location

• Location nodes, i 1 N (N 317)
• Initial state probabilities pi 1/N (could be
  anywhere)
• Transition probabilities aij (from previous 7
  slides)
• Observation probabilities from radio survey, but
  

RBFs to interpolate from survey nodes to location
nodes
Location nodes (317)
Radio survey (63)
35
Observation Probabilities
Observation Data It (true, false, true, true,
true) which access points were seen st (-58,
Ø, -57, -78, -79) signal strengths from each
visible access point
Likelihood of Observation Given Location

• accounts for seen and unseen access points
• assumes independence among access points (likely
  true)

36
What Happens
I0, s0
aij(0)
I1, s1
I2, s2
I3, s3
aij(1)
aij(2)
Viterbi algorithm efficiently computes optimal
path after every new rssi scan
N location nodes
37
Results

• Test Data
• 10 short walks over 700 m2 calibrated area
• Each walk 2 minutes, 25 seconds on average
• Ground truth by interpolating between turns
• Total 4586 test points

• Results
• Median error 1.53 meters

38
LOCADIO - Constraints
Make the client as smart as possible to reduce
calibration effort
No passing through walls
No speeding
We know when you move
39
LOCADIO - Results
Hidden Markov model gives median error of 1.53
meters
40
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

41
SPOT Watch
traffic
weather
dining
movies
Commercial FM transmit new data every 2 minutes
Filter on watch to take what it wants
Watch displays personalized data
42
Location-Sensitive Features

• Nice to have
• Local traffic
• Nearby movie times
• Nearby restaurants

Need to know location of device
43
Use FM Radio Signal Strengths
Scan signal strengths of 32 FM radio stations at
1 Hz
44
Clustering Approach
KMTT
Measured RSSI
A
B
C
KPLU
Input Power

• But
• Each watch scales signal strengths differently
• Impractical to calibrate every watch

45
Ranking Approach
Any monotonically increasing function of signal
strength preserves ranking
Redmond KPLU n radio stations ? n! possible rankings
e.g. s (12, 40, 38, 10) r (2, 4, 3, 1)
f (f1, f2, f3, , fn) scanned frequencies
s (s1, s2, s3, , sn) signal strengths
r (r1, r2, r3, , rn) ranks of signal
strengths
R(r) permutation hash code 0, 1, 2, n!-1
46
Test
Six suburbs and six radio stations
81.7 correct from 8 radio stations
47
Avoid Manual Training
Seattle
KMPS 94.1 MHz
KSER 90.7 MHz
48
Classify Into Grid Cell

• Find location in grid
• Use predicted signal strengths to avoid manual
  training

8 kilometers average error
Summer intern Adel Youssef, U. Maryland
49
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

50
NearMe
Find people and things nearby
printers
people
reception desk
bathroom
conferencerooms
51
NearMe Basic Idea
802.11 Access Points
NearMe Server
52
NearMe ? Location
Why compute absolute locations when you only need
relative locations?
Tomasi, Kanade, Shape and Motion from Image
Streams a Factorization Method, 1991
Hightower, Fox, Borriello, The Location Stack,
2003
53
Short Circuit
Get signal strengths
Get signal strengths
54
NearMe Screen Shots
1. Register with server
2. Report Wi-Fi signals

• NearMe Server
• SQL Server
• .NET Web Service

3. Nearby people
4. Nearby printer(s)
55
NearMe Distance Estimate
Estimate distance between two clients by
comparing Wi-Fi signatures
15 meters rms error
Distance f(nn,?s)
nn number of access points seen in common ?s
Spearman rank correlation of signal strengths
56
NearMe Applications
Device association (with Ken Hinckley, Microsoft
Research)
Look up URLs of nearby people/things
Send email to people nearby
57
Outline

• Introduction
• Video Tracking
• Active badge SmartMoveX
• Coarse Wi-Fi location Here I Am
• Fine Wi-Fi location LOCADIO
• SPOT Wristwatch Location RightSPOT
• Wi-Fi proximity NearMe
• Longhorn Location Service Sensor Fusion

58
Longhorn Location Service
Longhorn PC knows its location
GPS
your location
cell phone
Location Service
Wi-Fi
Bluetooth
other unanticipated
AN/PLR-3 Helmet-Mounted Radar (I am not making
this up.)
other location resolvers
59
Longhorn Location Service
Mgmt App (shell, netxp, OEM)
User Pref. Db.
LocMgmt API
User Resolver
Cache
App (Shell, OEM)
Notification Service
Master Resolver
Fuser
AD Resolver
Location API
AD
WinFS
Plugin Manager
Map Point Resolver
LocProv API
MapPoint Db / Serv.
Blue Tooth Provider
802.11 Provider
OEM Provider
Tracey Yao, PM Florin Teodorescu, Dev Vivek
Bhanu, Dev Jim Seifert, Test Madhurima Pawar, Test
OEM Service
Wireless (802.11) Zero Configuration Service
BT Configuration Service
60
Fusion
Proper fusion of measurements depends on
knowledge of uncertainty
Metric Measurements
Hierarchies
Weighted Hierarchical Voting
Kalman Filter
61
The End