Sensor%20Fusion%20Using%20Dempster-Shafer%20Theory

1
Sensor Fusion Using Dempster-Shafer Theory

• Huadong Wu, Mel Siegel
• The Robotics Institute, Carnegie Mellon
  University
• Rainer Stiefelhagen
• Interactive System Labs, University of Karlsruhe
• Jie Yang
• Interactive System Labs, Carnegie Mellon
  University

2
agenda

• describe sensor fusion requirement for
  context-aware computing applications
• describe application case study meeting
  participants focus-of-attention analysis
• review alternative sensor fusion approaches
• introduce Dempster-Shafer theory of evidence
• develop weighted D-S evidence combination
• demonstrate its effectiveness in the case study
• consider extensions of scale and scope

3
outline

• context aware computing
• sensor fusion
• system architecture
• case focus-of-attention sensor fusion
• Dempster-Shafer approach
• implementation details
• experiments analysis
• conclusions and future work

4
context aware computing

• best algorithm for human-computer interaction
  tasks depends on context
• context can be difficult to discern
• multiple sensors give complementary (and sometime
  contradictory) clues
• sensor fusion techniques needed
• (but best algorithm for sensor fusion tasks may
  depend on context!)

5
sensor fusion

• how to combine outputs of multiple sensor
  perspectives on an observable?
• modalities may be complementary,competitive,
  or cooperative
• technologies may demand registration
• variety of historical approaches, e.g.
• statistical (error and confidence measures)
• voting
• Bayesian (probability inference)
• neural network, fuzzy logic, etc

6
generalizable SF architecture

• for context-aware computing applications
• cartoon (next) illustrates typical configuration
• low level sensor fusion done locally
• sensor clusters deliver local perceptions
• implemented via LAN communication and shared
  global database technologies

7
system architecture to support sensor fusion for
context-aware computing
8
dynamic database

• example user identification and tracking
• tables (next) list basic information about
  environment (room) and parameters, e.g.,
• temperature, noise, lighting, available devices,
  number of people, segmentation of area, etc
• detail analyze focus-of-attention of each user
  in a meeting at a small conference table

9
context information architecturedynamic context
information database
Background-table-userHd Background-table-userHd
Name Huadong Wu (? 1.0)
Height 56 (s 0.5 )
Weight 144 lb (s 4 lb)
Preference Preference-table-userHd



Room-table NSH A417 Room-table NSH A417
Area Area-table
Detected people 6 (? gt 0.5)
Detected users User-table
Temperature 72 ºF (s 3 ºF)
Light condition Brightness grade
Noise level 60 db (s 6 db)
Devices Device-table
Current



Preference-table-userHd Preference-table-userHd
Name Huadong Wu (? 1.0)
Height 56 (s 0.5 )
Weight 144 lb (s 4 lb)
Preference Preference-table-userHd



History-table-userHd History-table-userHd
Name Huadong Wu
Time Place
906AM-1055AM NSH 4102
Preference Preference-table-userHd



Inside Area Inside Area
Of room NSH A417
Temperature 72 ºF (s 3 ºF)
Light condition Brightness grade
Noise level 60 db (s 6 db)
Devices Device-table
Detected people 4 (? gt 0.5)
Detected user User-table



User-table User-table User-table User-table User-table User-table User-table
Name Background Place Confidence Activity First detected history
Hd Background-table-userHd Entrance 0.5, 0.9 Activity-table-userHd 1032AM, 06/06/2001 History-table-userHd
Mel Background-table-userMel Entrance 0.3, 0.7 Activity-table-userMel 1148AM, 06/06/2001 History-table-userMel
Alan Background-table-userAlan Inside 0.9, 0.98 Activity-table-userAlan 248PM, 06/06/2001 History-table-userAlan
Chris Background-table-userChris Inside 0.4, 0.9 Activity-table-userChris 1045AM, 06/06/2001 History-table-userChris


Entrance Area Entrance Area
Of room NSH A417
Temperature 72 ºF (s 3 ºF)
Light condition Brightness grade
Noise level 60 db (s 6 db)
Devices Device-table
Detected people 2 (? gt 0.5)
Detected user User-table


10
system architecture

• create a sensor fusion mediator for each
  context
• mediator collects and processes sensor data
• in this test, sensor fusion implemented using
  Dempster-Shafer theory of evidence
• need to develop differential weighting scheme for
  sensors in which we have differing confidence
• allows combination of individual sensors
  observation with specified confidence level
• admits measures of ignorance as well as knowledge

11
system configuration
12
focus of attention

• recorded meeting activities of four people around
  a conference table
• whose focus-of-attention is on whom
  ground-truth found manually at 10 Hz
• microphones detect who is talking
• video imagery analyzed for each talker/listener
  pair

13
focus of attention analysis equipment
Panoramic View
14
video image analysis

• neural network estimates head poses
• focus of attention estimate based on head pose
  probability distribution analysis
• audio reports speaker, assumed to be focus of
  other participants attention
• situation is not easy to analyze due to, e.g.,
  dependence of behavior on discussion topic
• suggests we need more general fusion approach
  than provided by Bayesian

15
focus of attentionestimation from video and
audio sensors
16
comparison of sensor fusion alternatives
1. complementary
3. cooperative
Parametric template, Figures of merit, Syntactic
pattern recognition
Logical template AI rule-based reasoning, Heuristi
c inference Neural network
2. competitive
Classic Inference Sensor i Pi( x detected x appeared ) Simple effective for x vs. x problems Priori knowledge and pdf are required to combine multiple sensor outputs, priori assessments are not used, do not have enough reasoning power
Voting Fusion Associate pdf with confidence estimation, and provide a way to predict the result probabilities of their boolean combinations Though big improvement over Classic Inference method, still not powerful enough to reason at fine granularity
Bayesian Network Likelihood of a hypothesis is updated using a previous likelihood estimation and additional evidence cannot distinguish between lack of belief and disbelief, cannot address a problem like its likely either user A or user B
Fuzzy Logic No pdf required, very cheap in computation It doesnt make sense that a person is assigned as 0.6 membership of user A, 0.7 membership of user B, and 0.9 membership of either user A or B
Neural Network Flexible, powerful, no pdf needed, cheap computational cost in classification process Local minimal problem, results cannot be easily explained, not suitable for dynamic configuration of sensors
17
add Dempster-Shafer approach

• generalization of Bayesian approach
• theory of evidence
• implement via quantitative definitions of
  belief and plausibility

18
sensor fusion with Dempster-Shafer theory of
evidence algorithm

• Frame of discernment T L, O, R, L
  O, O R, L R, L O R
• Updated Belief

19
we add weighted Dempster-Shafer evidence
combination rule
20
weight prior probability
21
Sensor Fusion Results of Focus-Of-Attention
Experiments
22
conclusions and future research

• Dempster-Shafer evidence combination rule and
  weighted Dempster-Shafer evidence combination
  rule are generalized forms of the classic
  Bayesian inference method
• Dempster-Shafer method, especially the weighted
  Dempster-Shafer method, is suitable for sensor
  fusion tasks in context-sensing architectures
  with highly dynamic sensor configurations
• we expect the weighted Dempster-Shafer evidence
  combination rule will outperform linear summation
  and other sensor fusion methods in planned
  research that will involve additional sensors