Focus of Attention during dyadictriadic conversations

1
Focus of Attention during dyadic/triadic
conversations

• Study the visual and auditory behaviour to
  extract information on the focus of attention
  during social interactions.
• Monitor the temporal and spatial correlations
  between redirection of the gaze toward human
  faces and localization of object of interest.

2
Focus of Attention during dyadic/triadic
conversations

• Steps in the analysis
• (past) Detection of faces
• (past-present) Detection of the gaze direction
• (future) Detection and recognition of objects
• Based on images, gaze and audio analysis
• ? Enhance detection of the point/object of
  interest by multimodal analysis

3
The WEARCAM

• 2 wired cameras
• CCD PAL Cameras
• The interaction zone
• The manipulation zone
• 2 microphones
• Wide field of view
• 74 horizontal
• 112 vertical

4
Multi-View Face Detection on Low-Quality Videos
? Video-based detection of faces that are robust
against important degradation of the images
quality, and that are invariant to in-plane
rotations.
Analysis of Head Mounted Wireless Camera Videos
for Early Diagnosis of Autism Noris, B.,
Benmachiche, K., Meynet, J., Thiran, J-P. and
Billard, A. (2007). International Conference on
Recognition Systems - CORES07
5
Eye Gaze Direction Detection

• Appearance-based detection
• - Localization of the eyes regions with
• SVM (RBF-kernel) on database
• of cropped eye regions
• Create a mapping between the
• appearance of the eye and the image
• Coordinate, SVR (RBF-kernel)
• Performances results
• Eye location
• Gaze direction

Calibration-Free Eye Gaze Direction Detection
with Gaussian Processes Noris, B., Benmachiche,
K. and Billard, A. (2007). International
Conference on Computer Vision Theory and
Applications VISAPP08
6
Audio-Visual Analysis of Scene Analysis Joint
Work with IDIAP
Training Set Learning a color-based model of
skin and objects
7
Audio-Visual Analysis of Scene Analysis Eye Gaze
Direction Detection Object Detection
Future work Combining gaze detection with
audio-visual detection of objects and visual
detection of faces collaboration with IDIAP
Object Detection
Object Gaze Detection
8
Current and Future Work

• Current efforts
• Gaze detection in Real-Time
• Still needs a manual offline process
• Once calibrated, the system runs in real-time
• Looking for manpower
• Robustify Object detection
• Video-Audio analysis