Scene Understanding

1

• Scene Understanding
• perception, multi-sensor fusion, spatio-temporal
  reasoning
• and activity recognition.
• Francois BREMOND
• Orion project-team,
• INRIA Sophia Antipolis, FRANCE
• Francois.Bremond_at_sophia.inria.fr
• http//www-sop.inria.fr/orion/orion-eng.html
• Key words Artificial intelligence,
  knowledge-based systems,
• cognitive vision, human behavior representation,
  scenario recognition

2
A scene understanding platform real-time
interpretation of videos from pixels to events
Segmentation
Classification
Scenario Recognition
Tracking
Alarms
access to forbidden area
3D scene model Scenario models
A priori Knowledge
3
Video Understanding Applications
metro station monitoring
airport monitoring
train monitoring
building control access
4
ETISEO performance evaluation depends on sensors
(position, type) and scenes conditions
strong perspective
shadows
tiny objects
lighting conditions
clutter
close view
5
ETISEO selection of 6 sites for ETISEO video
acquisition.
6

• Classification into more than 8 classes (e.g.
  Person, Groupe, Train) based on 2D and 3D
  descriptors (position, 3D ratio height/width, )

7
Vandalism in metro station
8
Fighting in metro station
9
Posture Recognition silhouette comparison
Real world
Virtual world
10
Multi sensors information fusion Lateral Shape
Recognition
11
Bank agency monitoring
12
Parked aircraft monitoring in Toulouse

• Toulouse-Blagnac Airport
• France

Apron
13
Parked aircraft monitoring in Toulouse
Unloading Front Operation
14
Scenario recognition Disturbing people in a
train
15
HealthCare Monitoring
16
HealthCare Monitoring
17
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
18
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
19
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
20
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
21
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
22
HealthCare Monitoring Object detection and
tracking
(a) Segmentation
(b) Classification
(c) Tracking
23
Trajectory clustering in Torino subway (45min),
2052 trajectories
24

• People Classification (based on 3D
  parallelepiped) into 3 people classes 1Person,
  2Persons, 3Persons, Unknown

25
Posture Recognition Set of Specific Postures
Sitting
Bending
Lying
Standing
Hierarchical representation of postures