Aucun titre de diapositive

1
Learning and recognition of places using hidden
Markov models application to place-based
localization and to motion monitoring
Olivier Aycard LAPLACE Research Group UJF / IMAG
http//www-leibniz.imag.fr/aycard
Olivier.Aycard_at_imag.fr
2
Outline

• The mobile robot
• Motivations
• The hidden Markov models
• Learning and recognition of places
• Application to place-based localization
• Application to motion monitoring
• Conclusion and perspectives.

3
The mobile robot

• 3 driving and front wheels
• An arm with a clamp
• A ring of bumper
• 16 infrared sensors
• 16 ultrasonic sensors
• A laser rangefinder
• 2 cameras.

4
Motivationswhat we have numerical data
5
Motivationswhat we need symbolic data
6
Motivationsnumerical data -gt symbolic data
7
Hidden Markov modelsdefinition

• S a finite set of N states S s1, s2, , sN
• p a vector of initial probabilities over S pi
  P(si q1), 1 i N
• A a matrix of probabilities of transitions over
  SxS aij P(sj qt si qt-1), 1 i, j N
• V a finite set of M observations V v1, v2,
  , vM
• B a matrix of probabilities of observations
  associated with each state over SxV bi(k)
  P(vk ot si qt), 1 i N, 1 k M.

8
Hidden Markov modelsexample T-intersection
model ?T
9
Hidden Markov modelslearning of models

• Given k sequences of observations of a learning
  corpus Ok, the goal of the learning is to compute
  the model ? which maximizes (given a criteria)
  the probability that the model ? recognizes its
  corpus (i.e, ?k P(Ok ?)).
• Maximum likelihood criteria (the most used)

10
Hidden Markov modelsexample of learning
Model of T-intersection ?T  (AT, BT, pT, V, N)
11
Hidden Markov modelslearning of a sequence of
observation
observations
2
1
1
-2
12
Hidden Markov modelsmodels after learning
Model of T-intersection ?T (AT, BT, pT, V, N)
Model of corridor ?C (AC, BC, pC, V, N)
13
Hidden Markov modelsreestimation procedure

• ?k P(Ok ?T) ?k P(Ok ?T) and ?k P(Ok
  ?C) ?k P(Ok ?C)
• The last estimate is more likely than the
  previous one
• Reestimation procedure
• For i from 1 to N do
• Estimate ? using Ok (the learning corpus) and ?
• ? lt ?
• End_for
• Problem of the choice of N.

14
Hidden Markov modelsreestimation of the model
Baum-Welch algorithm (1/5)
15
Hidden Markov modelsreestimation of the model
Baum-Welch algorithm (2/5)
16
Hidden Markov modelsreestimation of the model
Baum-Welch algorithm (3/5)
17
Hidden Markov modelsreestimation of the model
Baum-Welch algorithm (4/5)
18
Hidden Markov modelsreestimation of the model
Baum-Welch algorithm (5/5)
19
Hidden Markov modelsrecognition of models
Viterbi algorithm

• Given a sequence of observations O, the goal of
  the recognition is to find the model ? in the set
  of all the models ? the probability of
  recognition (i.e, P(O ?)) is maximum (given a
  criteria)
• Use of a dynamic programming method.Similar
  to the ? quantity (max in place of ?)

20
Hidden Markov modelsexample of recognition
Recognition of the sequence (2 -1 0 -2)
Model of corridor
Model of T-intersection
2
-1
0
-2
2
-1
0
-2
0,71
0,17
0,32
0,07
0,07
0,4
0,28
0,72
0,29
0,71
0,71
0,0338
0,0078
0,07
0,0014
0,0004
state 2
state 2
state 1
state 1
state 1
state 1
0,72
0,71
0,72
0,32
0,42
0,21
0,4
0,2
0,32
0,68
0,33
0,67
0,0220
0,0108
0,0013
0,0002
0,2147
0,0099
state 3
state 2
state 2
state 3
state 2
state 2
0,68
0,67
0,11
0,72
0,29
0,21
1
1
0,0161
0,0116
0,0019
0,0004
state 3
state 3
state 3
state 3
21
Learning of placesmodel used to represent each
place

• Second order hidden Markov models with
  multi-dimensional and continuous observations
• aijk instead of aij
• instead of bi(k) P(vk ot si qt)
• Variation of the 16 ultrasonic sensors as
  observation

22
Learning of places construction of the 10
learning corpus

• 50 runs back and forth in a corridor (30
  meters)
• People wandering in the corridor
• Doors partially or completly open
• Different open doors on each run
• A T-intersection on each run
• Obstacles shelves, cardboard boxes...

23
Learning of places segmentation and labelization
of each run
24
Recognition of places interpretation of a
recognized sequence
recognized sequence
interpretation
sequence to recognize
beginning
T-intersection
substitution
corridor
recognition
corridor
door
recognition
door
recognition
corridor
corridor
insertion
door
T-intersection
corridor
insertion
T-intersection
recognition
corridor
recognition
omission
25
Recognition of places rate of recognition
10 different runs back and forth Spot the
recognized places when the robot is in a corridor.
26
Recognition of placesglobal rate of recognition
27
Application to place-based localizationprinciple
of the method

• Acquisition of sensors data
• Recognition of places
• Matching between the recognized places and the
  places of the environment
• Computation of the position of the robot in the
  environment

28
Application to place-based localizationproblems
to solve to performe the matching

• Dynamic environment
• A door can be closed, a T-intersection can be
  locked
• Impossible to know the places the robot will have
  to recognize
• Rate of recognition
• A recognized place can be a good (or a bad)
  recognition, or an insertion
• Impossible to know the places the robot really
  saw.

29
Application to place-based localizationmethod
Markov localization (1/2)

• Modelize the environment with a HMM one place in
  the environment is modelized by a state in the
  HMM
• S the set of N places in the environment
• V the set of M observations in the environment
• Discrete observations
• ? the knowledge on the initial position of the
  robot in the environment
• A matrix of probability to go from a given
  place to an other
• B matrix of probability of making an
  observation in a given place.

30
Application to place-based localizationmethod
Markov localization (2/2)

• Lt(i) probability for the robot of being at
  state si at time t
• Similar to the ? quantity for the Baum-Welch
  algorithm
• When a place is recognized recompute the
  distribution over the states of the model
• Define a strategy to choose the current position
  of the robot using Lt.

31
Application to place-based localizationthe model
used to represent the environment

• First order hidden Markov model the current
  position of the robot only depends on its
  previous position
• S the set of places present in the environment
• V the 10 possible places discrete observations
  provided by the recognition of places
• pi 1 for the state si where the robot is
• pj 0 for i?j.

32
Application to place-based localizationconstructi
on of the matrix A and B (1/4)
33
Application to place-based localizationconstructi
on of the matrix A and B (2/4)

• The transitions depend on the observations

34
Application to place-based localizationconstructi
on of the matrix A and B (3/4)

• P(seen place recognized place)
• P(right_door left_door) 4/84 0,04
• P(insertion left_door) 34/84 0,4.

35
Application to place-based localizationconstructi
on of the A and B matrix (4/4)

• Aij(O) P(sjqt1, vk O si qt)
• Ai(i1) P(bi O)
• Aii(O) P(insertion O)
• Aij(O) P(corridor O) / (j-i) with jgti.
• Before each recognition one or more place could
  have been omitted to be recognized
• Aij(O) P(omitted bk) P(bj-1 O)
• Aij(O) P(omitted bk) P(insertion O)
• Aij(O) P(omitted bk) P(corridor O) /
  (j-i).

36
Application to place-based localizationexample
37
Application to motion monitoring

• 4 possible motions go North, South, West, East
• Associate an motion to each place given a goal
  A
• Choose the motion associated with the most likely
  state in L.

38
Conclusion

• Learning and recognition of places
• 90 of recognition on a real robot
• Implementation on an outdoor robot NASA Ames
• Application to place-based localization
• Robustess
• Dynamic environment
• Determination of the state (i.e, open or closed)
  of the places in the environment
• Application to motion monitoring
• Very simple motion planning
• Always an action to execute recovering of
  failure.

39
Perspectives

• Learning and recognition of places
• Recognition of places using vision
• Fusion of sensor data
• Application to place-based localization
• Taking into account the uncertainty of the
  motions of the robot
• Application to motion monitoring
• Taking into account the uncertainty during the
  planification phase Markov decision Process.