Unsupervised Clustering in Multimodal Multiparty Meeting Analysis

1
Unsupervised Clustering in Multimodal Multiparty
Meeting Analysis
2
(No Transcript)
3
The aim

• to investigate the role of participant
  nonverbal signals exchanged in shaping the
  content of agreement

4
Challenges in multimodal meeting analysis

• No clear and explicit definitions, particularly
  for functional categorizations
• A lot more annotations need to be made than in
  the case of spoken dialogues

5
The experiment

• Six people A F sitting around a table, face to
  face they have to reach consensus on a given
  subject
• No pre-determined roles were imposed. None of
  the members had professional knowledge about the
  topic.
• Taking notes on a sheet of paper, no computers

6

• Head movements are calculated by looking at
  differences between a given frame and the
  previous frame
• Faces indicated by circles, head directions by
  lines

7
(No Transcript)
8
(No Transcript)
9
2 methods

• Method 1 classifies the amplitude of the optical
  flow into 3 classes and, excluding the class with
  the smallest amplitude, classifies the direction
  of the face into 3 classes.
• Method 2 decomposes the optical flow into
  horizontal and vertical amplitude, and classifies
  these data with 2 dimensional features into 4
  classes.

10

• Speech utterances, including verbal
  backchannels, are frequent except from the clerk
  A.
• Main speaker utterances are accompanied by
  nonverbal behaviors.
• Nonverbal responses given by B, E, and F appear
  even without accompanying verbal utterances.
• After D yields the main speaker role to C, D
  doesnt produce nonverbal behaviors.

11

• C produced a proposal
• B, D, F nonverbal responses, E almost no
  nonverbal responses
• B, D, F strong support of the proposal
• E didnt get the merit of the idea and directed
  to C a refinement question.

12
Conclusions

• By combining statistical and qualitative
  analysis, it was possible to obtain a clear
  picture of the interrelationships between types
  of movements, gaze shifts and head movements, and
  interaction organization functions of listener
  responses, degree of engagement and support.