Thematic Alignment of Static Documents with Meeting Dialogs

1
Thematic Alignment of Static Documents with
Meeting Dialogs

• Dalila Mekhaldi
• Diva Group
• Department of Computer Science
• University of Fribourg

2
Outline

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Thematic Segmentation
• Alignments Grouping
• Conclusion Perspectives

3
Introduction

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• In document-centric meetings (lectures,
  teleconferencing, press reviews, etc.)
• Static documents are present
• Should be integrated in a common multimedia
  archive

• ? Need to build links between documents and other
  media

4
Document Alignments

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Several way to link static documents with other
  meeting data
• Document/Image alignment
• Document/Speech alignment

5
Document/Speech Alignment

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Links static data (documents) to temporal data
  (audio).
• Enriches the documents with temporal indexes and
  thematic links.
• Helps
• Building document-based browsing interfaces.
• Improving documents search and retrieval.

6
Document/Speech Alignment

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• 3 alignment categories
• Thematic lexical similarity of document/speech
  parts
• Quotation of a document part
• Reference to a document part

• Text decomposition into segments
• Document
• Logical
• Syntactic
• Speech transcript
• Turns
• Utterances

Speech Transcript
Document
Logical
Turns
Syntactic
Utterances
7
Thematic Alignment

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

8
Thematic Alignment

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Similarity based matching
• ? Vectors of weighted terms
  S1?V1t1, t2,..
• S1?V1t1, t2,..
• a. Stop-words removing, Stemming
• b. Similarity metrics between units
• Jaccard V1 ? V1 / V1 ? V1
• Dice 2 V1 ? V1 / V1 V1
• Cosine V1 ? V1 / ? V1 V1
• Two strategies
• One-best and multiple alignments

9
One-best Alignment Evaluation

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Manual ground truth for 8 meetings
• N1 alignments to found (manual )
• N2 alignments found (automatic) gt
• N3 correct alignments found (automatic)

• Precision N3/ N2
• Recall N3/ N1

• Improve the similarity metrics with a semantic
  dictionary

10
Multiple Alignments Evaluation

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

11

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

Meeting Thematic Segmentation

• Thematic alignment (e.g sentences/utterances)
• Alignability ? arcs
• Similarity ? weights

12

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

Meeting Thematic Segmentation
13
Thematic Segmentation Evaluation

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Manual ground-truth for 22 meetings
• 1. Speech 2 main sets
• Stereotyped 2.7 utterances/turn (ratiogt2)
• Non-stereotyped 1.3 utterances/turn (ratiolt2)
• 2. Documents 2 main sets
• Mono-document
• Multi-documents
• Comparison with 2 mono-modal methods Texttiling,
  Baseline
• Speech Baseline turn-based segmentation
• Documents Baseline reflexive alignment/clusterin
  g

14
Thematic Segmentation Evaluation

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Pk (Beeferman) metric ? 0 for a perfect
  segmentation.

a. Speech b. Documents
15
Analysis

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

Document
Our bi-modal method outperforms standard
mono-modal methods

• more precise in computing the segments number

• detects the similar segments

• bridges the gaps between documents and speech
  transcript

? Documents greatly help structuring meetings
16
Alignments Grouping

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignment
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

Speech
T1 T2 U1 U2 U3
Document L1
L2 S1 S2

• Implementation of a framework that
• a. Combine the various levels, to correct
  the false alignments pairs, e.g.
• (sentences x utterances) (logical blocks x
  turns)

17
Alignments Grouping

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignment
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• 2. A tool for the visualization that
• Highlights the alignment categories (Thematic,
  Quotations, References)
• Represent the various structures of the
  documents/speech as Layers.

Speech
Document
18
Conclusion

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Thematic Alignment of documents with meeting
  dialog
• Is a solution for integrating static documents
  into multimedia archives
• Conference
• Lectures, etc.

19
Perspectives

• Introduction
• Thematic Alignment
• One-best Alignment
• Multiple Alignments
• Meeting Segmentation
• Alignments Grouping
• Conclusion Perspectives

• Automatic transcription of the speech
• Generalize the alignment on other
• documents types with few text (e.g. slides,
  agenda)
• meeting kinds where documents are discussed
  irregularly (e.g. conferences)