Towards OntologyDriven Discourse: From Semantic Graphs to Multimedia Presentations

1
Towards Ontology-Driven Discourse From Semantic
Graphs to Multimedia Presentations

• Joost Geurts, Stefano Bocconi, Jacco van
  Ossenbruggen, and Lynda Hardman

2
Intro Problem scope

• Improve presentation of information retrieved by
  Semantic Web Technology
• Importance of presentation often overlooked
• Large, heterogeneous sources of data, semantics
  and user profiles require some form of automation
  with knowledge
• Key Information presentation design -gt knowledge
  driven

3
Intro Paper scope

• Explore to what extent Semantic Web Technology
  can create a process to convey information
• Process
• 1) Assume a properly annotated semantic graph
  representing domain relations
• 2) Use Discourse Ontology to select, order, group
  and prioritize info
• 3) Use Discourse / Design Ontology to generate a
  multimedia presentation

4
(No Transcript)
5
Example Scenario

• Life and work of Rembrandt
• Structured Progression - discBiography
• Output medium non-interactive multimedia
• In this example
• Semantic relations from domain ontology form
  Semantic Graph
• discBiography uses set of rules to form
  Structured Progression
• Transitions are chosen to create the Multimedia
  Presentation

6
Semantic Graph to Structured Progression

• Not as simple as XSLT/CSS Style Sheets
• Required Knowledge on schema level to query
  domain/discourse knowledge
• Solution Use Sesame to store schema knowledge
• Allows us to make domain/discourse knowledge
  explicit
• As opposed to implicitly looking at XML on syntax
  level

7
Narrative Units

• Property that can structure and organize the
  information
• Contains associative rules to select matching
  content
• These rules can be applied recursively
• Example Spouse -gtSon

8
(No Transcript)
9
Additional Rules

• Instances can play different roles
• Spouse replaces main character
• Each role may contain extra slots as determined
  by Narrative Units
• MainCharacterPortrait, PortraitDescription

10
Structured Progression

• Information that is grouped, ordered and
  prioritized in terms of discourse
• Domain ontology is hidden from next stage of
  transformation
• Sufficient semantics must be carried on

11
Structured Progression to Multimedia Presentation

• Determine method and means of presenting
  information
• Use discourse and design knowledge
• 2 step method
• Structured Progression transforms into document
  structure
• Document structure transforms into tree of
  formatting objects

12
Document Structure Creation

• Defining Narrative structure
• Determine output medium
• Text chapters, sections
• Multimedia presentations scenes, sub-scenes

13
Tree of formatting Objects

• Specify format decisions
• Use Multimedia transformation engine Cuypers
  use this to verify multimedia conformity
• Detailed layout / Format
• Text typesetting used
• Multimedia presentation timing, transition
  effects
• Generalizing
• If conversion too complex
• Discpainter -gt Discartist
• DiscSculptors -gt Discartist

14
2 Step Method

• Advantages
• Generalized document structure allows more
  commonly applicable formats
• Disadvantages
• In some cases the detailed structure cannot be
  transformed into document structure
• Solution copy detailed structure into document
  structure

15
Conveying Discourse Semantics Directly

• More specific rules not handled by generic rules
• Need to be copied directly into document
  structure
• Example grouping specific paintings together
• Used as backup rules in case a rule cannot be
  applied at last stage (i.e. hardware constraints)

16
Related Work

• Domain Independence
• Explicit knowledge representation
• Generated narrative

17
Conclusions

• Use of Semantic Web Technologies for explicit
  domain, discourse and media knowledge to convey
  relations in multimedia presentations
• Focus
• Explicit rules to facilitate information
  flexibility
• Role based rules can aid expansion of narrative
  units
• Improvements
• Scaling
• User profile