A New Approach to Decisionmaking within an Intelligent MultiMedia Distributed Platform Hub

1
A New Approach to Decision-making within an
Intelligent MultiMedia Distributed Platform Hub
MediaHub

• Glenn Campbell, Tom Lunney,
• Aiden Mc Caughey, Paul Mc Kevitt
• School of Computing and Intelligent Systems
• Faculty of Engineering
• University of Ulster, Magee Campus
• Derry/Londonderry
• Northern Ireland
• Campbell-g8, TF.Lunney, P.McKevitt
  _at_ulster.ac.uk

2
Outline

• Goals and objectives
• Key research problems
• Software and tools
• Architecture of MediaHub
• Future development

3
Goals

• The primary objectives of this research are to
• Interpret/generate semantic representations of
  multimodal input/output
•  
• Perform fusion and synchronisation of multimodal
  data (decision-making)
•  
• Implement and evaluate a multimodal platform hub
  (MediaHub)

4
Key research problems

• Semantic representation?
• Semantic storage?
• Communication with other elements of a multimodal
  platform?
• Decision-making?

5
Software and Tools

• Main Programming Language
• Java
• Communication
• OpenAIR Specification using Psyclone
• Decision-making tool
• HUGIN GUI / API (Hugin 2006)
• Semantic Representation
• Likely to be XML-based
• XML, XHTML Voice, MPEG-7, OWL, OIL, SMIL, EMMA?

6
Psyclone

• Semantic Storage via Whiteboards implemented in
  Psyclone
• Psyclone uses OpenAIR Specification for
  communication
• Publish-Subscribe mechanism for communication
• Using Java implementation of OpenAIR specification

7
Hugin

• Tool for implementing Bayesian Networks as CPNs
  (Causal Probabilistic Networks)
• Hugin GUI
• Graphical user interface to Hugin decision engine
• Hugin API
• Library implemented in Java
• Allows programs to implement Bayesian Networks
  for decision-making

8
Bayesian Networks

• AKA Bayes nets, Causal Probabilistic Networks
  (CPNs), Bayesian Belief Networks
• Consists of nodes and directed edges between
  nodes
• Node represents a variable
• Influence between nodes represented by edges

               
 
9
MediaHub Example Network
10
Architecture of MediaHub

• Dialogue Manager
• Facilitates communication between other modules
• Synchronisation
• Whiteboard implemented in Psyclone
• Whiteboard and Dialogue Manager form kernel of
  MediaHub
• Decision-Making Module
• Bayesian Networks
• Semantic Representation Database
• Stores semantic representation of multimodal data

11
Architecture of MediaHub
12
Architecture of MediaHub
13
Decision-making in MediaHub

• Input
• Determining semantic content of input
• Fusing semantics of input
• Resolving ambiguity at input
• Output
• Synchronising multimodal output
• Best modality for output

14
Conclusion

• An intelligent multimodal distributed platform
  hub called MediaHub will be developed
• MediaHub will interpret/generate semantic
  representations of multimodal input and output
• MediaHub will perform fusion and synchronisation
  of multimodal data
• MediaHub will provide a new method of
  decision-making within a distributed platform hub
  

15
Future development

• Define necessary decisions
• Develop Bayesian decision-making using Hugin API
  for Java
• Finalise the semantic representation scheme
• Acquire (or create) multimodal corpora for testing

16
Questions?