IMSC Communication Vision Project

1
IMSC Communication Vision Project

• CSCI 586
• Professor Dennis McLeod
• Group Leader
• Bomi Song Jongeun Jun (JJ)
• Group Members
• Alok Desai, Gabriela Gutierrez
• Shivraj-pradeep Phatak, Dinesh Rajwani
• Jin-Kyu Shim

2
Contents

• Long Term Project Goals
• Our Approach
• Plan of Action
• Project Scenario
• Task Domain Ontology
• Ontology Structure
• Presentation
• Demo
• Future Work

3
Long Term Project Goals

• Create an immersive environment for trainees and
  experts to minimize social presence
• Social presence the distance between
  communicating people decays concentration
• Try to increase the concentration rate using
  two-way communication skill using visual and
  auditory participant localization and
  identification techniques

4
Our Approach

• Task Plan Representation
• Determine the specific task by using task domain
  ontology and user behavior sensing information
• Task Plan Action
• Generate a proper presentation

5
Plan of Action

• Implementing a Task Domain Ontology
• Determine complete scenario and model for Task
  Domain Ontology
• Implement using JAVA
• 3 group members Dinesh, Jin-Kyu and Shivraj
• Implementing the Presentation
• Implement sample presentation models and test
  which models are efficient for each step of the
  project
• 2 group members Alok and Gabriela

6
Project Scenario

• Fixing a Computer
• Example Replacing a PCI card
• Task Domain Ontology
• Divides the task into 8 steps
• Specifies terminology associated with each step,
  its exceptions and their handling
• Presentation
• Generates proper presentation according to a
  specific situation with multimedia data

7
Task Domain Ontology

• The task is divided into the following 8 steps
• 1. Unplug the power outlet
• 2. Pull the tray out completely
• 3. Unlock the screws and remove the cover
• 4. Locate and remove the COMTAL card
• 5. Insert a replacement in the empty PCI slot
• 6. Reattach top cover with screwdriver
• 7. Push the tray back into position
• 8. Plug in the power outlet

8
Ontology Structure
9
Ontology Structure (contd.)

• Example
• Step 3 Unlock screws and remove the cover
• Sub-step 1 Locate the screws
• Success Screws located
• Exception Screws cannot be located
• Exception handling Show image of positions.
• Key Words Synonyms
• Screw Coupling, Fastener, Latch, Peg.
•  

10
Ontology Structure (contd.)

• Example (contd.)
• Step 3 Unlock screws and remove the cover
• Sub-step 2 Unlock the screws
• Success Screws are unlocked
• Exception Screws are too tight, cannot be
  unlocked
• Exception handling Audio clip with images
  telling
• the trainee to use an electric/bigger
  screwdriver
• Key Words Synonyms
• Unlock Unbind, Unfasten, Loosen, Unscrew,
  Free, Separate.

11
Ontology Structure (contd.)
Step 3 Unlock screws and remove the
cover Sub-step 3 Remove the top cover Success
Cover removed Exception Cover too tight, cannot
be removed Exception handling Show a video
clip of the correct way of removing cover Key
Words Synonyms Remove/Release Detach,
Dislodge, Oust, Separate, Unseat. Cover Cap,
Coating, Hood, Overlay, Roof.    

12
Presentation

• Ontology data structure will contain a double in
  the form of (size, order) for each presentation
  element.
• Structure generator takes in ontology data
  structure and generates the presentation.
• Prototype contains first 3 tasks.
• Dialog manager will control navigation in future.

13
DEMO
14
Future Work

• Refining the task domain ontology.
• Building a presentation model for all 8 steps of
  the scenario.
• Developing a state-summary report.

15

• THANK YOU !!