Formal Description Technique for Multimodal Interactive Systems

1
Formal Description Technique for Multimodal
Interactive Systems

• Amélie Schyn
• Supervisor Philippe Palanque

2
Context

• WIMP Interfaces
• Simples Input/Output devices Keyboard/Mouse/Scree
  n
• Standard Events Click, Double-Click,
• Conventional interaction techniques DragDrop,
• A lot of well-defined formal methods
• Post-WIMP Interfaces
• Exotic and numerous Input/Output devices
• Non standard events
• Non conventional interaction techniques
• Few formal methods more or less adapted

3
Multimodal Interaction

• Reliability
• Between 35 and 50 less critical error during
  interaction
• If a device stops working, it is always possible
  to use another device to specify a critical
  command
• Efficiency
• Spatial commands

USE OF MM IN THE FIELD OF SAFETY CRITICAL SYSTEM
NEED FOR ADEQUATE FORMAL METHODS
4
Modelling Interface

• Reactive System
• State based
• Produce/receive events
• Internal state is function of received events
• Produced events are function of change of the
  internal state
• Reactive System lead by human
• Unpredictable Input
• Understandable Output
• Waiting time suitable for user

5
Modelling post-WIMP Interfaces

• Concurrent and Asychronise inputs
• Need to fusion/fission information
• Quantitative and qualitative temporal relations
• Dynamic creation/destruction of interaction
  objects
• Non-standard input devices
• Complex rendering

6
Impact on the formalism

• Need to explicitly represent state
• Need to handle input and output event
• Need to handle an infinite number of state
• Need to provide an architecture able to link
• Received event
• Internal state change
• Produced event
• ?Petri Nets are one of the few formalisms able to
  deal with theses constraints

7
ICO formalisme

• Set of Cooperative Classes (CO)
• Each class is formed with
• A behaviour part
• High level Petri net
• Fully described by
• Its offered services
• Their availability in function of its internal
  state
• A presentation part
• Specifies the perceptible appearance of the ICO.
• An activation function
• Links users' actions on the presentation part to
  services
• A rendering function
• Maintains the consistency between the internal
  state of the system and its external appearance

8
3)Activation
1)Presentation
2)Behaviour
4)Rendering
9
Extensions

• Quantitative temporal information (Stochastic
  Petri Nets)
• Communication mechanism based on production and
  consumption of event
• Methodological decomposition of the specification
  (in terms of level of abstraction)

10
Conclusion

• We have used the proposed extensions to describe
  post-WIMP applications
• Fusion/fission mechanism can be modeled
• Substitution of an input media
• Is relatively easy to do
• Have the same impact on the ICO specification
  than on the ARCH model

11
Perspectives

• On the formalism
• Validate our methodology on a big example to
  ensure scalability
• Rewrite semantic of the ICO
• Link our method with UML