A modelbased development for 3D User Interfaces

1
A model-based development for 3D User Interfaces

• Juan Manuel Gonzalez Calleros

Université catholique de Louvain, School of
Management (IAG) Information Systems Unit (ISYS)
Louvain-la-Neuve, 13th June, 2006
2
Contents

• Introduction
• Related Work
• Model-based development for 3D User Interfaces
• Case Study
• Conclusion

3
Introduction - Problem

• Current methods
• rarely provide the design knowledge
• are more focusing on the implementation
• Available tools for 3D UIs are Toolkits,
  interface builders, rendering engines

4
Introduction proposal

• We propose
• a user-centered approach
• model-driven architecture
• separation of concerns
• Our goal
• to demonstrate the feasibility of a
  MDE-compliant method that is user-centered as
  opposed to contents-centric for developing 3D UIs

5
Contents

• Introduction
• Related Work
• Model-based development for 3D User Interfaces
• Case Study
• Conclusion

6
Related work 3D UIs generation

• Programmatic approach OpenGL, Direct3D, C,
  MCNL, VRML, X3D,
• Toolkit programming Maya, Max3D, Crazy Eddie's
  GUI System, Anark, Alice
• Render engines Microsoft task gallery, infinite
  3D Cube, Sphere XP, looking glass, Clara browser,
  3DNA
• Model based ontoWEb (VR-Wise), ConGenIVE
  (VRIXML), InTML, Contigra

7
Related work Programmatic Approach, VRML
8
Related work Toolkit Approach, Anark
9
Related work Render Engines, 3DNA
10
Related work Model-based approach,
CoGenIVE-VRIXML

• ltGroupItem gt
• ltUISlider paramID "10"gt
• ltValue min "0" max"255"/gt
• ltTickstyle orientation " horizontal
  position " both "
• frequency "16"/gt
• lt/ UISlider gt
• ltPosition gt
• ltXgt1.0 lt/XgtltYgt1.5 lt/Ygt
• lt/ Position gt
• lt/ GroupItem gt

11
Related work Observations

• Methodological diversity
• Not user centered approaches
• Single entry and single output
• Transformations are hidden

12
Related work Shortcomings

• Lack of a methodology for developing 3D UIs
• Lack of user task models
• Lack of models independent to the final
  implementations
• Lack of a toolkit for such work
• Lack of genuine 3D UIs

13
Contents

• Introduction
• Related work
• Model-based development for 3D User Interfaces
• Case Study
• Conclusion

14
Model-based development of 3DUIs requirements

• Expressivity
• Machine processable
• Human readable
• Methodological explicitness, flexibility,
  separation of concerns

15
We rely on

• Cameleon Framework
• UsiXML
• UML class diagrams
• Graph transformations

16
Method Outline
17
Development paths
18
Graph transformation tool support AGG
19
Method Step1 Task and concepts description
20
Method Step1 - User Action Task Extension
21
Method Step1 3D Task Patterns

• A pattern is not a finished design that can be
  transformed directly into code
• It is a description or template for how to solve
  a problem that can be used in many different
  situations
• We propose some patterns for common 3D user tasks

22
Method Step1 3D Task Patterns

• Navigation pattern
• Travel
• Wayfinding

23
Method Step1 3D Task Patterns

• Navigation pattern
• Travel pattern
• Wayfinding

24
Method Step1 3D Task Patterns

• Navigation pattern
• Travel
• Wayfinding pattern

25
Method Step1 Domain Model
26
Method Step1 Software tool
27
Method Step2 Abstract Model

• An Abstract Individual Component (AIC) is an
  abstraction that allows the description of
  interaction objects in a way that is independent
  of the modality
• An Abstract Container (AC) is an entity allowing
  a logical grouping of other abstract containers
  or abstract individual components.

28
Method Step2 Tool support
29
Method Step3 To obtain a Concrete specification
30
Method Step3 Taxonomy of 3D UIs
31
Method Step3 -Taxonomy of 3D UIs

• Pure Reality
• Augmented reality
• Augmented Virtuality
• Virtual 3D GUI
• Digital 3D GUI
• 3D rendering of 2D GUI
• 2D GUI

32
Method Step3 - Lesson learnt from Taxonomy

• Source of inspiration
• Abstract characteristics
• Design options
• Gaps to fulfill
• Concrete User Interface Model

33
Method Step3 - Tool support - Alice
34
Method Step3 - Tool support - Maya
35
Method Step3 - Tool support - VUIToolkit
36
Method Step4 Final User Interface

• VRML
• X3D
• Java 3D

37
Contents

• Introduction
• Related Work
• Model-based development for 3D User Interfaces
• Case Study
• Conclusion

38
Case Study Task and domain definition
39
Case Study Concrete Definition
40
Case Study Concrete Model B
41
Case Study From Abstract to concrete Model
42
Case Study From Abstract to concrete Model
43
Mock-up of the Concrete User Interface
44
Case Study From Abstract to concrete Model

• ltGroupgt
• ltCubeFace id"C1"gt
• ltSphereTrigger defaultContent"Start"
  radious"1.5" solid"True" isVisible"true"
• isEnabled"true"gt
• ltTransform scale"8.23 8.23 8.23"
  translation"0.27 12.14 18.30"/gt
• ltTouchSensor id"TS1" enabled"True"/gt
• ltAppearance name"ButtonAppe" id"App1"gt
• ltMaterial
• diffuseColor"0.8 0.8 0.0"
  specularColor"0.11 0.11 0.11"
• emissiveColor"0.0 0.0 0.0"
  shininess"0.3"/gt
• lt/Appearancegt
• lt/SphereTriggergt
• lt/Groupgt

45
Case Study Editing the FUI in Maya
46
Case Study Editing the FUI in Alice
47
Case Study FUI generated with Alice
48
Case Study FUI generated with the VUIToolkit
49
Case Study FUI generated with the Maya
50
Case Study FUI generated with the Maya
51
Use of the virtual polling system Case Study in a
virtual world
52
Contents

• Introduction
• Related Work
• Model-based development for 3D User Interfaces
• Case Study
• Conclusion

53
Conclusion Model-based approaches
54
Conclusion

• The method based on graph transformation has
  been introduced, defined and illustrated
• We proposed a method for 3D UI development whose
  life cycle was divided into four steps ranging
  from the most abstract to the most concretes
  according to the principles of Model-Driven
  Engineering.
• The feasibility of the approach is much depending
  on the amount and the quality of the design rules
  that are encoded in UsiXML.

55
Conclusion

• Future work. Exploring more design options and
  encode them in UsiXML
• Models
• Transformational rules
• Software tools
• More FUI
• Consolidate concepts

56
Thank you

• Juan Manuel Gonzalez Calleros