A MDACompliant Environment for Developing User Interfaces of Information Systems

1
A MDA-Compliant Environment for Developing User
Interfaces of Information Systems

• Jean Vanderdonckt
• vanderdonckt_at_isys.ucl.ac.be
• Head of BCHI Lab, http//www.isys.ucl.ac.be/bchi

2
 Everything you can imagine is real  (Picasso)
 Everything you model can be turned into a
real interface
3
Outline

• Conceptual modeling of user interfaces
• Forward engineering
• Reverse engineering
• Lateral engineering
• Conclusion

4
1.1 Why is conceptual modeling of user interfaces
desirable?

• The presentation layer outside the scope of
  CSCD (Antoni Olivé)
• A domain model is certainly not a presentation
  model, but there could be a mapping between
• UIs remained for many years the poor parent of
  conceptual modeling and software engineering
• It emerged during late 80s
• Still growing
• The complexity and the variety of user interfaces
  (UI) is dramatically increasing
• The paradigm of  one system fits all  is no
  longer valid

5
1.1 Diversity of users

• Traditional users
• People with disabilities
• Visual, motor, cognitive

6
1.1 Variety of tasks
Forrester Research, Inc., 2003

• Users want to determine their path on each
  platform

7
1.1 Heterogeneousness of platforms
8
1.1 Multiplicity of contexts of use
9
1.1 Consequence

• Proliferation of developments

10
1.1 Consequence

• Why is it difficult?
• For the designer
• Multiplicity of contexts of use
• No systematic design method
• For the user
• Poor usability of resulting UI
• UI not adapted to the genuine context of use
• For the developer
• Increase of development and maintenance costs
• Increasing development complexity
• Little of no factoring out of common elements

Fold Drop technique P. Dragicevic
11
1.2 What does it cover?

• Conceptual modeling should cover
• Presentation external manifestation of the UI
  that is perceivable to the user (static)
• Dialog ordering in time and space of operations
  performed by the user/system (dynamic)
• Modalities of interaction
• Graphical
• Vocal
• Haptic, tactile

12
1.2 What does it cover?
13
1.3 Abstraction 1 the concrete UI

• Starting point FUI Final User Interface
• Mark-up languages HTML, WML, cHTML
• Programming languages Visual Basic, Java
• Many manifestations of the same object
• Abstraction with respect to the toolkit
• Presentation aspects
• Events generating, passing, controlling
• Attributes external (e.g., font) vs internal
  (e.g. state)
• Primitives life cycle

14
1.3 Abstraction 1 the concrete UI
Vanderdonckt Bodart, 1993

• Definitions
• Original Abstract Interaction Object
  abstraction of the same Concrete Interaction
  Objects independently of their underlying
  presentation and dialog
• We have introduced this abstraction in 1993!
• Current Concrete Interaction Object
  abstraction of interaction objects with respect
  to the underlying toolkit

15
1.3 Abstraction 1 the concrete UI
Limbourg, 2004

• Each graphicalCIO has specific properties such as
  isVisible, isEnabled, fgColor, and bgColor.
• Each graphicalCIO is sub-typed into one of the
  two possible categories graphicalContainer, and
  graphicalIndividualComponent

CUI Model
CUI Model
CIO
CIO
graphicalCIO
graphicalCIO
graphicalContainer
graphicalContainer
graphicalIndividualComponent
graphicalIndividualComponent
GrafiXML
16
1.3 Abstraction 2 the abstract UI

• Different CIOs can be used for the same purpose,
  but with different interaction modalities
• Definition
• Abstract Container set of Abstract Individual
  Component
• AIC abstraction of CIOs of the same type, but
  independently of any interaction modality
• Abstract User Interface (AUI) decomposition
  into ACAIC

17
1.3 Abstraction 2 the abstract UI
Limbourg, 2004
18
1.3 Abstraction 2 the abstract UI
Montero et al., 2005

• Notation based on canonical abstract prototypes
  Larry Constantine, Helmut Windl, James Noble,
  Lucy Lockwood http//www.forUse.com

IdealXML
19
1.3 Abstraction 3 the task
Paternò et al., 1997

• Task set of actions carried out by a user in a
  given context to reach a goal
• Logical decomposition of task into sub-tasks
• Temporal ordering LOTOS operators
• T1 gtgt T2 Enabling
• T1 gtgt T2 Enabling information passing
• T1 gt T2 Suspend/resume
• T1 T2 Choice
• T1 gt T2 Disabling (e.g. Form submit)
• T1 T2 Independence (any order, but finished)
• T1 Iteration
• T1n Finite iteration
• T1 T2 Concurrency
• T1 x T2 Concurrency information passing
• T Optional
• T Recursion

CTTE Editor
20
1.3 Abstraction 3 the task

• Task definition action object
• Action types
• Acquire, render, modify, publish, compute,
  derive,
• Object types
• Element, list, table, collection, compound,

21
1.3 Abstraction 3 the task
Limbourg, 2004
22
1.3 Abstraction 4 the context of use
Cameleon project, 2004

• Context of use triple (U,P,E) where
• U is a user model from cognitive psychology
• P is a platform model subset of UAProf (W3C)
• E is an environment model from ubiquitous
  computing

Pick Drop technique J. Rekimoto
23
1.3 Abstraction 4 the context of use
Limbourg, 2004
24
1.3 Mapping the models
Montero et al., 2005
25
1.3 Mapping the models
Limbourg, 2004

• Mapping the models with a mapping model (!!)

26
1.4 What do we have so far?
Method triggered download a file Object
computer file
Task

Task
Task

Task
Classes
Classes
Domain
Modality-independent
Control AIC
Abstract Individual Component
Abstract User
Abstract User
Interface (
AUI
)
Interface (
AUI
)
Digital control AIC
Physical control AIC
Toolkit-independent Concrete Interaction Object
Graphical 2D push button
Graphical 3D push button
Physical push button
Concrete
User
Concrete
User
Interface (CUI)
Interface (CUI)
HTML push button
Windows push button
OSF/Motif XmButton
VRML97/X3D button
Software key
Function key
Code
Code
ltinput typesubmit valueDownload" name
Final User
Final User
Interface (FUI)
Interface (FUI)
Rendering
Rendering
Down
Down
Down
Download
Download
Download
Load
Load
Load
27
1.4 What do we have so far?
SSource context of use
User S
Task and Domain S
UsiXML supported model
Abstract user Interface S
UsiXML unsupported model
Concrete user Interface S
Final user Interface S
28
2. Forward engineering
Limbourg, 2004

• 2.1 Task and domain models
• 2.2 From TD to AUI
• Model-to-model transformation
• 2.3 From AUI to CUI
• Model-to-model transformation
• 2.4 From CUI to FUI

29
2.1 Task and domain models
30
2.1 Task and domain models
Limbourg, 2004
31
2.1 Task and domain models
32
2.2 From TD to AUI
Limbourg, 2004

• Definition of AUI structure
• Which objects should be logically grouped?
• Definition of AIC types
• Which  functionnality  should assume AICs and
  what data do they manipulate ?
• Definition of spatio-temporal arrangement
• How should AIC be arranged in space and/or in
  time ?
• Definition of dialog control
• What is the valid flow of action on AIOs ?

33
2.2 From TD to AUI
Limbourg, 2004

• Definition of AUI structure

34
2.2 From TD to AUI
Limbourg, 2004

• Definition of AIC types

AC Abstract Component AIC Abstract Individual
Component CIC Concrete Interaction Component
35
2.2 From TD to AUI
Limbourg, 2004

• Definition of spatio-temporal arrangement

36
2.2 From TD to AUI

• All transformations are in UsiXML
• Each model instance of meta-model
• Each model graph as instance of graph type
• Each model-to-model transformation
• graph transformation
• Set of productions

37
2.2 From TD to AUI
Limbourg, 2004

• Typed model-to-model transformation

Meta-Meta-Model
Graph Structure
is instance of
Meta-Model
Our Meta-Model
Meta-Model Subset 1
Meta-Model Subset 2
e.g., TaskDomain Model
e.g., Concrete UI Model
Uses language
is instance of
is instance of
Initial UI Model
Resultant UI Model
Transformation Rule
e.g., MyTaskAndDomainModel
e.g., MyConcreteUIModel
Our transformation catalog
38
2.2 From TD to AUI

• TransformiXML

39
2.3 From AUI to CUI
Limbourg, 2004

• Definition of CUI structure
• Which AIC is a window?
• Definition of CIO type
• Which  widget  should represent which AIO ?
• Definition of placement
• What layout can be specified between CIOs
• Definition of navigation
• Which container can be started or closed from
  which container?
• Definition of dialog control
• What is the dialog local to each CIO?
• What is the valid flow of action on CIOs?

40
2.3 From AUI to CUI
Limbourg, 2004

• Definition of CIO structure

41
2.3 From AUI to CUI

• Definition of placement

Limbourg, 2004
42
2.4 From CUI to FUI

• Model-to-code transformation
• By rendering (interpretation)
• Java (InterpiXML)
• Tcl/tk (QtkiXML)
• Flash (FlashiXML)
• By code generation
• HTML
• Visual Basic

43
2.4 What do we have so far?
SSource context of use
User S
Task and Domain S
UsiXML supported model
Abstract user Interface S
UsiXML unsupported model
Concrete user Interface S
Final user Interface S
44
3. Reverse engineering

• 3.1 From FUI to CUI
• 3.2 From CUI to AUI, task domain

45
3.1 From FUI to CUI
Bouillon Vanderdonckt, 2004

• Do you have the source code?
• Markup languages (e.g., HTML) static code
  analysis (ReversiXML)
• Do you have the compiled code?
• Programming languages (e.g., compiled C)
  resource file extraction and static code analysis
• Do you have the running application?
• Video analysis computer vision
• Do you have only the documentation?
• Image analysis image processing

46
3.1 From FUI to CUI
Bouillon Vanderdonckt, 2004

• ReversiXML

47
3.2 From CUI to AUI, task domain
Limbourg, 2004

• Code-to-model and model-to-model
• CUI to AUI

48
3.2 From CUI to AUI, task domain

• Re-engineering reverse forward
• Possible re-interpretation by QtkiXML

Bouillon Vanderdonckt, 2004
49
3.2 What do we have so far?
SSource context of use
User S
Task and Domain S
UsiXML supported model
Abstract user Interface S
UsiXML unsupported model
Concrete user Interface S
Final user Interface S
50
3.2 What do we want to get?
Cameleon project, 2004
TTarget context of use
SSource context of use
User S
User T
http//www.plasticity.org
Task and Domain T
Task and Domain S
UsiXML supported model
Abstract user Interface T
Abstract user Interface S
UsiXML unsupported model
Concrete user Interface T
Concrete user Interface S
Final user Interface T
Final user Interface S
51
3.3 Lateral engineering

• Definition model-to-model transformation
  applied at the same level of abstraction
• Same context of use
• Across various contexts of use
• Examples
• Simple CUI adaptation widget replacement
• Design-time vs run-time adaptation

52
Example 1 widget replacement (1)
Limbourg et al., 2004
lt?xml version"1.0" encoding"UTF-8"
standalone"yes"?gt ltcuiModel name"MyModel"gt
ltversion modifDate"2004-03-24T170917.4020100"
xmlns""gt7lt/versiongt ltauthorName
xmlns""gtYourilt/authorNamegt ltwindow
height"500" width"600" name"Formulaire (2/5)"
id"window_1"gt ltbox relativeHeight"100"
name"box1_0" id"box1_0"gt ltbox
type"vert" name"boxTodo" id"boxTodo"gt
... ltbox type"horiz"
name"box_2_2_2_1" id"box_2_2_2_1"gt lttextCompone
nt defaultContent"Sexe" isBold"true"
id"label_2"/gt ltradioButton groupNamegrupo01"
defaultContent"Femme"
defaultState"false"
id"radiobutton_0"/gt ltradioButton
groupName"grupo01" defaultContent"Homme"

defaultState"true" id"radiobutton_1"/gt
lt/boxgt ... lt/boxgt
lt/boxgt lt/windowgt lt/cuiModelgt
Excerpt for an usiXML CUI specification.
53
Example 1 widget replacement (2)
54
Example 1 widget replacement (3)
The UsiXML graph before applying any rule
55
Example 1 widget replacement (4)
Limbourg et al., 2004
Rule 1 Create a new comboBox with the same id
and name as the name of the group of radioButtons.
NAC
RHS
LHS
56
Example 1 widget replacement (5)
Rule 1 Create a new comboBox with the same id
and name as the name of the group of radioButtons.
The usiXML graph after aplying the first rule
57
Example 1 widget replacement (6)
Limbourg et al., 2004
Rule 2 Convert every radioButton within the
group x into an item for the comboBox x, we
have just created.
LHS
RHS

58
Example 1 widget replacement (7)
Rule 2 Convert every radioButton within the
group x into an item for the comboBox x, we
have just created.
The usiXML graph after aplying the second rule
59
Example 1 widget replacement (8)
lt?xml version"1.0" encoding"UTF-8"
standalone"yes"?gt ltcuiModel name"MyModel"gt
ltversion modifDate"2004-03-24T170917.4020100"
xmlns""gt7lt/versiongt ltauthorName
xmlns""gtYourilt/authorNamegt ltwindow
height"500" width"600" name"Formulaire (2/5)"
id"window_1"gt ltbox relativeHeight"100"
name"box1_0" id"box1_0"gt ltbox
type"vert" name"boxTodo" id"boxTodo"gt
... ltbox type"horiz"
name"box_2_2_2_1" id"box_2_2_2_1"gt lttextCompone
nt defaultContent"Sexe" isBold"true"
id"label_2"/gt ltcomboBox id"comboBox001"
name"label_3" isDropDown"true"gt ltitem
id"radiobutton_0" name"radiobutton_0"
defaultContent"Femme"/gt ltitem
id"radiobutton_1" name"radiobutton_1"
defaultContent"Homme"/gt lt/comboBoxgt
... lt/boxgt
lt/boxgt lt/windowgt lt/cuiModelgt
Excerpt from the final transformated usiXML
specification
60
Example 1 widget replacement (9)
61
Example 2 Adaptation

• Two forms of adaptation
• Adaptability user-initiated
• Adaptivity system-initiated
• Four steps user, system, third party,
  combination
• Detection
• Computation
• Decision
• Execution
• 2 Adaptivity forms
• UI remain the same vectorial UIs
• UI change at run-time plastic UIs

62
Example 2 Plastic UI
Grolaux, Van Roy, Vanderdonckt, 2002
63
Example 3 multimodal UI
Stanciulescu, Limbourg, Michotte, Vanderdonckt,
2005

• Need for additional abstraction
• Additional transformation rules

64
Example 4 Crazy UIs
Molina, Vanderdonckt, Gonzalez, 2005

• Use the same model
• for 3D UI Cube
• for migration across platforms in virtual
  reality MigriXML

65
5. Conclusion

• A multi-path support of MDA

TransformiXML
IdealXML
FlashiXML
Rendering
UsiXML model Abstract user interface
UsiXML model Concrete user interface
UsiXML models task, domain
Graph transformations
Graph transformations
Final user interface
Generative programming
VisualiXML
Derivation rules
GrafiXML VisiXML SketchiXML FormiXML
KnowiXML
ReversiXML
66
5. Conclusion

• Conceptual modeling of UIs
• Separation of concerns is feasible
• Correlability of models is required
• It is possible to have multiple abstractions on
  demand
• Not all models should be involved
• Everything is in UsiXML
• Extreme modeling or extreme engineering
• Everything is in the model relatively easy
• Model-to-model transformation harder
• Model-to-code generation hardest
• Industrial practise
• Multi-path development
• Sketching of UI

67
Future trend sketching?
Coyette Vanderdonckt, 2005
68
I would like to dedicate this talk to

• Prof. Em. François Bodart (Univ. of Namur, BE)
  for
• Inoculating me the virus of (meta-)modeling and
  structured design of information systems
• Sharing with me his vision about user interfaces
  and interactive information systems

69
I would like to thank

• BCHI team members for their involvement and their
  constant effort in the UsiXML initiative among
  others
• Laurent Bouillon ReversiXML, UsiXML
• Benoît Collignon PlastiXML
• Adrien Coyette SketchiXML
• Murielle Florins Graceful degradation
• Monica Gemo multi-platform UIs and annotations
• Juan Gonzalez 3D UsiXML
• Donatien Grolaux detachable UIs, DistriXML
• Josefina Guerrero UIs for workflows
• Quentin Limbourg multi-path development, UsiXML,
  all tools
• Céline Mariage MetroWeb, guidelines
• Benjamin Michotte GrafiXML, TransformiXML,
  UsiXML
• José Pascual Molina MigriXML, VUIToolkit
• Francisco Montero IdealXML
• Adrian Stanciulescu ModaliXML, UsiXML
• Daniela Trevisan augmented reality

70
I would like to thank

• João Falcão e Cunha, Oscar Pastor, Nuno Jardim
  Nunes
• The CAiSE Advisory Committee
• Janice Bubenko
• Collette Roland
• Arne Solvberg

71
and you for your attention
Free download and register USer Interface
eXtensible Language http//www.usixml.org
SIMILAR, the European task force making user
interfaces similar to human-to-human
communication http//www.similar.cc
Home Page of BCHI Lab http//www.isys.ucl.ac.be/bc
hi