Reverse Engineering of Declarative User Interfaces

1
Reverse Engineering of Declarative User
Interfaces
Laurent Bouillon Ph.D. in Management
Sciences Information systems

• Examination committee
• Prof. Bernard Fortz, Université catholique de
  Louvain, Examiner
• Prof. Jean-Luc Hainaut, Facultés Universitaires
  Notre-Dame de la Paix, Reader
• Prof. Manuel Kolp, Université catholique de
  Louvain, Examiner
• Prof. Christophe Kolski, Université de
  Valenciennes et du Hainaut-Cambrésis, Reader
• Prof. Jean Vanderdonckt, Université catholique de
  Louvain, Advisor
• Louvain-la-Neuve, 21/6/2006

2
Presentation Outline

• The reengineering problem
• Objectives and working hypotheses
• Reference framework
• Retargeting
• Our notation for RE rules
• Reverse engineering of
• Web pages (HTML)
• Phone interfaces (WML VoiceXML)
• Windows applications (resource files)
• Results evaluation
• Conclusion

3
1.The Reengineering Problem

• Interactive Software evolution context of use
  (U,P,E)

time
4
1.The Reengineering Problem

• Reengineering of information systems
• Focuses on reverse engineering
• Forward engineering has been achieved in several
  other researches
• Centered on UI
• It represents 50 of the total code of an
  information system
• Other aspects of information systems are already
  treated
• Less portable code which has most of time to be
  rewritten entirely

Functional core
Communication
Data Base
User Interface
Abstract Representation
Functional core
Reverse Engineering
Forward Engineering
Functional core
Communication
Communication
Data Base
Data Base
User Interface
User Interface
Source Information System
Target Information System
5
1. Shortcomings of Current Approaches

• Lack of generality approaches are generally
  dedicated to peculiar couples of languages or
  limited scope.
• Lack of flexibility RE rules and techniques are
  applied in a rigid way
• Lack of controllability designer can not
  control the process in the RE / non-explicit
  transformations

6
2.Objectives and Working Hypotheses

• The reverse engineering of UI at a higher level
  of abstraction than the code level supports UI
  reengineering with more flexibility while
  preserving predictability, more generality and
  controllability in the process than with
  code-to-code (transcoding) approaches or current
  reverse engineering approaches.
• Working Hypotheses
• Scope declarative languages (HTML, WML,
  VoiceXML, resources files)
• Methodology model-based approach / model
  recovery
• Technique static analysis
• Output language XML compliant UI description
  language
• Context focus on platform

7
3.Reference Framework for Multi-Context UIs
8
4.Retargeting

• Retargeting RE rules tailored for a specific
  context of use.

Task
Concepts
Task
Concepts
Task
Concepts
Task
Concepts




Abstract UI
Abstract UI
Abstract UI
Abstract UI




Concrete
UI
Concrete
UI
Concrete
UI
Concrete
UI





Final UI

Final UI

Final UI

Final UI
Source
platform
Target
platform
Source
platform
Target
platform
9
Retargeting (2)

• Gain
• Reduce the scope of possible results and
  translation operations gain in efficiency
• Benefits from code to refine RE and allows to
  remove code know as platform specific gain in
  adequacy to the new context of use
• Allow automated reengineering(no human
  intervention)
• Loss
• Loss of generality and maintanability
• Loss of distinction of RE and translation
• Loss of the platform independence of the produced
  model

10
5. Our notation for RE rules

• Requirements
• Make explicit the reverse engineering process
• Use of a common representation for rules for
  various source (declarative) UIs
• Outline rules categories reusable for several
  types of UIs.
• Other techniques were considered (UML-OCL, TXL,
  graph transformation)

11
5. Our notation for RE rules

• Based on several notations and on graph / tree
  theories.
• Notation consists of derivation rules (described
  thanks to sets of operations-functions) applied
  on tree reprensentation of UI.
• Derivation rules consider the source language of
  the FUI and abstract it to another level,
  allowing to recover a CUI or AUI model thanks to
  a series of mappings between languages
  meta-models.
• Set of basic operations
• Addnode(x), ModifyAttribute(x.a), DeleteArc(x,y)
• Set of functions
• ParentNode(x), IsLeaf(x), IsInPath(x)

12
5.Example of Derivation Rules

• Examples
• For a textbox in HTML/WML
• ? x ? Ts x input ? (x.typetext ?
  x.typepassword ? x.typeNULL)? Addnode
  (textComponent, idtext)
• where idtext NodeAmount(Tt)
• For a table in HTML/WML
• ? x ? Ts x table ? x.bordergt0
• ? Addnode (table, idtable) where idtable
  NodeAmount(Tt)
• ? ? x ? Ts x table ? x.border0 ?
  Addnode (box, idbox) where idbox
  NodeAmount(Tt)
• ? AddAttribute (idbox, type, vertical)

Input
TextComponent
Namein1 Maxlength15 Valuelogin Typetext
Namein1 Maxlength15 defaultContentlogin Idin1

Element Y
Element B
13
5.Derivation rules categories

• Rules can be classified into different
  categories, outlining the common issues in a RE
  process for various source UIs.
• Element recovery
• Attribute detection
• Layout / temporal ordering analysis
• Dialog recuperation
• Hierarchy recovery
• Multi-model transformations
• Retargeting operations

? x ? to Tti ,y ? to Tt0 x window ? (ybox ?
ywindow) ?x.filename y.targetfile
? CloneNode(x.id, idnew, Tt0) where idnew ?
node ? Tt0 ? RemoveNode(x, x.id) ?
RemoveArc(ParentNode(x).id, x.id) ? zroot(Tti) ?
Remove Node(z,z.id) ? AddArc(y.id, idnew) ? x
? to Tti ,y ? to Tt0 x vocalGroup ?
yVocalGroup ?x.filename y.insertFile
? CloneNode(x.id, idnew, Tt0) where idnew ?
node ? Tt0 ? RemoveNode(x, x.id) ?
RemoveArc(ParentNode(x).id, x.id) ?
zroot(Tti) ? Remove Node(z,z.id)
? AddArc(y.id, idnew)
Multi-model transformations
14
6.1.RE of Web PagesVaquita

• First RE study with the following major working
  hypotheses
• Focus on presentational aspects
• No recovery of layout and dialog
• XIML presentation model as ouput
• Static flexible RE of Web Pages

15
6.1.RE of Web PagesVaquita
Flexibility ensured thanks to reverse
engineering options

• - Implemented in VB 6
• About 10,000 LOC
• Local Web page as input

16
6.1.Reengineering with Envir3D
HTML Page
ltPresentation_element Id"First(give_Label-8"gt ltNa
megta Labellt/NamegtltFeaturesgt ltAttribute_Statement
Definition"LabelString"gt First (given)
namelt/Attribute_Statementgt ltAttribute_Statement
Definition"LabelSize"gt 7lt/Attribute_Statementgt ltA
ttribute_Statement Definition"LabelFont"gt sans-se
riflt/Attribute_Statementgt ltAttribute_Statement
Definition"LabelColor"gt 000000lt/Attribute_Statem
entgt ltAttribute_Statement Definition"LabelStyle"gt
normallt/Attribute_Statementgt ltAttribute_Statement
Definition"LabelJustif"gt Leftlt/Attribute_Stateme
ntgt ltAttribute_Statement Definition"Margin"gt 0lt/A
ttribute_Statementgtlt/Featuresgt lt/Presentation_elem
entgt
Vaquita reverse engineering
XIML Page
Envir3D virtualisation of non-virtual user
interface
VRML97/X3D file
VRML97-enabled browser
17
6.1.RE of Web PagesReversiXML

• 2nd reverse engineering tool to overcome
  shortcomings of Vaquita
• Modified working hypothesis

18
6.1.RE of Web PagesReversiXML
1. Web Page Extraction
Temporary Pages
Server
UI Model DB
2. Syntax Correction
Tidy
Configuration DB
3.Selection of Options
4. Global Scan
Temporary Information
5. Local Scan
WhoAmI
6. Layout Relations
7. Dialog Relations
8.Model Correction
Context DB
UI Model DB
9. Context Model
10. Save Model
19
6.1.RE of Web PagesReversiXML

• Written in php 5
• On-line RE
• About 12,000 LOC
• Set of libraries

20
6.2.RE of Phone Interfaces

• Major working hypotheses
• Static RE of WML 1.1 and voiceXML 2.0 up to the
  CUI in USIXML 1.4.6
• Recovery of the layout and hierarchy/temporal
  ordering.
• Dialog-Navigation analyzed (but not scripts)
• No retargeting operations
• Similar method to HTML applied for WML VoiceXML
  reverse engineering
• Description of languages meta-models and
  derivation rules.

21
6.2.RE of Phone Interfaces

• WML
• Several derivation rules identical - or less
  constraining- to HTML (combined rules starts with
  TH/W)
• RE prototype developed in Cui05
• VoiceXML
• Derivation rules completely different from HTML
  and WML
• Several losses due to ecmascripts integration
  (conditions, events).

22
6.2.RE of WML Phone Interfaces

• Example
• Prototype Cui05 using XSLT XPATH
• Local process allowing no design alternatives

WML ltpgt Yahoo! ID ltinput name"login" value""
format"m" /gtPassword ltinput type"password"
name "passwd" value"" format"m" /gt ltanchor
title"OK"gtSubmit ltgo method"post" href"/raw?"gt
USIXML lttextComponent id "textComponent_1"
glueHorizontal "left isVisible"true"
isEnabled "true" defaultContent"Yahoo!
ID lttextComponent id "textComponent_2"
glueHorizontal "left" defaultContent""
isEditable "true" isVisible"true"/gt
23
6.3.RE of Windows Interfaces

• Major working hypothesis
• Static RE of .RC files up to the CUI in USIXML
  1.4.6
• No recovery of dialog
• No layout, hierarchy and retargeting not covered
• Resource file have to be modified in order to
  obtain a tree structure, so that derivation rules
  can be applied
• Major difficulties
• Compact attribute specifications shortcuts
  notations / hexadecimal
• Layout expressed in absolute positions /
  overlapping elements
• Hierarchy of elements not clearly defined
• Attributes values defined/modified in the
  functional core

24
7.Results of the Evaluation

• Theoretical validation of the HTML RE approach
• Coverage
• 93 of HTML tags recovered
• Coherence
• Coherent application of rules and, by analogy, of
  the entire rules set
• Performance
• 20 web pages reverse engineered in a classical RE
  and with retargeting
• 25 (up to 65) of size reduction thanks to
  retargeting and 20 of process time diminution
• Correction
• Manual RE of 5 web pages
• Most important categories are always fully
  recovered
• Retargeting preserve correctness of models

25
7.Results of the Evaluation

• Exploratory Study
• qualitative study achieved by 17 students of
  LINF2356 course
• Case studies illustrating various HTML RE
  techniques
• Study of reengineering of the Sedan-Bouillon
  Website thanks to two FE tools Teresa and QtkXML

26
7.Results of the Evaluation

• With Teresa
• RE applied without retargeting
• USIXML CUI model used as input in Teresa that
  performs translations for another context of use
• Produces the Web site designed for Pocket PC
  (XHTML)

27
7.Results of theEvaluation

• With QtkiXML

Original UI
Reengineered UI
28
8.Conclusion - Contributions

• Conceptualization
• Development of a flexible method in a model-based
  approach addressing the problem of declarative UI
  RE processBoui2a
• Retargeting concept Boui02b
• Operationalization
• 2 HTML RE tools RIHM,Boui02c, Boui03
• Study of full reengineering with Teresa,
  Envir3DBoui04, QtkiXML

29
8.Conclusion - Contributions(2)

• Test
• The validation against four criterion (coverage,
  coherence, correction, performance)
• Exploratory study of the use of the tool and
  RE/retargeting evaluation
• Generalization
• Semi-formal notation for RE derivation rules
• Meta-models and derivation rules for XIML, HTML,
  WML, VoiceXML, Windows resource files and
  identification of common RE subproblems Boui05.

30
8.Conclusion - Projects Contributions

• Cameleon (2002-2005)
• Approach for Web RE and tool support
• Contributions to USIXML
• Similar (2005-2006)
• Extension of the approach to phone UI and windows
  applications
• Extension of USIXML

31
8. Conclusion - limitations

• Dialog aspects of the UI (imperative language).
• Static RE implies losses, dynamicity needed to
  complete models.
• Retargeting concept.
• Notation for declarative UI with simple CF and
  representable as a tree.
• Limited automated reengineering (usability).

32
8.Perspectives

• Short term
• Extend the coverage of the RE sub-problems for
  windows UI.
• Implement two new tools to RE VoiceXML and
  Windows resources files.
• Expand the existing implementation of ReversiXML.
• Long term
• Widen scope of sources languages (Visual Basic,
  Delphi).
• RE of scripts to recover dialogue entirely.
  Explore new RE techniques.
• Complete RE of context model.
• Study of RE to other languages (UIML)

33
Perspectives
ltWindow id1 name1 isVisibletrue
isEnabledtrue defaultBorderTitleCalculator
borderWidth1height358 width309gt ltBox
id2 name2 type verticall
isVisibletrue isEnabledtruegt ltbutton
isEnabledtrue

• Begin VB.Form Calculator BorderStyle 1 'Fixed
  Single Caption "Calculator" ClientHeight 2970
  ClientLeft 2580 ClientTop 1485 ClientWidth
  3270 ClipControls 0 'False BeginProperty Font
  name "System" charset 1 weight 700 size
  9.75 underline 0 'False italic 0 'False
  strikethrough 0 'False EndProperty Height
  3375 Icon "CALC.frx"0000 Left 2520 LinkMode
  1 'Source LinkTopic "Form1" MaxButton 0
  'False ScaleHeight 2970 ScaleWidth 3270 Top
  1140 Width 3390 Begin VB.CommandButton Number
  Caption "7" Height 480 Index 7 Left 120

USIXML
34

• The interactive systems of today are the legacy
  systems of tomorrow. Therefore, reverse
  engineering of information systems will never
  become an obsolete problem itself.
• THANK YOU FOR YOUR ATTENTION

35
(No Transcript)
36
6.1.RE of Web PagesVaquita
Web page extraction
HTML Page
ReverseEngineering
Tidy
XML File
Manual modification
Msxml3.dll
Detection
VAQUITA
XIML Presentationmodel
Structuration
Translation
Forward Engineering
New DOM writing
User Interface Generator
.RES Resource file
WML, VoiceXML,HTM files
37
6.3.RE of Windows Interfaces
ltBox id18 name18 typevertical
isVisibletrue isEnabledtrue height90
width55 defaultBorderTitleDirectiongt ltBox
id19 name19 typehorizontal
isVisibletrue isEnabledtruegt ltradtioButton
id20 name234 isVisibletrue
isEnabledtrue defaultMnemonicu
/gt lttextComponent id21 name9
defaultContentUp isVisibletrue
isEnabledtrue textSize8 textFontMs Shell
Dlg /gt lt/boxgt ltBox id22 name22 type
horizontal isVisibletrue isEnabledtruegt ltRa
dioButton id23 name235 isVisibletrue
isEnabledtrue defaultMnemonicd
/gt lttextComponent id24 name24
defaultContentDown isVisibletrue
isEnabledtrue textSize8 textFontMs Shell
Dlg /gt lt/boxgt lt/boxgt ltbutton id25 name205
isVisibletrue isEnabledtrue
defaultMnemonicf defaultContentFind Next
/gt

• 14 GROUPBOX "Direction", -1, 135, 22, 60,
  34, WS_GROUP
• 15 AUTORADIOBUTTON "Up", 234, 140, 30, 45,
  12, NOT WS_TABSTOP WS_GROUP
• 16 AUTORADIOBUTTON "Down", 235, 140, 42,
  45, 12, NOT WS_TABSTOP

38
6.3.RE of Windows Interfaces
Windows .RES file
Windows .RC file
Decompilation
UI detection
Elements recovery
KB of shortcut notations
Attribute recuperation
Hierarchy detection
Layout deduction
Dialog relations
USIXML CUI
Save model
39
Architecture ReversiXML 1.0-1.1
40
Architecture ReversiXML 1.2
41
UIML other languages

• Valable car USIXML a des capacités dexpression
  très similaire à UIML (contexte).
• Peu de changement conceptuellement, les principes
  restent les mêmes (détections éléments,
  attributs, layout)
• Différences majeures
• UIML plus proche du FUI (incorporation de
   code ).
• Plus grande séparation entre présentation et
  contenu - (mais resource model en USIXML).
• Connections avec les autres parties de
  linterface et comportement des objets plus
  précis
• USIXML plus riche en termes de modèles,
  différents niveaux dabstraction et outils
  non-commerciaux.

42
Conclusion

• Contributions categorized into four levels
• In the conceptualization
• The development of a flexible method thanks to a
  model-based approach addressing the problem of
  declarative UI RE process
• The adaptation of the RE definitions to the UI
  domain and their positions in the conceptual
  framework (section 2.2).
• The retargeting concept (section 2.3) and some
  retargeting rules (appendix G).
• The constitution of a state of the art in RE
  (chapter 3)
• The identification of shortcomings of the output
  language that led to modifications of the USIXML
  language.
• The specification of meta-models illustrating the
  complete structure of XIML, HTML, WML, VoiceXML,
  Windows resources files in UML class diagrams
  (appendix A and B).

43
Conclusion (2)

• The development of derivation rules based on
  these meta-models for HTML (6.3), WML (7.1.3),
  VoiceXML (7.2.3), Windows resource files (section
  8.3) completed by appendix C. Derivation tables
  for HTML towards XIML presentation model were
  also described (5.3).
• For the operationalization
• Vaquita for the static flexible RE of HTML files
  into an XIML presentation model.
• ReversiXML an online tool allowing the flexible
  RE of HTML files into a CUI or AUI model
  expressed in USIXML.
• A prototype of RE tool able to extract a CUI
  model from WML files.
• In the test phase
• The validation against four criterion coverage,
  correction, coherence and performance(section
  9.1)

44
Conclusion (3)

• An exploratory study describing a qualitative
  evaluation of the produced models, the tool, and
  the advantages of using retargeting (section
  9.2.1).
• Three case studies of various complexities
  (section 9.2.2)
• An investigation of the full reengineering by
  combining the results of ReversiXML with
  TeresaXML and QtkXML (section 9.2.2) and the
  reengineering in a virtualization process is also
  described in section 5.1.4.
• For the last step of generalization
• A semi-formal notation for RE derivation rules
  (chapter 4 and appendix C) for various source UI.
• A generalization of the reverse engineering
  method thanks to an analysis of the process
  applied to a large scope of UI and the
  identification of common RE tasks to be achieved
  (section 4.4.8).

45
ReversiXML Shortcomings

• CSS support
• Javascript RE
• Embedded components (flash, java...)
• Universal access
• Box reduction
• Use of statistics to refine RE
• Pursue implementation
• Site RE
• Evolution to architecture 1.2
• Context model selection mechanism
• Complete the RE of AUI

46
6.2.RE of Phone Interfaces
WML or VoiceXML file
Elements attribute recovery
Hierarchy/temporal sequence detection
Temporary Information
Dialog relations
Model Correction
Model links
USIXML CUI
Save model
47
1.The Reengineering Problem

• Benefits of reengineering
• Practical benefits
• Economical benefits
• Theoretical benefits

• Importance of reengineering
• Amount of legacy systems (60)
• New needs for reengineering
• New context of use (internet enabled platforms)

48
Integration with other works
49
7.Results Evaluation

• Exploratory Study
• qualitative study achieved by 17 students of
  LINF2356 course
• Aim use ReversiXML and GrafiXML to produce
  previews of UI suited for mobile phones and
  pocket PC, by using
• retargeting operations
• manual modifications
• Conclusion
• Produces correct and expected results with
  various configuration
• Flexible process/retargeting really important as
  it allows to save time and efforts during the
  reengineering process

50
MDA compliance

• Models are expressed in a unified notation and
  form the cornerstone to understanding software
  systems (semantics of the models are based on
  meta-models).
• ? UsiXML models are well-formed models based on
  XML schema. The semantics of the models are based
  on meta-models expressed in terms of UML
  diagrams, from which the XML schema is derived.
• The building of software systems can be organized
  around a set of models by applying a series of
  transformations between models, organized into an
  architectural framework of layers and
  transformation.
• ? Model-to-model transformations are specified
  in UsiXML. Model-to-code transformations are
  ensured thanks to a set of forward engineering
  tools. Code-to-model transformations are achieved
  by derivation rules that are based on the mapping
  between the meta-model of the source language and
  the meta-model of UsiXML.

51
MDA Compliance (2)

• A formal underpinning for describing models in a
  set of meta-models facilitates meaningful
  integration and transformation among models, and
  is the basis for automation
• ?All transformations are explicitly defined,
  based on a series of predefined semantic
  relationship and a set of the primitive ones
  (abstraction, reification and translation).
• Acceptance and adoption of this model-driven
  approach requires industry standards to provide
  openness to consumers, and foster competition
  among vendors.
• ? The last principle is on the way, as the
  potential wide adoption of the above techniques
  will validate the principle.
• Four levels of abstraction are almost similar
  with the various layers of MDA.

52
8.Conclusion rules categories

• Elements and attributes
• 46 rules sets
• Layout / Temporal ordering
• 6 rules sets/ 10 rules sets
• Dialog
• 11 rules sets
• Hierarchy
• 6 rules sets
• Multi-Model transfomation
• 6 rules sets
• Retargeting
• 14 rules sets

53
MDE

• MDE
• Attention focuses first on the application's
  business functionality and behavior
• Same principle but focus on processing/communicati
  on
• UML MOF

54
TransformiXML
IdealXML
FlashiXML
Rendering
UsiXML model Abstract user interface
UsiXML model Concrete user interface
UsiXML models task, domain
Graph transformations
Graph transformations
Final user interface
Generative programming
VisualiXML
Derivation rules
ReversiXML
GrafiXML VisiXML SketchiXML FormiXML
KnowiXML
55
QVT

• QVT
• UML to UML mappings
• Too complex for the objectives of our notation
• Tools applies on UML diagrams, based on UML
  meta-models, not on code based on UML meta-models
• Open question, as released recently