CUSPIS HMI aspects: User interfaces, Context awareness and 3D fruition

1

A Cultural Heritage Space Identification
System Galileo and the European Cultural Assets
a European infrastructure serving another
European infrastructure

• CUSPIS HMI aspects User interfaces, Context
  awareness and 3D fruition

2
HMI Aspects
The Cultural Asset Fruition (CAF) service must
support three usage cases a) pre-travel usage
of the service b) on-travel usage c)
after-travel usage
3
CAF scenarios

• Scenario a) pre-travel usage of service
• Main functions
• Registering
• Personal settings
• POI selection
• POI guided tour
• Show POI on map
• Display information on POI
• Send a POI to a friend

4
CAF scenarios

• Scenario b) on-travel usage of service
• Main functions
• POI search (by keyword, by location, etc)
• POI guided tour
• Route guidance
• POI selection
• Show POIs on map
• Show users position on map
• Display information on POIs
• Send a POI to a friend

5
CAF scenarios

• Scenario c) after-travel usage of service
• Main functions
• Trip replay
• Show information on visited POIs

6
CAF scenarios

• General Guidelines on all scenarios
• The CAF user can have a personal tourist guide
  that gives him
• Real-time and updated information
• Multimedial contents (images, sound, text,
  videos)
• Location based information

7
HMI Requirements

• HMI Main Requirements
• provide a dialog interface for user registering
  (accepting personal data and preference settings)
• provide an interactive geographic map with
  navigation capabilities (zoom in/out, move
  north/south/west/east, etc), combined with
  interactive menus for direct city/area choice
• display a list of POIs, categorized and sorted
  according to users preferred categories
• allow the user to browse the list of POIs and
  select a specific one
• allow the user to search for a POI by specifying
  keywords
• display information related to a POI selected by
  the user, including images
• allow for playing video and audio contents (if
  available), or any other multimedial information
  associated to a POI
• display a list of suggested POIs (Guided Tour)
• display a navigable map of the area around a
  POI, with the POI positioned on the map
• show the current users position on a map
• display a list of POIs located near the users
  current position, categorized and sorted
  according to users preferences
• display POIs located near the users current
  position on a map
• display a route planning and/or route guidance
  indications on a map of the city or area of
  interest
• allow the user to modify his personal settings
  (change preferences, profile) and to add/remove
  favorite POIs at any time
• allow the user to send a POI reference (by
  email) to another registered user
• provide 3D visualization and navigation
  capabilities, if 3D contents are available for a
  particular CA or POI

8
HMI Requirements
Other important general requirements are
represented by Multilingual interfaces the
system must support different language
versions Multi-channel delivery the content
must be able to be offered to different systems,
such as PDAs, smart-phones, etc., and to any
other channel that may be suitable for the
end-user.
9
HMI Architecture
10
HMI Considerations

• Different users
• Different requirements
• Different Devices

How to create a suitable interface ?

• How to adapt interface to
• The process
• The context
• The device
• In a real environment ?

Usability If the HMI shows many functionalities
at the same time the user interface will result
ununderstandable
Hardware Different portable device, Different
displays (different Quality and different
size) How to adapt ?
Functionality lackness I dont need all the
functionalities At the same time, but when I
Need a functionality I want to Retrieve it
easily
The key are adaptivity and extensibility
11
Adaptivity

• The client application must be able to receive
  the user interface and the linked functionalities
  in dynamic manner
• The interface description should be independent
  from the implementation
• The applicative functionalities must be
  dynamically linked to the user interface

• XUL has been chose as user interface description
  language
• A parsing engine for XUL description has been
  implemented
• The applicative functionalities are dynamically
  linked to the user interface through the Java
  reflection mechanism

12
Context definition

• The Control centre knows step by step these
  contextual information
• The user profile
• The user status
• The user position
• The user device type
• These information drive the building of the
  dynamic interface during which the interface
  aspect and the relative functionalities are chosen

13
Interface generation process

• Once received the XUL description and the
  functional classes the client application provide
  to
• Generate the HMI through a XUL parsing engine.
• Instantiate the functionalities class linked to
  the HMI.

14
Interface generation process
XUL Specification
Window
lt?xml version1.0?gt lt?xml-stylesheet
hrefchrome//global/skin/ typetext/css?gt ltwi
ndow idrootWindow titleEsempio width240
height300gt lttext idButton1text
valueText1 x1 y20 width120
height130/gt lttext idButton2text
valueText2 x120 y20 width120
height130/gt ltbutton idbutton1
valuebottone1 onClickFunction1 x1
y150 width120 height150/gt ltbutton
idbutton2 valuebottone2 onClickFunction2
x120 y150 width120 height150/gt
Box
Box
Dom Parsing
Text
Text
Button
Button
Optimized data structure
Functionalities classes instantiation
Window X0 Y0 Width240 Height320 TitleEsempio
Interface generation
Functionalities binding
Box X0 Y0 Width240 Height160
Box X0 Y160 Width240 Height160
Functionalities classes
Text value Text1
Button Onclick action1
Text value Text2
Button Onclick action2
performAction()
15
HMI Components
Multimedia content player (audio, video)
3D explorer
POI content viewer
Map Viewer
16
Map viewer

• Accurate map viewer
• Vectorial maps
• Raster maps
• Position following capability
• Zoom pan
• POI highlight

17
HMI Components
3D Explorer on Mobile device This components
allows for a 3D visualization and navigation of
Cultural Assets and Points of Interests

• Examples where users could benefit from a 3D
  content visualization are represented by
•  
• Virtually reconstructed cultural assets ancient
  buildings, monuments or other cultural assets are
  often characterized by the fact that large and
  important parts are missing
• Intuitive 3D exploration of a site the use of
  3D visualization and navigation capabilities can
  help the user to better understand his current
  location within a complex environment (e.g. a
  museum, an archeological site, etc.) and to
  easily identify nearby areas of interest.

18
3D visualization on mobile devices
Virtually reconstructed cultural assets
Ancient buildings, monuments or other cultural
assets are often characterized by the fact that
large and important parts of the original
structure are missing A three-dimensional
virtual model could give the user a realistic
view of the original structure and appearance of
the cultural asset in its entireness.
Example 3D Model of Temple of Isis, Pompei
archeological site - Italy
19
3D visualization on mobile devices
Intuitive 3D exploration of a site
The use of 3D visualization and navigation
capabilities can help the user to better
understand his current location within a complex
environment (e.g. a museum, a large building, an
archeological site, etc.) and to easily identify
nearby areas of interest.
20
3D visualization on mobile devices
Technology

• Embedded C OpenGL ES low-level approach
  fine control on the application, but needs to
  code all aspects of the graphical application
  difficult to import common 3D file formats (e.g.
  VRML, 3DS)
• Java Micro Edition (J2ME) M3G (Mobile 3D
  Graphics) supports both high-level and low-level
  approaches can import M3G files (exported from
  3D Studio Max with M3G plug-in)
• Cortona Pocket PC JavaScript simplest solution
  with VRML support allows viewing and navigation
  of VRML models, but offers limited control on the
  application (can only use JavaScript).

For the development of the 3D mobile HMI, we
propose a solution based on Java language (J2ME),
and the M3G mobile graphics technology
21
3D visualization for CA monitoring
Monitoring tourists visiting Cultural Assets
Position of CAF users, who have Galileo-enabled
mobile terminals, is updated in real-time
CAF users
22
3D visualization for CA monitoring
23
3D visualization for CA monitoring
Technology

• Java OpenInventor (commercial version)
  high-level approach, fast and easy development,
  supports VRML format
• Java3D high-level approach, fast and easy
  development, does not support VRML (needs extra
  libraries)
• C/C and scene-graphs (e.g. OpenSG,
  OpenSceneGraph, Performer) open-source, best for
  performances, supports several 3D file formats
  (VRML, 3DS, )

24
3D visualization for CA monitoring
For the development of the CA monitoring HMI, we
propose a solution based on C OpenSG (Open
Source Scene-Graph), which allows high
flexibility and performances, while supporting
several 3D file formats (VRML, OBJ, )
25
CUSPIS Technical Responsibile Matteo
Pace NEXT-Ingegneria dei Sistemi SpA Responsible
of Research and Development Matteo.Pace_at_next.it
Alessandro Boccalatte Space Applications
Services N.V. Alessandro.boccalatte_at_spaceapplicat
ions.com