DisplayWall Software Architecture

1
DisplayWall Software Architecture

• Yuqun Chen, Grant Wallace, Kai Li

2
Motivation

• Weve achieved high resolution
• Next stage
• multiple active surfaces
• many types of inputs
• new ways of interacting with the surfaces
• new ways to visualize information
• Stanfords Interactive Space

3
New Application Framework

• aware of multiple surfaces
• aware of multiple inputs
• aware of reliability
• allows dynamic grouping
• aware of multiple platforms (PDA, laptop, PC)

4
Future Applications

• Collaborative Engineering
• 3D models
• 2D blue prints/maps
• video conferencing
• writing workspace (documents, code)
• multiple simultaneous user inputs
• different types of devices (laptop, PDA, etc.)
• many apps work cooperatively

5
Orthogonal Design Spaces

• Multiple Applications
• Multiple Inputs
• Multiple Displays

6
Desired Properties

• Applications
• multiple apps cooperate
• take multiple inputs
• scalable
• Input
• route many inputs to many apps
• Display/System
• multiple surface management
• re-configurable and scalable
• fault tolerance (timely recovery)
• hardware/OS platform independent

7
Application Space

• Synchronization and communication
• high-level events among multiple apps
• large-scale data exchange
• Select among multiple input events
• system support for multiplexing inputs
• Application-level adaptability
• deal with join/leave on the fly
• new site joins a video conferencing
• bring the new apps states up to date

8
Application Scalability

• Scalable to actual screen size aspect ratio
• e.g., imageviewer
• Scalable with number of physical nodes
• walk-through
• load-balancing

9
Inputs Space

• Multiple physical inputs gt one virtual inputs
• (gyro-mouse, hand position) gt virtual mouse
• (PDA, keyboard, keyboard) gt virtual keyboard
• Multiple virtual inputs
• flight simulator captain and co-pilot

10
Physical to Virtual Input Routing
app
virtual keyboard 1
PC keyboard
virtual keyboard 2
wireless keyboard
remote keyboard
virtual mouse 1
mouse
virtual mouse 2
app
gyro
virtual speech
virtual body
speech recog
11
Display System Space

• Surface (virtual screen) management
• which virtual screen to route application output
• virtual screen space configuration/grouping
• Per-surface reconfiguration
• assign physical screens to a surface
• surface resolution
• Fault-tolerance
• should not worry about connections
• state recovery
• Hardware/OS independent gt LINUX

12
Common Properties

• Communication abstraction is the key
• Sequencing of data/events
• One-to-many multicast
• high-level app events, graphics primitives
• Many-to-one point-to-point
• input devices
• Avoid explicit connection management
• Safe storage for fault tolerance

13
Communication Choices 1

• MPI-style message passing
• advantages
• people are familiar with messages
• virtual nodes
• disadvantages
• mutual knowledge between send and receiver
• even though physical nodes can change underneath
• lack of a concept of a shared space

14
Communication Choice 2

• Shared (Virtual) Memory
• advantages
• single shared memory space
• at API level, no knowledge about the each others
• disadvantages
• memory management from many nodes
• platform dependent (big vs. small endian)
• coherence may come at a high cost

15
Communication Choices 3

• Linda-like tuple space (T Space)
• a named storage space independent of any nodes
• in(name, data), out(name)-gtdata, read(name)-gtdata
• messages without direct sender-receiver knowledge
• anybody can read from the space

16
Virtual Frame Buffer/Pixelmap

• Well-defined mappings between virtual areas and
  the physical memory/frame buffer
• Coordinated access to the virtual space
• everybody reads, barrier, then write
• Multiple virtual frames to switch between

17
Use of Virtual Pixelmap
declare virtual map A and B optional global
frame update A gt B barrier() every node read
a region from A every node write to
B barrier() swap A and B barrier()
18
X Server vs. VDD

• X Window System
• virtual screen management
• multiple sources
• multiple platforms
• multiple inputs ??
• VDD (partially developed)
• Windows Apps
• smaller API

19
Software Architecture
app
app
App synchronization
XML
Virtual inputs
rendering
inputs
OpenGL
local
VDD
X
Virtual FB
mouse
keyboard
hand
System communication