Title: DGL / 3DITs presentation
1DGL / 3DITs presentation
2Schedule
- 900-1000 - DGL Overview and programming
guide - 1000-1010 - Break
- 1010-1030 - Discussion on potential usages of
DGL - with your software projects
- 1030-1100 - Overview of available VR hardware
- 1100-1130 - Introduction to 3D Interaction
techniques - 1130-1140 - Break
- 1140-1200 - Discussion and final wrap-up
- 1200 - Lunch
3DGL
- Capabilities
- Case studies
- Output
- OpenGL, Open SceneGraph, Coin, VTK
- Input
- Mouse, keyboard
- Communication with other processes
4Capabilities of DGL
- Extends DIVERSEs reach into OpenGL based
rendering and toolkits built to render to OpenGL - Primary thrust is to provide VR capabilities to
Open Scene Graph, while allowing more flexibility
than DPF did with OpenGL Performer - Improves upon some parts of the DPF API
5Case study 1
- Road crossing experiment using VRML
- Elderly citizens crossing the street in a city
- Controlled environment with no unreasonable risk
for the participant - VRML animation for cars / streetlights
- Coin used to render the project
- Coinfly example was all it took to run the
project
6Case study 2
- Benefits of immersion study
- 3D mathematical equations
- Different levels of immersion (walls on / off)
- Sometimes head tracker used / sometimes not
- Used to help understand 3D math and what it looks
like in its natural surroundings
7Case study 3
- AMADEUS Underground mining visualization tool
- Designed to view statistically generated
fractures to simulate what the rock looks like
before it is tunneled through - Somewhat simple novel computer graphics algorithm
to cut different types of the tunnels out of the
rock and be able to see them in true 3D
8Screenshot
9Result of graphics algorithm
10DGL Output capabilities
- Based around a draw callback that is added to a
dtkAugment - Allows for OpenGL to be drawn before and after
navigation and has an overlay capability to
ensure that it is drawn after all the other
OpenGL - How does this model work with Open SceneGraph /
Coin / VTK?
11DGL Augment Interface
- Same exact process as making your own dtkAugment
- Derive off the draw callback
- Pass in the type of dglAugment you want it to be
(i.e. where it is drawn) - Load the DSO into your favorite application
- Done
12Porting existing programs to DGL
- DGL is designed to ease porting old OpenGL
programs. - There are C based wrappers that offer the same
functionality as the C interface - dglDisplayCallback
- dglInitGLCallback
- dglPostframeCallback
- dglSet/GetData
13Problems with porting old GL programs
- Timers arent implemented in DGL
- Input probably is going to have to be redone
- Depending on the application, it may have to be
re-architected a bit to handle cluster
development - May also have to worry about multi-context issues
14Open Scene Graph usage
- Step 1 Develop OSG application
- Step 2 Initialize DOSG (DOSGinit())
- Step 3 Insert top level OSG root node into
DOSGgetWorld()-gtaddChild(here) - Done
15Open Scene Graph
- DGL handles structuring Open Scene Graphs
internals so that all you have to do is add a
child to a node in a scenegraph that is drawn by
DGL - Structure of scenegraph is
- scene
- nav ether
- world
16Coin
- Very similar to osg, except there is no nav node
- scene
- world ether
- DCoininit Make the system
- DCoingetWorld()-gtaddChild(here)
- DCoinloadFile(model.iv/wrl)
-
17VTK
- 4 step process
- Create a render window
- (DVtkRenderWindowNew())
- Create a renderer (DVtkRendererNew())
- Add the renderer to the window (AddRenderer(pointe
r)) - Add your actors to the scene
- Flip the scene to OpenGL coordinates if desired
(DVtkRenderer-gtRotateSceneX(90))
18Input for DGL
- Now that we have graphical development taken care
of, we need to get how the user develops
applications! - Same as DPF for wand / joystick / buttons / head
tracker (dtkSharedMemory) - Has a different mouse / keyboard interface due to
a different input system (Producer)
19Mouse input
- DGLMouse and DGLMouseButton
- DGLMouse allows for setting the mouse position,
and reading the mouses current position or the
queued position of the mouse (setMouse(x,y)
readMouse(float x, float y) getMouse(float x,
float y)
20Mouse Input
- DGLMouseButton Tells you what happened to the
mouse button, and the position of the mouse when
the button action happened - Based on top of shared memory
- Concept of Actions, state for each mouse button
- int readMouseButton(x,y, action)
- Returns 0 if it doesnt have data, otherwise it
tells you the position it was changed at and what
the change is - Actions are press / release / unknown
- Allows for the ability to fake mouse events if
there is a need
21Keyboard Input
- It uses the same actions concept as mouse input
did - DGLKeyboardgetState(DGLKeyboardgetKeyCharacter
(a)) - Pass an action pointer (to be written to) into
readKeyCharacter and it will return the action /
character that was pressed - Do have to switch between char and KeyCharacter
due to Producer syntax
22The raw deal
- If you dont like the helper classes created then
there is always shared memory. - It is what the system is based off of.
- dgl/mouse dgl/keyboard dgl/mousebutton are the
segments - The structure is usually an integer for the data
and an integer for the action
23Nuts and bolts
- Building dgl programs is the same as with dpf
programs, dgl-config is your friend - dgl-config --include and --libs
- dgl-config --osg-include --osg-libs
- dgl-config --coin-include --coin-libs
- dgl-config --vtk-include --vtk-libs
- Example makefiles in the examples directory on
how they are used
24Design strategies for DGL programs
- Decide the type program that you are developing
OSG , OpenGL, etc - Decide the level of interactivity
- Often having a DSO that reads / displays data is
the best approach because you can reuse DGL shell
programs (i.e. dosg-fly) - Full applications give the most flexibility and
opportunity
25DGL design strategies
- Often I break the problem into these specific
parts - Reading input data
- Transforming input data
- Displaying data
- Interacting with data
- Often need to be careful about structure of data
storage / accessibility because these decisions
often interactivity
26DGL design strategies
- Often interactivity places the most demands on
applications - The earlier in the development process that you
consider it, usually the best - Aim for several quick iterations that can test
different features / abilities - Get users involved as soon as possible in the
development cycle
27Questions / Comments
- Capabilities?
- Design decisions?
- Application design?
- Building applications?
2810 minute Break
29VR Hardware
- Often what you do in VE creation is heavily
influenced by the input devices you have
available - Very large range of devices, some very well
supported, some not so much. - In CAVE environments usually light weight one
handed devices are preferable - Often have to consider how equipment will affect
others in the CAVE environment (for demos)
30Outline for VR hardware
- Text entry
- Positional trackers
- Pointing devices (other than wands)
- Niche areas
- Haptics
31One handed text entry
A one handed text entry method through chording
Press multiple buttons to obtain different
letters of the alphabet Advantages Only needs
16 keys to operate Can be used as a cheap input
device that has 16 different buttons Can be
combined with a velcrod tracker Disadvantages R
equires training Will not be as fast as regular
keyboard text entry
32Frog pad
33Software text entry
On screen keyboards are often used for text
entry. Basically make a rectangle with all of
the different keys you want. Stick a ray on
the end of the wand, put the ray into the letter
you want. Press a button on the wand to select
the letter.
34TULIP
- Tulip menus / text entry
- Imagine wearing pinch gloves and having a
floating keyboard in front of you in the world - Reach out and pinch the letter in front of you
- 11 -12 WPM for expert users
35Optical tracking
Different way of tracking Put markers (little
balls) on a set of glasses or a hat. The camera
then recognizes them and generates a 3D position
based on the Info. Advantages Cheaper than
Intersense Can easily make any object
trackable Disadvantages Requires users to wear
something Marker has to be visible to camera
36Magnetic tracking
- Tracking through electromagnetic fields
- Doesnt work well with metal objects in the
environment - Higher latency than with Intersense trackers
- More portable though
37Acoustical tracking
Acoustical tracking Low latency Not sensitive to
metal in the environment Not cheap or portable
(has to be calibrated for each environment. Very
good for fixed setups such as CAVEs
38Computer vision tracking
- Through the use of markers and a standard webcam
you can get 6DOF tracking - Software process
- Useful and cheap for passive haptics
39Gyroscopic mice
The gyroscopic mice provide 2D in air movement
and generally do not require drivers, just plug
in and the OS / USB does the rest (even on
Linux) Advantages -Allows for a cheaper (70)
more restricted wand movement (only desktop
X,Y) -Is possible to have multiple mice
influencing the VE at the same time -Scenario
(All data on front wall, multiple clickable
options). Disadvantages -Does not have
absolute tracking -Has to be used with care, a
flick of the wrist during a demo could be
problematic
40Pinch Gloves
Pinch gloves allow for fingers to be
pinched together to produce different
combinations for input into the VE. Combine with
a 6DOF tracker to know where the users hands
are. Somewhat cheap 2,000 device Advantage
More natural style of interaction than with a
wand Disadvantage Does not track finger movement
41Data gloves
- Allows for true finger positioning to be reported
to the VE - Can do sign language type gestures
- Has calibration issues
- More expensive
- Doesnt always work right
- Very expensive
42Shape tape
- A piece of material that can be shaped and have
this shape reported to the computer - Useful for art, engineering (measuring / curve
placement) - Can be used for interfaces when combined with
trackers - Also can be used to imitate snakes
43Omni Directional Treadmills
Imagine being able to walk around in an
environment naturally and not have to worry about
the running into the CAVE walls. Advantages Natu
ral movement in a VE Perfect for types of
training Disadvantages Expensive Noisy May be
harder than regular walking and may not react
well to running
44Haptic devices
- Provide force feedback
- Sometimes in a pen form
- Sometimes a skeleton framework over a glove
- Often easier to use passive haptics (pit
experiment)
45Discussion on Input devices
- Would adding a different input device be useful
for your application? - Make for interesting demos
- Can even go so far as hang-gliders with fans /
tilt recognition or walking in place harnesses /
floor tiles that move around - Questions / Comments
463DIT
- Interacting with VEs makes them not only more
usable, but more useful - Helps increase the sense of presence
- Many different types that provide very
interesting abilities.
473DIT Input
- Require input devices to work
- Does not require a 3D input device to do a 3D
task (mouse / virtual sphere) - Input devices do enable / limit different types
of input though
483DIT Realism
- Natural versus Magic techniques
- Think of having a virtual cup of water on the
table in front of you. - Reach out and grab it
- Now imagine it being 15 feet in front of you
- Both are possible in 3DITs
- Can be mixed to have some of both
493D Selection and Manipulation
- Tasks are mainly composed of selection,
positioning, and rotation - Selection is composed of indicating which object
is desired, confirming that it is the object that
is desired to be selected, and then providing
feedback to the user. - Can be done through a couple of different methods
(ego / exo centric)
50Different types of selection techniques
- Simple infinite (or finite) ray to select an item
- Two handed pointing
- Flashlight / aperture addition to raycasting
- Flexible ray casting
- Image plane techniques
- Fishing reel technique
51Selection and Manipulation
- Once you select an object, what do you normally
want to do with it? Manipulate it. - Think back on the water cup example
- Imagine being able to stick your hand out 50 feet
and grab objects - Take a hand tracker, and use a non-linear scaling
function on the object it is attached to
52World in Minature
- Imagine being in a room and then having a top
down view that is slightly bigger than your hand. - Combine with a tracked tablet and can be used
with a wand to pick and move objects - Can be modified to provide an overview of the
world on the floor of a cave and then warping
based on that - Can be used to replicate maps and then travel
based on that
53Voodoo Dolls
- Select an object in the world
- Create a miniature copy of the object in front of
you - Use tracked pinch gloves scale, rotate, and
position the object
54Travel techniques
- Exploration, search, maneuvering
- Scaling movement from slow to fast to cover long
distances - Velocity
- Location guides
- Reset to start usually essential
- Users get lost in VEs all the time, plan for it
55Different types of travel techniques
- Point and fly based travel
- Gaze directed
- FPS style
- Button versus joystick
- Camera in hand technique
- Maps to place self in world
- Set of guided points
56System Control
- There are whole hosts of options that can be
configured with VEs - How do you do this inside of a VE?
- 2D in 3D (Ring / XWand / VEWL type interactions /
Pen Tablet) - 3D menus (Tulip, cone trees, Command and control
cube)
57Different ways to combinetechniques
- 3DITs are often best used when combining with
other 3DITs - Example HOMER
- Select an object with ray casting, then
manipulate with GoGo. Allows for selecting a far
away object and moving it even farther. - Flavors
- Adding small nuances to improve usability of
technique. I.e. having the ray in ray casting
snap to the nearest object in the world. - Adding scaling and scrolling to the WIM.
58Implementation strategies
- Where possible 3DITs should be created as DIVERSE
DSOs - Plugs into different applications
- Static interfaces are your friend
- Need to get a handle to the world (easier if the
world is loaded as a DSO)
59Implementation strategies
- Make sure they can be turned on and off
- Think about creating a master 3DIT DSO that turns
sub parts on and off - Dont be afraid to try different options
- You can only do so much with a simulator on the
desktop, you need to use the real device in the
CAVE to make sure that it works well
60DGLMessage structure
- Think of having the visualization component as
only interacting through messages sent to it - You need a protocol for 3DITs and the Viz core to
talk to each other - One way is to have messages designed to be sent
to a specific class to do something particular.
61Structure of message usage
User
Action
3DIT
Message
Message Center
Delivers message
Vis1-3 register with the Message Center at
starting time
Output shown to user
Vis1
Vis2
Vis3
62Message wrapup
- Advantage Works through shared memory so it can
be used on cluster systems rather easily - Disadvantages Not meant for large amounts of
data transfer, mainly just directions on what to
do. Takes planning, usage of multiple
inheritance, and has overhead in implementation.
Can pay off in the long term though.