Virtual Reality Artificial Reality

1
Virtual Reality (Artificial Reality)
2
Definition

• A computer system used to create an artificial
  world in which the user has the impression of
  being in that world and with the ability to
  navigate through the world and manipulate objects
  in the world.
• C. Manetta and R. Blade

3
Motivations

• A display connected to a digital computer gives
  us a chance to gain familiarity with concepts not
  realizable in the physical world. It is a looking
  glass into a mathematical wonderland.
• The ultimate display would, of course, be a room
  within which a computer can control the existence
  of matter.
• - Ivan E. Sutherland

4
Two Requirements of VR

• Immersive
• Interactive

5
Two Related Courses of VR

• Computer Graphics
• Computer Vision

6
Two Ways to construct VR
7
Two Types of VR
Immersive 3D environment seen through a
head-mounted display (HMD). In a completely
immersive system the user has no visual contact
with the physical world. Semi-immersive Most
advanced flight, ship and vehicle simulators are
semi-immersive. The cockpit, bridge, or driving
seat is a physical model, whereas the view of the
world outside is computer-generated (typically
projected).
8
Two Parts of VR

• Augmented Virtuality
• Augmented Reality

9
Evolution of VR
Adventure games, MUD(Multi-Users Dungeon)
Textually described virtual worlds where the user
perceives the virtual environment through mental
images like reading a novel. Desktop 3D
virtual environment graphically displayed on a
desktop computer monitor.
10
Evolution of VR
Projected 3D environment projected onto a
screen. Enables a single user to demonstrate
concepts to a group of people. A CAVE (developed
by University of Illinois), where several screens
are used to surround the user with images, is the
most advanced form of projected VR in use today.
11
VR Technology

• Hardware capable of rendering real-time 3D
  graphics and high-quality stereo sound.
• Input devices to sense user interaction and
  motion.
• Output devices to replace user's sensory input
  from the physical world with computer-generated
  input.
• Software that handles real-time input/output
  processing, rendering, simulation, and access to
  the world database in which the environment is
  defined.

12
Head Mounted Display (HMD)

• Can display either stereo or mono images
• Stereo images (binocular disparity)
• Same image twice (binocular concordance)
• Single image (uniocular)
• May be totally immersive or semi-immersive
  (see-through)
• May include a built-in head-tracker
• May have built-in stereo headphones

13
Each eye sees a different field of view
14
Alternative Displays

• LCS (Liquid Crystal Shutter) glasses
• Display shows left and right images alternately,
  switching at high speed between images
• Stereoscopic image is seen when the display is
  viewed with special glasses
• Typical 'Fishtank VR'
• Particularly good for large audiences in a
  theatre

15
Alternative Displays

• BOOM(Binocular Omni-Orientation Monitor) by
  Fakespace
• Uses a CRT to provide high-resolution display
• It is comfortable to use, since it does not have
  to be worn
• Has fast, accurate, built-in tracking

16
Alternative Displays

• VRD (Virtual Retinal Display)
• Image is projected directly onto the retina (by
  Microvision)

17
Motion Tracking Types

• Mechanical
• Usually a mechanical arm attached to the tracked
  object
• Very accurate, short lag, but restrict movement
• Electromagnetic
• Measures strength of magnetic fields in coils
  attached to objects
• Fast, short lag, but often prone to interference
  
• Limited range

18
Motion Tracking Types

• Optical
• Typically, pulsating LEDs monitored by a camera
  at a fixed position
• Fast, reasonably short lag, but often prone to
  interference caused by ambient lighting
  conditions
• Line of sight problems
• Acoustic
• Use ultrasound waves to measure position and
  orientation
• Slow and often imprecise

19
Interaction Devices

• 3D Mice
• Spaceballs
• Eye tracking
• Video camera and shadows
• Voice recognition
• Biological sensors

20
Interaction Devices
Light Pen
21
Interaction Devices

• Data Gloves

22
Interaction Devices

• Data Gloves
• Hand and gesture tracking
• Enables natural interaction with objects
• Can use hand-signs to execute actions
• Full body suits
• Motion capture

23
Feedbacks

• Motion platforms
• Tactile feedback provides a sense of touch
  through, typically, vibrating nodules or
  expanding air bubbles inside a glove or suit
• Force feedback provides physical constraints
• Exoskeletons
• Joysticks, Hand controllers

24
Sound

• Important to create a sense of atmosphere
• Can greatly enhance feeling of presence
• Can be used to provide valuable depth cues,
  aiding navigation
• Enables the user to perceive events that occur
  outside the immediate field of view
• Audio feedback

25
Software

• Need modeling tools to create objects (AutoCAD,
  3D Studio, etc.)
• Designing objects is time-consuming
• Objects often need to be optimized for VR use

26
Applications

• Architecture
• Design and Prototyping
• Education Conferencing
• Training
• Medical
• Business and Visualization
• Art and Leisure

27
Architecture

• Walkthroughs to evaluate design decisions and
  present designs to customers
• Demonstrate how a planned construction fits into
  the environment in which it is intended to be
  built

28
Design and Prototyping

• Use to create rapid prototypes rather than make
  clay models or full-scale mock-ups
• Simulate assembly lines. For example, to
  evaluate the risk of interference or collision
  between robots, cars, and car parts

29
Education Conferencing

• Education
• Visualize concepts
• Visualize the past ('Virtual Heritage')
• Virtual lectures and classrooms
• Conferencing
• Collaborative work over the Internet
• Virtual work groups
• Virtual conferences

30
Training

• Civilian and military training simulators
• Driving simulators
• Flight simulators
• Ship simulators
• Tank simulators

31
Training

• Train for hazardous or difficult operations
• Nuclear plant maintenance
• Learn to move in zero gravity
• Practice locating and fixing faults in equipment

32
Medical

• Surgery
• Practice performing surgery
• Perform surgery on a remote patient
• Rehabilitation
• Phobia therapy
• Use VR input devices and tele-presence to enable
  handicapped people to do things that would
  otherwise be impossible for them to do
• Enable people to visit/see places that they
  would be otherwise unable to experience
• Use VR to teach new skills in a safe,
  controlled, environment

33
Business and Visualization

• Business
• 3D visualization of complex financial
  information
• Demonstrate customizable products to customers
• Scientific Visualization
• View complex data sets to gain greater insight
  and understanding of structure
• View complex molecular and geological structures

34
Art and Leisure

• Art
• Virtual galleries and museums
• Virtual theatres
• Leisure
• Games
• Sport simulators

35
(No Transcript)
36
HSCT
37
XVS
38
Challenges of VR
39
Hardware Technology

• More realistic graphics and audio (this is also
  a software problem)
• Greater processor power
• Less lag
• Less obstructive input/output devices
• Better display technology

40
Building worlds

• Creating models takes a long time and is a skill
  that needs to be acquired
• We need better software tools to create objects,
  add behavior to objects, and to handle
  interaction
• Optimizing models takes time and synchronizing
  modifications can be difficult (CAD tools to/from
  VR environment builder)

41
Building worlds

• Implementing scripts and interaction is often
  difficult for non-programmers
• Making accurate models of the physical world is
  difficult. Devices such as laser scanners are
  making this easier, but human intervention is
  still a necessary and time consuming part of the
  modeling process
• Can we build knowledge-based systems to help us
  capture the physical world?

42
User Interaction Design

• The user interface for a virtual environment
  needs to be carefully designed to take into
  account the functional requirements of the
  application and the limitations of current
  technology
• How should the user interact with the virtual
  environment?
• How should the computer provide feedback?
• What is the most meaningful way to visualize an
  abstract object/process/structure?
• What does cyberspace (the Internet) look like?
• How do we improve navigational and spatial
  awareness?

43
Human Factors

• There are a number of factors that should be
  taken into consideration when designing the user
  interface of a virtual environment in order to
  minimize physical discomfort for the user
• Latency causes user discomfort (simulation
  sickness, nausea)
• Well design and optimized models reduce latency
  and improve rendering consistency
• Eye strain problems?
• Long-term physical and psychological effects?

44
Distributed Virtual Environment

• How should users communicate?
• How do I know you are real and not an agent?
• Social effects of distributed VR?
• Technological Lag, large numbers of users
  simultaneously, etc.

45
Conclusion

• VR is a powerful user interface technology
• Choosing the best way to visualize information
  is important
• VR enables the user to interact directly with
  information
• VR enables the user to see/experience things in
  new ways
• VR does not have to be immersive to be useful