Augmented Reality

1
Augmented Reality

• Overview and Current Systems

2
Motivation A Brief Scenario
3
Motivation A Brief Scenario

• The Problems
• The surgeon cant look at the patient and the
  screen at the same time.
• Looking at the screen while working on the
  patient isnt ergonomic.
• The screen is 2D whereas the patients body is
  3D.

4
Motivation A Brief Scenario

• The Vision
• The information should be provided where it is
  needed.
• The user should be enabled to access and interact
  with the information ergonomically.
• The information should be in 3 dimensions like
  the applications subject.

5
Roadmap

• Defining Augmented Reality
• A Taxonomy of Reality and Virtuality
• Technologies
• Fields of Application
• Current Systems

6
Defining Augmented Reality

• Combining Real and Virtual
• Interactive in Real Time
• Registered in 3-Dimensions

7
Defining Augmented RealityCombining Real and
Virtual

• Attention
• Combining does not neccesarily mean adding.
• Combining does not only include visual
  information (Navi-Project).

8
Defining Augmented Reality Interactive in Real
Time

• Using AR technology, users can interact with a
  combination of real and virtual objects in a
  natural way.
• In Confluence of Computer Vision and Interactive
  Graphics for Augmented Reality

9
Definining Augmented Reality Registered in
3-Dimensions
Lightning reacts to the position of the tangible
sun.
10
A Taxonomy of Reality and Virtuality

• The Reality-Virtuality Continuum

11
A Taxonomy of Reality and Virtuality
                

• Centricity Exocentric or Egocentric?

12
A Taxonomy of Reality and Virtuality

• Control-Display Congruence
• Question How intuitive is the control?
• In AR When I move the real object to the right
  hand side, in what direction does the virtual one
  go?
• Conclusion
• AR is situated near the real-role of the
  continuum.
• AR uses an egocentric viewpoint in most cases.
• AR tries to maximize the control-display
  congruence.

13
Technologies

• User Interfaces
• Visualization
• Tracking

14
User Interfaces

• Visual
• Acoustic
• Haptic/Tangible

15
User Interfaces Visual

• Gesture Recognition
• Example
• MITs KidsRoom Temporal integration

16
User Interfaces Acustic

• 2 Parts
• Speech Synthesis
• Speech Recognition
• Odyssey 2001s dream has not come true, yet.
• So, Acoustic UIs are combined with other
  modalities.
• Example
• SHEEP

Insert!
17
User Interfaces Haptic/Tangible

• Joystick, 3D-Mouse
• Tangible Objects
• Examples
• CAR, SHEEP and the Studierstubes PIP
• Touch gloves

18
Visualization Ways of Visualizing

• Visualization Devices
• HMDs
• PDAs
• Projection
• Monitors
• 3-Dimensionality
• Separeted displays
• Polarized projection
• Anaglyphic projection
• Time division

19
VisualizationMixing Real and Virtual

• Optical see through
• Video see through
• Chroma keying Mixing with video hardware
• Mixing in the Renderer
• What about occlusion?
• Tracking all real objects
• Depthmaps construction via stereo images
• Special Projection Techniques

20
Visualization Challenges

• Contrast
• Brightness
• Resolution
• Field of View
• Safety

21
Tracking Ways of Tracking

• Optical Tracking
• Markers
• IR-reflecting spheres
• Cards
• Types of optical Tracking
• Inside out
• Outside in

22
Tracking Ways of Tracking

• Time of Flight
• GPS, DGPS, Galileo
• Ultrasound
• Magnetic Field Sensing

• Inertial Tracking
• Gyro Meters
• Accelerometer
• Special Eye Tracking

23
Tracking Challenges

• Static Errors
• Optical distortion
• Improper parameter configuration
• Mechanical misalignment
• Strategies
• Carefullness
• Algorithms for autocalibration

• Dynamic Errors
• Hardware related delays
• Software related delays
• Strategies
• Higher processing speed
• Video stream matching
• Rendering in advance
• Predicting the future

24
Fields of Application

• Annotating the Environment
• Displaying Auxiliary Sensor Information
• Visualizing Artificial Data

25
Annotating the Environment

• Navigation
• Columbias Touring Machine
• Police and Military
• BARS

the success of a military operation in an
urbanized environment depends crucially on being
able to provide navigation and coordination
information to the individual marine level. The
US-Army's Concepts Division of the Marine Corps
Combat Development Command
26
Annotating the Environment

• Repair, Maintenance and Production
• BMW The Intelligent Welding Gun
• Boeing Augmented wiring

27
Displaying Auxiliary Sensor Information

• Medical
• University of North Carolina Ultrasound images
  of a fetus.
• The heARt-Project Minimal invasive heart
  surgery.
• MIT MRI and EEG to support neurosurgery.
• Successfully used

28
Visualizing Artificial Data

• Design
• Studierstubes Virtual Showcase

29
Visualizing Artificial Data

• Scientific Visualization
• Education
• Studierstube 3D-Construct Application

30
Current Systems

• Libraries
• ARToolKit
• Frameworks
• Studierstube
• DWARF

31
Current SystemsDWARF a Sample Framework

• DWARF, a network of distributed dynamically
  cooperating services.
• Distribution
• Heterogenity
• Modularity

32
Current SystemsDWARF a Sample Framework

• Services
• Needs
• Abilities
• Service Descriptions
• Middleware
• Service Manager
• Visual Frontend

33
Current SystemsDWARF a Sample Framework

• Subsystems
• Tracking
• Presentation
• Interaction
• Context
• World Model
• Application

34
The Future

• HMDs in the near future?
• AR in sports and television

35

• ??????? ??????? ?? ???? ????????.
• Vielen Dank für Ihre Aufmerksamkeit!
• Thank you very much for paying attention.

36
Discussion

• Questions?
• Excursus
• CAR, SHEEP and Co. as motion pictures.
• A close look at DWARF The Source Code of a
  Service.
• What does it cost? A sample HMD and a sample
  Tracking System.