Visual Systems in Simulation Applications

1
Visual Systems in Simulation Applications

• A Brief History and Primer

Michael Coleman AVT Simulation
2
Michael Coleman - Vitae

• Visual Systems Engineer, AVT Simulation
• 14 years of simulation industry experience
• AVT
• Raydon
• Evans and Sutherland
• CSC
• Southwest Research Institute
• B. S. Computer Science, Trinity University, 1996
• Graduate Certificate, Project Engineering, UCF,
  2005
• M.S., Modeling and Simulation, University of
  Central Florida, 2008

3
Outline

• History
• Visual System Components
• Displays
• Image Generators
• Databases

4
History of Visual Systems in Simulation
5
Dawn of Simulation

• 1929 Link Aviation Trainer
• Instrument flying in adverse weather
• No visual system
• Later placed in painted rooms

6
Early Visual Systems

• 1950s-1960s
• Closed-circuit video and film
• Model boards

7
Computer Graphics in Simulation

• 1964 - GE system for Apollo program
• 1972 McDonnell Douglas VITAL II certified for
  training commercial airline night landings
• Visual simulation technology largely advanced by
  commercial aviation in the 1970s

8
Commercial Simulation Technology

• 1970s - Raster/calligraphic systems
• Collimated mirror displays and motion platforms

9
Military Simulation Technology

• 1980s saw technology being advanced by military
  market
• Raster-based displays
• Domes and head-mounted systems

10
Visual Systems Today

• Technology driven by consumer entertainment
  industry
• PC-based graphics
• Off-the-shelf projection technologies

11
Visual Systems Concepts
12
Visual System Components

• Display
• Image Generator
• Database

13
Displays
14
Display

• For flight simulation, display requirements drive
  visual system design
• Military target detection requirements
• Commercial FAA guidelines
• Display may include night vision and infrared
  devices as well as out-the-window (OTW)

15
OTW Display Components

• Dome
• Faceted dome
• Mirror / back projection screen
• Head-mounted

16
Display Projection Technology

• In the past, simulation requirements advanced
  projector technology (CRT, calligraphic lights)
• Technology now driven by entertainment industry
  (HDTV, digital projection, LCoS)

17
Sensor Displays

• Forward-Looking InfraRed (FLIR)
• Auto-acquisition modes
• Auto-tracking modes
• Night Vision Goggles (NVG)
• Stimulated (actual goggles)
• Simulated (dedicated IG channel/computer)

18
A Few Display Parameters

• Field of View
• Resolution
• Luminance
• Contrast
• Collimation
• Screen Characteristics

Black, 2003
19
Field of View

• Volume subtended by light reaching the eyepoint
• Physical FOV
• Orientation of eyepoint relative to display
  device (monitor, dome surface, head-mounted
  display)
• Measured by theodolite

20
Viewing Volume
21
Viewing Volume

• The viewing volume frustum captures everything
  drawn by the image generator between near
  clipping plane (screen surface) and far clipping
  plane (horizon)
• Critical that the viewing volume matches the
  physical field of view in simulation and training
  applications
• Minification objects appear smaller than actual
  size
• Magnification
• FOV not overly critical in gaming industry

22
Measurement of Resolution

• Units Arc-minutes per optical line pair
  (arcmin/OLP)
• Ability to separate two or more small objects
• Not the same as video format or line standard
  (720p, 1080i, etc.)
• Determination of number of lines that can be seen
  given a number of pixels (considering Kell
  factor, MTF)

23
Johnsons Criteria

• Industry standard for determining resolution
  requirements
• Number of OLP required for
• Detection
• Orientation
• Recognition
• Identification

24
Johnsons Criteria
25
Resolution Applications

• Once resolution requirements (Johnsons criteria)
  and FOV requirements are known, one can determine
• Number of lines to be displayed
• Number of lines to be drawn by IG
• Number / type of display required
• Number / type of IG required

26
Resolution Example
27
Resolution Example
28
Resolution Theory and Practice

• Eye-limiting resolution 1 arcmin per OLP
• Not practical for entire dome (12 up to 40
  diameter) display
• Number of projectors (cost, maintenance)
• Number of IG channels

29
Resolution Compromises

• Overlay/target projectors
• Area-of-interest
• Head-tracked AOI
• IG/Database approaches

30
Image Generators (IGs)
31
Image Generators

• The graphics computers
• Also provide for interaction of vehicles and
  people with the virtual world (collisions, etc.)

32
IG Capabilities

• Mission Functions
• Height-above-terrain (HAT)
• Collision detection (terrain / objects)
• Line-of-Sight
• Environment
• Time-of-Day
• Weather
• Maritime
• Sensor Simulation

33
Sensor Simulation
34
IG Architecture

• Traditionally, proprietary software required to
  exploit proprietary hardware
• Today, runtime / scene graph packages run on
  any suitably equipped PC
• Linux, Windows
• OpenGL, Direct3D APIs
• Similar to (if not) game engines

35
IG Database Rendering

• Traditional Image Generators
• 3,000 triangles or else
• Overload management to guarantee frame rate
• Today, PC graphics cards have different limits
• Millions of polys per 60 Hz frame (estimated)
• Textures, internal structure important
• Limited overload management

36
Beyond the standard PC

• 40 PC-IGs per trainer not uncommon
• Display/Monitor synchronization (genlock)
• Backward compatibility with old legacy systems

37
Traditional IG (circa 1998)
38
PC-IG (circa 2005)
39
Databases
40
Databases

• Virtual representation of real-world (or
  geotypical) locations
• Terrain
• Culture
• Texture
• Models
• Correlated databases for sensors and other
  systems

41
Database Gaming Areas

• Flight simulation databases hundreds of
  geocells (square degrees) to whole-earth
• Ground vehicle simulation hundreds of
  kilometers on a side
• PC, console games hundreds of meters

42
Database Engineering

• One size does not fit all
• IG limitations restrict terrain and culture
  fidelity, number of models
• Flight simulation databases typically
  texture-based (satellite imagery)
• Ground warfare databases typically culture-based
• As much an art as it is a science

43
(No Transcript)
44
Database Terrain

• Much data now freely available
• Before PC graphics, terrain triangle count was
  critical

45
Database Culture

• From 2-D data
• Extraction from imagery
• Cut into terrain and/or placed atop imagery

46
Database Imagery

• One meter per pixel (or better) for areas of
  interest
• Care must be taken in using imagery from
  different sources (cloud cover, time of year,
  etc.)

47
Database Models

• Vehicles and people
• Buildings on terrain
• Some game industry tools used

48
Correlated Databases

• Other versions of the database for other
  computers in the simulation
• Correlated databases should be generated
  simultaneously with the visual database
• Correlation issues must be considered when
  designing the visual database

49
Correlated Databases

• Visual Database vs. Radar Database

Spuhl, 2003
50
Industry Trends

• Vertical consolidation of visual system and
  component providers by prime contractors
• FlightSafety buys Glass Mountain Optics (Jan.
  2009)
• Rockwell Collins buys ES (2006), SEOS (2008)
• CAE forms Presagis from Multigen-Paradigm, Terrex
  and others (2007)

51
Industry Trends

• Databases become major cost components due to
  advent of PC graphics and digital projection
• Databases remain labor-intensive, but re-use
  efforts gaining
• Increased use of technology from entertainment
  industry
• Increase in application of serious games to
  training tasks

52
Simulation Industry

• Prime Contractors
• Boeing
• Lockheed Martin
• L-3 Link
• Rockwell Collins
• FlightSafety
• Visual Systems providers
• Aechelon (IG, databases)
• Quantum3D (IG, databases)
• Barco (projectors, displays)
• VDC (projectors, displays)

53
Educational Paths in Simulation

• University of Utah
• University of Illinois, Urbana-Champaign
• Texas AM University
• Old Dominion University (VA)
• University of Central Florida

54
Educational Paths to Simulation

• Display Systems
• Mechanical Engineering
• Physics/Optics
• Electrical Engineering
• Image Generation
• Electrical Engineering
• Computer Science
• Database
• Computer Science
• Geography/Geographic Information Systems (GIS)
• Graphic Arts
• Digital Media

55
Discussion
56
Notes

• Adapted from presentation to IDS 5717C, UCF,
  March 2006
• References include those consulted for March 2006
  presentation

57
References

• http//www.link.com/history.html
• http//accad.osu.edu/waynec/history/lesson13.html
  
• Weinberg, R. Computer Graphics in Support of
  Space Shuttle Simulation. SIGGRAPH, August
  1978, p. 83

58
References

• Visual Systems Technology Seminar, Evans and
  Sutherland, October 2002
• http//homepage.ntlworld.com/bleep/SimHist9.html
• http//www.aechelon.com/media/images_pcnova2.html
• http//www.sgbent.com/MR1-224.htm

59
References

• Black, S. Fundamentals of Display Systems for
  Visual Simulation. IMAGE 2003 tutorial
• http//www.sogitec.fr/en/solutions/projecteurs_cib
  le.htm
• Introduction to the Hardware Graphics Pipeline,
  http//download.nvidia.com/developer/presentations
  /2004/Eurographics/EG_04_IntroductionToGPU.pdf

60
References

• http//www.caci.com/mtl/tools/practiss.shtml
• http//www.peostri.army.mil/PM-CATT/CCTT_Graphics.
  jsp
• http//www.aechelon.com/products/databases/swus/ve
  gas.html

61
References

• http//www.terrex.com/groundpics.htm
• Spuhl, K. Sensor Simulation Fundamentals
  Correlation Considerations When Simulating Sensor
  and Visual Imagery. IMAGE 2003 tutorial
• http//gis.washington.edu/esrm250/lessons/create_f
  _layers/exercise/index.htmlcreate_line_theme

62
References

• http//www.terrex.com/aterrain.htm
• http//www.spaceimaging.com/products/ikonos/index.
  htm
• http//www.multigen.com/products/database/creator/
  more.shtml

63
References

• http//www.facets3dmodels.com/
• http//www.terrex.com/saf.htm
• http//www.link.com/simusphere.html
• http//vdcds.com/products/calligraphics.html
• http//news.sel.sony.com/en/press_room/b2b/broadca
  st_production/display_systems/release/8816.html
• http//www.leica-geosystems.com/metrology/en/ndef/
  lgs_788.htm
• http//www.lighthouse3d.com/opengl/viewfrustum/ind
  ex.php?gaplanes