Image Fusion

1
Image Fusion

• In Real-time, on a PC

2
Goals

• Interactive display of volume data in 3D
• Allow more than one data set
• Allow fusion of different modalities (e.g.
  CTPET)
• Allow simultaneous display of all data sets

3
Considerations

• Must run on consumer level hardware
• APIs such as OpenGL Volumizer already available
  on High End machines ()
• Must run at interactive frame rates
• Must allow flexibility in visualization of data

4
Requirements

• Allow for
• Arbitrary viewpoints
• Arbitrary slicing to show internal detail
• Flexible transfer functions
• User adjustable for
• Colors
• Transparency
• Arbitrary mixing of multi modal data sets

5
Problems

• More More flexibility, so
• Consumer cards have less than 128 MB of shared
  memory (texture, frame buffer etc)
• PCs have more main memory but it is slow
• Dont have X units working together
• Need processing power, (lots of it)

6
Workarounds

• Limited memory on video card
• Roam around a high resolution data set by
  selecting smaller areas that fit in available
  memory
• Still slow, because when the viewpoint moves new
  data needs to be loaded (from main memory)
• Down sample data to fit in memory
• Loss of detail

7
Workarounds

• Compress textures
• Implementations will allow for a 81 compression
  of textures
• Will quality be affected? How much?
• Will it compromise flexibility? How much?
• Processing power needed
• Use programmability of newer GPUs to leverage
  available high speed memory
• Do transfer functions on the GPU? In parallel?

8
Details

• Allow user to choose blending between the data
  sets
• Assign colors, transparencies and mixing values
• Each data set must have its own transfer
  function(s)
• Registration
• Not handled

9
Rendering

• How?
• Volume data? Volume textures (3D Textures)
• Project onto 2D screen by overlaying slices
• Slices are perpendicular to the screen
• Slices are cut from the volume at fine enough
  intervals and blended together to produce a
  single viewpoint image
• Repeat for different views (rotations/scale) etc

10
Rendering

• More datasets
• Could render image for each data set and overlay
  on top (multipass slow)
• Render once, applying both textures at once
• Limited to 2 textures on most cards
• Use shaders to program the GPU
• Very flexible, 2 textures at once
• - Flexibility?

11
Rendering

• Lighting
• Current rendering mode does not help with
  lighting
• Must be done at a lower level
• Surfaces
• No real surfaces generated, so lighting is hard
  to do, must be done at the voxel level
• Lighting at voxel level, will show appearance of
  surfaces

12
Conclusions

• Limited Memory Large data sets
• Want lighting lack of surfaces
• But we have a programmable GPU to our advantage
• Dataset manipulation in fast video memory
  Faster rendering
• Alternative rendering methods?
• Most are CPU bound

13
References

• http//www.opengl.org
• http//www.sgi.com/software/volumizer/
• http//www.cs.utah.edu/jmk/simian/
• http//www.nvidia.com
• http//www.ati.com
• http//www.cla.sc.edu/psyc/faculty/rorden/render.h
  tml