Advanced Graphics Part 3

1
Advanced Graphics Part 3

• What now?
• So far
• Revised OpenGL
• Discussed performance optimisation
• Now the OpenGL photorealism/special effects
  toolkit.
• Some revision some new material

2
Reference

• David Blythe
• Advanced OpenGL programming
• One of the SIGGRAPH lecture courses.

3
The advanced OpenGL toolkit

• Most of the OpenGL special effects are based on
  five basic ideas or combinations of them.
• Textures (incl multitextures)
• Blending (using alpha channel)
• Accumulation buffer (adding images together)
• Stencil buffer ("the cookie cutter")
• Fog/depth cues (colours affected by distance from
  viewer)

4
Plan ...

• Look at each of these individually, then look at
  how they can be combined to do special effects,
  like shadows, reflection, caustics, etc.

5
Textures

• Staple of current polygonal graphics
  architectures.
• Why? Allows you to add detail easily, without
  additional geometry.
• Will quickly revise some aspects.
• Quick demos Nate's texture, VRML demos.

6
Texture use

• OpenGL supports 1D, 2D, 3D textures.
• Most of the time, textures are images, not
  procedural. OpenGL supports only images.
• Texel is a pixel in a texture image. Often used
  to avoid confusion with pixel. We'll be talking
  about texels mapping to pixels, so this is
  useful.

7
Using textures

• Each vertex in a polygon is defined as having
  texture coordinates.
• Texture is stretched (i.e mapped) onto each
  polygon.
• Texturing is one thing that really benefits from
  hardware acceleration.

8
Steps in using textures

• Creating the texture
• Indicate how texture is applied
• Enable texture mapping
• Draw scene with both glVertex and glTexCoord
  calls.

9
Creating textures

• Messy!
• Involves loading stuff up into memory and then
  telling system to accept it as texture.

10
glTexImage2D()

• One ugly function
• glTexImage2D(target, level, internalFormat,
  width, height, border, format, type, data)
• target ?
• level for mipmaps more later
• internal format how OpenGL stores the data
  internally.
• width height obvious, but note must be of
  form 2m 2b where b is ...

11
glTexImage2D cont'd

• ... border. Must be either 0 or 1. Specify
  borders on side? OpenGL must support at least
  64x64, but may support more.
• format type how is image data stored in
  memory?
• Finally, the data.

12
Notes on format

• Textures don't have to be RGB.
• Can be RGB, RGBA, LUMINANCE or ALPHA or
  INTENSITY.
• What's the "A" in RGBA? What's alpha? To do with
  blending. Talk about in a little while

13
What if image is not a power of 2?

• Use gluScaleImage() to correct.
• Note Textures can be, say 64x32.

14
Funky stuff

• Can read current frame buffer directly for
  texture map
• glCopyTexImage2D(target, level, internalFormat,
  x, y, width, height, border).
• Can be useful sometimes ... many special effects
  use this as a hack.

15
Repetition and Clamping

• Can either say want lots of repetitions or just
  one copy.
• Can control separately for each texture dimension
  (e.g. can repeat vertically, but not
  horizontally).
• How to set glTexParameteri(GL_TEXTURE_2D,
  GL_WRAP_ST, GL_REPEAT GL_CLAMP).

16
What's with the target parameter?

• Target is used for optimisation.
• Why? Graphics card only has certain amount of
  memory.
• Remember right place at right time stuff?
• target has two possible values GL_TEXTURE_2D, or
  GL_PROXY_TEXTURE_2D.
• Proxy is to make enquiries about space on
  graphics card, efficiency, etc.

17
1D and 3D textures

• 1D textures useful, e.g. paintbrush or something.
  
• 3D textures often arise in medical data (e.g. CAT
  scan, MRI etc). Examples of volume rendering.
• HUGE amounts of data.

18
Different texture modes

• Replace Overwrite previous values.
• Modulate Modify existing colour using textures
  (eg for lighting)
• Decal Like a sticker. Anyone built model cars?
  Attach sticker to outside, but transparent.
• Blended Mixed in some way.
• Example Nates texture.

19
Different modes

• How to think of them existing value of pixel is
  f and texture is t, c is the current texture
  colour
• Then replace is C Ct, A At (ie only texture
  matters).
• Modulate is C Cf.Ct, A Af.At
• Decal is CCf.(1-At)Ct.At, AAf
• Blend is CCf.(1-Ct)Cc.Ct A Af.At

20
Uses of different mode

• Replace In 2D.
• Modulate Typical for use with lighted polygons.
  Most commonly, polygon used is white.
• Decal Insignias etc.
• Blend Not very frequently used, but can be used
  to mix in a "background colour". Kind of like
  "modulate".

21
Enable texture mapping

• glEnable(GL_TEXTURE_D)

22
Supplying coordinates

• glTexCoord1234sifdv(coordinate).
• 4? Yes, you can specify coordinates in homogenous
  form as well. (s,t,r,q).
• Options Repeating vs clamped.
• Can do this independently for each texture.
• glTexParameter(target, pname, param)
• target GL_TEXTURE_D, pname, value).

23
Automatic coord generation

• Can use OpenGL to work out texture coords for
  you.
• But quite complicated and less efficient.
• How?
• You supply a linear equation for each texture
  coord, expressed in terms of a point's position
  coords of the form
• (p1,p2,p3,p4) depend on application.

24
Texture coord generation

• Can specify in either object coordinates (i.e.
  when glVertex() gets called), or eye coordinates.
  
• 99 per cent of time, you want object
  (pre-transform) coordinates.
• When would you need eye coordinates? Some effects
  depend on eye position, e.g. some textures.

25
The OpenGL code

• glTexGenifd(coord, pname, param)
• coord is GL_S, GL_T, GL_R or GL_Q
• pname is GL_TEXTURE_GEN_MODE then either
  GL_OBJECT_LINEAR, GL_EYE_LINEAR or GL_SPHERE_MAP
• pname is GL_OBJECT_PLANE or GL_EYE_PLANE, then
  the next parameter is the vector p1, p2, p3, p4

26
Demo

• Have a look at texgen.c

27
So much to textures ...

• Still to do
• Texture filtering
• Mipmaps
• Multitexturing
• Sphere maps/Environment Maps
• Specular reflection and textures
• Light maps
• Bump maps

28
Texture management

• Use texture objects.
• Can speed things up.
• Usage very similar to display lists.

29
Initialisation

• Get some texture ids
• glGenTextures(n, textureids)
• n is number of textures you want, textureids is
  an array of ints identifying the textures.

30
Usage

• Usage is a little odd. Think of current texture
  as part of the context.
• glBind(type, textureid) sets the current texture.
• Any definitions affect the current textureid. So
  to set up call glTexParameteri() and
  glTexImage2D().
• To change current texture, call glBind on another
  texture
• Look at texbind.c

31
Texture usage and hardware

• Textures can live in memory or on graphics card.
• If in memory have to be copied onto card whenever
  they need to be used. SLOW!
• Why not keep all textures on card? Because card
  has limited memory.
• Resident texture lives on graphics card.
• glAreTexturesResident(n, textureids,
  residentstates) can be used to work

32
But I want CONTROL!

• Not much use if you don't have control.
• First of all, glDeleteTextures(n, textureids) is
  useful.
• glPrioritizeTexures(n, textureids, priorities)
• priority 1 gotta be on the card, 0 who cares?
• LRU algorithms are sometimes used if textures
  have equal priority.

33
Filtering

• General problem Texels and pixels are not going
  to be the same size.
• Three possibilities
• Magnification of texture one texel maps to many
  pixels.
• Minification of texture many texels map to one
  pixel.
• Exact match Yeah right!

34
Magnification vs Minification
35
Solving the magnification problem

• Magnification leads to pixelation effects ... we
  can see the texels on the screen.
• Ugly!
• How to fix?
• Linearly interpolate Each intensity value is the
  centre of a pixel and then take a weighted
  average of surrounding four pixels.
• Called bilinear filtering.

36
Bilinear filtering (mag)
Each box is one texel

• Interpolate along AB to get intensity at E.
• Interpolate along DC to get intensity at F.
• Interpolate along EF to get intensity at G.

B
E
A
G
Centre of pixel maps to this point in texture
space
C
D
F
Texture Space (zoomed in to pixel level)
Texture space
37
Comparison

• On previous slide, if using point sampling
  (GL_NEAREST), then it will be coloured black.
• If using bilinear filtering, it is a weighted sum
  of the four surrounding pixels, depending on
  distance.
• In this case, it would be coloured grey.

38
What about minification?

• Ok, linear interpolation fixes problem with close
  up, but what about far away?
• Can also use linear interpolation, but what use
  is that?
• Point of using linear interpolation was to
  "smooth" edges and pixels.
• But real problem with min is the opposite.
• Ideal solution average values of pixels covered.
• Obviously not feasible

39
MIPMaps

• A precomputation approach
• Say we have a 64x64 texture, then we calculate
  32x32, 16x16, 8x8, 4x4, 2x2 and 1x1 ahead of
  time.
• Each is created by filtering higher resolution
  images down, so it is nicely rounded.
• Fits efficiently into memory.
• Depending on distance to object, we will use a
  different mipmap.

40
MIPMap idea
From OpenGL Programming Guide
41
MIPMap issues

• Idea At different distances use the most
  appropriate MIPMap.
• Generally, the most appropriate is the one that
  gives the closest to 11 texelpixel ratio.
• But can also interpolate between MIPMap levels.
• Called trilinear mipmapping.

42
How does this fix things?

• Prevents aliasing. If we use point sampling, then
  basically it picks a random texel as colour.
• This precomputes averages.

43
Mipmaps in OpenGL

• Two options
• Do it yourself Rememeber glTexImage2D(target,
  level, internalFormat, width, height, border,
  format, type, data)? The level is the mipmap
  level.
• Get gluBuild2DMipmap() to do the work for you.
• Why not use gluBuild2Dmipmap?
• Can use better filters
• Might want to play tricks
• Some textures should be mipmapped carefully

44
Filtering ...

• How to handle the issue of minification and
  magnification?
• Have to apply a filter ... something that will
  combine data from multiple texels.
• Note Think of texel as providing value in CENTER
  of texel, not for whole texel.

45
Magnification

• When magnifying we can do two ways
• Point sampling
• Map pixel centre into texel space. Pick nearest
  texel as colour of pixel. glTexParameteri(GL_TEXTU
  RE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST)
• Linear sampling
• Map pixel centre into texel space. Pick neareast
  4 texels and average using bilinear
  interpolation. glTexParameteri(", ", GL_LINEAR)

46
Minification

• Point sampling and linear sampling can be used as
  before. But glTexParameteri(GL_TEXTURE_2D,
  GL_TEXTURE_MIN_FILTER, GL_NEARESTGL_LINEAR)
• But can now use mipmaps!!

47
Minification types

• GL_NEAREST_MIPMAP_NEAREST (choose nearest mipmap
  level, then choose nearest pixel in that mipmap)
• GL_LINEAR_MIPMAP_NEAREST (choose nearest mipmap
  level, then do bilinear interpolation)
• GL_NEAREST_MIPMAP_LINEAR (interpolate between
  nearest point in two mipmaps)
• GL_LINEAR_MIPMAP_LINEAR (trilinear interp)

48
Environment mapping

• Reminder Environment mapping is a hack to add
  reflection to polygon rendering.
• Basic idea
• Render/photograph the world around the object of
  interest onto an enclosing shape. Turn this into
  a texture map
• Reflect the ray from the viewer off the surface
  (like ray tracing)
• Use the reflected ray as an index for the texture
  map.
• Q What shape is texture map.

49
Environment mapping
Representation of scene
Reflected Ray
Object
Pixel in cube map that is mapped on to object
N
Vector from viewer
50
Environment mapping in practice

• Uses texture maps.
• Is an example of view dependent textures depends
  on position on viewer.
• Coordinates are calculated whenever view changes.
• Means that every time viewer's position changes,
  so does texture coordinates.

51
Reflected ray

• Calculations done in eye coordinate space. (do on
  OHP).
• Final equation

52
Representing surrounding world

• Different ways of representing the world around
  the object.
• Ideal solution would be to use a spherical
  texture.
• Problem Representing a 3D surface like a sphere
  in 2D always leads to distortion.
• Some techniques
• Cube mapping
• Sphere mapping
• Dual paraboloid mapping (only in passing)

53
Cube map

• Scene is represented as six sides of a cube.
• The cube is the unit cube ie from (-1,-1,-1) to
  (1,1,1).

54
Cube map
Cube map
Reflected Ray
Object
Pixel in cube map that is mapped on to object
N
Vector from viewer
55
Cube maps coords

• How to work out coordinates in cube map?
• Two subproblems
• Which face of the cube to look at?
• What (s,t) coords of that face to use?
• Solution
• Use longest coordinate
• Normalise by length, then adjust other
  coordinates. Then get to usual texture
  coordinates.

56
What are coordinates?

• Which side depends on normal vector.
• E.g. if vector is (0.7, 0.5, 0.5) then this means
  it must intersect with the face on the positive
  x-axis
• The coordinates on the face will be s 0.5/0.7,
  t 0.5/0.7

57
Different representations

• Texture maps are in 0,1x0,1
• Faces of cube are -1,1x-1,1.
• Question How to map from cube face to texture
  map?

(1,1)
(1,1)
?
(0,0)
(-1,-1)
(-1,-1)
58
Mapping from face to texture

• Simple

59
Making a cube map

• How to make a cube map?
• Can just take six pictures of a scene.
• But what if we have an artificial scene?
• Even easier! Render six images one for each face.
• Hence can do real-time reflection (except each
  will take 7 renders).

60
Using Cube Maps in OpenGL

• Added in version 1.3
• Was an ARB extension before
• Specify each of the textures (one for each of the
  sides)
• Basically a 2D texturing technique
• How to calculate?

61
Making a cube map

• Use the existing glTexImage2D() call
• Uses a new target. Remember Target is usually
  GL_TEXTURE_2D
• Usually something like GL_TEXTURE_CUBE_MAP_SIGN
  _AXIS
• SIGN POSITIVENEGATIVE
• AXIS XYZ
• glEnable(GL_TEXTURE_CUBE_MAP)

62
Coordinate generation

• A new mode GL_REFLECTION_MAP
• glTexGenfv(GL_S, GL_TEXTURE_GEN_MODE,
  GL_REFLECTION_MAP)
• Similarly for t and r!
• Also support for GL_NORMAL_MAP (talk about that
  later).

63
Manual definition

• Don't HAVE to use either of these. Can specify
  texture coordinate as glTexCoord3f(s,t,r). This
  will then be treated the same as the reflection
  vector.
• Hence can use for arbitrary vector lookup.
• A very powerful capability ... can be used for
  many special effects (look at bump mapping
  later).

64
Demos

• Have a look at GLUT demos
• TexCyl
• NewWave

65
Sphere maps

• An alternative approach.
• Older, more space efficient.
• Easier to implement -- doesn't involve choosing
  an individual side.
• But ... harder to understand.

66
What is a sphere map?

• Imagine taking perfect sphere putting it in scene
  and taking picture
• See Kilgard's slides

67
Sphere maps in OpenGL

• To generate coordinates, use glTexGeni(GL_S,
  GL_TEXTURE_GEN_MODE, GL_SPHERE_MAP)
• Similarly for T (but not r!)
• How are coordinates actually generated?

68
Equations for sphere map

• Proof OHP

69
Environment maps In practice

• Unless surface is really reflective, lo-fi
  envmaps work well.
• In real-time applications (i.e. mostly games)
• Envmap is only low res (even 64x64x6 for cube
  maps)
• Envmap is not updated every frame ... e.g. every
  10 frames - but doesn't work too well with
  moving viewer
• Sphere maps are old hat, cubes are in fashion,
  dual paraboloid still theoretical.

70
Environment maps Limitations

• Sphere map's singularity is a problem.
• Cubes still have problems in corners.
• All only give one level of reflection (of course,
  you can iterate process ...)
• Bit of a performance hit, but can really enhance
  the appearance.
• Have a look at Spin.java

71
Multitexturing

• Won't go into details too much
• But some graphics hardware supports applying more
  than one texture in a single pass.
• Can always accomplish same effect in multiple
  passes
• OpenGL supports multitexturing.
• Why gt 1 texture? Envmapping, bump mapping, light
  maps