Andries van Dam September 5, 2000 Introduction to Computer Graphics 17

1
Image Processing Antialiasing Part II
2
Outline

• Overview
• Example Applications
• Jaggies Aliasing
• Sampling Duals
• Convolution
• Filtering
• Scaling
• Reconstruction
• Scaling, continued
• Implementation

3
Getting down to details

• Specific topics in depth
• Stages 2, 3 and 4 contain numerous important
  topics that will now be discussed in more depth
• First anti-aliasing
• What is aliasing?
• What is anti-aliasing?
• Why is it necessary?
• How is it performed?

4
Jaggies-Manifestation of Aliasing

• Representing a line with discrete pixel values
  means sampling a continuous function
• Jaggies are a manifestation of sampling error and
  loss of information (aliasing)
• Doubling resolution in x and y only lessens the
  problem
• Costs 4 times memory, bandwidth and scan
  conversion time!

(a) Standard midpoint line on a bilevel display.
(b) Same line on a display that has twice the
linear resolution
5
Line as 1-Pixel Width Rectangle

• Sample unit rectangle rather than mathematical
  (infinitesimally thin) line, and switch to
  representing pixels as squares centered at (x, y)
  intersections rather than circles (they are
  neither they are mathematical samples
  reconstructed in display dependent ways. As Alvy
  Ray Smith says A pixel is not a little square)
• Midpoint algorithm pick single pixel closest to
  center line of rectangle (0 lt slope lt 1).
• this is a form of point sampling sample the
  mathematical center line at each of the integer X
  values.
• As with our original midpoint line algorithm, we
  pick a single pixel to represent the lines
  intensity.
• Only single value of intensity used regardless of
  amount of overlap between each pixel and
  rectangle. Therefore all but one pixel in each
  column is off, and all have same (max)
  brightness.

6
Aliasing Examples (1/2)
Low frequency pattern no spatial aliasing
High frequency pattern spatial aliasing
High frequency pattern temporal aliasing
http//www.fundza.com/rman_shaders/high_freq_alias
ing/index.html
7
First Antialiasing Attempt

• Set each pixels intensity value proportional to
  its area of overlap covered by primitive
• sample sub-areas of rectangle (that represents
  line) covered by rectangle representing pixel.
• Note more than one pixel/column for lines of
• 0 lt slope lt 1
• This is a form of un-weighted area sampling
• only pixels covered by primitive can contribute
• only total amount of area of overlap matters,
  regardless of the distance of any individual
  sub-area of overlap from pixels center
• Area sampling smoothes sharp transitions

Figure 3.36 Intensity of a pixel is
proportional to its area covered by the line
8
Box Filter Represents
Unweighted Area Sampling
W

• Weight function W(x, y) gives weight for
  incremental area dA centered at (x, y)
• Box filter is constant over all area and is a
  single pixel wide here, but could vary in width.
  
• For each pixel intersecting the line, intensity
  contributed by each sub-area dA is W(x,y)dA
• Then total intensity of the pixel (between 0 and
  1) integrated over area of overlap is
• Integral is volume over area of overlap (in above
  figure, a rectangular wedge).

W(x, y) dA
9
Cone Filter for Weighted Area Sampling (1/2)

• Area sampling, but the overlap between filter and
  primitive is weighted so that sub-areas with dA
  closer to center of pixel count more.
• revolve -45 degree slope to get 3D cone shape
  linear falloff with distance, and circular
  symmetry
• Calculate contribution by
• calculate volume of wedge of filter whose base is
  area of overlap between support and primitive.
• dA has greater weight if closer to center of
  pixel, Normalize so that volume 1
• subvolume subtended by intersection between line
  and filter is a conical wedge.

W
Cone filter for circular pixel with diameter of
two grid units.
10
Weighted Area Sampling (2/2)
2-unit circular support of filter
Pixel center ()
Area of overlap between support and primitive
Primitive
Differential area dA1
Differential area dA2
W
W(x, y) is the weight which is multiplied with dA
at (x, y) normalize W to make volume under cone
1
11
Cone Filter for Weighted Area Sampling (2/2)

• 2 unit support implies up to 3 pixels/column
• Gupta-Sproull algorithm provides fast
  anti-aliased lines via table look-up (see book)
• Weighted area sampling does a better job of
  smoothing than unweighted area sampling.

Fig. 3.60 One-unit-thick line intersects 3-pixel
supports.
12
Another Look at Problems of
Point Sampling

• This simplistic scan conversion algorithm only
  asks if a mathematical point is inside the
  primitive or not
• bad for sub-pixel detail which is very common in
  high-quality rendering where there may be many
  more micro-polygons than pixels!

Fig. 14.10 Point-sampling problems. Samples are
shown as black dots. Object A and C are sampled,
but corresponding objects B and D are not.
13
Another look at Unweighted AreaSampling (1/3)
Fig. 14.11 Unweighted area sampling. (a) All
sub-areas in the pixel are weighted equally. (b)
Changes in computed intensities as an object
moves between pixels.

• The box filter sets intensity proportional to
  area of overlap
• Get winking of adjacent pixels which eye can
  detect
• eye is very forgiving of TV-type noisy images
  with small changes between frames
• eye is very sensitive to pixels winking in images
  of more uniform appearance

14
Another look at Weighted Area Sampling (2/3)

• Pyramid filter approximates circular cone to
  emphasize area of overlap close to center of
  pixel
• Support is equal to pixel area

(b)
Fig. 14.12 Weighted area sampling. (a)
sub-areas in the pixel are weighted differently
as a function of distance to the center of the
pixel. (b) Changes in computed intensities as an
object moves between pixels.
15
Another Look at Weighted Area Sampling (3/3)
Fig. 14.13 Weighted area sampling with overlap.
(a) Typical weighting function. (b) Changes in
computed intensities as an object moves between
pixels.

• Support of symmetric cone filter greater than
  pixel area for greater smoothness

16
Pseudocode and Results

• for each pixel p
• place filter centered over p
• for each pixel q under filter
• weight filter value over q
• intensity_p weight intensity_q
• aliased antialiased

17
Aliasing Examples (2/2)
Close-up of original, aliased render
Antialiasing Techniques
Blur filter weighted average of neighboring
pixels
Supersampling - sample multiple points within a
given pixel and average the result
Supersampling and Blurring
Images made by Nong Li 08, Jeff Cohen 08, and
Michael Frederickson 08
18
Outline

• Overview
• Example Applications
• Jaggies Aliasing
• Sampling Duals
• Convolution
• Filtering
• Scaling
• Reconstruction
• Scaling, continued
• Implementation

19
Sampling of Images

• Scan converting a geometric scene or scanning in
  a natural image is digitizing a sequence of a
  continuous intensity function (one per scan line)
• Fig. 14.8 Image (a) Graphical primitives. (b)
  Mandrill. (c) Intensity plot of scan line a in
  (a). (d) Intensity plot of scan line a in (b).
  (Part d is courtesy of George Wolberg, Columbia
  University.)
• Step functions abound at the edges of geometric
  primitives

?
?
20
The Sampling/Reconstruction/Display Pipeline
Fig. 14.9 The original signal is sampled, and
the samples are used to reconstruct the signal.
(Sampled 2D image is an approximation, since
point samples have no area.) (Courtesy of George
Wolberg, Columbia University.)

• Scanning in a photograph and displaying it
• scan in one sequence of intensity functions per
  line
• digitize each by sampling at discrete points,
  losing both spatial and intensity information
• reconstruct the signal, e.g., with continuous
  analog voltage wave form to CRT or LCD (via DAC)

21
Waveform Synthesis
Fig. 14.14 A signal in the spatial domain is the
sum of phase-shifted sines. Each component is
shown with its effect on the signal shown at its
right. Approximation of Fig. 14.8(d) (Courtesy
of George Wolberg, Columbia University.)
22
Digression Music

• Intensity of sound is pressure wave amplitude
• Frequency of pressure wave pitch
• Lowest frequency is fundamental frequency
• Integer multiples of the fundamental frequency
  are called harmonics
• Power of two multiples of the fundamental
  frequency are called octaves
• For more, applets to interact with these
  concepts
• http//homepages.gac.edu/huber/fourier/index.html
  
• http//www.phy.ntnu.edu.tw/ntnujava/index.php?topi
  c17
• http//www.chem.uoa.gr/applets/AppletFourier/Appl_
  Fourier2.html

23
Digression Analogous Operation

• Easier to visualize filtering in the frequency
  domain
• Many problems are solved easier by transforming
  to another problem (must also be a way to go
  back!)
• characterize a signal by its frequency spectrum
• signal and frequency domains are duals and
  represent identical information.
• easier to filter in frequency domain than in
  signal domain, as we shall see.
• Take a familiar problem multiplication of
  numbers.
• Can take logarithm of number, perform operations
  on log(number), then move back using
  antilogarithm
• Dual of multiplication is addition in logarithmic
  space
• If ab c then
• log (a) log (b) log (c)
• This invertible transformation makes
  slide-rules such effective tools for
  multiplication manipulating sliders corresponds
  to manipulating numbers via their logs.

24
Frequency Spectrum of a Signal

• Sine wave is characterized by amplitude and
  frequency
• Frequency of a sine wave is number of cycles per
  second for audio, or number of cycles per unit
  length (e.g., inter-pixel distance) for images
• Can characterize any waveform by enumerating
  amplitude and frequency of all its component sine
  waves (Fourier transform see chapter 14)
• This can be plotted as a frequency spectrum,
  a.k.a. power spectrum, (we ignore negative
  values, but they are needed for mathematical
  correctness)

For fun with Fourier transforms see
http//www.ysbl.york.ac.uk/cowtan/fourier/magic.h
tml
25
Sampling The Nyquist Limit
We must sample at a rate that is higher than 2
times the highest frequency in the signal (the
Nyquist limit)

• Here is an approximate analog sine wave
• The sine wave sampled at an acceptable rate
• (4 times the highest frequency)
• Reconstructed wave based on these samples

26
Aliasing Know Thine Enemy

• Here is our analog sine wave again
• Here is the sine wave sampled at too low a rate
• Here is the reconstructed wave based on these
  samples
• The reconstruction isnt even close!

Aliasing occurs when we sample a signal at less
than twice maximum frequency
27
Sampling At the Nyquist Limit

• Sampling right at the Nyquist limit can be
  problematic
• Here is our perfect analog sine wave again
• Here is the sine wave sampled at the Nyquist
  limit. This time it works fine
• Here is the same sine wave sampled at the Nyquist
  limit, with the sample points shifted. Now we
  get no signal

28
The Enemy is Recognized
Aliasing is shown in bottom diagrams on previous
slides Signals that are sampled at too low a
rate can reconstruct high frequencies as low
frequencies

• These low frequencies are aliases of high
  frequencies
• The low sampling rate data could not adequately
  represent the high frequency components, so it
  represented them incorrectly, as low frequencies
  
• So, we just sample above the Nyquist limit,
  right?
• Regrettably, we cant always do that
• What about this?
• Lets try by using Fourier Synthesis!

5
harmonics
25
125
29
Infinite Frequencies

• We cant. Pure and simple. ? 2 ?, and we
  cant sample at an infinite rate (unfortunately,
  infinite frequencies are the rule in synthetic
  computer graphics - discrete transitions between
  adjacent pixels)
• So, do inverse operation. Instead of increasing
  our sampling frequency to meet signal, we
• pre-filter out high frequencies we cant show
• the signal is guaranteed to consist only of
  frequencies we can represent and reconstruct with
  reasonable accuracy
• this isnt same signal that came in, but its
  closer than version that would cause aliasing
• reconstructing the pre-filtered approximate
  signal will yield a better result than
  reconstructing, with corrupting aliases, the
  original signal
• The more high frequencies we pre-filter out, the
  lower the sampling frequency needed but the less
  the filtered signal resembles original
• Note pre-filtering is often just abbreviated as
  filtering, but the prefix pre helps remind us
  that post-filtering (i.e., another stage of
  filtering after image computation or
  transformation) is also practiced. If it is done
  on the reconstructed samples of the original
  signal, it will blur in the aliases present in
  the corrupted reconstruction!

Square wave has infinite frequencies of sine
waves in it, right at jumps from low to high and
high to low (see Fig 14.15). How can we sample
correctly?
30
Image Scaling Aliasing, or
Why do we have to prefilter?
This doesnt look right at all. There are no
stripes and the image now has a blacker average
31
Scale Aliasing II, or
Close, but no cigar?
Better, but not perfect
32
Image Scaling Aliasing III, or Why is it still
wrong?

• The filter made scaled image have same relative
  brightness, but no stripes
• Filter removed high frequencies from image
• discontinuities that were stripes
• Given number of points to represent image once
  scaled, not enough points to represent high
  frequencies
• Well never be able to represent frequencies
  higher than ½ our sampling rate. We cant do
  better than this blurred approximation
• remember Nyquist limit
• Now, pre-filtering in practice
• first show effect of low-pass filtering in both
  signal and frequency domains to eliminate
  unwanted high frequencies
• then discuss mathematics of filtering, called
  convolution

The scaled image with pre-filtering looked a
little better, but we still couldnt see any
stripes
33
Outline

• Overview
• Example Applications
• Jaggies Aliasing
• Sampling Duals
• Convolution
• Filtering
• Scaling
• Reconstruction
• Scaling, continued
• Implementation

34
Low-Pass Filtering(Spatial Domain)
Fig. 14.20 The sampling pipeline with filtering.
(Courtesy of George Wolberg, Columbia
University.)
35
Low-Pass Filtering(Frequency Domain)
1
1
Fig. 14.13 Multiplying with the box function in
the frequency domain. (a) Original spectrum.
(b) Low-pass filter. (c) Spectrum with filter.
(d) Filtered spectrum. (Courtesy of George
Wolberg, Columbia University.)
36
Convolution Math! Or Just Play with the Applets

• Convolving signal f(x) with filter function g(x)
• value of h(x) at each point x is integral of
  product of f(x) and g?(x) where g?(x) is g(x)
  flipped about vertical axis and shifted so origin
  is at x replace x by t and g?(x)g(x-t)
• t is variable of integration that takes x as
  local origin
• diagram shows how g?(x) is constructed using a
  box filter
• if filter g?(x) has finite support, it does
  weighted average of all pixels within support,
  centered at x
• The mathematical flip doesnt matter for
  symmetric functions like ours (see dirac-delta
  applet)

37
Simple Convolution Example
Convolution is a lot like multiplication
f(x) f(x)

1111
1111
1111
1111
1111
1111
1234321
Try the applet http//www.cs.brown.edu/explorator
ies/freeSoftware/repository/edu/brown/cs/ explorat
ories/applets/twoBoxConvolution/two_box_convolutio
n_guide.html
38

• Convolution applets

39
Sinc
Sinc in 2D Sinc in 3D


40
Duals
Spatial
Frequency
Multiplication
Convolution
Convolution
Multiplication
Box
Sinc
Box
Sinc
41
Outline

• Overview
• Example Applications
• Jaggies Aliasing
• Sampling Duals
• Convolution
• Filtering
• Scaling
• Reconstruction
• Scaling, continued
• Implementation

42
Low-Pass Filtering Convolving with sinc
Fig. 14.23 Low-pass filtering in spatial domain.
(a) Original signal. (b) Sinc filter. (c)
Signal with filter, with value of filtered signal
shown as black dot at filters origin. (d)
Filtered signal. (Courtesy of George Wolberg,
Columbia University.)

• Note that in theory sinc has infinite extent,
  however small the contributions and negative
  lobes, but weights contributions at the center
  most heavily. Practically, it is decently
  approximated with gaussian (normal) distribution,
  or even triangle, with finite extents and weights
  greater than or equal to zero.

43
What Does Filtering/Convolution Do?

• Select points at which to sample, e.g., pixel
  centers
• As per 14.23, slide filter over each successive
  sample point and compute convolution integral at
  that point (the area under product curve).
• This is weighted average of current pixel and its
  nearby neighbors (most useful graphics filters
  are symmetric about their origin and fall off
  rapidly from their center).
• In color lecture I, slide 35, we did something
  similar when we calculated response of a color
  receptor to incoming intensity distribution I(?)
  by computing the R(?) I(?)f(?) product curve,
  where I(?) is the Incoming Light Distribution
  (i.e., the signal), and  f(?) is response
  function (i.e., filter function) of receptor.
• Weighted average is value of pixel for filtered
  image
• These illustrations are 2D, but for pixels one
  should think of sinc as three-dimensional surface
  of revolution and primitive as having area. see
  slide 36 Of course, were not actually going to
  compute the integral, we merely look up discrete
  values of filter function and do discrete
  multiplication and summing to approximate it.
• Use term filter strictly in signal-processing
  sense, while Unix and Photoshop have their own
  notions of filters. Some of Photoshops filters
  are signal-processing filters, while some are
  arbitrary transformations of image.

44
Filtering Summary (1/4)

• Filtering is nothing more than weighted averaging
  around a sample point on the image, to smooth
  image
• The dual of sinc function in spatial domain is
  box (aka pulse) function in frequency domain, and
  dual of box function in frequency domain is sinc
  in spatial domain
• The dual of box function in spatial domain is
  sinc function in frequency domain, and vice versa
• Multiplying frequency distribution with box
  function in frequency domain provides exact
  low-pass filtering, i.e., filtering out all high
  frequencies past a specified one. Hence, optimal
  filter is sinc in spatial domain (see figure
  14.24)
• Conversely convolving/filtering with a box filter
  in the spatial domain corresponds to multiplying
  with sinc in the frequency domain, which does
  attenuate high frequencies but not perfectly, and
  introduces artifacts because of the negative
  lobes.
• Multiplication of F(x) and G(x) in frequency
  domain corresponds to convolution of their duals,
  f(x) and g(x), in spatial domain, and vice versa

45
Filtering Summary (2/4)
Fig. 14.24 (a) Sinc in spatial domain corresponds
to pulse in frequency domain. (b) Truncated sinc
in spatial domain corresponds to ringing pulse in
frequency domain. (Courtesy of George Wolberg,
Columbia University)
46
Filtering Summary (3/4)

• To get low-pass filtering (i.e., filter out high
  frequencies), we often use convolution with
  triangle function in spatial domain to
  approximate ideal sinc
• Properties of triangle function
• easy to compute, unlike sinc, which has an
  infinite support (or even a Gaussian
  approximation to sinc)
• its dual, sinc2, is acceptable approximation to a
  box function although it has infinite extent (see
  fig. 14.25b on next slide)
• cause inaccurate representation of all
  frequencies and therefore some degree of
  corruption/aliasing must occur.
• this is not as bad as using box as a filter with
  sinc as its dual in the frequency domain. In
  other words, weighted area sampling of any kind,
  providing it is roughly cone-shaped, is better
  than unweighted area sampling with a box filter,
  which is better in turn than point sampling

47
Filtering Summary (4/4)
Fig. 14.25 Filters in spatial and frequency
domains. (a) Pulse sinc. (b) Triangle sinc2.
(c) Gaussian
Gaussian. (Courtesy of George Wolberg, Columbia
University)
48
The Filtering Pipeline

• Construct continuous function (e.g.,
  rectangles representing lines, polygons)
• Geometric primitives lead to infinite frequencies
  at edges
• Low-pass filter function to generate
• Ideally, in the frequency domain Multiply dual
  of with box in the frequency domain
• Equivalently, in spatial domain Convolve
  with sinc
• (i.e., evaluate integral at an infinite number of
  points)
• Sample pre-filtered continuous function
  to generate
• Construct a continuous function
• Simultaneously sample and filter to generate
• Equivalent to generating new function, then
  evaluating it only at sample points (i.e., at
  pixels)

To filter, we have used three-step pipeline
f(x)
f(x)
f(x)
As programmers, we save work
f(x)
49
Relation between Weighted Area sampling and Area
under Curve (this data is mostly here for the
mathematically inclined)

• Answer Convolution h(x) and filtering compute
  area under p(x) f(t) g(x) product curve
• Discrete Convolution is weighted area sampling
• Each box is ideally hi tall and wide
• Total area
• We use because we actually split the
  interval into 9 subintervals in above image
  (Riemann sum)