Forensic Image Processing

1
Forensic Image Processing
Peter Kovesi School of Computer Science
Software Engineering UWA
http//www.csse.uwa.edu.au/pk/Forensic/UnderGrad
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12

• Application areas
• Image enhancement
• Image metrology
• Biometric identification

13
Image Enhancement

• Common problems in images
• Poor resolution, especially in video images.
• Poor contrast due to under or over-exposure.
• Corruption with noise.
• Motion blur or poor focus.

14
Image Metrology

• Measurements from stereo pairs of images, or
  sequences of images.
• Measurements from single views.
• Rectification of planar surfaces obtaining a
  plan view of a surface from a perspective image.
• Non-contact 3D shape measurement (footprint
  impressions, wound shapes)

15
Biometric Identification

• Fingerprint and palmprint images.
• Face recognition.
• Iris recognition.
• Automated/semi-automated image indexing(sorting/s
  earching security video recordings)

16
Image Enhancement
The images that almost always need enhancement
are video images. Video images are very poor
with an effective resolution that may be no more
than about 250x300 pixels. But video cameras are
used everywhere
17
Surveillance Cameras in Public Areas in New York
18
(No Transcript)
19
Image Enhancement Video surveillance images are
typically of poor quality
camera
video recorder
monitor
The quality of the display on the monitor can be
misleading. The video recorder runs 24
hours/day. ? The tape gets worn and the recording
heads get dirty. ? The images that are recorded
can be very poor.
20
Getting video images into a computer
Composite video signal (analogue voltages)
The frame grabber converts the analogue voltages
representing the video image into an array of
numbers, forming a digital image. The quality of
the frame grabber can make a big difference.
21
The Digital Image An image is represented in the
computer by a grid of numbers (pixels). The
number at each grid point indicates the
brightness at that point. Typically a value of 0
represents black and 255 represents white. In a
colour image 3 numbers will be stored for each
grid point to indicate the red, green and blue
values at that point.
22
6 2 4 4 2 0 2 0 2
4 4 4 4 8 12 4 16 16 12
20 0 0 4 0 4 0 6 4 4
2 6 4 6 8 6 14 10 12 12
16 0 2 8 6 6 10 14 10 8
6 8 12 6 8 12 14 16 14 8
16 0 8 4 2 8 6 4 6
10 8 6 6 6 14 14 12 12 10
16 18 4 2 0 2 2 10 6 2
10 4 4 8 8 12 10 10 10 12
10 12 0 2 4 4 8 10 4 4
6 10 0 6 12 10 6 10 12 16
14 16 4 0 2 4 6 8 4
4 10 12 8 8 12 10 18 16 20
18 26 16 0 2 6 4 4 2 6
6 8 4 22 8 0 4 12 12 14
14 12 14 2 0 0 4 0 10 8
2 18 86 96 118 88 8 14 10 14
14 16 14 4 0 4 6 6 0
0 2 76 146 130 128 158 82 12 10
12 10 14 14 6 2 6 8 4 2
2 44 148 22 2 0 30 154 38 12
12 14 14 16 6 6 0 4 6 6
6 84 76 0 2 8 0 74 150 0
12 14 24 20 2 0 0 4 6
4 4 140 4 4 10 12 8 0 160
50 18 14 20 12 2 4 4 6 6
4 2 162 0 2 0 6 12 4 84
140 4 12 16 8 6 4 4 4 14
12 8 160 0 12 12 6 16 14 92
154 60 12 14 18 0 0 2 6
12 6 6 154 14 8 4 8 8 8
100 124 168 10 16 22 2 0 8 2
6 8 0 90 90 0 10 8 14 0
132 132 176 92 6 12 0 0 6
2 2 6 0 10 144 38 0 2 0
64 148 146 148 168 38 6 4 2 6
8 8 14 8 0 46 146 120 92 104
130 152 160 158 172 158 26 4 2 10
6 8 4 2 4 0 26 76 124 160
160 162 152 148 158 200 178 2 6
8 4 6 6 4 0 6 12 0 2
0 24 140 196 156 150 164 198 2 2
0 4 4 4 2 10 10 8 6 0
0 0 0 50 146 172 172 160 4 0
2 12 8 4 8 10 12 14 12 4
8 12 10 8 10 164 178 170 2 2
0 8 4 10 8 10 8 16 8 6
4 16 14 4 4 2 162 162 2
0 10 2 0 2 2 2 8 10 4
12 12 6 10 8 18 0 80 164 2
2 10 2 4 6 2 10 8 14 4
6 8 8 12 8 14 14 2 12 0
2 6 4 2 2 6 8 6 6 10
10 8 10 4 12 16 22 12 6
0 2 4 4 0 2 6 0 4 4
4 12 6 4 4 12 12 4 10 14
0 0 4 6 0 4 0 4 6 4
8 6 6 10 16 12 8 6 12 10
6 6 6 4 4 8 8 8 10 8
10 10 0 4 6 12 2 2 8 12
Pixel values for the top left corner of this image
23
It is not possible to "blow up" a digital image
to achieve finer and finer detail. The level of
detail is fixed once and for all when the digital
image is first created. There is a trade-off
high resolution images require more storage and
more processing effort than low resolution
images. The human eye has over 100 million
light sensitive cells in the retina even the
most expensive digital camera has nowhere near
this resolution.
24
A video image may have been sampled and digitised
at, say, a resolution of 640x480 pixels. However
the underlying resolution of a video image is no
more than about 250x300 pixels. This is fixed by
the television transmission standard. Video
images are also interlaced - one field of the
image displayed in the even numbered rows of the
image and the other field in the odd numbered
rows. These two fields are recorded 1/50th of a
second apart. A complete frame, made up of two
fields, is generated every 1/25th of a second.
Interlacing can be seen where there is motion.
25
(No Transcript)
26
(No Transcript)
27
Field extracted from odd rows
Field extracted from even rows
Images are deinterlaced by extracting every 2nd
row and filling in the missing rows by taking the
average of the rows above and below the missing
ones.
28
Each field is recorded to tape by a separate
head. The image recorded by each head can have
very different characteristics if the recorder
has not been well maintained.
29
(No Transcript)
30
Field 1
Field 2
31
after conversion to greyscale and some contrast
adjustment
32
Poor resolution
33
Poor resolution Zooming in does not help!
34
Create a higher resolution image by
mathematically interpolating pixel values
35
Interpolation produces an image that is more
pleasing to look at, but you typically do not get
any new information that you could not see in the
original image. Unfortunately video resolution
is such that it is useless for the identification
of people.
36
4.8 m
37
(No Transcript)
38
Super-resolution from multiple low resolution
images
Images must be aligned accurately. Object should
be planar.
Irani and Peleg 1991 Capel and Zisserman 1998
39
Super-resolution from multiple low resolution
images
Results from synthetic data can be impressive
40
Real data Car number plate - low resolution
blur and noise.
41
Super-resolution from multiple low resolution
images
Number plate reconstruction
42
Correcting for poor exposure
The range of light intensities in a scene is
often very much greater than can be captured and
presented by a photo or video image. In bright
light, you must either expose for the sunlight
and lose detail in the shadows, or vice versa.
Video cameras operate 24 hours a day under
widely varying lighting conditions. Video
cameras usually have electronic gain adjustment,
but this can only (attempt to) compensate for
changes in overall brightness levels
43
Enhancing Contrast by Adjusting Grey Values in
the Image
A number of techniques, such as histogram
equalisation and gamma correction, involve
remapping grey values in the image.
contrast suppressed
output grey values
mapping function
contrast enhanced
0
original grey values
dark bright
44
Input-Output Transformations
45
(No Transcript)
46
Gamma Gamma correction involves a remapping curve
given by the equation
(Divide by 255 to get a value between 0 and 1,
raise to a power, then rescale back to 0 to 255.)
47
Histogram Equalisation The distribution of
luminance intensities in an image is called its
histogram. The horizontal axis represents
intensity levels and the vertical axis represents
the number of pixels at each intensity level.
Most well-lit scenes have a wide dynamic range
of intensities, with hills and valleys.
Histogram equalisation involves spreading out
the image histogram to one that is more uniform.
48
(No Transcript)
49
greyscale enhanced image
original image (from Crime Stoppers Web Page)
50
original image
51
Grey scale enhanced image
52
Deinterlaced and then denoised
53
Spatial Image Enhancement Techniques

• There are two main categories of techniques
  for image enhancement
• Spatial Domain Methods - which operate directly
  on pixels.
• Frequency Domain Methods - which operate on the
  Fourier Transform, or Wavelet Transform, of the
  image.

54
Spatial Domain Methods The value of a pixel at
location (x,y) in the enhanced image is the
result of performing some operation on the pixels
in the neighbourhood of (x,y) in the input image.
T operation
g(x,y)
neighbourhood around f(x,y)
input image f
output image g
For computational reasons the neighbourhood is
usually square but it can be any shape.
55
Linear Filtering A common application of linear
filtering is image smoothing using an averaging
filter, or averaging mask, or kernel.
Each point in the smoothed image, g(x,y) is
obtained from the average pixel value in a
neighbourhood of (x,y) in the input image.
56
Convolutions Computationally, spatial filtering
is implemented in a computer program via a
process known as convolution. If we assume a 3x3
neighbourhood the smoothing by averaging would
correspond to convolving the image with the
following filter, or mask. This filter is
convolved with the image by placing it over a 3x3
portion of the image, multiplying the overlaying
pixel values and adding them all up to find the
value that replaces the original central pixel
value.
57
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 -
- -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
58
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 -
- -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
59
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
60
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
61
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
62
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 2.5 -
-

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
63
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 2.5 -
- 4.4

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
64
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 2.5 -
- 4.4 3.0

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
65
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 2.5 -
- 4.4 3.0 2.7

x.1 x.1 x.1
x.1 x.1 x.1
x.1 x.1 x.1
Filtered image
Input image
66
The mask is moved across the image until every
pixel has been covered. We convolve the image
with the mask.
.1 .1 .1
.1 .1 .1
.1 .1 .1
Filter
3 2 1 2 2
4 2 5 3 3
8 6 3 1 1
4 7 2 1 6
6 5 3 2 8
- - - - -
- 3.4 2.5 2.1 -
- 4.1 3.0 2.5 -
- 4.4 3.0 2.7 -
- - - - -
Filtered image
Input image
67
Smoothing can be quite useful in eliminating
"salt and pepper" noise. - but the image is also
blurred
68
Gaussian smoothing Sometimes we want to emphasize
the fact that the closest pixels are the ones
most influencing the result in an area transform
process. Thus we use a smoothing filter that
places more weight on the central pixel, and
which lessens the weight the further one moves
from the centre. When the distribution of
weights follows the shape of a bell curve, the
filter is called a Gaussian filter.
69
A 5x5 mask approximating a Gaussian
70

• Averaging filters are typically not useful for
  enhancement
• However they are often a useful model of how an
  image has been degraded.
• Averaging over a circular region models out of
  focus blur.
• Averaging along a line of pixels models motion
  blur
• Given a mathematical model of the degradation we
  can attempt to invert the degradation process.

71
Motion blur can be modelled as a sharp image that
has had averaging applied along a line of pixels
72
Median Filtering Median filtering is an
alternative form of area process transformation
where the central pixel value in a neighbourhood
is replaced by the median value of those pixels.
The median of a list of numbers is the
value such that half the numbers are less than
this value and the other half are greater. For
example, the list 1 1 1 1 3 3 3 3 5 has median 3.
73
Median filtering is particularly good at removing
"salt and pepper" noise whilst preserving edge
structure.
74
Using Embossing Filters to Enhance Latent
Handwriting Impressions
75
Computationally, to implement edge detection we
need to design a filter that looks for places in
an image where differences are large, and then
mark these places. A filter that subtracts one
neighbour and adds the other will highlight such
places. The smallest filter that can detect
vertical edges looks like When this filter
is convolved with an image with a sharp vertical
edge, it generates an output image that
highlights the edge and ignores all uniform
areas. This corresponds to finding the
horizontal intensity gradients in the image and
produces an embossing result.
-1 0 1
76
By varying the values in the filter different
effects can be achieved. A diagonal embossing
effect can be obtained using a filter like this

-1 0 0
0 0 0
0 0 1
-1
0
1
Vertical embossing filter
-1 0 1
Horizontal embossing filter
77
-1 0 1
Horizontal embossing example
0 0 0 1 1 1 0 0 0
0 0 0 1 1 1 0 0 0
0 0 0 1 1 1 0 0 0
0 0 0 1 1 1 0 0 0
0 0 0 1 1 1 0 0 0
original image
0 0 1 1 0 -1 -1 0 0
0 0 1 1 0 -1 -1 0 0
0 0 1 1 0 -1 -1 0 0
0 0 1 1 0 -1 -1 0 0
0 0 1 1 0 -1 -1 0 0
filtered image
78
original image
filtered image
-1 0 1
79
Page from street directory recovered from
Brammers car
80
Illuminated from the right
Illuminated from the left
81
Illuminated from the top
Illuminated from the bottom
82
Right illuminated image Filter 1 0 -1
Left illuminated image Filter -1 0 1
83
Top illuminated image Filter 1 0
-1
Bottom illuminated image Filter -1
0 1
84
Adding all the edge detected images we get
Enhanced impression of someone practicing Sid
Wallaces signature