Sensors

1
Sensors
CSC 59866CD Fall 2004

• Lecture 4
• Sensors

Zhigang Zhu, NAC 8/203A http//www-cs.engr.ccny.c
uny.edu/zhu/ Capstone2004/Capstone_Sequence2004.h
tml
2
Acknowledgements

• The slides in this lecture were adopted and
  modified from lectures by
• Professor Allen Hanson
• University of Massachusetts at Amherst

3
Sensors

• Static monocular reflectance data (monochromic or
  color)
• Films
• Video cameras (with tapes)
• Digital cameras (with memory)
• Motion sequences (camcorders)
• Stereo (2 cameras)
• Range data (Range finder)
• Non-visual sensory data
• infrared (IR)
• ultraviolet (UV)
• microwaves
• Many more

4
The Electromagnetic Spectrum
C f l E ? f
Visible Spectrum
700 nm
400 nm
5
The Human Eye
6
The Eye
Retina

• The Retina
• rods (low-level light, night vision)
• cones (color-vision)
• synapses
• optic nerve fibers
• Sensing and low-level processing layer
• 125 millions rods and cones feed into 1 million
  nerve fibers
• Cell arrangement that respond to horizontal and
  vertical lines

7
Film, Video, Digital Cameras

• Black and White (Reflectance data only)
• Color (Reflectance data in three bands - red,
  green, blue)

8
Color Images
Spatial Resolution Spectra Resolution Radiometric
Resolution Temporal Resolution
Dimensions of an Image
Spatial (x,y) Depth (no. of components) Number of
bits/channel Temporal (t)
Pixel
9
Across the EM Spectrum
Crab Nebula
10
Across the EM Spectrum
Cargo inspection using Gamma Rays Mobile Vehicle
and Cargo Inspection System (VACIS)
Gamma rays are typically waves of frequencies
greater than 1019 Hz Gamma rays can penetrate
nearly all materials and are therefore difficult
to detect CourtesyScience Applications
International Corporation (SAIC),
11
Across the EM Spectrum
Cargo inspection using Gamma Rays Mobile Vehicle
and Cargo Inspection System (VACIS)
Gamma rays are typically waves of frequencies
greater than 1019 Hz Gamma rays can penetrate
nearly all materials and are therefore difficult
to detect CourtesyScience Applications
International Corporation (SAIC),
12
Across the EM Spectrum
Cargo inspection using Gamma Rays Mobile Vehicle
and Cargo Inspection System (VACIS)
Gamma rays are typically waves of frequencies
greater than 1019 Hz Gamma rays can penetrate
nearly all materials and are therefore difficult
to detect CourtesyScience Applications
International Corporation (SAIC),
13
Across the EM Spectrum

• Medical X-Rays

14
Across the EM Spectrum

• Chandra X-Ray Satellite

15
Across the EM Spectrum

• From X-Ray images to 3D Models CT Scans

16
Across the EM Spectrum

• Flower Patterns in Ultraviolet

Dandelion - UV
Potentilla
17
Across the EM Spectrum

• Messier 101 in Ultraviolet

18
Across the EM Spectrum

• Traditional images

19
Across the EM Spectrum

• Non-traditional Use of Visible Light Range

20
Across the EM Spectrum

• Scanning Laser Rangefinder

21
Across the EM Spectrum

• IR Near, Medium, Far (heat)

22
Across the EM Spectrum

• IR Near, Medium, Far (heat)

23
Across the EM Spectrum

• IR Finding chlorophyll -the green coloring
  matter of plants that functions in photosynthesis

24
Across the EM Spectrum

• (Un)Common uses of Microwaves

Exploding Water Movie
CD Movie
25
Across the EM Spectrum

• Microwave Imaging Synthetic Aperture Radar (SAR)

Tibet Lhasa River
San Fernando Valley
Red L-band (24cm) Green C-band (6 cm) BlueC/L
Athens, Greece
Thailand Phang Hoei Range
26
Across the EM Spectrum

• Radar in Depth Interferometric Synthetic
  Aperture Radar - IFSAR

(elevation)
27
Across the EM Spectrum

• Low Altitude IFSAR

IFSAR elevation, automatic, in minutes
Elevation from aerial stereo, manually, several
days
28
Across the EM Spectrum

• Radio Waves (images of cosmos from radio
  telescopes)

29
Stereo Geometry

• Single Camera (no stereo)

30
Stereo Geometry
LEFT CAMERA
RIGHT CAMERA
P(X,Y,Z)
B Baseline
31
Stereo Geometry
LEFT IMAGE
RIGHT IMAGE
P
Pr(xr,yr)
Pl(xl,yl)
Disparity xr - xl
depth
32
Stereo Images

• A Short Digression

Stereoscopes
33
Stereo Images
Darjeeling Suspension Bridge
34
Picture of you?
35
Stereo

• Stereograms

36
Stereo X-Ray
37
Range Sensors

• Light Striping

David B. Cox, Robyn Owens and Peter
Hartmann Department of Biochemistry University of
Western Australia
http//mammary.nih.gov/reviews/lactation/Hartmann0
01/
38
Mosaics

• A mosaic is created from several images

39
Mosaics

• Stabilized Video

40
Mosaics

• Depth from a Video Sequence (single camera)

GPS
Height H from Laser Profiler
P(X,Y,Z)
41
Mosaics

• Brazilian forest..made at UMass CVL

42
Why is Vision Difficult?

• Natural Variation in Object Classes
• Color, texture, size, shape, parts, and relations
• Variations in the Imaging Process
• Lighting (highlights, shadows, brightness,
  contrast)
• Projective distortion, point of view, occlusion
• Noise, sensor and optical characteristics
• Massive Amounts of Data
• 1 minute of 1024x768 color video 4.2 gigabytes
• (Uncompressed)

43
The Need for Knowledge
Variation
Knowledge
Motion
Context
Function
Shape
Shape
Purpose
Specific Objects
Generic Objects
Structure
Size
44
The Figure Revealed
45
The Effect of Context
46
The Effect of Context - 2
47
Context, cont.

• .a collection of objects

48
Context

• The objects as hats

49
Context

• And as something else..
• To interpret something is to give it meaning in
  context.

50
Vision System Components

• ..at the low (image) level, we need
• Ways of generating initial descriptions of the
  image data
• Method for extracting features of these
  descriptions
• Ways of representing these descriptions and
  features
• Usually, cannot initially make use of general
  world knowledge

IMAGE (numbers)
DESCRIPTION (symbols)
51
Vision System Components

• .at the intermediate level, we need
• Symbolic representations of the initial
  descriptions
• Ways of generating more abstract descriptions
  from the initial ones (grouping)
• Ways of accessing relevant portions of the
  knowledge base
• Ways of controlling the processing
• Intermediate level processes should be capable of
  being used top-down (knowledge-directed) or
  bottom-up (data-directed)

IINTERMEDIATE DESCRIPTIONS
IMAGE
KNOWLEDGE
52
Vision System Components

• .at the high (interpretation) level, we need
• Ways of representing world knowledge
• Objects
• Object parts
• Expected scenarios (relations)
• Specializations
• Mechanisms for Interferencing
• Beliefs
• Partial matches
• Control Information
• Representations of
• Partial interpretations
• Competing interpretations
• Relationship to the image descriptions

53
Next

• Anyone who isn't confused really doesn't
  understand the situation.

--Edward R. Murrow
Next Image Formation
Reading Ch 1, Ch 2- Section 2.1, 2.2, 2.3,
2.5 Questions 2.1. 2.2, 2.3, 2.5 Exercises 2.1,
2.3, 2.4