COMPUTER VISION

1
COMPUTER VISION

• S. Bugra Karahan

2
Main Topics

• Optical Tracking and Eye Tracking
• Range Scanning
• Facial Recognition

3
Optical Tracking

• Overview
• How they work
• Advantages
• Disadvantages
• Eye Tracking
• Links

4
(OT) Overview

• The mostly used trackers use magnetic field. The
  biggest problem with the magnetic trackers is
  their limited range. As the range increases so
  the possibility of distortion also increases.
  They are also very sensitive to the environment.
  Metals and electromagnetic interference cause
  distortion as a result.
• The optical trackers are used to overcome these
  limitations.

5
(OT) Overview

• Optical trackers are mainly used for two
  purposes in computer applications
• For computer animation (human movement analysis)
• Virtual environment (to capture the precise
  information about the position and the
  orientation of the users head)
• Early examples of optical trackers, such as
  Op-Eye and SelSpot were used by MIT and New York
  Institute of Technology in 1982-1983.

6
(OT) How They Work

• Optical tracker systems use either reflective or
  IR-emitting markers and video cameras to monitor
  the tracking space.
• For body motion analysis 20 - 30 markers are
  attached to the body (especially to the joints).
  Number of the markers depends on the the desired
  resolution. More markers give more accurate
  results.
• The markers are small spheres or disks covered in
  reflective material. Can be distinguished by
  their shape, brightness and size.

7
(OT) How They Work
8
(OT) How They Work
9
(OT) How They Work
10
(OT) How They Work

• The markers are imaged by high speed digital
  cameras. The number of the cameras depends on
  the type of motion capture.
• Facial motion capture usually uses one or two
  cameras. Full body motion capture may use four
  to six cameras to provide full coverage of the
  active area.
• To enhance contrast, each camera is equipped
  with IR-emitting LED

11
(OT) How They Work

• IR pass filters are placed over the camera
  lenses.
• Cameras are attached to the controller cards,
  typically in a PC chassis.
• Before motion capture begins, a calibration
  frame -a carefully measured 3D array of markers-
  is recorded. This defines the frame of reference
  for the motion capture session

12
(OT) How They Work

• During the motion capture session, the computer
  is presented with each camera view.
• After the motion capture session, the recorded
  2D motion data is converted to 3D position data
  by using triangulation approach.
• This resultant data is typically applied to an
  inverse kinematics system, to animate a skeleton.
  

13
How They Work (wide area tracking)

• In a virtual environment, to provide the user
  with the impression of being immersed in the
  simulated 3D environment, precise information
  about the user s head is required.
• For this purpose, another optical tracking system
  is used and is called Wide Area Tracking.
• This system uses ceiling panels housing LEDs, a
  miniature camera cluster called HiBall and a
  single-constraint-at-a-time (SCAAT) algorithm
  which converts individual LED position into
  position and orientation data.

14
How They Work (wide area tracking)

• Ceiling
• Current applications cover up to 4,000 cubic
  feet.(500 square feet X 8 feet). But can be
  easily expanded by adding new tiles.

15
How They Work (wide area tracking)

• HiBall

16
How They Work (wide area tracking)

• HiBall is a cluster of 6 lenses and 6
  photo diodes
  arranged so that each
  photo diode can view LEDs
  thorough
  several lenses.
• SCATT algorithm computes the position
  of the user by using
  the LED sightings
  provided by HiBall.

17
Advantages (Optical Body Motion Tracking)

• Large Possible Active Area Unlike magnetic
  tracking system, depending on the system used and
  the precision required, the motion capture area
  can be arbitrarily large.
• Unencumbered Subject The subject is not
  physically attached to the tracking system.
• Markers are passive Since markers are the
  active elements of the system, additional markers
  cost very little. Hundreds of markers can be
  used for a motion track.

18
Advantages (Optical Body Motion Tracking)

• High enough sampling rate for most sport moves
  At 120 to 200 Hz sampling rate, most human
  motions are easily measured.
• Free from electromagnetic interference.

19
Disadvantages (Optical Body Motion Tracking)

• Cost Most expensive tracking systems.
• Bioengineering Technology Systems (Superfluo)
  Uses passive markers. 135,600 (50Hz) 33,000
  (for upgrade to100Hz)
• Selspot AB(Selspot II) IR LEDs - 37,000
• Northern Digital (Optorack) IR LEDs - 80,000
• Motion Analysis Corp (Expert Vision 3D) 38,
  500

20
Disadvantages (Optical Body Motion Tracking)

• Sensitivity to light Background, clothing,
  ambient illumination affect the accuracy.
• Sensitivity to reflection Wet or shiny
  surfaces (mirrors, floors, jewelry, and so on)
  can cause false marker readings.
• Marker Occlusion Since a marker must be seen
  by at least two cameras (for 3D data), the
  occlusion caused by subject (human), materials
  in the environment and the other markers can
  result in lost, noisy, displaced or swapped
  markers.

21
Disadvantages (Optical Body Motion Tracking)

• Tracking time Tracking time can be much greater
  than the actual capture session and may vary
  unpredictably, depending on accuracy
  requirements, motion difficulty, and the quality
  of the raw data captured.
• Non real-time device Since there is no
  immediate feedback , it is impossible to know if
  a motion is adequately captured. More than two
  sessions may be needed.
• Sensitivity to calibration Since multiple
  cameras, the frame of reference for each camera
  must be accurately measured.

22
Eye Tracking

• Are similar to optical trackers. Using infrared
  illumination and lightweight high-resolution
  video sensors.
• The IR waves created by IR LEDs are reflected by
  the eye. This reflection is captured by video
  sensors and white and black colors are used to
  calculate the position of the pupil.

23
Eye Tracking(SensoMotoric Instruments products)

• Headband/Helmet-mounted Eye tracking Device
• Can record eye movement with unrestricted head
  movement

24
Eye Tracking(SensoMotoric Instruments products)

• Remote Eye tracking Device (R.E.D.)
• Eye movements can be acquired without physical
  contact to the subject.
• The R.E.D., placed in front of the subject below
  the line of sight, automatically tracks the
  subject¹s eye within the range of natural head
  movements.

25
Eye Tracking(SensoMotoric Instruments products)

• Head Mounted Display with integrated eye
  tracking (H.M.D.)
• Integrated with Head-mounted display (HDM).
  Useful for virtual reality applications.

26
Eye Tracking(Quick Glance)

• Consists of two IR LEDs and a camera
• The camera and light sources are
  mounted on the
  computer's monitor.
• Examines the reflections from the user's eye
  which is illuminated by LEDs . The reflected
  light is focused onto the camera. By analyzing
  the position of the light reflections and the
  center of the pupil contained in the image, the
  gaze point is determined. Duration can also be
  derived. With that information, the software
  controls the location of the cursor according to
  the gaze point and its duration.

27
Eye Tracking(ISCAN)
28
Range Scanning Outline

• Overview
• Optical Triangulation
• Imaging Radar
• Range Images and Range Surfaces
• Range Image Registration
• Reconstruction
• Future of Range Scanning

29
Range Scanning

• Computer Vision researchers have long studied the
  problem of determining the shape of a scene from
  a set of photographs.
• They attempt to take advantage of a wealth of
  visual cues present in the human visual system
  stereo and motion parallax, occlusion,
  perspective, shading, focus and so on.
• These methods assume that the sensor simply
  records light that already exists in the scene.
• What about active or structured light sensing?

30
Optical Triangulation

• A focused beam of light illuminates a tiny spot
  on the surface of an object.
• For a fairly matte surface, this light is
  scattered in many directions, and a camera
  records an image of the spot.
• We can compute the center pixel for this spot,
  and trace a line of sight through that pixel
  until it intersects the illumination beam at a
  point on the surface of the object. (Figure a)

31
Optical Triangulation

• How do we modify the design to scan the surface
  of an object?
• One method is to scan the light spot over the
  surface using mirrors.
• Another approach is to fan the beam into a plane
  of laser light (Figure b)
• Both approaches need to capture many frames while
  sweeping the light over the object.The temptation
  is to project many points or stripes of light at
  once to capture as much shape as possible in one
  shot.

32
Optical Triangulation
33
Optical Triangulation
34
Imaging Radar

• Time of Flight Radar Systems The scanner emits a
  focused pulse of laser light and waits for it to
  return to a center.
• Amplitude Modulation Imaging Radar The laser is
  operating continuously, but the power of the beam
  is being modulated sinusoidal over time.Compute
  the phase difference between the emitted and
  reflected power signals.
• More recently systems send a plane of light.

35
Many Range Scanners use laser illumination ,
because
1Lasers can be focused tightly over very long
distances(tight beams, narrow stripes) 2 Lasers
have an extremely narrow radiation
spectrum(relatively high immunity to ambient
illumination in the environment)
36
Range Images and Range Surfaces
A range image is like a conventional camera
image, except that each pixel stores a depth
rather than a color. From a single range image,
we can create a range surface by connecting
nearest neighbors with triangular facets. To
avoid making bad assumptions about the shape, we
can apply an edge length criterion and omit long
skinny triangles that would bridge
discontinuities.
37
From Range Image to Range Surface
38
Range Image Registration

• To acquire the shape from all sides, indeed to
  see all into every nook and cranny, many scans
  may be necessary.
• The problem of finding each of the rigid
  transformations to the common coordinate system
  is called range registration or alignment.
• When aligning two range images, we find the 3D
  translation and 3D rotation that bring the points
  as close together as possible.

39
Range Image Registration
40
Reconstruction

• Once all of the range data is precisely
  registered into a common coordinate system, we
  can fuse the data into a single shape, e.g. , a
  dense triangle mesh. This problem is called
  surface reconstruction.
• Numerous solutions have been developed

Compute a surface from the cloud of range
points. Convert images to surfaces, then merge
the surfaces.
- Stitch or zipper the triangle meshes together -
Blend the range surfaces in a sampled volumetric
space
41
Reconstruction
42
What next on Range Scanning?

• The volumetric method is well-suited for
  manufacturing high resolution hardcopies using
  layered manufacturing technologies such as
  stereolithography, thus yielding a 3D Fax.
• A significant effect on how professionals create
  models for the entertainment industry.(replace
  real clay for sculptors?)
• RGBZ Cameras. (This real-time Z channel will
  assist in the compositing process by separating
  image layers based on their relative depths.)

43
Facial Recognition

• Achieving Face Recognition
• Face Recognition Efforts
• Future Work

44
Achieving Face Recognition

• Today, face recognition is not only technically
  feasible but also practical. There are several
  companies that sell commercial face recognition
  software.
• The dominant representational approach that has
  evolved is descriptive, rather than generative
  (we can use example face images to obtain a
  simple mathematical model of facial appearance in
  image data).
• Once you obtain a low dimensional representation
  of face class, you can use standard statistical
  parameter estimation methods to learn the range
  appearances that the target exhibits in the new,
  low dimensional coordinate system.
• Face recognition capitalizes on regularities that
  are peculiar to humans.

45
Face Recognition Efforts

• The most famous early example is that of Teuvo
  Kohonen(HELSINKI Univ. of Tech.), who
  demonstrated that a simple neural net could
  perform face recognition for aligned and
  normalized images of faces.
• In the following years many researchers tried
  face recognition schemes based on edges,
  interfeature distances and other neural-net
  approaches.
• Kirby and Sirovich later introduced an algebraic
  manipulation that made it easy to directly
  calculate the eigenfaces.
• Turk and Pentland then demonstrated that the
  residual error when coding with the eigenfaces
  could be used to detect faces in cluttered
  natural imagery and to determine precise location
  and scale of faces in an image.

46
Current Work

• University of Southern California

47
Current Work

• Massachusetts Institute of
  Technology

p g

• The system collects a database of face images.
• It generates a set of eigenfaces by performing
  principal component analysis on the face images.
  Approximately 100 eigenvectors are enough to code
  a large database of faces.
• The system then represents each face image as a
  linear combination of the eigenfaces.
• Given a test image, the system approximates it as
  a combination of eigenfaces. A distance measure
  indicates the similarity between two images.

48
Current Work

• Massachusetts Institute of Technology

p g
49
Current Work

• Rockefeller University

They used local feature analysis. The parts
marked on the image to the left correspond to
receptive fields for the (a) mouth, (b) nose, (c)
eyebrow, (d) jaw line, and (e) cheekbone
50
Future Work

• All current face recognition algorithms fail
  under the vastly varying conditions in which
  humans can and must identify other people.
• Next-generation recognition systems will need to
  recognize people in real time and in much less
  constrained situations.
• Future smart environments should use the same
  modalities as humans and have approximately the
  same limitations.

51
Links

• Optical Tracking
• http//www.cs.unc.edu/tracker
• http//www.ndigital.com
• http//www.peakperform.com
• http//www.motionanaliysis.com
• http//www.actisystem.fr
• Eye Tracking
• http//www.smi.de/iv/index.html
• http//www.gkc.co.uk/vr-systems/borgtext.htm
• http//www.dinf.org

52
links

• Range Scanning
• Facial Recognition

• www.visint.com
• www.3dvsystems.com
• www.faceit.com
• www.viisage.com
• www.miros.com