CC Lab: Computer Vision with Processing

1
CC Lab Computer Vision with Processing
2
What is Computer Vision?
Computer Vision is the study and application of
methods which allow computers to "understand"
image/ video content. The term "understand"
means here that specific information is being
extracted from the image data for a purpose. -
Wikipedia
3
What is Computer Vision?
Computer vision refers to a broad class of
algorithms that allow computers to make
intelligent assertions about digital images and
video.
4
Commercial Artistic Uses
5
Surveillance
Computer Vision based systems give a smart eye
to your surveillance systems, allowing for a
24/24 access control and monitoring over
critical areas.
6
Iris Recognition

• Combines computer vision, pattern recognition,
  statistical inference, and optics.
• Its purpose is real-time, high confidence
  recognition of a person's identity by analysis of
  the iriss random patterns.
• Because it is a protected internal organ whose
  random texture is stable throughout life, it can
  serve as a kind of living passport.

7
Face Recognition
8
EYETOY PlayStation Controller
9
Myron Krueger Video Place 1969

• One of the original pioneers of virtual reality
  and interactive art
• Created "responsive environments" that responded
  to the movement and gesture of the viewer through
  an elaborate system of sensing floors, graphic
  tables, and video cameras

10
Daniel Rozin Wooden Mirror 1999

• Materials 830 square pieces of wood, 830 servo
  motors, control electronics, video camera,
  computer, wood frame
• This piece explores the line between digital and
  physical, using a warm and natural material such
  as wood to portray the abstract notion of digital
  pixels. - Rozin

http//www.smoothware.com/danny/woodenmirror.htm
l
click to play movie
11
Adam Frank Shadow 2004

• Interactive installation with transparent
  interface (the user is not aware of any hardware
  controls)
• Projects a disembodied, autonomous, human shadow
  on the ground
• Uses real-time, computer generated 3D graphics
  and video sensing

http//www.adamfrank.com/shadow/shadow.htm
12
Standard CV Algorithms
Many low-level computer vision algorithms are
geared toward distinguishing which pixels, if
any, belong to people or other objects of
interest in the scene.

• Three basic techniques to accomplish this
• Frame-differencing
• Background Subtraction
• Brightness Threshholding

13
Frame Differencing
Each pixel in a video frame F1 is compared with
its corresponding pixel in the subsequent frame
F2. The difference in color and/or brightness
between these two pixels is a measure of the
amount of movement in that particular location.
14
Background Subtraction

• Makes it possible to detect the presence of
  people or other objects in a scene, and to
  distinguish the pixels which belong to them from
  those which do not.
• Compares each frame of video with a stored image
  of the scene's background, captured at a point in
  time when the scene was known to be empty.

15
Brightness Threshholding

• Each video pixel's brightness is compared to a
  threshold value, and tagged as foreground or
  background.
• Effective when objects of interest are
  considerably darker than, or lighter than, their
  surroundings.
• Done with the aid of controlled illumination
  (such as backlighting) and/or surface treatments
  (such as high-contrast paints)

16
CV Libraries/Tools

• Myron - the cross-platform, cross-language, open
  source, video capture and computer vision plugin
  (what well work with today) http//webcamxtra.sou
  rceforge.net/index.shtml
• cv.jit - a collection of max/jitter tools for cv
  applications http//www.iamas.ac.jp/jovan02/cv/
• Eyesweb - a collection of modules for real-time
  motion tracking and extraction of movement
  http//www.infomus.dist.unige.it/eywindex.html

17
Working with Myron and Processing

• Myron is a cross platform computer vision
  library that was created by Josh Nimoy a graduate
  of ITP and named in honor of Myron Kruger.
• The Myron library can be used in Java, Director,
  C and MAX.
• Go to http//webcamxtra.sourceforge.net/download.
  shtml and download it now.

18
Installing and including Myron

• Copy the JMyron folder from the JMyron0023
  directory into the Processing/libraries
  directory. If Processing is already running, then
  restart it.
• To import the library into your sketch, choose
  the menu "Sketch gt Import Library gt JMyron" and
  you will see "import JMyron." appear at the top
  of the sketch.

19
Arrays review

• When we work with Myron we will be using arrays
  and multidimesional arrays so before we look at
  the example code lets review how arrays work.

20
One dimensional arrays

• Arrays are a data type which can hold a series
  of values.
• They are notated using brackets and the
  elements in an array are ordered starting at
  zero.
• Think of arrays as shopping lists
• ShoppingList " Peas, Milk, Bread"
• ShoppingList0 Peas"
• ShoppingList1 Milk"
• ShoppingList2 Bread"

21
Multi dimensional arrays

• A Multi dimensional array is an array of arrays.
• Think of multi-dimensional arrays in terms of a
  table holding columns and rows.
• For example myArray23 would have 3 columns
  and 4 rows.
• In the example above the location myArray 03
  would hold "bmw".

22
Code Examples

• Depending on how many cameras were brought to
  class break off into groups.
• Attach your camera to your groups computer and
  open up Processing.
• Today we will be looking at 4 code examples, 3
  of which can be found in the Jmyron0023gt JMyron
  Examples folder.

23
Code Examples

• Example 1 Myron_simpleCamera
• Reads the value from the camera and draws the
  output to our sketch.
• Example 2 Myron_simple_invert
• Reads the value from the camera and inverts each
  pixels value.
• Example 3 Myron_BoundingBoxes
• Tracks white blobs and draws their bounding
  boxes.
• Example 4 WhiteSq
• Simple tracking application that moves a white sq
  in four directions based
• upon a tracked objects location.