Motivation

1
Motivation

• Problem Amateur photographers often take
  low-quality pictures with digital still camera
• Personal use
• Professionals who need to document (realtors,
  architects)
• Solution Find alternatives to picture being
  acquired by automating photographic composition
  rules
• Analyze scene, including detection of main
  subject
• Adapt camera settings automatically to follow
  rules
• Contribution Automated detection of main subject
• Independent of indoor/outdoor setting and scene
  content
• Low implementation complexity, fixed-point
  computation

2
Digital Still Cameras

• Converts optical image toelectric signal using
  chargecoupled device
• Camera settings under software control
• Focus, e.g. auto-focus filter
• Zoom
• White balance Corrects color distortions
• Shutter aperture and speed
• Produces JPEG compressed images

3
Main Subject Detection Methods

• Two differently focused photographs Aizawa,
  Kodama, Kubota 1999-2002
• One has foreground in focus, and other has
  background in focus
• Significant delay involved in changing the focus
• Bayes nets based training Luo, Etz, Singhal,
  Gray 2000-2001
• Bayesian network trained on example set and
  tested later
• Training time involved suited for offline
  applications
• Multi-level wavelet coefficients Wang, Lee,
  Gray, Wiederhold 1999-2001
• Expensive to compute and analyze wavelet
  coefficients
• Iterative classification from variance maps Won,
  Pyan, Gray 2002
• Optimal solution from variance maps and
  refinement with watershed
• Suitable for offline applications involving
  iterative passes over image

4
Proposed Algorithm

• User starts image acquisition
• Focus main subject using auto-focus filter
• Partially blur background and acquire resulting
  picture
• Open shutter aperture (by lowering f-stop) which
  takes about 1 s
• Foreground edges stronger than background edges
• While acquiring user-intended picture, process
  blurred background picture to detect main subject
• Generate edge map (subtract original image from
  sharpened image)
• Apply edge detector (Canny edge detector performs
  well)
• Close boundary (e.g. gradient vector flow or
  proposed approximation)

5
Generate Edge Map

fsmooth(x,y)
-
g(x,y)
fsharp(x,y)
Smoothing filter


f(x,y)

Sharpening filter
Model for an image sharpening filter

• Symmetric 3 x 3 sharpening filter
• For integer a and b, coefficients are
• Integer when dropping 1/(1 a) term
• Fractional when -1 2a ? b lt 1 and 1/(1 a) is
  power-of-two
• Generate edge map
• Subtract original image from sharpened image
• Main subject region now has sharper edges

6
Boundary Closure

• Gradient vector flow method Xu, Yezzi, Prince
  1998-2001
• Compute gradient
• Outer boundary of detected sharp edges is initial
  contour
• Change shape of initial contour, depending on
  gradient
• Shape converges in approximately 5 iterations
• Disadvantage computationally and memory
  intensive
• Approximate lower complexity method
• Select leftmost rightmost ON pixel and make row
  between them ON
• Can detect convex regions but fails at concavities

7
Implementation Complexity

• Number of computations and memory accesses per
  pixel
• Sharp region calculation convolution with
  symmetric 3x3 filter with parameters a 0.5 and
  b 3.5 subtraction
• Canny edge detector gradient computation with
  symmetric 3x3 filter non-maximal suppression
• Digital still cameras use 160 digital signal
  processor instruction cycles per pixel

8
Results
Original image with main subject(s) in focus
Detected strong edges with proposed algorithm
Detected main subject mask with gradient vector
flow
9
Conclusion

• Developed automated low-complexity one-pass
  method for main subject detection in digital
  still cameras
• Processes picture taken with blurred background
• Detects main subject by detecting frequency
  content difference between main subject and
  background
• Requires 18 multiply-accumulates, 4 comparisons,
  and10 memory accesses per pixel
• All calculations in fixed-point arithmetic
• Applications digital still cameras,
  surveillance, constrained image compression, and
  transmission and display
• Copies of MATLAB code, poster, and paper,
  available at
• http//www.ece.utexas.edu/bevans/papers/2003/sti
  llCameras

10
AUTOMATIC MAIN
SUBJECT DETECTION FOR DIGITAL
Serene Banerjee and Brian L. Evans
Embedded Signal Processing LaboratoryThe
University of Texas at Austin
11
STILL CAMERAS