Codebook-based Background Subtraction (BGS)

1
Codebook-based Background Subtraction (BGS) for
Visual Surveillance
Kyungnam Kim, Thanarat Horprasert, David Harwood,
Larry Davis, Computer Vision Lab
Key features of our BGS algorithm
Color and Brightness
Layered modeling and detection

• resistance to artifacts of acquisition,
  digitization and compression.
• capability of coping with local and global
  illumination changes.
• adaptive and compressed background model that
  can capture structural background motion over a
  long period of time under limited memory. This
  allows us to encode moving backgrounds or
  multiple changing backgrounds.
• unconstrained training that allows moving
  foreground objects in the scene during the
  initial training period.
• automatic parameter estimation
• layered modeling and detection allowing us to
  have multiple layers of background representing
  different depths
• postprocessing, incorporating spatial shape
  information to obtain better silhouettes.

Basic color distortion metric (having uncertainty
in dark colors)

• The scene can change after initial training.
  These changes should update the background model.
• Additional model cache - The values
  re-appearing for a certain amount of time enter
  the background model as non-permanent, short-term
  backgrounds.

BG model
Input

Add brightness as a factor in computing color
distortion
absorbed into BG

Detection
Result

detected against both box and desk
(a) The woman placed the box on the desk and then
it has been absorbed into the background model as
non-permanent. Then the purse is put in front of
the box. It is detected against both the box and
the desk.
Background (BG) modeling
Results on compressed image sequence and moving
trees
Input sequence
BG Model
(width) x (height) Codebooks
(b) time-indexed detection with different color
labeling unloading two boxes from car

• Each pixel _ 1 codebook (B)
• Each B _ M codewords (wm)
• Each wm _ monochromatic images 4-tuple ltI,
  f, l,tgt
• _ color images 8-tuple ltr,g,I, Imin, Imax,
  f, l, tgt

(a) input image from MPEG sequence
(b) zoomed image
last access time
frequency
maximum negative run-length
Temporal filtering The true background, which
includes both static pixels and moving background
pixels, usually is quasi-periodic.
(c) unattended suspicious objects
Future work
(c) single mode BGS method
(d) our method

• Background subtraction (BGS)
• Clipping problem, Region-based approach,
  Temporal(motion) filtering, Parameter estimation
  for shadow highlight, etc.
• Region- and layer-based BGS
• High-level analysis (for activity recognition)
• - Key frame segmentation
• Rule-based analysis (expert system)
• Decision and control by logic programming

• Input image
• including moving trees

(c) our method with postprocessing
(b) our method without postprocessing