Motion Detection using PCA

1
Motion Detectionusing PCA

• Roland Miezianko
• rmiezian_at_temple.edu
• Video Analysis Project
• Spring 2004
• Advisor Prof. Dr. Longin Jan Latecki

2
Agenda

• Motion Detection
• Input Video
• Algorithm Steps (2-D and 3-D blocks)
• Results
• Sample Videos and Results
• Video with 8x8 Detection Blocks
• Video with 32x32 Detection Blocks
• Matlab Code
• Source Code

3
Input Video

• MPEG video converted to 2688 JPEG image frames
• Full RGB color

4
Algorithm Steps

• Reshape image to 8x8 blocks
• Collect blocks from every frame, normalize and
  reshape array from 3-D 8x8 blocks
• Compute PCA projection matrix per block
• Compute PCA score by projecting blocks from each
  frame onto that blocks 3-PCA projection
• Compute EV values with W3 for each block
• Generate global threshold based on all blocks and
  frames
• Generate local dynamic threshold for each
  block/frame with W3
• Generate motion matrix based on local and global
  dynamic threshold for all blocks-frames

5
Step 1 - Details

• Read the color image
• Resize the image by scale factor of 0.5
• Convert the image to gray scale
• Reshape the image into 8x8 distinct blocks
• Transpose and save the data
• Note save per frame block data

6
Step 1 - Reshape Image
7
Step 1 - Code
fileName ... imN imread(fileName) imN
imresize(imN,0.5) imN rgb2gray(imN) imCols
im2col(imN,bH bW,'distinct') imT imCols'
8
Step 1 - 8x8 Block Size
Block size relative to image size Block
26x25 Image Size 36x48 blocks
9
Step 2 - Collect Blocks

• Collect same block from all the frames
• Create a single matrix for each block location
• Reshape vector from 3-D 8x8 blocks
• There are 1728 matrices holding pixel values
• Each matrix is 2688 x 64 (frames x pixels/block)

10
Step 2-Normalize Blocks

• Normalize each block by its mean value
• Each block has its mean subtracted from each of
  the 64 pixel values
• Store the normalized block data to be used in
  Step 3 and Step 4

11
Step 2 - Code
fileName ...load(fileName, '-mat') matrix
is BlockX BlockN will be the normalized version
of BlockX BlockN BlockX BlockXMean
(mean(BlockX'))' for FrameIndex FrameStart
FrameEnd BlockN(FrameIndex,) BlockX(FrameI
ndex,) - BlockXMean(FrameIndex,1) End
store normalized block matrix as BlockN
12
Step 2 - Block Matrix
Each block X of 1728 total blocks has a matrix
representation of size 2688x64 Each block is
normalized by its mean value N 2688
13
Step 2 3-D 8x8 Blocks

• Take 3 rows of Block matrix from previous slide
  3x64
• Reshape into 1x192 vector
• 3-D blocks are overlapping
• New 3-D Block Matrix is used in computing PCA
  scores and projection matrices

14
Step 3 - Compute PCA

• Load normalized block matrix from Step 2 and
  compute the PCA projection matrix for this block
  sequence
• Code

fileName ... load(fileName, '-mat') matrix
is BlockN pc,latent,explained
pcacov(cov(BlockN))
15
Step 3 - PCA Projection

• The principal components projection matrix
  contains 64 rows representing each pixel location
  in the block and 64 columns representing 64
  principal components
• Only the first three components are used in
  projection (first 3 columns)

16
Step 4 - Compute Score

• Load normalized block matrix from Step 2 and
  project it onto the PCA projection matrix
  computed in Step 3
• Only the first 3 PCA projections are used

17
Step 4 - Code
fileName ... load(fileName, '-mat') matrix
is BlockN fileName ... load(fileName,
'-mat') matrix is pc Score double(BlockN)
pc(,13)
18
Step 5 - Compute EV

• For each block sequence, load the PCA score
  matrix computed in Step 4
• Compute a covariance matrix using a moving window
  of size 3
• Compute eigenvalues (EVs)
• Sort to get the larges EV value
• Store the data in one EV matrix, representing all
  blocks and all frames

19
Step 5 - EV Matrix

• EV matrix will contain a single EV value for a
  block-frame spatiotemporal location

20
Step 5 - Code
fileName ...load(fileName, '-mat') matrix
is Score dd Scoreevx zeros(FrameEnd,3)W
3 for iW1length(dd)-W cc
dd(i-WiW,) cm cov(cc) evx(i,)sort(eig(c
m)')end
21
Step 6- Global Threshold

• Load EV matrix from Step 5
• Compute mean and standard deviation
• Find all entries in the EV matrix that are below
  mean2std
• Update the EV matrix

22
Step 6 - Code
fileName ...load(fileName, '-mat') matrix is
ev gmean mean(mean(ev'))gstd
std(mean(ev')) LessThanThr find(ev lt
(gmean2gstd))ev(LessThanThr) 20
23
Step 7 - Local Threshold

• Use the updated EV matrix from Step 6
• Compute a local dynamic threshold using window
• Generate a Motion matrix of same size as the EV
  matrix with a simple 0/1 values (1motion)

24
Step 7 - Assumptions

• Assume that first 100 frames have no detectable
  motion
• Compute mean and std of first 100 frames for each
  block
• Compute local threshold for each block using a
  moving window (W3)
• Adjust local threshold, when no moving object is
  detected

25
Step 7 - Code 1
for BlockIndex 1 NumBlocks W3 current
sample FrameStart 100 first frames no
motion meanlmean(ev(BlockIndex,1FrameStart))
stdlstd(ev(BlockIndex,1FrameStart)) movingobje
ct0 for i FrameStartW FrameEnd-W ...
(next slide) end i end BlockIndex
26
Step 7 - Code 2 ()
lwev(BlockIndex,i-Wi) left windowrwev(BlockI
ndex,i1iW) right window if
mean(rw)-meanlgt50stdl mobin(i)mean(rw)-meanl
moving obj detected movingobject1 Motio
n(BlockIndex,i) 1else if movingobject0
meanl0.9meanl0.1mean(lw)
stdl0.9stdl0.1std(lw)
end end
27
Step 8 - Motion Matrix

• Motion matrix is of size 1728x2688, same size as
  the EV matrix
• It contains values 0 or 1, where 1 motion
  detected
• Use the Motion matrix to create sample videos
  showing blocks where motion was detected

28
Detected Motion
No motion Detected Motion (red blocks)
29
Conclusion

• The method of motion detection using principal
  component analysis combined with dynamic
  thresholding yields very good results in
  detecting motion
• Future projects will include processing images
  with variation in size of the blocks

30
Questions Answers

• Sample Videos
• 8x8 Blocks
• 32x32 Blocks
• Principal Component Analysis
• Matlab Code