Adaptive%20Methods%20for%20Motion%20Characterization%20and%20Segmentation%20of%20MPEG%20Compressed%20Frame%20Sequences

1
Adaptive Methods for Motion Characterization and
Segmentation of MPEG Compressed Frame Sequences

• C. Doulaverakis, S. Vagionitis, M. Zervakis,
• E. Petrakis
• Technical University of Crete (TUC)
• Chania Crete Greece

2
Problem Definition

• Video segmentation
• Abrupt (cuts) Gradual transitions
• Zoom, Pan/Tilt
• Contribution segmentation and identification of
  camera effects
• Processing on MPEG video
• Partially decompressed block data
• DC intensity approximation for blocks
• Coherent motion vectors for I, P, B frames

3
Twin Comparison (TC)

• Shot boundaries ltgt peaks on histogram
  differences
• Thresholds Ta, Tb
• Ta µ a s
• ?b bµ
• Requires pre-processing,does not adapt to signal

?a
Tb
4
Sliding Window (SW)

• Processing over W frames
• One Threshold
• Ta(i) µ(i) a s(i)
• Cut lt 5 frames
• Gradual gt 5 frames
• No preprocessing, adapts to signal

Ta
5
Adaptive Method (AM)

• Ta(i) µ(i) a s(i)
• µ(i) µ(i-1)-c(µ(i-1)D(i))
• s(i) µ(i)2 ?(i)1/2
• ?(i)?(i-1)c (?(i-1)D(i))2
• c0.05
• Ta is computed at each i and depends on previous
  values
• No preprocessing, adapts to signal

Ta
6
Direction Histogram

• Histogram of angles of motion vectors
• 8 angles multiples of p/4 for moving vectors
• Plus 0-th value for static vectors v lt 1

static
moving
7
Motion Characterization

• Analysis of variance s motion histogram
• Normalized by number of intracoded vectors
• Zooming the vectors are spread uniformely (max
  s)
• Panning-Tilting the vectors are concentrated at
  a single bin (min s)
• Static camera the vectors are concentrated at
  bin 0

8
Example

pan pan zoom static
camera

Ta

9
Video Segmentation Method

• Cuts the number of intracoded vectors in frame
  exceeds threshold
• Gradual transitions combines motion and
  intensity information
• Difference of intensity histogram exceeds
  threshold
• Magnitude of motion vectors exceed threshold

10
Experiments

• Measurements over 17 videos
• Competitive methods correspond to thresholding by
  TC, SW, AM
• Each method is represented by its
  precision/recall curve as a function of the
  threshold parameter a

11
Abrupt Transitions (cuts)
12
Gradual Transitions
13
Future Work

• More accurate threshold estimation
• SW or AM gets trapped in local minima
• Detection of Cuts is fairly stable
• More elaborate methods for detection of gradual
  transitions and for cleaning-up false positives
  due to camera effects

14
Zoom Detection
a4.5
a2
15
Pan/Tilt Detection
a2
a4.5