Applying 3-D Methods to Video for Compression

1
Applying 3-D Methods to Video for Compression

• Salih Burak Gokturk
• Anne Margot Fernandez Aaron
• March 13, 2002
• EE 392J Project Presentation

2
OUTLINE

• MOTIVATION
• PREVIOUS WORK
• ALGORITHMS
• RESULTS
• CONCLUSION

3
BACKGROUND

• Most common video compression scheme
• Blockwise transform (i.e. 2D-DCT) for spatial
  compression
• Blockwise Motion Estimation and Compensation for
  temporal redundancy
• Used in MPEG
• Motion Compensated Predictive Coding is better
  than Image based coding since it uses a good
  prediction from the previous frame.

4
OBJECTIVE
Exploit Temporal Redundancies MORE!

• Why?

• How?
• Use 3-D Transformations and Methods.

5
3-D Transform Methods

• Good for ideal case smooth motion, global
  motion
• Bad for non-uniform motion
• Need large number of frame stores

6
PREVIOUS WORK

• 3-D DCT for Video
• Bauer and Sayood, Chan and Sui, Servais and de
  Jager
• 3-D Wavelets for Video
• Pearlman et al. , Taubman et al.,
• Motion Compensated Coding

7
ALGORITHMS IN OUR EXPERIMENTS

1. 2D DCT
2. 3D DCT
3. (Block) Motion Compensated Coding
4. 3D (Volume) Motion Compensated Coding
5. Other Transformation as a prediction scheme
   (PCA, Wavelet)

8
2-D versus 3-D DCT

• 2-D Blocks versus 3-D Blocks

• 2-D DCT versus 3-D DCT
• When would 3-D DCT not work?

Markov-1 chain P(XnXn-1) p Xn uniform 0, 255
9
3D Volume Motion Compensated Coding
depth
Current stack of frames
Previous stack of frames
For each volume block
Divide stack into volume blocks
Find 3D motion vectors
Calculate residual
3D DCT on residual
Q
10
3D Volume Motion Compensated Coding

• Finding the motion vectors
• Full Search
• Measurement criterion
• MAD
• standard deviation of residual
• When it does not work
• No smooth motion (acceleration present)
• Cases when 2D MC does not work (occlusion,
  illumination, non-uniform motion)
• Block size (depth) along time axis is too small
  or too big
• Centroid Effect Center of the block usually
  has a better prediction than edges of the block.

11
Initial Prediction Results
2D MC Prediction
3D MC Prediction
12
3D Principal Component Analysis
Current stack of frames
1 Apply PCA on a large collection of blocks
A U ? VT

13
Experimental Setup

• Techniques
• 2D DCT by 8x8 Blocks
• 3D DCT by 8x8x8 Blocks
• Block Motion Compensated Coding (8x8 and 16x16
  followed by 2-D DCT)
• Volume Motion Compensated Coding (8x8x4, 8x8x6
  and 8x8x8) followed by 3-D DCT.
• Principal Component Analysis (use principal
  components of 8x8x8 blocks as prediction followed
  by 3-D DCT)

• Bit Rate vs PSNR Comparison
• DCT coefficients were quantized using a uniform
  quantizer
• PSNR was computed over the same support region.
• Bit rate was calculated by finding the
  theoretical entropy of the quantized DCT
  coefficients and motion vectors.

14
Miss Am
15
Miss Am
0.14 bit/pixel
2D-DCT
2D-MC
3D-DCT
3D-PCA
3D-MC
16
Salesman 2D-MC vs 3D-MC
2D-MC
3D-MC
0.11 bit/pixel
17
Salesman 3D-DCT vs 3D-MC
3D-DCT
3D-MC
0.16 bit/pixel
18
Foreman
2D-MC
3D-MC
3D-DCT
0.56 bit/pixel
19
Bus
20
CONCLUSION

• There is redundancy across temporal domain.
• There is a lot of room for improvement if we use
  3-D compression methods!
• 2-D motion compensated coding is good for movies
  with A LOT of motion.
• When movements are smooth, 3-D methods will
  perform better than 2-D motion compensated coding.

FUTURE WORK

• Find Better Residual Coding Schemes for 3D MC.
  (That deals with centroid effect better)
• Other 3D Methods ?