Manipulation and Compositing of MC-DCT Compressed Video

1
Manipulation and Compositing of MC-DCT Compressed
Video

• S-F. Chang
• D.G. Messerschmitt
• Presented by Alex Li

2
Outline

• Motivations
• MC-DCT based compression system
• Manipulations in DCT domain
• Manipulation of MC-compressed video
• Manipulation of MC-DCT compressed video
• Results
• Conclusion

3
Motivations

• Video data is compressed for storage and
  transmission
• Spatial domain video manipulations can be
  computationally expensive
• involves full decoding and encoding that brings
  higher data rate and more computations
• Manipulations in compressed domain is proposed
• NO full decoding and encoding, thus lower data
  rate and fewer computations
• low-end computing/communication devices can
  ignore less significant components without much
  degradation (reduced computation and bandwidth)

4
MC-DCT based compression system
5
Manipulations in DCT domain

• Overlap
• Spatial Pnew(i,j) a Pa(i,j)(1-a) Pb(i,j)
• DCT DCT(Pnew) a DCT(Pa)(1-a) DCT(Pb)
• Pixel Multiplication
• Spatial Pnew(i,j) a(i,j) Pa(i,j)(1-a(i,j))
  Pb(i,j)
• need to figure out DCT of Pnew(i,j) Pa(i,j)
  Pb(i,j)
• 2N-point circular convolution of DCT(Pa) DCT(Pb)

6
Manipulations in DCT domain

• Translation
• blockwise translation only replaces the DCT
  blocks
• pixelwise translation is not trivial
• Spatial B B13 B24 B31 B42
• DCT DCT(B) DCT(B13)DCT(B24)DCT(B31
  )DCT(B42)
• How do we extract subblocks?
• B24 H1B4H2 where H1 and H2 are extraction
  matrices

7
Manipulations in DCT domain

• Translation (contd)
• DCT DCT(B24) DCT(H1)DCT(B4)DCT(H2)
• DCT(B) Si DCT(Hi1)DCT(Bi)DCT(Hi2)
• Linear Filtering, Scaling, and Shearing
• Different matrix multiplications, similar
  operations
• Spatial Y Si Gi Xi Hi
• DCT DCT(Y) Si DCT(Gi) DCT(Xi) DCT(Hi)
• Rotation is likely too cumbersome

8
Manipulation of MC-compressed video
foreground
Indirectly affected area
background
Directly affected area
Problem Blocks in DAA could reference blocks
in FG, need to find new reference blocks. This
leads to recalculations. Solution Reduce
number of recalculations by restricting them to
blocks in DAA. Turns out 10 to 15 of the blocks
in DAA need recalculations.
9
Manipulation of MC-compressed video
foreground
D
Indirectly affected area
D2
D1
B
background
Directly affected area
Problem Require considerable amount of
computations to find best-matched block again for
B. Solution Jain and Jains algorithm -- D1
and D2 are likely the less distorted blocks Just
pick between D1 and D2, saves a LOT of
computations
10
Manipulation of MC-DCT compressed video
Original Prec(t,x,y) DCT-1(DCT(e(t,x,y)))
Prec(t-1,x-dx,y-dy) Now DCT(Prec(t,x,y))
DCT(e(t,x,y)) DCT(Prec(t-1,x-dx,y-dy))
11
Results

• 10 to 30 faster than spatial domain
• However, only blockwise translation

12
Results (contd)

• 3 to 6 times faster than spatial domain

13
Conclusion

• 2nd quantization degrades image quality.
• DCT-domain manipulation produces variable
  throughput.
• Can skip high-order coefficients.
• More zero motion vectors during encoding can
  reduce computations further (10 to 20 speedup).
• Combine multiple processes into one through
  precomputed matrices.
• Future work involves compromising video
  compression and manipulation flexibilities.

14
Questions
?