Error Concealment for Stereoscopic Sequences ITG Fachausschusstagung 3'2, Juni 2006

1
Error Concealment for Stereoscopic Sequences ITG
Fachausschusstagung 3.2, Juni 2006
2
Introduction

• Monoscopic Error Concealment strategies are not
  well suited for stereoscopic scenario
• Assumtions
• independently coded views of a stereo image pair
• remaining redundancies between the channels,
  which can be utilized for error concealment
• block based coding (16x16 blocks)

3
Algorithm Overview

• Identification of corresponding region
• feature extraction
• feature matching along epipolar line
• selection of matches (M-estimator/RANSAC)

• Projective Transformation
• initial parameter set from matches
• optimization by Newton Method

• Smoothing
• only in case of discontinuities of depth

4
Matching and Transformation
projective transformation
5
Selection of feature matches

• M-estimator
• uses all matches with different weights
• In some cases the transformation fails, because
  pixels from outside the image were warped into
  the erroneous burst.
• RANSAC (random sample consensus)
• uses a number of subsamples (four feature
  matches)
• minimize the sum of squared residues of the
  boundary region
• RANSAC yields better results than
  M-estimator

6
Adapted Newton Method

• Find the optimal transformation parameter by
  minimizing a cost function C(k)
• b is the Border Region of the erroneous block
  burst

7
Adapted Newton Method

• Iteration step
• Problem I Local minimum solution
• Initial Parameter set is of prime importance

Cost function C(k) over horizontal and vertical
translation parameter
8
Adapted Newton Method

• Problem II Convergence of Newton method
• Successivly decreasing of border pixel size L
  after every minimum search

Speed of convergence of the Newthon algorithm for
different border sizes L
9
3D Block Smoothing

• In case of great discontinuities in depth
  (variation of disparity) we perform a linear
  smoothing algorithm towards the surrounding pixel
  region (3D-BS)
• Minimization of the intersample variance between
  neighboring samples and to the block borders

10
HQ EC Results / Example
11
Results / Subjective Evaluation

• Subjective Simulation Results
• Double Stimulus Continuous Quality Scale Method
  (DSCQS) as phsychovisual test with 15 subjects
• Shutterglasses (StereoGraphics)

DMOS
12
Fast EC Algorithm Overview

• Block Search
• Directional Diamond Search
• SAD

• Side Match Distortion

13
Fast EC Matching Example Hall

• SMD determined for each possible position
• Position with minimum SMD selected
• Block used for reconstruction

14
Fast EC Subjective Evaluation
15
Simple Matching - Example

• Balloons 720x480, corrupted frame and features

16
Simple Matching - Example

• Reference frame (temporal) with features

17
Simple Matching - Example

• Reference blocks

18
Simple Matching - Example

• Reconstructed frame

19
Publications

• K. Günther, C. Clemens, and T. Sikora
• A Fast Displacement-Estimation Based Approach
  For Stereoscopic Error Concealment
• PCS 2004, San Francisco
• C. Clemens, M. Kunter, S. Knorr, and T. Sikora
• A hybrid approach for error concealment in
  stereoscopic images
• WIAMIS '04, Lissabon
• M. Kunter, S. Knorr, C. Clemens, and T. Sikora
• A gradient based approach for stereoscopic error
  concealment
• ICIP '04, Singapore
• S. Knorr, C. Clemens, M. Kunter, and T.
  SikoraRobust Concealment for Erroneous Block
  Bursts in Stereoscopic Images3D Data Processing,
  Visualization, and Transmission (3DPVT'04),
  Thessaloniki, Greece