Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality

1
Real-Time Non-Rigid Shape Recovery via AAMs for
Augmented Reality

• Jackie Zhu
• Oct. 24, 2006

2
Outline

• Introduction
• Extended Active Appearance Models (AAMs) Fitting
• Algorithm
• Offline construction of 3D shape model
• Estimate 3D pose and non-rigid shape parameters
• Experiemental Results
• Conclusion

3
Backgroud

• Rigid Object
• L. Vacchetti et al. (PAMI04) proposed an
  efficient solution for 3D rigid object tracking
• Two 2D AAMs approach for rigid object pose
  estimation
• Non-rigid Object
• V.Blandz 3D Morphable Models
• J.Ahlberg 3D AAM with generic Model.
• Jing X. (CVPR05) 2D3D AAM

4
Overview
5
Extended AAMs Fitting Algorithm
6
AAM Fitting Sample

• The AAMs are built up with 140 still face
  image belonging to 20 individuals, 7 images for
  each. The fitting experiment is performed on an
  AAM with 14 shape parameters, 68 texture
  parameters, and 36335 color pixels.

7
Algorithm

• Building offline basis
• Acquire the 2D shape of objects using the AAM
  fitting algorithm, then construct the 3D shape
  basis.
• Online tracking
• Estimate the 3D pose and shape parameters
  simultaneously via local bundle adjustment by
  building up the point correspondences between 2D
  and 3D.

8
Algorithm Offline 3D Model
9
Algorithm Online Pose Estimation

• The optimization problem can be derived as
• Where

10
Experiemental Results I
11
Experimental Results II
12
Experimental Results
13
Conclusion

• A novel two-stage scheme for online non-rigid
  shape recovery toward Augmented Reality
  applications using AAMs.
• Obtain unbroken point correspondences across
  multiple frames to construct 3D shape models
• Provide 2D to 3D vertex correspondences in the
  online tracking.
• An efficient algorithm is proposed to estimate
  both 3D pose and non-rigid shape parameters via
  local bundle adjustment.

14
References

• Ahlberg, J. Using the active appearance
  algorithm for face and facial feature tracking.
  In Recognition, Analysis, and Tracking of Faces
  and Gestures in Real-Time Systems, 2001.
  Proceedings. IEEE ICCV Workshop on. (2001) 68-72
• Xiao, J., Baker, S., Matthews, I., Kanade, T.
  Real-time combined 2d3d active appearance
  models. In IEEE CVPR'2004. Volume 2. (2004)
  535-542
• Vacchetti, L., Lepetit, V., Fua, P. Stable
  real-time 3d tracking using online and offline
  information. IEEE Trans. PAMI 26 (2004)
• Cootes, T., Edwards, G., Taylo, C. Active
  appearance models. IEEE Trans. PAMI 23 (2001)
• C.Bregler, A.Hertzmann, H.Biermann Recovering
  non-rigid 3d shape from image streams. In IEEE
  CVPR'2000. Volume 2. (2000) 690-696
• Jianke Zhu, Steven C.H. Hoi and Michael R. Lyu
  The Real-Time Non-Rigid Shape Recovery via Active
  Appearance Models for Augmented Reality. In
  ECCV'2006. Volume 1. LNCS 3951. (2006) 186-197.

15
The End

• Thank You !