IEEE International Conference on Computer Vision (ICCV 2015)
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
Kaiwen Guo1, Feng Xu1, Yangang Wang2, Yebin Liu1, Qionghai Dai1
Tsinghua University1
Microsoft Research2
We present a new motion tracking technique to robustly reconstruct non-rigid geometries and motions from a single view depth input recorded by a consumer depth sensor. The idea is based on the observation that most non-rigid motions (especially human-related motions) are intrinsically involved in articulate motion subspace. To take this advantage, we propose a novel L0 based motion regularizer with an iterative solver that implicitly constrains local deformations with articulate structures, leading to reduced solution space and physical plausible deformations. The L0 strategy is integrated into the available non-rigid motion tracking pipeline, and gradually extracts articulate joints information online with the tracking, which corrects the tracking errors in the results. The information of the articulate joints is used in the following tracking procedure to further improve the tracking accuracy and prevent tracking failures. Extensive experiments over complex human body motions with occlusions, facial and hand motions demonstrate that our approach substantially improves the robustness and accuracy in motion tracking.
Abstract

Results
Figure. 2: Results of our technique. For each result, we show a color image, the input depth and the reconstruction results. Notice that the color image is only for viewing the captured motion. It is not used by our system.

Video Results
Primary Video
Secondary Video

ICCV 2015 Poster

[primary video]
[secondary video]
[primary video]
[secondary video]

Technical Papers
Citation
@inproceedings{guo2015robust,
  title={Robust Non-Rigid Motion Tracking and Surface Reconstruction Using L0 Regularization},
  author={Guo, Kaiwen and Xu, Feng and Wang, Yangang and Liu, Yebin and Dai, Qionghai},
  booktitle={ICCV},
  pages={3083--3091},
  year={2015}
}
Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu and Qionghai Dai, "Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization". IEEE International Conference on Computer Vision 2015.
IEEE Transactions on Visualization and Computer Graphics (TVCG 2017)
Figure. 1: The pipeline of the our method. The red circles indicate the newly detected joints for each anchor frame.
TVCG 2017
ICCV 2015

Source Code and Data
Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu and Qionghai Dai, "Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization". IEEE Transactions on Visualization and Computer Graphics 2017.
@article{guo2017robust,
  title={Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization},
  author={Guo, Kaiwen and Xu, Feng and Wang, Yangang and Liu, Yebin and Dai, Qionghai},
  journal={IEEE transactions on visualization and computer graphics},
  year={2017},
  publisher={IEEE}
}