Online Dense Non-Rigid 3D Shape and Camera Motion Recovery
In Proceedings British Machine Vision Conference 2014
AbstractThis paper describes a sequential solution to dense non-rigid structure from motion that recovers the camera motion and 3D shape of non-rigid objects by processing a monocular image sequence in an online fashion. We propose to model the time-varying shape with a probabilistic linear subspace of mode shapes obtained from continuum mechanics. To encode efficiently the deformations of dense 3D models, the dense rest shape is downsampled to a sparse mesh where modal analysis is applied at a low computational cost. The sparse mode shapes are subsequently grown back to dense. With this probabilistic low-rank constraint, we estimate camera pose and non-rigid shape in each frame using expectation maximization over a sliding window of frames. Since the time-varying weights are marginalized out, our approach only estimates a small number of parameters per frame, and hence can potentially run in real-time. We evaluate our algorithm on both synthetic and real sequences with 3D ground truth data for different objects ranging from sparse to dense shapes. We show the advantages of our approach with respect to competing methods.
FilesExtended Abstract (PDF, 1 page, 758K)
Paper (PDF, 12 pages, 7.8M)