Dyna: A Model of Dynamic Human Shape in Motion
To look human, digital full-body avatars need to have soft tissue deformations like those of real people. We learn a model of soft-tissue deformations from examples using a high-resolution 4D capture system and a method that accurately registers a template mesh to sequences of 3D scans. Using over 40,000 scans of ten subjects, we learn how soft tissue motion causes mesh triangles to deform relative to a base 3D body model. Our Dyna model uses a low-dimensional linear subspace to approximate soft-tissue deformation and relates the subspace coefficients to the changing pose of the body. Dyna uses a second-order auto-regressive model that predicts soft-tissue deformations based on previous deformations, the velocity and acceleration of the body, and the angular velocities and accelerations of the limbs. Dyna also models how deformations vary with a person’s body mass index (BMI), producing different deformations for people with different shapes. Dyna realistically represents the dynamics of soft tissue for previously unseen subjects and motions. We provide tools for animators to modify the deformations and apply them to new stylized characters.
This video shows all the data used to train our Dyna model of human soft-tissue motion — all 40,000 meshes. It is not the most thrilling video but there is something remarkable about watching how human soft tissue really moves.
We capture raw 3D scans at 60 frames per second using a custom 3dMD active stereo scanner. We then precisely register a common template mesh to all the scans in a process we call 4Cap – for 4D motion capture. This is like having really dense mocap that captures the subtle details of how the body moves. The video shows these registered meshes. Once all the meshes are in correspondence like this, we can model soft-tissue motions in our Dyna model.
SIGGRAPH 2015 Paper Video:
Pons-Moll, G., Romero, J., Mahmood, N. and Black, M.J.,
ACM Transactions on Graphics, (Proc. SIGGRAPH), 2015.