Virtual markers help smartphone cameras reconstruct lifelike multi-person movement, from hugs to martial arts

For decades, the gold standard for capturing human movement and transferring these data into the digital world has required actors to wear suits with physical markers that cameras track in a specialized studio. While accurate, this capture process is time-consuming and expensive. Markers need to be placed manually, they can fall off, and if two people get too close—such as during a hug or a dance—the system fails to track all markers as one person blocks the other from the camera’s view, and accuracy drops significantly. Extensive post-processing is needed to recover the motion in such challenging sequences. The creators of the Hollywood blockbusters “Avatar” or “The Lord of the Rings” will know what we mean.