Microsoft is working hard on improving realistic face animation as the company’s researchers are developing some of the most advanced 3D face-capturing technology to date. Microsoft hopes this improved technology makes a big impact in the video game and film industry.
“We are very familiar with facial expressions, but also very sensitive in seeing any type of errors. That means we need to capture facial expressions with a high level of detail and also capture very subtle facial details with high temporal resolution,” Microsoft scientist Xin Tong stated.
According to Tong, Microsoft is working on realistic face animation since human facial expressions are the “holy grail” of computer graphics. Tong’s group of researchers are working on a new method to capture not just the main features of different expressions, but also in-depth details such as skin wrinkles and pores.
Using three actors with highly mobile faces, the researchers first used marker-based motion capture, applying about 100 reflective dots to each actor’s face. With video rolling, the actors made a series of pre-determined facial expressions that enabled the collection of rough data on how faces change with different expressions, for use in 3-D scans.
Top images show how markers create a set of correspondences across all the face scans. The bottom row shows how a two-step face-scan registration produces dense, consistent surface correspondences across all the face scans.
Also, by analyzing the captured marker-based data, the team determined the minimum number of scans required for accurate facial reconstruction.In the next step, the research team used a laser scanner to capture high-fidelity facial scans. Those scans then were aligned with the corresponding frames in the marker-based facial data. Using a new algorithm, the facial scans were registered with each other.
Lastly, the team combined the motion-capture information with the face scans to reconstruct the actual expressions as they were performed. The resulting images capture both the “big” movements of a face and fine details such as the texture and movement of the skin.
Microsoft is continuing to work on this technology as the company has yet to master the capture of eye and lip movement. You can read the full article at Microsoft Research here.
Further reading: Microsoft
