Chinese scientists enable more realistic digital humans by building high-precision 3D facial database

SHENZHEN -- Chinese scientists have developed a high-precision three-dimensional (3D) face database and achieved a breakthrough in personalized modeling, which will strongly support more natural human-computer interaction.

To enable virtual humans to express vivid emotions, recognize human identities and demonstrate embodied intelligence, the key technology is 3D facial keypoint detection.

However, the lack of large-scale and precisely annotated 3D facial datasets means that most current 3D facial landmark detection algorithms rely on 2D texture assistance or non-photorealistic digital 3D faces. Such 3D facial keypoint detection has long been hindered by insufficient data and poor generalization ability.

A research team from the Shenzhen Institutes of Advanced Technology of the Chinese Academy of Sciences and Fujian University of Technology has developed a new curvature-fused graph attention network (CF-GAT) capable of predicting facial landmarks directly from raw point clouds, which helps achieve an essential improvement from "one-size-fits-all" to personalized modeling.

This study was published last week in the journal IEEE Transactions on Circuits and Systems for Video Technology.

The research team built a custom 3D/4D facial acquisition system and conducted standardized data collection, creating what it said is the industry's largest high-precision, accurately annotated 3D facial database to date, comprising approximately 200,000 high-fidelity 3D facial scans.

On this basis, the database system also includes a multi-expression 3D face dataset, a standardized 3D facial landmark dataset, a high-precision 3D human body dataset, and a dynamic 4D facial expression dataset.

"These databases have become core support in the key technology chain of humanoid robots, providing basic data for high-fidelity perception, expression modeling and behavior generation," the corresponding author Song Zhan said.

"In the future, these datasets will further serve the data-driven large-model humanoid robot system to build more natural and intelligent human-robot interaction capabilities," he added.