Date of Award
8-2025
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Computer Science
Committee Chair/Advisor
Dr. Eric Patterson
Committee Member
Dr. Daljit Singh Dhillon
Committee Member
Dr. Matias Volonte
Committee Member
Dr. Joseph T. Kider Jr.
Abstract
Creating re-topologized 3D facial meshes is a critical step in high-quality facial animation pipelines, yet it remains a labor-intensive and time-consuming task. Traditional approaches typically rely on multiview stereo reconstruction and specialized photometric environments to acquire accurate geometric and reflectance data under controlled conditions. This dissertation presents work toward more efficient capture of production-ready meshes including (1) developmental aspects of VarIS, a custom-designed light sphere capable of capturing high-resolution stereo geometry and reflectance maps—including diffuse, specular, and normal components under programmable illumination; (2) a study of the effects of camera parameters on automatic 2D and 3D landmarking methods, (3) methods for using synthetic data to train neural face regression, and (4) techniques proposed to improve neural multi-view regression of face shape.
While VarIS enables photorealistic face capture, its operational cost and the need for manual processing of its acquired data highlight the need for a more scalable solution. To address this, a deep learning–based framework is proposed, enabling direct prediction of re-topologized facial meshes from synthetic multiview images. Training data was generated using Visage Craft, an in-house rendering system built upon a physically based Appearance 3D Morphable Model (A3DMM). The method infers dense mesh geometry in a standardized format ready to rig and animate.
Results demonstrate that integrating precise camera intrinsics and extrinsics during training markedly improves landmark accuracy and geometric consistency, and incorporating 3D landmarks themselves in the regularization of the network also improves results. The final system presents a robust, data-driven alternative to conventional face analysis/synthesis workflows, capable of producing facial meshes with minimal human supervision.
Recommended Citation
Li, Xiang, "Multiple View Neural Regression of a Facial Shape Model" (2025). All Dissertations. 3969.
https://open.clemson.edu/all_dissertations/3969