基于表情驱动的张量辐射场的动态人脸重建

doi:10.3969/j.issn.1000-5641.2026.04.007

摘要/Abstract

摘要：

提出了一种动态人脸的重建方法, 将张量辐射场作为场景表达的基础网络结构, 并提出了一种基于多层神经网络的隐式表情驱动方式, 将张量辐射场的表达能力拓展到人脸动态场景中, 同时优化损失函数, 进一步提升了重建效果. 与已有的方法相比, 能够实现速度更快、质量更好的动态人脸的表示. 定性与定量的实验结果表明, 相对于已有方法, 本文方法能够节省计算资源, 提升了训练速度以及推理速度, 并维持高质量的图像重建效果.

关键词: 人脸重建, 张量辐射场, 体渲染

Abstract:

This paper presents a dynamic face reconstruction method, which takes the tensorial radiance field as the basic network structure of scene expression, and proposes an implicit expression-driven approach based on a multi-layer neural network, extending the expressive capability of the tensorial radiance field to dynamic facial scenes, and optimizes the loss function to further improve the reconstruction effect. Compared with the current method, it can achieve dynamic face representation with higher speed and better quality. Qualitative and quantitative experimental results show that compared with the original methods, our method can save computing resources slightly, improve the training speed by about three times and the inference speed by about ten times, and maintain high-quality image reconstruction results.

Key words: face reconstruction, tensorial radiance field, volume rendering

中图分类号:

TP391

张铭杨, 翁锦程, 李洋. 基于表情驱动的张量辐射场的动态人脸重建[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 63-72.

Mingyang ZHANG, Jincheng WENG, Yang LI. Dynamic face reconstruction based on expression-driven tensorial radiance field[J]. J* E* C* N* U* N* S*, 2026, 2026(4): 63-72.

图/表 7

图1

图2

表1

表2

图3

表3

图4

参考文献 21

1	Hu L W, Ma C Y, Luo L J, et al.. Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics, 2015, 34 (4): 55.
2	Beeler T, Hahn F, Bradley D, et al.. High-quality passive facial performance capture using anchor frames. ACM Transactions on Graphics, 2011, 30 (4): 1- 10.
3	Gafni G, Thies J, Zollhöfer M, et al. Dynamic neural radiance fields for monocular 4D facial avatar reconstruction [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 8645-8654.
4	Grassal P W, Prinzler M, Leistner T, et al. Neural head avatars from monocular RGB videos [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 18653-18664.
5	Zheng Y F, Abrevaya V F, Buhler M C, et al. IMavatar: implicit morphable head avatars from videos [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 13545-13555.
6	Gao X, Zhong C L, Xiang J, et al.. Reconstructing personalized semantic facial NeRF models from monocular video. ACM Transactions on Graphics, 2022, 41 (6): 1- 12.
7	Xu Y L, Wang L Z, Zhao X C, et al. AvatarMAV: fast 3D head avatar reconstruction using motion-aware neural voxels [C]//Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. ACM, 2023: 1-10.
8	Chen C H, O’Toole M, Bharaj G, et al. Implicit neural head synthesis via controllable local deformation fields [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 416-426.
9	Guo Y D, Chen K Y, Liang S, et al. AD-NeRF: audio driven neural radiance fields for talking head synthesis [C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 5784-5794.
10	Liu X, Xu Y H, Wu Q Y, et al. Semantic-aware implicit neural audio-driven video portrait generation [C]//Computer Vision – ECCV 2022. 2022: 106-125.
11	Blanz V, Vetter T. A morphable model for the synthesis of 3D faces [C]//Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 1999: 187-194.
12	Chen A P, Xu Z X, Geiger A, et al. TensoRF: tensorial radiance fields [M]//Computer Vision – ECCV 2022. 2022: 333-350.
13	Mildenhall B, Srinivasan P P, Tancik M, et al. NeRF: representing scenes as neural radiance fields for view synthesis [C]//Computer Vision – ECCV 2020. 2020: 405-421.
14	Müller T, Evans A, Schied C, et al.. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics, 2022, 41 (4): 1- 15.
15	吉银帅, 续晋华.. 基于表面高度和不确定性的单目 3D 物体检测. 华东师范大学学报 (自然科学版), 2025 (1): 72- 81.
16	Ichim A E, Bouaziz S, Pauly M.. Dynamic 3D avatar creation from hand-held video input. ACM Transactions on Graphics, 2015, 34 (4): 45.
17	Drobyshev N, Chelishev J, Khakhulin T, et al. MegaPortraits: one-shot megapixel neural head avatars [C]//Proceedings of the 30th ACM International Conference on Multimedia. ACM, 2022: 2663-2671.
18	Zielonka W, Bolkart T, Thies J. Instant volumetric head avatars [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 4574-4584.
19	Li X, De Mello S, Liu S, et al. Generalizable one-shot 3D neural head avatar [C]//Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023: 47239-47250.
20	Thies J, Zollhöfer M, Stamminger M, et al. Face2Face: real-time face capture and reenactment of RGB videos [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 2387-2395.
21	Xie Z K, Yang X D, Yang Y J, et al. S3IM: stochastic structural SIMilarity and its unreasonable effectiveness for neural fields [C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 18024-18034.

方法	训练时间/min	渲染时间/s	参数量/(×10⁶)
INSTA	7	1.8	26.706
NeRFace	22	7.5	5.867
本文方法	6	0.45	5.609

数据集	方法	MAE	PSNR/dB	SSIM
数据集1	INSTA	9.63	22.89	0.86
	NeRFace	7.98	24.52	0.87
	本文方法	6.53	26.95	0.89
数据集2	INSTA	4.93	29.78	0.93
	NeRFace	3.99	32.01	0.94
	本文方法	3.10	33.32	0.94
数据集3	INSTA	5.16	27.75	0.90
	NeRFace	4.21	29.04	0.90
	本文方法	3.32	31.15	0.93

方法	MAE	PSNR/dB	SSIM
方法A	9.98	22.32	0.81
方法B	4.86	29.52	0.90
方法C	3.76	31.56	0.92
方法D	3.47	32.88	0.93
方法E (本文方法)	3.10	33.32	0.94

[1]	陈宁, 林荣胜, 袁成, 尚瑨, 白帆, 黄定江. 基于深度学习的配电柜压板状态自动核对方法[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 51-62.
[2]	宫泽正, 沈富可, 魏同权. 基于大语言模型的测试用例图谱生成研究[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 143-153.
[3]	殷炜, 俞黎阳. 基于语言相似度的低资源机器翻译迁移学习框架[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 134-142.
[4]	龚子豪, 陈志云. FusionSeg: 基于语义特征融合的主题分割模型[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 123-133.
[5]	谢子阳, 高岩. 结合空间特征融合与特征增强的遥感道路提取[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 112-122.
[6]	吕欣樾, 黄新力. 基于条件变分自编码器的联邦推荐系统冷启动问题研究[J]. 华东师范大学学报（自然科学版）, 2025, 2025(6): 53-62.
[7]	何德鑫, 韩凡宇, 王伟. 大语言模型在开源项目主题标注中的应用与评估研究[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 14-24.
[8]	游明东, 彭佳恒, 韩凡宇, 王伟. 基于DTA的GitHub高星仓库活跃度评估方法[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 140-150.
[9]	赵生宇, 王伟, 韩凡宇, 彭佳恒, 游兰. 开源评价学: 基于评价学理论的开源生态系统评价框架与方法[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 151-161.
[10]	陈小伟, 王伟, 韩凡宇, 包光磊, 董菲, 霍昊, 刘辰. OSS Insight: 开源生态时空数据分析和智能洞察平台[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 170-182.
[11]	庞瑞洋, 陆雪松. 基于智能体的可交互数据结构和算法可视化实现[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 32-42.
[12]	谢林娜, 陆雪松. 基于学生开源社区行为的数字岗位就业预测[J]. 华东师范大学学报（自然科学版）, 2025, 2025(5): 87-98.
[13]	陈洁芳, 黄昶. 基于匹配点和薄板样条函数模型优化的视差图像拼接[J]. 华东师范大学学报（自然科学版）, 2025, 2025(4): 15-27.
[14]	徐星星, 黄昶. 基于高完备性自建数据集的集装箱锁销识别[J]. 华东师范大学学报（自然科学版）, 2025, 2025(4): 28-37.
[15]	胡正, 黄昶. 有损通道中高鲁棒性的快速视频隐写算法[J]. 华东师范大学学报（自然科学版）, 2025, 2025(4): 38-48.