Dynamic face reconstruction based on expression-driven tensorial radiance field

doi:10.3969/j.issn.1000-5641.2026.04.007

Abstract

Abstract:

This paper presents a dynamic face reconstruction method, which takes the tensorial radiance field as the basic network structure of scene expression, and proposes an implicit expression-driven approach based on a multi-layer neural network, extending the expressive capability of the tensorial radiance field to dynamic facial scenes, and optimizes the loss function to further improve the reconstruction effect. Compared with the current method, it can achieve dynamic face representation with higher speed and better quality. Qualitative and quantitative experimental results show that compared with the original methods, our method can save computing resources slightly, improve the training speed by about three times and the inference speed by about ten times, and maintain high-quality image reconstruction results.

Key words: face reconstruction, tensorial radiance field, volume rendering

CLC Number:

TP391

Mingyang ZHANG, Jincheng WENG, Yang LI. Dynamic face reconstruction based on expression-driven tensorial radiance field[J]. J* E* C* N* U* N* S*, 2026, 2026(4): 63-72.

Figures/Tables 7

Fig.1

Fig.2

Table 1

Table 2

Fig.3

Table 3

Fig.4

References 21

1	Hu L W, Ma C Y, Luo L J, et al.. Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics, 2015, 34 (4): 55.
2	Beeler T, Hahn F, Bradley D, et al.. High-quality passive facial performance capture using anchor frames. ACM Transactions on Graphics, 2011, 30 (4): 1- 10.
3	Gafni G, Thies J, Zollhöfer M, et al. Dynamic neural radiance fields for monocular 4D facial avatar reconstruction [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 8645-8654.
4	Grassal P W, Prinzler M, Leistner T, et al. Neural head avatars from monocular RGB videos [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 18653-18664.
5	Zheng Y F, Abrevaya V F, Buhler M C, et al. IMavatar: implicit morphable head avatars from videos [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 13545-13555.
6	Gao X, Zhong C L, Xiang J, et al.. Reconstructing personalized semantic facial NeRF models from monocular video. ACM Transactions on Graphics, 2022, 41 (6): 1- 12.
7	Xu Y L, Wang L Z, Zhao X C, et al. AvatarMAV: fast 3D head avatar reconstruction using motion-aware neural voxels [C]//Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. ACM, 2023: 1-10.
8	Chen C H, O’Toole M, Bharaj G, et al. Implicit neural head synthesis via controllable local deformation fields [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 416-426.
9	Guo Y D, Chen K Y, Liang S, et al. AD-NeRF: audio driven neural radiance fields for talking head synthesis [C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 5784-5794.
10	Liu X, Xu Y H, Wu Q Y, et al. Semantic-aware implicit neural audio-driven video portrait generation [C]//Computer Vision – ECCV 2022. 2022: 106-125.
11	Blanz V, Vetter T. A morphable model for the synthesis of 3D faces [C]//Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 1999: 187-194.
12	Chen A P, Xu Z X, Geiger A, et al. TensoRF: tensorial radiance fields [M]//Computer Vision – ECCV 2022. 2022: 333-350.
13	Mildenhall B, Srinivasan P P, Tancik M, et al. NeRF: representing scenes as neural radiance fields for view synthesis [C]//Computer Vision – ECCV 2020. 2020: 405-421.
14	Müller T, Evans A, Schied C, et al.. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics, 2022, 41 (4): 1- 15.
15	吉银帅, 续晋华.. 基于表面高度和不确定性的单目 3D 物体检测. 华东师范大学学报 (自然科学版), 2025 (1): 72- 81.
16	Ichim A E, Bouaziz S, Pauly M.. Dynamic 3D avatar creation from hand-held video input. ACM Transactions on Graphics, 2015, 34 (4): 45.
17	Drobyshev N, Chelishev J, Khakhulin T, et al. MegaPortraits: one-shot megapixel neural head avatars [C]//Proceedings of the 30th ACM International Conference on Multimedia. ACM, 2022: 2663-2671.
18	Zielonka W, Bolkart T, Thies J. Instant volumetric head avatars [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 4574-4584.
19	Li X, De Mello S, Liu S, et al. Generalizable one-shot 3D neural head avatar [C]//Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023: 47239-47250.
20	Thies J, Zollhöfer M, Stamminger M, et al. Face2Face: real-time face capture and reenactment of RGB videos [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 2387-2395.
21	Xie Z K, Yang X D, Yang Y J, et al. S3IM: stochastic structural SIMilarity and its unreasonable effectiveness for neural fields [C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 18024-18034.

方法	训练时间/min	渲染时间/s	参数量/(×10⁶)
INSTA	7	1.8	26.706
NeRFace	22	7.5	5.867
本文方法	6	0.45	5.609

数据集	方法	MAE	PSNR/dB	SSIM
数据集1	INSTA	9.63	22.89	0.86
	NeRFace	7.98	24.52	0.87
	本文方法	6.53	26.95	0.89
数据集2	INSTA	4.93	29.78	0.93
	NeRFace	3.99	32.01	0.94
	本文方法	3.10	33.32	0.94
数据集3	INSTA	5.16	27.75	0.90
	NeRFace	4.21	29.04	0.90
	本文方法	3.32	31.15	0.93

方法	MAE	PSNR/dB	SSIM
方法A	9.98	22.32	0.81
方法B	4.86	29.52	0.90
方法C	3.76	31.56	0.92
方法D	3.47	32.88	0.93
方法E (本文方法)	3.10	33.32	0.94

[1]	Ning CHEN, Rongsheng LIN, Cheng YUAN, Jin SHANG, Fan BAI, Dingjiang HUANG. Deep learning-based method for automatic verification of platen status in distribution cabinet [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 51-62.
[2]	Zezheng GONG, Fuke SHEN, Tongquan WEI. Automatic generation of use case diagrams from product requirement documents using large language models [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 143-153.
[3]	Wei YIN, Liyang YU. Transfer learning framework based on linguistic similarity for low-resource neural machine translation [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 134-142.
[4]	Zihao GONG, Zhiyun CHEN. FusionSeg: Topic segmentation model based on semantic feature fusion [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 123-133.
[5]	Ziyang XIE, Yan GAO. Spatial feature fusion and enhancement for remote sensing road extraction [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 112-122.
[6]	Xinyue LYU, Xinli HUANG. Cold-start problem of federated recommendation systems based on conditional variational autoencoder [J]. J* E* C* N* U* N* S*, 2025, 2025(6): 53-62.
[7]	Dexin HE, Fanyu HAN, Wei WANG. Application and evaluation of large language models in open source project topic annotation [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 14-24.
[8]	Mingdong YOU, Jiaheng PENG, Fanyu HAN, Wei WANG. A DTA based activity evaluation method for high star GitHub repositories [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 140-150.
[9]	Shengyu ZHAO, Wei WANG, Fanyu HAN, Jiaheng PENG, Lan YOU. Open source evaluatology: A framework and methodology for evaluating open source ecosystems based on evaluatology [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 151-161.
[10]	Xiaowei CHEN, Wei WANG, Fanyu HAN, Guanglei BAO, Fei DONG, Hao HUO, Chen LIU. OSS Insight: A platform for open source ecosystem spatiotemporal data analysis and insights [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 170-182.
[11]	Ruiyang PANG, Xuesong LU. Interactive data structure and algorithm visualization based on AI agents [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 32-42.
[12]	Linna XIE, Xuesong LU. Student employment prediction for digital jobs based on behavior in open-source communities [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 87-98.
[13]	Jiefang CHEN, Chang HUANG. Parallax image mosaic based on matching points and thin plate splines function model optimization [J]. J* E* C* N* U* N* S*, 2025, 2025(4): 15-27.
[14]	Xingxing XU, Chang HUANG. Container lock pins recognition based on a self-built dataset with high completeness [J]. J* E* C* N* U* N* S*, 2025, 2025(4): 28-37.
[15]	Zheng HU, Chang HUANG. A high robust fast video steganography algorithm in lossy channels [J]. J* E* C* N* U* N* S*, 2025, 2025(4): 38-48.