面向身份相互关系一致性的人脸去识别化方法

doi:10.3969/j.issn.1000-5641.2023.06.005

摘要/Abstract

摘要：

智能手机、监控摄像头等智能设备的普及导致了严重的人脸隐私安全问题. 人脸去识别化技术被认为是解决上述问题的有效途径, 该技术旨在将人脸图像中的身份信息去除以有效保护人脸的隐私. 然而, 现有的研究工作大多缺乏对去识别化人脸图像在身份上显式的控制与可控的变化, 导致去识别化后的图像无法应用于人脸验证和检索等与身份相关的任务. 因此, 考虑到去识别化人脸图像在身份相关任务上的可用性, 提出了一种新的面向身份相互关系一致性的人脸去识别化任务, 即任意两个被去识别化的人脸之间的身份相互关系与去识别化前的人脸保持一致. 为解决这一问题, 引入了一种任务驱动的身份相互关系一致性生成对抗网络算法. 该方法设计了一种基于特征向量旋转的去识别化模块, 将原有的身份特征可控修改为具有身份相互关系一致性的去识别化特征. 此外, 还引入了身份控制损失, 以保证去识别化后生成的人脸身份的准确性. 大量的定性和定量结果表明, 提出的方法在人脸的去识别化和去识别人脸的身份相互关系一致性上的表现远优于现有的方法.

关键词: 隐私保护, 人脸去识别化, 生成对抗网络, 身份相互关系一致性

Abstract:

The popularity of intelligent devices such as smartphones and surveillance cameras has led to serious face privacy problems. Face de-identification is considered an effective tool for protecting face privacy by concealing identity information. However, most de-identification methods lack explicit control and controllable changes in identifying de-identified face images, resulting in de-identified images that are inapplicable to face authentication and retrieval and other identity-related tasks. Therefore, this study proposes an identity inter-relationship-consistent face de-identification task in which the identity inter-relationship between two arbitrary de-identified faces maintained the same as before de-identification. To this end, a task-driven identity inter-relationship consistent generative adversarial network is introduced to generate de-identified faces with a consistent identity inter-relationship. A rotation-based de-identifier was designed to modify the original identity features to be de-identified with identity inter-relationship consistency. In addition, identity control loss is introduced to guarantee a precise identity generation using a de-identified generator. Qualitative and quantitative results show that our method achieves improvements compared with exiting methods for de-identifying de-identified faces as well as for maintaining their identity inter-relationship consistent.

Key words: privacy protection, face de-identification, generative adversarial network, identity inter-relationship consistency

中图分类号:

TP391

步一凡, 王晓玲, 贺珂珂, 卢兴见, 王文萱. 面向身份相互关系一致性的人脸去识别化方法[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 49-60.

Yifan BU, Xiaoling WANG, Keke HE, Xingjian LU, Wenxuan WANG. Towards an identity inter-relationship-consistent face de-identification method[J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 49-60.

图/表 10

图1

图2

图3

表1

表2

表3

表4

图4

图5

表5

参考文献 25

1	NEWTON E M, SWEENEY L, MALIN B A.. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering(TKDE), 2005, 17 (2): 232- 243.
2	LI Y F, NISHANT V, BART P K, et al. Blur vs. block: Investigating the effectiveness of privacy-enhancing obfuscation for images [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR workshops). IEEE , 2017: 1343-1351.
3	CHEN R, CHEN X, NI B, et al. SimSwap: An efficient framework for high fidelity face swapping [C]// 2020 ACM International Conference on Multimedia. ACM, 2020: 2003-2011.
4	XU Z L, YU X Y, HONG Z B, et al. FaceController: Controllable attribute editing for face in the wild [C]// 2021 AAAI Conference on Artificial Intelligence. AAAI , 2021: 3083-3091.
5	WANG Y H, CHEN X, ZHU J W, et al. HifiFace: 3D shape and semantic prior guided high fidelity face swapping [C]// 2021 Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI). IJCAI, 2021: 1136-1142.
6	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// NIPS 14: Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. Cambridge, MA USA: MIT Press, 2014: 2672-2680.
7	WU Y F, YANG F, XU Y, et al.. Privacy-protective-GAN for privacy preserving face de-identification. Journal of Computer Science Technology, 2019, 34 (1): 47- 60.
8	GAFNI O, WOLF L, TAIGMAN Y. Live face de-identification in video [C]// 2019 IEEE International Conference on Computer Vision (ICCV). IEEE, 2019: 9377-9386.
9	MAXIMOV M, ELEZI I, LEAL-TAIXÉ L. CIAGAN: Conditional identity anonymization generative adversarial networks [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 5446-5455.
10	GONG M G, LIU J, LI H, et al.. Disentangled representation learning for multiple attributes preserving face deidentification. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (1): 244- 256.
11	GU X Y, LUO W X, MICHAEL S, et al. Password-conditioned anonymization and deanonymization with face identity transformers [C]// European Conference on Computer Vision, Computer Vision – ECCV 2020. Cham: Springer, 2020: 727-743.
12	PROENÇA H. The UU-Net: Reversible face de-identification for visual surveillance video footage [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(2): 496-509.
13	GROSS R, SWEENEY, BAKER S, et al. Semi-supervised learning of multi-factor models for face de-identification [C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2008. DOI: 10.1109/CVPR.2008.4587369. https://ieeexplore.ieee.org/document/4587369
14	TOLOSANA R, VERA R R, FIERREZ J.. Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 2020, 64 (1): 131- 148.
15	ZHANG K P, ZHANG Z P, LI Z F, et al.. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23 (10): 1499- 1503.
16	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 2242-2251.
17	OLAF R, PHILIPP F, THOMAS B. U-Net: Convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.
18	ZHOU W, ALAN C B, HAMID R S, et al.. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Process, 2004, 13 (4): 600- 612.
19	DENG J, GUO J, XUE N, et al. Arcface: Additive angular margin loss for deep face recognition [C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019: 4690-4699.
20	LIU Z, LUO P, WANG X, et al. Deep learning face attributes in the wild [C]// 2015 IEEE International Conference on Computer Vision (ICCV). IEEE, 2015: 3730-3738.
21	KARL R, TAMIRAT T.. MORPH: A longitudinal image database of normal adult age-progression. IEEE International Conference on Automatic Face and Gesture (FGR), 2006, 1 (1): 341- 345.
22	HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments [EB/OL]. [2022-04-05]. http://vis-www.cs.umass.edu/papers/lfw.pdf.
23	GUO Y D, ZHANG L, HU Y X, et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition [C]// European Conference on Computer Vision (ECCV), Computer Vision – ECCV 2016. Cham: Springer, 2016: 87-102.
24	FLORIAN S, DMITRY K, JAMES P. FaceNet: A unified embedding for face recognition and clustering [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015: 815-823.
25	VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE [J]. Journal of Machine Learning Research, 2008(9): 2579-2605.

人脸检测器	检测率/%
人脸检测器	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
MTCNN	100	90.6	0	97.4	97.8	98.7
Dlib	100	95.5	22.2	95.9	96.6	97.9

人脸识别器	去识别化率(tpr@0.001)
人脸识别器	原始图像	Gafni	CIAGAN	IIRC-GAN
FaceNet-VGGFace2	0.986 $ \pm $ 0.010	0.038 $ \pm $ 0.015	0.034 $ \pm $ 0.016	0.027 $\boldsymbol{\pm}$ 0.008
FaceNet-CASIA	0.965 $ \pm $ 0.016	0.035 $ \pm $ 0.011	0.019 $\boldsymbol{ \pm }$ 0.008	0.027 $ \pm $ 0.014

数据集	人脸识别器	验证精度/%
数据集	人脸识别器	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
CelebA	FaceNet	96.34	61.10	49.68	76.85	77.18	94.28
CelebA	InsightFace	94.25	60.55	60.02	77.49	80.01	84.85
Morph	FaceNet	99.88	57.13	50.08	94.08	89.73	99.64
Morph	InsightFace	97.75	74.80	73.53	86.69	85.79	92.83

指标	准确率/%
指标	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
top-1	94.34	20.28	11.99	59.12	53.62	91.18
top-5	96.62	37.19	23.77	76.64	74.90	96.01
top-10	97.44	52.05	35.24	84.53	84.15	97.13

评测指标	基准模型	$+{L}_{\rm{ID\_fake} }$	$+{L}_{\rm{ID\_deID}}$	${ + L}_{\rm{ID\_fake}} + {L}_{\rm{ID\_deID}}$	IIRC-GAN
验证准确度/%	63.43	84.49	85.29	93.37	94.36