Towards an identity inter-relationship-consistent face de-identification method

doi:10.3969/j.issn.1000-5641.2023.06.005

Abstract

Abstract:

The popularity of intelligent devices such as smartphones and surveillance cameras has led to serious face privacy problems. Face de-identification is considered an effective tool for protecting face privacy by concealing identity information. However, most de-identification methods lack explicit control and controllable changes in identifying de-identified face images, resulting in de-identified images that are inapplicable to face authentication and retrieval and other identity-related tasks. Therefore, this study proposes an identity inter-relationship-consistent face de-identification task in which the identity inter-relationship between two arbitrary de-identified faces maintained the same as before de-identification. To this end, a task-driven identity inter-relationship consistent generative adversarial network is introduced to generate de-identified faces with a consistent identity inter-relationship. A rotation-based de-identifier was designed to modify the original identity features to be de-identified with identity inter-relationship consistency. In addition, identity control loss is introduced to guarantee a precise identity generation using a de-identified generator. Qualitative and quantitative results show that our method achieves improvements compared with exiting methods for de-identifying de-identified faces as well as for maintaining their identity inter-relationship consistent.

Key words: privacy protection, face de-identification, generative adversarial network, identity inter-relationship consistency

CLC Number:

TP391

Yifan BU, Xiaoling WANG, Keke HE, Xingjian LU, Wenxuan WANG. Towards an identity inter-relationship-consistent face de-identification method[J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 49-60.

Figures/Tables 10

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Fig.4

Fig.5

Table 5

References 25

1	NEWTON E M, SWEENEY L, MALIN B A.. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering(TKDE), 2005, 17 (2): 232- 243.
2	LI Y F, NISHANT V, BART P K, et al. Blur vs. block: Investigating the effectiveness of privacy-enhancing obfuscation for images [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR workshops). IEEE , 2017: 1343-1351.
3	CHEN R, CHEN X, NI B, et al. SimSwap: An efficient framework for high fidelity face swapping [C]// 2020 ACM International Conference on Multimedia. ACM, 2020: 2003-2011.
4	XU Z L, YU X Y, HONG Z B, et al. FaceController: Controllable attribute editing for face in the wild [C]// 2021 AAAI Conference on Artificial Intelligence. AAAI , 2021: 3083-3091.
5	WANG Y H, CHEN X, ZHU J W, et al. HifiFace: 3D shape and semantic prior guided high fidelity face swapping [C]// 2021 Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI). IJCAI, 2021: 1136-1142.
6	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// NIPS 14: Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. Cambridge, MA USA: MIT Press, 2014: 2672-2680.
7	WU Y F, YANG F, XU Y, et al.. Privacy-protective-GAN for privacy preserving face de-identification. Journal of Computer Science Technology, 2019, 34 (1): 47- 60.
8	GAFNI O, WOLF L, TAIGMAN Y. Live face de-identification in video [C]// 2019 IEEE International Conference on Computer Vision (ICCV). IEEE, 2019: 9377-9386.
9	MAXIMOV M, ELEZI I, LEAL-TAIXÉ L. CIAGAN: Conditional identity anonymization generative adversarial networks [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 5446-5455.
10	GONG M G, LIU J, LI H, et al.. Disentangled representation learning for multiple attributes preserving face deidentification. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (1): 244- 256.
11	GU X Y, LUO W X, MICHAEL S, et al. Password-conditioned anonymization and deanonymization with face identity transformers [C]// European Conference on Computer Vision, Computer Vision – ECCV 2020. Cham: Springer, 2020: 727-743.
12	PROENÇA H. The UU-Net: Reversible face de-identification for visual surveillance video footage [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(2): 496-509.
13	GROSS R, SWEENEY, BAKER S, et al. Semi-supervised learning of multi-factor models for face de-identification [C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2008. DOI: 10.1109/CVPR.2008.4587369. https://ieeexplore.ieee.org/document/4587369
14	TOLOSANA R, VERA R R, FIERREZ J.. Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 2020, 64 (1): 131- 148.
15	ZHANG K P, ZHANG Z P, LI Z F, et al.. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23 (10): 1499- 1503.
16	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 2242-2251.
17	OLAF R, PHILIPP F, THOMAS B. U-Net: Convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.
18	ZHOU W, ALAN C B, HAMID R S, et al.. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Process, 2004, 13 (4): 600- 612.
19	DENG J, GUO J, XUE N, et al. Arcface: Additive angular margin loss for deep face recognition [C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019: 4690-4699.
20	LIU Z, LUO P, WANG X, et al. Deep learning face attributes in the wild [C]// 2015 IEEE International Conference on Computer Vision (ICCV). IEEE, 2015: 3730-3738.
21	KARL R, TAMIRAT T.. MORPH: A longitudinal image database of normal adult age-progression. IEEE International Conference on Automatic Face and Gesture (FGR), 2006, 1 (1): 341- 345.
22	HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments [EB/OL]. [2022-04-05]. http://vis-www.cs.umass.edu/papers/lfw.pdf.
23	GUO Y D, ZHANG L, HU Y X, et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition [C]// European Conference on Computer Vision (ECCV), Computer Vision – ECCV 2016. Cham: Springer, 2016: 87-102.
24	FLORIAN S, DMITRY K, JAMES P. FaceNet: A unified embedding for face recognition and clustering [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015: 815-823.
25	VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE [J]. Journal of Machine Learning Research, 2008(9): 2579-2605.

人脸检测器	检测率/%
人脸检测器	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
MTCNN	100	90.6	0	97.4	97.8	98.7
Dlib	100	95.5	22.2	95.9	96.6	97.9

人脸识别器	去识别化率(tpr@0.001)
人脸识别器	原始图像	Gafni	CIAGAN	IIRC-GAN
FaceNet-VGGFace2	0.986 $ \pm $ 0.010	0.038 $ \pm $ 0.015	0.034 $ \pm $ 0.016	0.027 $\boldsymbol{\pm}$ 0.008
FaceNet-CASIA	0.965 $ \pm $ 0.016	0.035 $ \pm $ 0.011	0.019 $\boldsymbol{ \pm }$ 0.008	0.027 $ \pm $ 0.014

数据集	人脸识别器	验证精度/%
数据集	人脸识别器	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
CelebA	FaceNet	96.34	61.10	49.68	76.85	77.18	94.28
CelebA	InsightFace	94.25	60.55	60.02	77.49	80.01	84.85
Morph	FaceNet	99.88	57.13	50.08	94.08	89.73	99.64
Morph	InsightFace	97.75	74.80	73.53	86.69	85.79	92.83

指标	准确率/%
指标	原始图像	模糊化	像素化	PPGAN	CIAGAN	IIRC-GAN
top-1	94.34	20.28	11.99	59.12	53.62	91.18
top-5	96.62	37.19	23.77	76.64	74.90	96.01
top-10	97.44	52.05	35.24	84.53	84.15	97.13

评测指标	基准模型	$+{L}_{\rm{ID\_fake} }$	$+{L}_{\rm{ID\_deID}}$	${ + L}_{\rm{ID\_fake}} + {L}_{\rm{ID\_deID}}$	IIRC-GAN
验证准确度/%	63.43	84.49	85.29	93.37	94.36