华东师范大学学报（自然科学版） >

2023 , Vol. 2023 >Issue 6: 39 - 48

DOI: https://doi.org/10.3969/j.issn.1000-5641.2023.06.004

计算机科学

基于双视图特征融合的糖尿病视网膜病变分级

  • 姜璐璐 ,
  • 孙司琦 ,
  • 邹海东 ,
  • 陆丽娜 ,
  • 冯瑞
展开
  • 1. 复旦大学 工程与应用技术研究院, 上海　200433
    2. 上海市眼科疾病精准诊疗工程技术研究中心,上海　200080
    3. 复旦大学 计算机科学技术学院 上海市智能信息处理重点实验室, 上海　200433
    4. 复旦大学 上海市智能视觉计算协同创新中心, 上海　200433
    5. 上海交通大学附属第一人民医院, 上海　200080
    6. 上海市眼病防治中心, 上海　200040

收稿日期: 2022-06-09

  网络出版日期: 2023-11-23

基金资助

国家自然科学基金 (62172101); 上海市科委项目 (19DZ2250100, 20DZ1100205)

收起

Diabetic retinopathy grading based on dual-view image feature fusion

  • Lulu JIANG ,
  • Siqi SUN ,
  • Haidong ZOU ,
  • Lina LU ,
  • Rui FENG
Expand
  • 1. Academy for Engineering and Technology, Fudan University, Shanghai　200433, China
    2. Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai　200080, China
    3. School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai　200433, China
    4. Shanghai Collaborative Innovation Center of Intelligent Visual Computing, Fudan University, Shanghai　200433, China
    5. Shanghai General Hospital, Shanghai　200080, China
    6. Shanghai Eye Disease Prevention Center, Shanghai　200040, China

Received date: 2022-06-09

  Online published: 2023-11-23

Fold

摘要

基于双视图眼底图像的诊断方法被广泛应用于糖尿病视网膜病变 (diabetic retinopathy, DR) 的筛查, 该方法可以有效地解决单视角下图像遮挡和视场受限的问题. 针对如何有效融合不同视图信息来提高DR分级准确率, 提出了一种基于注意力机制的多视角图像之间特征融合的学习方法. 针对眼底图像中病灶占比率较小的问题, 引入了自注意力机制以加强局部病灶特征的学习; 针对双视图眼底图像分类场景, 提出了一种跨视图注意力机制, 有效地利用了双视图之间的信息. 在内部数据集DFiD和公开数据集DeepDR上进行的实验, 验证了所提方法能够有效提高DR分级精度, 可用于大规模DR筛查, 辅助医生实现高效诊断.

关键词： 眼底图像; 特征融合; 双视图融合; 注意力机制; 糖尿病视网膜病变

本文引用格式

姜璐璐 , 孙司琦 , 邹海东 , 陆丽娜 , 冯瑞 . 基于双视图特征融合的糖尿病视网膜病变分级[J]. 华东师范大学学报（自然科学版）, 2023 , 2023(6) : 39 -48 . DOI: 10.3969/j.issn.1000-5641.2023.06.004

Abstract

The diagnostic method based on dual-view fundus imaging is widely used in diabetic retinopathy (DR) screening. This method effectively solves the problems of image occlusion and limited field of view under single-view. This paper proposes a learning method of feature fusion between dual-view images based on the attention mechanism to improve the accuracy of DR classification by effectively integrating different view information. Due to the small proportion of lesions in fundus images, the self-attention mechanism was introduced to enhance the learning of local lesion features. Moreover, a cross-attention mechanism is proposed to effectively utilize information between dual-view images to improve the classification of dual-view fundus images. Experiments were performed on the internal DFiD dataset and public DeepDRiD dataset. The proposed method can effectively improve the accuracy of DR classification and can be used for large-scale DR screening to assist doctors in achieving an efficient diagnosis.

Key words： fundus image; feature fusion; dual-view image fusion; attention mechanism; diabetic retinopathy

参考文献

1 LI Y Z, TENG D, SHI X G, et al.. Prevalence of diabetes recorded in mainland China using 2018 diagnostic criteria from the American Diabetes Association: National cross sectional study. BMJ(British Medical Journal), 2020, 369, m997.
2 VAN TULDER G, TONG Y, MARCHIORI E. Multi-view analysis of unregistered medical images using cross-view transformers [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention – MICCAI 2021, Lecture Notes in Computer Science, vol 12903. Cham: Springer, 2021: 104-113.
3 WU N, PHANG J, PARK J, et al.. Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Transactions on Medical Imaging, 2020, 39 (4): 1184- 1194.
4 WANG G T, ZHAI S W, LASIO G, et al.. Semi-supervised segmentation of radiation-induced pulmonary fibrosis from lung CT scans with multi-scale guided dense attention. IEEE Transactions on Medical Imaging, 2022, 41 (3): 531- 542.
5 WANG H Y, FENG J, ZHANG Z Z, et al.. Breast mass classification via deeply integrating the contextual information from multi-view data. Pattern Recognition, 2018, 80, 42- 52.
6 WANG Z, YIN Y X, SHI J P, et al. Zoom-in-net: Deep mining lesions for diabetic retinopathy detection [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention ? MICCAI 2017, Lecture Notes in Computer Science, vol 10435. Cham: Springer, 2017: 267-275.
7 LIN Z W, GUO R Q, WANG Y J, et al. A framework for identifying diabetic retinopathy based on anti-noise detection and attention-based fusion [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention – MICCAI 2018, Lecture Notes in Computer Science, vol 11071. Cham: Springer, 2018: 74-82.
8 ZHAO Z Y, ZHANG K R, HAO X J, et al. Bira-net: Bilinear attention net for diabetic retinopathy grading [C]// 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019: 1385-1389.
9 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6000–6010.
10 WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 7794-7803.
11 GENG Z Y, GUO M H, CHEN H X, et al. Is attention better than matrix decomposition? [C]// Proceedings of the 9th International Conference on Learning Representations. ICLR, 2021. https://openreview.net/forum?id=1FvkSpWosOl.
12 YUAN L, CHEN Y P, WANG T, et al. Tokens-to-token vit: Training vision transformers from scratch on imagenet [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 558-567.
13 WANG W H, XIE E Z, LI X, et al. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 548-558.
14 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 770-778.
15 PASZKE A, GROSS S, MASSA F, et al. Pytorch: An imperative style, high-performance deep learning library [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2019: 8026-8037.
16 Deep diabetic retinopathy [EB/OL]. (2022-06-01)[2022-05-01]. https://github.com/deepdrdoc/DeepDRiD.
17 HE J J, LI C, YE J, et al. Classification of ocular diseases employing attention-based unilateral and bilateral feature weighting and fusion [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020: 1258-1261.
18 LI C, YE J, HE J J, et al. Dense correlation network for automated multi-label ocular disease detection with paired color fundus photographs [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020: 1250-1253.
19 SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 618-626.
Options
文章导航

/

〈 〉