Journal of East China Normal University(Natural Science) >

2023 , Vol. 2023 >Issue 6: 39 - 48

DOI: https://doi.org/10.3969/j.issn.1000-5641.2023.06.004

Computer Science

Diabetic retinopathy grading based on dual-view image feature fusion

  • Lulu JIANG ,
  • Siqi SUN ,
  • Haidong ZOU ,
  • Lina LU ,
  • Rui FENG
Expand
  • 1. Academy for Engineering and Technology, Fudan University, Shanghai　200433, China
    2. Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai　200080, China
    3. School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai　200433, China
    4. Shanghai Collaborative Innovation Center of Intelligent Visual Computing, Fudan University, Shanghai　200433, China
    5. Shanghai General Hospital, Shanghai　200080, China
    6. Shanghai Eye Disease Prevention Center, Shanghai　200040, China

Received date: 2022-06-09

  Online published: 2023-11-23

Fold

Abstract

The diagnostic method based on dual-view fundus imaging is widely used in diabetic retinopathy (DR) screening. This method effectively solves the problems of image occlusion and limited field of view under single-view. This paper proposes a learning method of feature fusion between dual-view images based on the attention mechanism to improve the accuracy of DR classification by effectively integrating different view information. Due to the small proportion of lesions in fundus images, the self-attention mechanism was introduced to enhance the learning of local lesion features. Moreover, a cross-attention mechanism is proposed to effectively utilize information between dual-view images to improve the classification of dual-view fundus images. Experiments were performed on the internal DFiD dataset and public DeepDRiD dataset. The proposed method can effectively improve the accuracy of DR classification and can be used for large-scale DR screening to assist doctors in achieving an efficient diagnosis.

Key words： fundus image; feature fusion; dual-view image fusion; attention mechanism; diabetic retinopathy

Cite this article

Lulu JIANG , Siqi SUN , Haidong ZOU , Lina LU , Rui FENG . Diabetic retinopathy grading based on dual-view image feature fusion[J]. Journal of East China Normal University(Natural Science), 2023 , 2023(6) : 39 -48 . DOI: 10.3969/j.issn.1000-5641.2023.06.004

References

1 LI Y Z, TENG D, SHI X G, et al.. Prevalence of diabetes recorded in mainland China using 2018 diagnostic criteria from the American Diabetes Association: National cross sectional study. BMJ(British Medical Journal), 2020, 369, m997.
2 VAN TULDER G, TONG Y, MARCHIORI E. Multi-view analysis of unregistered medical images using cross-view transformers [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention – MICCAI 2021, Lecture Notes in Computer Science, vol 12903. Cham: Springer, 2021: 104-113.
3 WU N, PHANG J, PARK J, et al.. Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Transactions on Medical Imaging, 2020, 39 (4): 1184- 1194.
4 WANG G T, ZHAI S W, LASIO G, et al.. Semi-supervised segmentation of radiation-induced pulmonary fibrosis from lung CT scans with multi-scale guided dense attention. IEEE Transactions on Medical Imaging, 2022, 41 (3): 531- 542.
5 WANG H Y, FENG J, ZHANG Z Z, et al.. Breast mass classification via deeply integrating the contextual information from multi-view data. Pattern Recognition, 2018, 80, 42- 52.
6 WANG Z, YIN Y X, SHI J P, et al. Zoom-in-net: Deep mining lesions for diabetic retinopathy detection [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention ? MICCAI 2017, Lecture Notes in Computer Science, vol 10435. Cham: Springer, 2017: 267-275.
7 LIN Z W, GUO R Q, WANG Y J, et al. A framework for identifying diabetic retinopathy based on anti-noise detection and attention-based fusion [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention – MICCAI 2018, Lecture Notes in Computer Science, vol 11071. Cham: Springer, 2018: 74-82.
8 ZHAO Z Y, ZHANG K R, HAO X J, et al. Bira-net: Bilinear attention net for diabetic retinopathy grading [C]// 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019: 1385-1389.
9 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6000–6010.
10 WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 7794-7803.
11 GENG Z Y, GUO M H, CHEN H X, et al. Is attention better than matrix decomposition? [C]// Proceedings of the 9th International Conference on Learning Representations. ICLR, 2021. https://openreview.net/forum?id=1FvkSpWosOl.
12 YUAN L, CHEN Y P, WANG T, et al. Tokens-to-token vit: Training vision transformers from scratch on imagenet [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 558-567.
13 WANG W H, XIE E Z, LI X, et al. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 548-558.
14 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 770-778.
15 PASZKE A, GROSS S, MASSA F, et al. Pytorch: An imperative style, high-performance deep learning library [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2019: 8026-8037.
16 Deep diabetic retinopathy [EB/OL]. (2022-06-01)[2022-05-01]. https://github.com/deepdrdoc/DeepDRiD.
17 HE J J, LI C, YE J, et al. Classification of ocular diseases employing attention-based unilateral and bilateral feature weighting and fusion [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020: 1258-1261.
18 LI C, YE J, HE J J, et al. Dense correlation network for automated multi-label ocular disease detection with paired color fundus photographs [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020: 1250-1253.
19 SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 618-626.
Options
Outlines

/

〈 〉