华东师范大学学报(自然科学版) ›› 2023, Vol. 2023 ›› Issue (6): 39-48.doi: 10.3969/j.issn.1000-5641.2023.06.004

• 计算机科学 • 上一篇    下一篇

基于双视图特征融合的糖尿病视网膜病变分级

姜璐璐1,2, 孙司琦3,4,*(), 邹海东2,5,6, 陆丽娜2,6, 冯瑞1,2,3,4   

  1. 1. 复旦大学 工程与应用技术研究院, 上海 200433
    2. 上海市眼科疾病精准诊疗工程技术研究中心,上海 200080
    3. 复旦大学 计算机科学技术学院 上海市智能信息处理重点实验室, 上海 200433
    4. 复旦大学 上海市智能视觉计算协同创新中心, 上海 200433
    5. 上海交通大学附属第一人民医院, 上海 200080
    6. 上海市眼病防治中心, 上海 200040
  • 收稿日期:2022-06-09 出版日期:2023-11-25 发布日期:2023-11-23
  • 通讯作者: 孙司琦 E-mail:siqi_sun@fudan.edu.cn
  • 基金资助:
    国家自然科学基金 (62172101); 上海市科委项目 (19DZ2250100, 20DZ1100205)

Diabetic retinopathy grading based on dual-view image feature fusion

Lulu JIANG1,2, Siqi SUN3,4,*(), Haidong ZOU2,5,6, Lina LU2,6, Rui FENG1,2,3,4   

  1. 1. Academy for Engineering and Technology, Fudan University, Shanghai 200433, China
    2. Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai 200080, China
    3. School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai 200433, China
    4. Shanghai Collaborative Innovation Center of Intelligent Visual Computing, Fudan University, Shanghai 200433, China
    5. Shanghai General Hospital, Shanghai 200080, China
    6. Shanghai Eye Disease Prevention Center, Shanghai 200040, China
  • Received:2022-06-09 Online:2023-11-25 Published:2023-11-23
  • Contact: Siqi SUN E-mail:siqi_sun@fudan.edu.cn

摘要:

基于双视图眼底图像的诊断方法被广泛应用于糖尿病视网膜病变 (diabetic retinopathy, DR) 的筛查, 该方法可以有效地解决单视角下图像遮挡和视场受限的问题. 针对如何有效融合不同视图信息来提高DR分级准确率, 提出了一种基于注意力机制的多视角图像之间特征融合的学习方法. 针对眼底图像中病灶占比率较小的问题, 引入了自注意力机制以加强局部病灶特征的学习; 针对双视图眼底图像分类场景, 提出了一种跨视图注意力机制, 有效地利用了双视图之间的信息. 在内部数据集DFiD和公开数据集DeepDR上进行的实验, 验证了所提方法能够有效提高DR分级精度, 可用于大规模DR筛查, 辅助医生实现高效诊断.

关键词: 眼底图像, 特征融合, 双视图融合, 注意力机制, 糖尿病视网膜病变

Abstract:

The diagnostic method based on dual-view fundus imaging is widely used in diabetic retinopathy (DR) screening. This method effectively solves the problems of image occlusion and limited field of view under single-view. This paper proposes a learning method of feature fusion between dual-view images based on the attention mechanism to improve the accuracy of DR classification by effectively integrating different view information. Due to the small proportion of lesions in fundus images, the self-attention mechanism was introduced to enhance the learning of local lesion features. Moreover, a cross-attention mechanism is proposed to effectively utilize information between dual-view images to improve the classification of dual-view fundus images. Experiments were performed on the internal DFiD dataset and public DeepDRiD dataset. The proposed method can effectively improve the accuracy of DR classification and can be used for large-scale DR screening to assist doctors in achieving an efficient diagnosis.

Key words: fundus image, feature fusion, dual-view image fusion, attention mechanism, diabetic retinopathy

中图分类号: