小样本实例分割综述

doi:10.3969/j.issn.1000-5641.2022.05.012

华东师范大学学报（自然科学版） ›› 2022, Vol. 2022 ›› Issue (5): 136-146.doi: 10.3969/j.issn.1000-5641.2022.05.012

• 供应链知识图谱构建与分析 • 上一篇

小样本实例分割综述

周雪茗, 黄定江*()

华东师范大学数据科学与工程学院, 上海　200062

收稿日期:2022-07-16 接受日期:2022-07-16 出版日期:2022-09-25 发布日期:2022-09-26
通讯作者: 黄定江 E-mail:djhuang@dase.ecnu.edu.cn
基金资助:
国家自然科学基金(U1711262, 62072185, U1811264)

Survey of few-shot instance segmentation methods

Xueming ZHOU, Dingjiang HUANG*()

School of Data Science and Engineering, East China Normal University, Shanghai　200062, China

Received:2022-07-16 Accepted:2022-07-16 Online:2022-09-25 Published:2022-09-26
Contact: Dingjiang HUANG E-mail:djhuang@dase.ecnu.edu.cn

摘要/Abstract

摘要：

实例分割是计算机视觉领域中的一项重要任务, 近年来元学习和小样本学习的发展推动了小样本与计算机视觉任务的结合, 突破了对人工标注难、标注成本高的目标检测与分类瓶颈. 虽然在小样本图像分类、小样本语义分割和小样本目标检测上都取得了较大的发展, 但是基于小样本学习的实例分割近年来才成为研究热点. 从小样本实例分割的相关概念出发, 对现有小样本实例分割方法, 按照基于锚框和无锚框两类分别进行了系统性的概述, 并介绍了小样本实例分割常用的数据集及评价指标. 通过对算法性能和优缺点的分析对比, 以及研究现状的整理归纳, 对小样本实例分割未来发展方向和面临的挑战进行了展望.

关键词: 小样本学习, 实例分割, 计算机视觉

Abstract:

Instance segmentation is an important task in computer vision. In recent years, the development of meta- and few-shot learning has promoted the combination of computer vision learning tasks, which has overcome the bottleneck of detection and classification with regard to objects that are difficult to manually label and those with high labeling costs. Although great progress has been made with few-shot semantic segmentation and object detection, instance segmentation based on few-shot learning has not become a research hotspot until very recently. Beginning with an overview of few-shot instance segmentation, existing approaches are divided into categories of anchor-based and anchor-free algorithms. The architectures and primary technologies behind those approaches are respectively discussed, and common datasets and evaluation indices are described. Additionally, advantages and disadvantages of algorithm performance are analyzed, and future development directions and challenges are presented.

Key words: few-shot learning, instance segmentation, computer vision

中图分类号:

TP183

周雪茗, 黄定江. 小样本实例分割综述[J]. 华东师范大学学报（自然科学版）, 2022, 2022(5): 136-146.

Xueming ZHOU, Dingjiang HUANG. Survey of few-shot instance segmentation methods[J]. Journal of East China Normal University(Natural Science), 2022, 2022(5): 136-146.

图/表 8

图1

图2

图3

图4

图5

表1

表2

表3

参考文献 27

1	YAN X, CHEN Z, XU A, et al. Meta r-cnn: Towards general solver for instance-level low-shot learning [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 9577-9586.
2	SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning [EB/OL]. (2017-06-19)[2022-07-10]. https://arxiv.org/abs/1703.05175.
3	KOCH G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition [C]// ICML Deep Learning Workshop. 2015.
4	GARCIA V, BRUNA J. Few-shot learning with graph neural networks [EB/OL]. (2018-02-20)[2022-07-10]. https://arxiv.org/abs/1711.04043.
5	SHABAN A, BANSAL S, LIU Z, et al. One-shot learning for semantic segmentation [C]// British Machine Vision Conference. 2017.
6	RAKELLY K, SHELHAMER E, DARRELL T, et al. Conditional networks for few-shot semantic segmentation [C]// International Conference on Learning Representations. 2018.
7	ZHANG X, WEI Y, YANG Y, et al. Sg-one: Similarity guidance network for one-shot semantic segmentation. IEEE Transactions on Cybernetics, 2020, 50 (9): 3855- 3865.
8	KARLINSKY L, SHTOK J, HARARY S, et al. Repmet: Representative-based metric learning for classification and one-shot object detection [EB/OL]. (2018-11-18)[2022-07-10]. https://arxiv.org/abs/1806.04728.
9	KANG B, LIU Z, WANG X, et al. Few-shot object detection via feature reweighting [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 8420-8429.
10	LIU S, JIA J, FIDLER S, et al. Sgn: Sequential grouping networks for instance segmentation [C]// Proceedings of the IEEE International Conference on Computer Vision. 2017: 3496-3504.
11	GAO N, SHAN Y, WANG Y, et al. Ssap: Single-shot instance segmentation with affinity pyramid [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 642-651.
12	PINHEIRO P O, COLLOBERT R, DOLLÁR P. Learning to segment object candidates [EB/OL]. (2015-09-01)[2022-07-10]. https://arxiv.org/abs/1506.06204v2.
13	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN [C]// Proceedings of the IEEE International Conference on Computer Vision. 2017: 2961-2969.
14	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 3431-3440.
15	TIAN Z, SHEN C, CHEN H, et al. Fcos: Fully convolutional one-stage object detection [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 9627-9636.
16	VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning [C]// Advances in Neural Information Processing Systems. 2016.
17	MICHAELIS C, USTYUZHANINOV I, BETHGE M, et al. One-shot instance segmentation [EB/OL]. (2019-05-28)[2022-07-10]. https://arxiv.org/abs/1811.11507.
18	FAN Z, YU J G, LIANG Z, et al. Fgn: Fully guided network for few-shot instance segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 9172-9181.
19	GANEA D A, BOOM B, POPPE R. Incremental few-shot instance segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 1185-1194.
20	NGUYEN K, TODOROVIC S. Fapis: A few-shot anchor-free part-based instance segmenter [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 11099-11108.
21	CHOI D, YE-BIN M, KIM J, et al. FoxInst: A Frustratingly simple baseline for weakly few-shot instance segmentation [C]// International Conference on Learning Representations. 2021.
22	WANG X, HUANG T E, DARRELL T, et al. Frustratingly simple few-shot object detection [EB/OL]. (2019-05-28)[2022-07-10]. https://arxiv.org/abs/2003.06957.
23	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 2015, 28.
24	TIAN Z, SHEN C, WANG X, et al. Boxinst: High-performance instance segmentation with box annotations [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 5443-5452.
25	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: Common objects in context [C]// European Conference on Computer Vision. Cham, Zug: Springer, 2014: 740-755.
26	EVERINGHAM M, VAN GOOL L, WILLIAMS C K I, et al. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 2010, 88 (2): 303- 338.
27	DENG J, DONG W, SOCHER R, et al. ImageNet: A large-scale hierarchical image database [C]// IEEE Conference on Computer Vision and Pattern Recognition. 2009: 248-255.

指标	定义
AP	AP均值
AP₅₀	IOU=0.50
AP₇₅	IOU=0.75
AP_S	小目标area≤32²
AP_M	中目标32²<area≤96²
AP_L	大目标area>96²

算法	年份	类别	骨干	检测(AP)	检测(AP₅₀)	分割(AP)	分割(AP₅₀)
Siamese Mask R-CNN	2018	基于锚框	ResNet-50		23.90		13.80
Meta R-CNN	2019	基于锚框	ResNet-50		20.10		12.50
FGN	2020	基于锚框	ResNet-101		30.68		16.20
FAPIS	2021	无锚框	ResNet-50		21.20		19.00
MTFA	2021	基于锚框	ResNet-50	9.99	21.68	9.51	19.28
iMTFA	2021	基于锚框	ResNet-50	11.47	22.41	8.57	16.32
FoxInst	2021	无锚框	ResNet-50	17.79	33.08	12.17	26.24

算法	年份	创新点	优势	缺陷
Siamese Mask R-CNN	2018	首个单样本实例分割算法	实现了基础的单样本实例分割, 目标分割效果较好	算法易产生过拟合, 在新类别检测上效果欠佳
Meta R-CNN	2019	提出一种基于RoI特征的新型元学习模式	模型轻量级, 实现了少样本实例分割	算法捕捉目标的全局轮廓, 在新类别分割上存在局限
FGN	2020	提出基于注意力的全引导模块	类间泛化适应性强	内存消耗高, 考虑新类别数量有限
FAPIS	2021	首个无锚框基于局部的小样本实例分割方法	关注多目标共享的局部轮廓, 提升对新类学习的性能	边缘信息不够清晰
IMTFA	2021	首个增量式小样本实例分割方法	采用微调方法, 训练成本低, 内存消耗少	对类未知的定位和分割表现欠佳
FoxInst	2021	首个弱监督小样本实例分割方法	标注成本低, 不受锚框限制, 采用微调方法	弱监督上分类精度欠佳

小样本实例分割综述

Survey of few-shot instance segmentation methods

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 27

相关文章 1

编辑推荐

Metrics

本文评价