平面壁画高分辨率图像的快速特征匹配方法

doi:10.3969/j.issn.1000-5641.2021.06.008

摘要/Abstract

摘要：

既有的图像特征匹配算法比较适合于一般分辨率的图像, 且是在灰度图像上进行的. 洞窟壁画图像的特点是分辨率非常高, 并且还可能存在具有相同灰度纹理和不同颜色的区域. 针对这类特殊图像, 提出了一种面向高分辨率壁画图像的高速化特征匹配算法(简称NeoKPM算法). NeoKPM算法有2个主要特点: ①通过降采样图像获得原图像粗配准的单应变换矩阵, 极大地降低了后续特征匹配的时间复杂度; ②提出了一种基于灰度和颜色不变量的特征描述符, 能很好地区分具有相同灰度纹理和不同颜色的特征点, 提高了特征匹配的正确性. 在实际壁画图像库上, 对NeoKPM算法的性能进行了实验. 实验结果表明, NeoKPM算法在分辨率为8 000万像素的壁画图像上, 其每对图像的正确匹配点数量平均比SIFT (Scale Invariant Feature Transform)算法高出了近10万个; 其特征点匹配平均处理速度是SIFT算法的20倍; 其基于图像单个像素的双图像平均误差小于0.04像素.

关键词: 壁画数字化, 特征匹配, 暴力匹配, 颜色不变量

Abstract:

Existing methods of key points matching were invented for grayscale images and are not suitable for high resolution images. Mural images typically have very high resolution, and there may be areas with the same gray textures and different colors. For this special kind of image, this paper proposes a high-speed algorithm of key points matching for high-resolution mural images (NeoKPM for short). NeoKPM has two main innovations: (1) first, the homography matrix of rough registration for the original image is obtained by downsampling the image, which substantially reduces the time required for key points matching; (2) second, a feature descriptor based on gray and color invariants is proposed, which can distinguish different colors of texture with the same gray level, thereby improving the correctness of key points matching. In this paper, the performance of the NeoKPM algorithm is tested on a real mural image library. The experimental results show that on mural images with a resolution of 80 million pixels, the number of correct matching points per pair of images is nearly 100 000 points higher than that of the SIFT (Scale Invariant Feature Transform) algorithm, the processing speed of key points matching is more than 20 times faster than that of the SIFT algorithm, and the average error of dual images based on a single pixel of the image is less than 0.04 pixels.

Key words: mural digitizing, key points matching, brute force matching, color invariance

中图分类号:

TP391

章昕烨, 童卫青, 李海晟. 平面壁画高分辨率图像的快速特征匹配方法[J]. 华东师范大学学报（自然科学版）, 2021, 2021(6): 65-80.

Xinye ZHANG, Weiqing TONG, Haisheng LI. A fast key points matching method for high resolution images of a planar mural[J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 65-80.

图/表 20

表1

表2

表3

图1

图2

图3

图4

表4

图5

图6

表5

表6

表7

图7

图8

表8

图9

图10

图11

图12

参考文献 21

1	中华人民共和国国家文物局. 古建筑壁画数字化测绘技术规程: WW/T 0082—2017 [S]. 北京: 文物出版社, 2017.
2	LOWE D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
3	MATAS J, CHUM O, URBAN M, et al. Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 2004, 22 (10): 761- 767. doi: 10.1016/j.imavis.2004.02.006
4	BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 2008, 110 (3): 346- 359. doi: 10.1016/j.cviu.2007.09.014
5	RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An efficient alternative to SIFT or SURF [C]// 2011 International Conference on Computer Vision. IEEE, 2011: 2564-2571.
6	MUKHERJEE D, WU Q M J, WANG G H. A comparative experimental study of image feature detectors and descriptors. Machine Vision and Applications, 2015, 26 (4): 443- 466. doi: 10.1007/s00138-015-0679-9
7	ABDEL-HAKIM A E, FARAG A A. CSIFT: A SIFT descriptor with color invariant characteristics [C]// 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). IEEE, 2006: 1978-1983.
8	GEUSEBROEK J M, VAN DEN BOOMGAARD R, SMEULDERS A W M, et al. Color invariance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23 (12): 1338- 1350. doi: 10.1109/34.977559
9	ALCANTARILLA P F, BARTOLI A, DAVISON A J. KAZE features [C]// Computer Vision – ECCV 2012, ECCV 2012, Lecture Notes in Computer Science. Berlin: Springer-Verlag, 2012: 214-227.
10	ALCANTARILLA P F, NUEVO J, BARTOLI A. Fast explicit diffusion for accelerated features in nonlinear scale spaces [C]// Proceedings of British Machine Vision Conference. BMVC, 2013: 13.1-13.11.
11	CHO Y J, KIM D, SAEED S, et al. Keypoint detection using higher order Laplacian of Gaussian. IEEE Access, 2020, (8): 10416- 10425.
12	SAVINOV N, SEKI A, LADICKY L, et al. Quad-networks: Unsupervised learning to rank for interest point detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 3929-3937.
13	DE TONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: Self-supervised interest point detection and description [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2018: 337-349.
14	ONO Y, TRULLS E, FUA P, et al. LF-Net: Learning local features from images [C]// 32nd Conference on Neural Information Processing Systems (NIPS 2018). 2018: 6234-6244.
15	FISCHLER M A, BOLLES R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24 (6): 381- 395. doi: 10.1145/358669.358692
16	ROUSSEEUW P J. Least median of squares regression. Journal of the American Statistical Association, 1984, 79 (388): 871- 880. doi: 10.1080/01621459.1984.10477105
17	FOTOUHI M, HEKMATIAN H, KASHANI-NEZHAD M A, et al. SC-RANSAC: Spatial consistency on RANSAC. Multimedia Tools and Applications, 2019, 78 (7): 9429- 9461. doi: 10.1007/s11042-018-6475-6
18	LIN W Y, WANG F, CHENG M M, et al. CODE: Coherence based decision boundaries for feature correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (1): 34- 47. doi: 10.1109/TPAMI.2017.2652468
19	CHOU C C, SEO Y W, WANG C C. A two-stage sampling for robust feature matching. Journal of Field Robotics, 2018, 35, 779- 801. doi: 10.1002/rob.21778
20	HARTLEY R, ZISSERMAN A. Multiple View Geometry in Computer Vision [M]. 2nd ed. New York: Cambridge University Press, 2003.
21	MOREL J M, YU G S. ASIFT: A new framework for fully affine invariant image comparison [J]. SIAM Journal on Imaging Sciences, 2009, 2(2): 438-469.

降采样率/%	特征点数量	降采样率/%	特征点数量
100	1 159 349	40	75 112
90	882 019	30	39 907
80	593 612	20	19 134
70	369 532	10	6 001
60	221 638	9	4 869

降采样率/%	$\overline \varepsilon $ /像素	平均匹配时间 /s
100	0	4 544.00
90	0.006	2 611.00
80	0.010	1 392.00
70	0.022	608.00
60	0.039	262.00
50	0.064	109.00
40	0.070	45.00
30	0.129	17.00
20	0.256	6.00
10	0.820	1.00
9	0.976	1.00
8	1.058	0.89
7	1.207	0.75

降采样率/%	$\overline \varepsilon $ /像素	平均匹配时间/s
100	0	139.80
90	0.0006	88.54
80	0.0020	56.98
70	0.0060	34.81
60	0.0120	20.61
50	0.0180	11.76
40	0.0290	6.57
30	0.0640	3.56
20	0.1430	1.77
10	1.0500	0.69
9	1.3400	0.64
8	1.8100	0.58
7	3.3700	0.54

算法	平均处理时间/s
算法	(10288×7730)像素	(5087×3703)像素
NeoKPM	461.66	63.37
SIFT	9177.71	118.41
SURF	2677.03	96.96
ORB	3967.46	428.66
SC-RANSAC	13286.36	120.13

算法	平均匹配点数
算法	(10288×7730)像素	(5087×3703)像素
NeoKPM	588874	61299
SIFT	500250	52611
SURF	303450	71770
ORB	208672	79777
SC-RANSAC	482390	51011