High resolution panorama generation method for irregular cylindrical murals

doi:10.3969/j.issn.1000-5641.2022.06.011

Abstract

Abstract:

The issue of how to unfold an irregular cylindrical mural from the top surface of a cave corridor into a panorama is a challenge for researchers involved with ancient mural protection and secondary development. This paper presents a method of dividing cylindrical murals into many overlapping small areas for sampling firstly, and then stitching these sampled images into a panorama. The constituents of this method include the following key elements: ① Reconstructing the 3D model with the sampled image set; ② Mapping the image texture to the 3D model; ③ Fitting the reconstructed irregular 3D cylindrical surface to the ideal cylindrical surface which is closest to the original form; and ④ Projecting the mural of the ideal cylindrical surface to a panorama. The method proposed in this paper was verified on an actual cave image set. The experimental results showed that the proposed method can generate the panorama in full; moreover, there was no evidence of stitching traces or texture deformation on the panorama. The proposed method offers practical value for mural protection.

Key words: panorama, cave murals, 3D reconstruction, image stitching

CLC Number:

TP391.41

Wei HE, Weiqing TONG. High resolution panorama generation method for irregular cylindrical murals[J]. Journal of East China Normal University(Natural Science), 2022, 2022(6): 102-122.

Figures/Tables 36

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Table 1

Fig.18

Fig.19

Fig.20

Fig.21

Fig.22

Table 2

Fig.23

Fig.24

Fig.25

Fig.26

Fig.27

Table 3

Fig.28

Fig.29

Fig.30

Fig.31

Fig.32

Table 4

References 34

1	郑晓瑞. 林区无人机航拍图像拼接算法研究 [D]. 北京: 北京林业大学, 2020.
2	袁兴峰. 基于快速标定的车载全景影像系统设计与实现 [D]. 合肥: 合肥工业大学, 2020.
3	邹宇. 基于鱼眼相机的车载全景图像拼接技术研究 [D]. 南京: 南京航空航天大学, 2019.
4	王曼, 李涛. 视频拼接系统与我院手术示教系统的结合应用. 中国医疗设备, 2018, 33 (11): 168- 171.
5	代少升, 姚俐. 基于ROI的高精度红外全景拼接方法研究. 半导体光电, 2020, 39 (4): 306- 314.
6	ANUTA P E. Spatial registration of multispectral and multi-temporal digital imagery using fast Fourier transform techniques. IEEE Transactions on Geoscience Electronics, 1970, 8 (4): 353- 368.
7	KUGLIN C D, HINES D C. The phase correlation image alignment method [C]//Proceedings of IEEE International Conference on Cybernetics and Society. 1975: 163-165.
8	DE CASTRO E, MORANDI C. Registration of translated and rotated images using finite fourier transforms.. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1987, 9 (5): 700- 703.
9	DJAMDJI J P, BIJAOUI A, MANIERE R. Geometrical registration of images: The multiresolution approach. Photogrammetric Engineering and Emote Sensing, 1993, 59 (5): 645- 653.
10	LOWE D G. Object recognition from local scale-invariant features [C]//Proceedings of the 7th IEEE International Conference on Computer Vision. IEEE, 1999(2): 1150-1157.
11	LOWE D G. Distinctive image features from scale-invariant key-points [J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
12	赵向阳, 杜利民. 一种全自动稳健的图像拼接融合算法. 中国图象图形学报, 2004, 9 (4): 417- 422.
13	BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 2008, 110 (3): 346- 359.
14	RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An efficient alternative to SIFT or SURF [C]//2011 International Conference on Computer Vision. IEEE, 2011: 2564-2571.
15	BROWN M, LOWE D G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 2007, 74 (1): 59- 73.
16	GAO J H, KIM S J, BROWN M S. Constructing image panoramas using dual-homography warping [C]//Conference on Computer Vision and Pattern Recognition (CVPR) 2011. IEEE, 2011: 49-56.
17	LIN W Y, LIU S, MATSUSHITA Y, et al. Smoothly varying affine stitching [C]//Conference on Computer Vision and Pattern Recognition (CVPR) 2011. IEEE, 2011: 345-352.
18	ZARAGOZA J, CHIN T J, BROWN M S, et al. As-projective-as-possible image stitching with moving DLT [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2013: 2339-2346.
19	CHEN Y S, CHUANG Y Y. Natural image stitching with the global similarity prior [C]//European Conference on Computer Vision. Cham : Springer, 2016: 186-201.
20	YI K M, TRULLS E, LEPETIT V, et al. Lift: Learned invariant feature transform [C]//European Conference on Computer Vision. Cham : Springer, 2016: 467-483.
21	陈小杜. 基于改进AKAZE和RANSAC的全景图像拼接算法研究 [D]. 南昌: 南昌大学, 2020.
22	MELEKHOV I, YLIOINAS J, KANNALA J, et al. Relative camera pose estimation using convolutional neural networks [C]//International Conference on Advanced Concepts for Intelligent Vision Systems. Cham: Springer, 2017: 675-687.
23	FU Y, GUO T T, ZHAO X F. Intelligent splicing method of virtual reality Lingnan cultural heritage panorama based on automatic machine learning [J]. Mobile Information Systems, 2021: 8693436. DOI: 10.1155/2021/8693436.
24	刘礼铭. 基于三维重建的超高分辨率壁画快速数字化技术研究 [D]. 杭州: 浙江大学, 2016.
25	张园林. 基于三维模型的碑刻数字拓片生成技术及应用 [D]. 杭州: 浙江大学, 2018.
26	闫冰. 基于深度图的三维重建算法的优化研究 [D]. 杭州: 浙江大学, 2019.
27	李敏. 基于非朗伯光度立体的三维重建研究 [D]. 杭州: 浙江大学, 2020.
28	李淑芬. 基于局部几何约束的自适应特征匹配技术研究 [D]. 杭州: 浙江大学, 2020.
29	ULLMAN S. The interpretation of structure from motion. Proceedings of the Royal Series B, 1979, 203 (1153): 405- 426.
30	LOURAKIS M I A, ARGYROS A A. SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software (TOMS), 2009, 36 (1): 1- 30.
31	SHEN S. Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes. IEEE Transactions on Image Processing, 2013, 22 (5): 1901- 1914.
32	JANCOSEK M, PAJDLA T. Exploiting visibility information in surface reconstruction to preserve weakly supported surfaces [J/OL]. International Scholarly Research Notices, 2014: 798595. (2014-08-11)[2021-08-01]. https://doi.org/10.1155/2014/798595.
33	WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! Large-scale texturing of 3D reconstructions [C]//European Conference on Computer Vision. Cham: Springer, 2014: 836-850.
34	TORR P H S, ZISSERMAN A. MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 2000, 78 (1): 138- 156.

算法	复原完整性	拼接处痕迹	图像变形	图像重影	图像模糊	色差
CPG	基本还原	无	无	无	无	无
OpenCV	无法还原	×	×	×	×	×
OpenPano	局部还原	无	有	有	有	无
PhotoShop	局部还原	有	有	无	无	无

算法	复原完整性	拼接处痕迹	图像变形	图像重影	图像模糊	色差	拼接时间/min
CPG	基本还原	无	无	无	无	无	185
OpenCV	局部还原	无	有	无	无	无	58
OpenPano	局部还原	无	有	有	有	无	24
PhotoShop	基本还原	有	有	有	无	有	63

算法	复原完整性	拼接处痕迹	图像变形	图像重影	图像模糊	色差	拼接时间/min
CPG	基本还原	无	无	无	无	无	755
OpenCV	局部还原	无	有	无	无	无	7
OpenPano	局部还原	无	无	有	有	无	97
PhotoShop	基本还原	有	有	无	无	有	270

算法	复原完整性	拼接处痕迹	图像变形	图像重影	图像模糊	色差	拼接时间/min
CPG	基本还原	无	无	无	无	无	270
OpenCV	局部还原	无	有	无	无	无	717
OpenPano	局部还原	无	有	有	有	无	315
PhotoShop	基本还原	有	有	无	无	有	235

[1]	Ruibo CUI, Feng WANG. Momentum-updated representation with reconstruction constraint for limited-view 3D object recognition [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 61-72.
[2]	SHEN Sha-sha. A novel palmprint recognition algorithm based on Curvelet invariant features of the surface [J]. Journal of East China Normal University(Natural Sc, 2015, 2015(3): 98-104.
[3]	LIU Dan, TONG Wei-qing. Nose tip location based on nose contour [J]. Journal of East China Normal University(Natural Sc, 2014, 2014(4): 54-61.
[4]	华东师范大学计算机科学与技术系;上海 00. Real time near infrared light face detection based on morphology [J]. Journal of East China Normal University(Natural Sc, 2010, 2010(3): 39-47.
[5]	FAN Jin-song;PI Ling;LI Fang;SHEN Chao-min. Intrinsic Image Segmentation on a Riemannian Manifold(Chinese) [J]. Journal of East China Normal University(Natural Sc, 2007, 2007(5): 70-77.
[6]	LI Fang. Image Restoration Combining the Total Variation Minimization and a SecondOrder Functional(English) [J]. Journal of East China Normal University(Natural Sc, 2007, 2007(5): 62-69.
[7]	CHEN You-Guang;ZHANG Wei;HUANG Sui. Algorithm for Extracting Contour and Generating Chain Code Tree(Chinese) [J]. Journal of East China Normal University(Natural Sc, 2006, 2006(3): 77-85.
[8]	ZHANG Wei;CHEN You-guang;GU Guo-qing. Calculation of Euclidean Distance between Vertices Based on VCC(Chinese) [J]. Journal of East China Normal University(Natural Sc, 2005, 2005(2): 33-38.
[9]	WANG Di-qiong;ZHANG Wei;GU Guo-qing. Calculating the Compact and Posture Ratio of An Image Region Based on VCC [J]. Journal of East China Normal University(Natural Sc, 2005, 2005(1): 59-62.