A landscape simulation modeling method based on remote sensing images

doi:10.3969/j.issn.1000-5641.2023.02.010

Abstract

Abstract:

Traditional virtual terrain modeling commonly uses a procedural generation method based on manual design, which cannot be used for competent simulation modeling tasks that need to restore real environments, such as in military applications. In this paper, we proposed a landscape simulation modeling method based on remote sensing images. The core of our proposed method is a landscape blended texture generation network (LBTG-Net); this method uses a blended texture generator (BTG) to generate landscape blended textures with the supervision of a style discriminator (SD) and multi-stage classification loss. Then, we procedurally build the complete virtual environment based on the blended texture generated by LBTG-Net. Our method has two main features: (1) accurate land-cover classification ability of remote sensing image inputs; and (2) high quality landscape blended texture outputs to guarantee virtual landscape modeling quality. We used multispectral image data from the Sentinel-2 satellite as the experimental dataset. The experimental results showed that our method offered high performance under mainstream land-cover classification evaluating indicators and can accurately reproduce the environmental distribution of input remote sensing images while completing high-quality virtual terrain simulation modeling.

Key words: landscape simulation modeling, generative adversarial networks, landscape blended texture

CLC Number:

TP391

Zehua WANG, Yan GAO, Mingang CHEN. A landscape simulation modeling method based on remote sensing images[J]. Journal of East China Normal University(Natural Science), 2023, 2023(2): 82-94.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Table 1

Fig.4

Fig.5

Table 2

Fig.6

Table 3

Fig.7

Table 4

Fig.8

Fig.9

References 22

1	RODRIGUEZ-GALIANO V F, CHICA-OLMO M, ABARCA-HERNANDEZ F, et al. Random forest classification of Mediterranean land cover using multi-seasonal imagery and multi-seasonal texture. Remote Sensing of Environment, 2012, 121, 93- 107.
2	ZHANG T X, SU J Y, XU Z Y, et al. Sentinel-2 satellite imagery for urban land cover classification by optimized random forest classifier. Applied Sciences, 2021, 11 (2): 543. doi: 10.3390/app11020543
3	RAKHLIN A, DAVYDOW A, NIKOLENKO S. Land cover classification from satellite imagery with u-net and lovász-softmax loss [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2018: 262-266.
4	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
5	ZHANG W, TANG P, ZHAO L. Fast and accurate land-cover classification on medium-resolution remote-sensing images using segmentation models. International Journal of Remote Sensing, 2021, 42 (9): 3277- 3301.
6	WAMBUGU N, CHEN Y, XIAO Z, et al. A hybrid deep convolutional neural network for accurate land cover classification. International Journal of Applied Earth Observation and Geoinformation, 2021, 103, 102515.
7	LI R, ZHENG S Y, DUAN C X, et al. Land cover classification from remote sensing images based on multi-scale fully convolutional network. Geo-Spatial Information Science, 2022, 25 (2): 278- 294.
8	SMELIK R M, DE KRAKER K J, GROENEWEGEN S, et al. A survey of procedural methods for terrain modelling [C]// Proceedings of the CASA Workshop on 3D Advanced Media in Gaming and Simulation (3AMIGAS). 2009: 25-34.
9	陈晓铮, 李震霄. 一种基于表面积比例的地形纹理贴图算法. 计算机仿真, 2007, 24 (8): 174- 177.
10	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680
11	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 2242-2251. DOI: 10.1109/ICCV.2017.244.
12	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 1125-1134. DOI: 10.1109/CVPR.2017.632.
13	LI W, LIANG Z X, MA P, et al. Hausdorff GAN: Improving GAN generation quality with Hausdorff metric [J]. IEEE Transactions on Cybernetics, 2022, 52(10): 10407-10419. DOI: 10.1109/TCYB.2021.3062396.
14	ZHANG H, KOH J Y, BALDRIDGE J, et al. Cross-modal contrastive learning for text-to-image generation [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 833-842. DOI: 10.1109/CVPR46437.2021.00089.
15	LEE K S, TRAN N T, CHEUNG N M. Infomax-GAN: Improved adversarial image generation via information maximization and contrastive learning [C]// 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2021: 3942-3952. DOI: 10.1109/WACV48630.2021.00399.
16	CHEN J, LIU G, CHEN X. AnimeGAN: A novel lightweight gan for photo animation [C]// International Symposium on Intelligence Computation and Applications. Singapore: Springer, 2019: 242-256.
17	XU W J, LONG C J, WANG R S, et al. DRB-GAN: A dynamic resblock generative adversarial network for artistic style transfer [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021: 6383-6392. DOI: 10.1109/ICCV48922.2021.00632.
18	XIAN W Q, SANGKLOY P, AGRAWAL V, et al. Texturegan: Controlling deep image synthesis with texture patches [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 8456-8465. DOI: 10.1109/CVPR.2018.00882.
19	FRÜHSTÜCK A, ALHASHIM I, WONKA P. Tilegan: Synthesis of large-scale non-homogeneous textures. ACM Transactions on Graphics (ToG), 2019, 38 (4): 0:1- 0:11.
20	KIM H, DISCHLER J M, RUSHMEIER H, et al. Edge-based procedural textures. The Visual Computer, 2021, 37 (9): 2595- 2606.
21	BECKHAM C, PAL C. A step towards procedural terrain generation with GANs [EB/OL]. (2017-07-11)[2022-02-20]. https://arxiv.org/pdf/1707.03383v1.pdf.
22	DRUSCH M, DEL BELLO U, CARLIER S, et al. Sentinel-2: ESA’s optical high-resolution mission for GMES operational services. Remote Sensing of Environment, 2012, 120, 25- 36.

序号	上采样单元	降采样单元
1	Conv2D 3 × 3	Conv2D 3 × 3
2	batch normal	batch normal
3	ReLU	ReLU
4	Conv2D 3 × 3	Conv2D 3 × 3
5	batch normal	batch normal
6	ReLU	ReLU
7	Deconv2D 2 × 2	maxpooling2D 2 × 2

波段	波长/nm	地面分辨率/m	描述
B₁	433~453	60	ultra blue
B₂	458~523	10	blue
B₃	543~578	10	green
B₄	630~680	10	red
B₅	698~713	20	VNIR
B₆	733~748	20	VNIR
B₇	773~793	20	VNIR
B₈	785~900	10	VNIR
B₉	935~955	60	SWIR
B₁₀	1360~1930	60	SWIR
B₁₁	1565~1655	20	SWIR
B₁₂	2100~2280	20	SWIR

类别	样本数量/像素点	类别	样本数量/像素点
水体	69850904	林地	9309214
裸露土壤	12040157	岩石	8728849
草地	8461388	雪地	16636007

类别	准确率/%	敏感度/%	特异度/%
水体	98.64	85.30	98.86
裸露土壤	83.42	71.76	87.30
草地	83.31	72.45	86.67
林地	93.46	63.31	93.43
岩石	88.26	59.55	93.00
雪地	95.69	40.03	99.19

[1]	Zhiwei JIN, Chang HUANG, Ruihong ZHU. Fast establishment of a point cloud model for a lock pin based onhigh overlapping views [J]. Journal of East China Normal University(Natural Science), 2023, 2023(2): 95-105.
[2]	Tiantian ZHANG, Man LAN. Target-dependent event detection from news [J]. Journal of East China Normal University(Natural Science), 2023, 2023(2): 60-72.
[3]	Xiaoqiu LU, Yang CAI, Jiajun CHEN, Xi ZHOU, Xueming ZHOU, Yunzhe TANG, Dingjiang HUANG. Fault location algorithm based on Kirchhoff ’s law and a Boolean equation [J]. Journal of East China Normal University(Natural Science), 2023, 2023(2): 143-154.
[4]	Yang ZHANG, Yejing LAI, Dingjiang HUANG. Device component state recognition method of power distribution cabinet based on a residual networks [J]. Journal of East China Normal University(Natural Science), 2023, 2023(2): 132-142.
[5]	Ziyin HUANG, Yuanbin WU. Distant supervision relation extraction via the influence function [J]. Journal of East China Normal University(Natural Science), 2022, 2022(6): 79-86.
[6]	Wei HE, Weiqing TONG. High resolution panorama generation method for irregular cylindrical murals [J]. Journal of East China Normal University(Natural Science), 2022, 2022(6): 102-122.
[7]	Ming WANG, Te LI, Dingjiang HUANG. Text matching based on multi-dimensional feature representation [J]. Journal of East China Normal University(Natural Science), 2022, 2022(5): 126-135.
[8]	Jizhou LI, Xin LIN. Neural architecture search algorithms based on a recursive structure [J]. Journal of East China Normal University(Natural Science), 2022, 2022(4): 31-42.
[9]	Xiaozhen ZHAO, Weiqing TONG, Yongmei LIU. A graph convolutional neural network for garment pattern classification [J]. Journal of East China Normal University(Natural Science), 2022, 2022(4): 56-66.
[10]	Shucheng LU, Yan GAO, Changbo WANG. Modeling and simulation technology of roads for a battlefield environment [J]. Journal of East China Normal University(Natural Science), 2022, 2022(4): 79-94.
[11]	Zejie WANG, Chaomin SHEN, Chun ZHAO, Xinmei LIU, Jie CHEN. Recognition of classroom learning behaviors based on the fusion of human pose estimation and object detection [J]. Journal of East China Normal University(Natural Science), 2022, 2022(2): 55-66.
[12]	Qinmin YANG, Zhisong JIANG. Design and implementation of automatic correction for college mathematics assignments [J]. Journal of East China Normal University(Natural Science), 2022, 2022(2): 76-83.
[13]	Guofang ZHANG, Lili WEN, Meng WU, Tongyu LIU, Kuanyun ZHENG, Fuxing HUANG, Peisen YUAN. Anomaly detection of transformer loss data based on a robust random cut forest [J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 135-146.
[14]	Xinye ZHANG, Weiqing TONG, Haisheng LI. A fast key points matching method for high resolution images of a planar mural [J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 65-80.
[15]	Dongqing HUANG, Liyang YU, Jue CHEN, Tongquan WEI. Research on joint computation offloading and resource allocation strategy for mobile edge computing [J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 88-99.