A unified pansharpening model based on prompt learning

doi:10.3969/j.issn.1000-5641.2026.04.010

Abstract

Abstract:

Pansharpening is an image processing technique primarily used to enhance the spatial resolution of remote sensing images by fusing high spatial resolution panchromatic images with low spatial resolution multispectral images. With the development of deep learning, building deep models has gradually become the mainstream method for pansharpening. However, current mainstream technologies are based on training models using datasets from specific satellites, limiting the application of the trained models to fuse images from only those satellite datasets, without covering satellites with limited datasets. Addressing this issue, this paper proposes a unified pansharpening approach that leverages prompt learning technology to train models on datasets from different satellites. The trained unified model can be applied to fuse images from various satellite datasets. Experimental results demonstrate that the model achieves state-of-the-art performance and exhibits good generalization capabilities.

Key words: pansharpening, remote sensing images, image fusion, deep learning, prompt learning

CLC Number:

TP751.1

Hongfei ZHAO, Tingting WANG, Faming FANG, Guixu ZHANG. A unified pansharpening model based on prompt learning[J]. J* E* C* N* U* N* S*, 2026, 2026(4): 92-101.

Figures/Tables 7

Fig.1

Table 1

Table 2

Table 3

Fig.2

Table 4

Fig.3

References 26

1	Zhang B, Wu D, Zhang L, et al.. Application of hyperspectral remote sensing for environment monitoring in mining areas. Environmental Earth Sciences, 2012, 65 (3): 649- 658.
2	Asokan A, Anitha J.. Change detection techniques for remote sensing applications: a survey. Earth Science Informatics, 2019, 12 (2): 143- 160.
3	Sharma R, Kamble S S, Gunasekaran A, et al.. A systematic literature review on machine learning applications for sustainable agriculture supply chain performance. Computers & Operations Research, 2020, 119, 104926.
4	Vivone G, Dalla Mura M, Garzelli A, et al.. A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (1): 53- 81.
5	Meng X C, Shen H F, Li H F, et al.. Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: practical discussion and challenges. Information Fusion, 2019, 46, 102- 113.
6	Ghassemian H.. A review of remote sensing image fusion methods. Information Fusion, 2016, 32, 75- 89.
7	Gao H, Yang J, Zhang Y, et al.. Prompt-based ingredient-oriented all-in-one image restoration. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34 (10): 9458- 9471.
8	Zhang C, Zhu Y, Yan Q S, et al. All-in-one multi-degradation image restoration network via hierarchical degradation representation [C]//Proceedings of the 31st ACM International Conference on Multimedia. ACM, 2023: 2285-2293.
9	Masi G, Cozzolino D, Verdoliva L, et al.. Pansharpening by convolutional neural networks. Remote Sensing, 2016, 8 (7): 594.
10	Bandara W G C, Patel V M. HyperTransformer: a textural and spectral feature fusion transformer for pansharpening [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 1757-1767.
11	Li B Y, Liu X, Hu P, et al. All-in-one image restoration for unknown corruption [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 17431-17441.
12	Chen L Y, Lu X, Zhang J, et al. HINet: half instance normalization network for image restoration [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2021: 182-192.
13	Li R T, Tan R T, Cheong L F. All in one bad weather removal using architectural search [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 3172-3182.
14	Zhou K Y, Yang J K, Loy C C, et al. Conditional prompt learning for vision-language models [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 16795-16804.
15	Chen J Y, Dong J X, Ju Y K, et al. PromptRestorer: a prompting image restoration method with degradation perception [C]//Advances in Neural Information Processing Systems 36. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023: 8898-8912.
16	Khan S, Potlapalli V, Shahbaz Khan F, et al. PromptIR: prompting for all-in-one image restoration [C]//Advances in Neural Information Processing Systems 36. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023: 71275-71293.
17	Zamir S W, Arora A, Khan S, et al. Restormer: efficient transformer for high-resolution image restoration [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 5718-5729.
18	Meng X C, Xiong Y M, Shao F, et al.. A large-scale benchmark data set for evaluating pansharpening performance: Overview and implementation. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (1): 18- 52.
19	Peng S Y, Zhu D, Gao Q W, et al.. PSCF-net: deeply coupled feedback network for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61, 5401812.
20	Wang J M, Lu T, Huang X, et al.. Pan-sharpening via conditional invertible neural network. Information Fusion, 2024, 101, 101980.
21	Wang H, Gong M Q, Mei X G, et al.. Deep unfolded network with intrinsic supervision for pan-sharpening. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38 (6): 5419- 5426.
22	Yang J F, Fu X Y, Hu Y W, et al. PanNet: a deep network architecture for pan-sharpening [C]//2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 1753-1761.
23	Jin Z R, Zhang T J, Jiang T X, et al.. LAGConv: local-context adaptive convolution kernels with global harmonic bias for pansharpening. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36 (1): 1113- 1121.
24	Yuan Q Q, Wei Y C, Meng X C, et al.. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11 (3): 978- 989.
25	Xu S, Zhang J S, Zhao Z X, et al. Deep gradient projection networks for pan-sharpening [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 1366-1375.
26	Cai J J, Huang B.. Super-resolution-guided progressive pansharpening based on a deep convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59 (6): 5206- 5220.

方法	GF1			QB
方法	PSNR↑/dB	SSIM↑	ERGAS↓	PSNR↑/dB	SSIM↑	ERGAS↓
PSCF	39.6091	0.9861	6.2063	43.6585	0.9895	3.8693
PSCINN	40.0811	0.9867	5.7596	42.1693	0.9871	4.7166
DISPNet	46.6074	0.9949	2.9815	48.1770	0.9940	2.4140
PNN	45.1481	0.9931	3.4656	47.0071	0.9912	2.7640
PanNet	46.1890	0.9947	3.1255	47.7010	0.9940	2.3748
LAGConv	47.6586	0.9952	2.7133	47.8898	0.9937	2.3554
MSDCNN	46.2428	0.9941	3.1067	47.6549	0.9927	2.5328
GPPNN	43.7842	0.9916	4.1566	46.2982	0.9912	3.0878
SRPPNN	46.9454	0.9953	2.8718	48.5581	0.9945	2.1984
本文方法	48.5660	0.9957	0.8423	48.5080	0.9947	0.6159

方法	WV2			WV4
方法	PSNR↑/dB	SSIM↑	ERGAS↓	PSNR↑/dB	SSIM↑	ERGAS↓
PSCF	40.3956	0.9821	17.1725	42.1347	0.9918	3.5732
PSCINN	38.0735	0.9761	21.3113	32.4285	0.9776	10.922
DISPNet	36.7243	0.9663	25.6244	38.1584	0.9781	5.5745
PNN	40.4835	0.9813	16.8310	40.7926	0.9895	4.1806
PanNet	39.5244	0.9785	18.7344	40.3312	0.9876	4.4710
LAGConv	39.9678	0.9804	17.8407	40.1784	0.9876	4.4442
MSDCNN	40.4649	0.9820	16.8003	41.5396	0.9905	3.8784
GPPNN	40.7654	0.9826	16.4375	42.2707	0.9913	3.5499
SRPPNN	39.8992	0.9797	18.0026	40.7506	0.9891	4.2770
本文方法	39.8846	0.9797	18.2235	41.4290	0.9898	3.9733

方法	WV3			IKONOS
方法	PSNR↑/dB	SSIM↑	ERGAS↓	PSNR↑/dB	SSIM↑	ERGAS↓
PSCF	29.6898	0.9147	47.1054	33.6036	0.9207	9.7883
PSCINN	25.7267	0.8128	70.7151	32.6893	0.8960	10.7471
DISPNet	30.5656	0.9203	43.9822	37.3609	0.9462	6.4953
PNN	30.1288	0.9184	44.5506	36.4540	0.9343	7.3178
PanNet	29.9108	0.8591	50.0833	36.7974	0.9397	6.9012
LAGConv	31.1914	0.9084	42.2248	37.7622	0.9333	7.6344
MSDCNN	30.0524	0.9250	44.8807	35.7193	0.9341	7.8613
GPPNN	26.0173	0.8344	69.8300	33.1218	0.9192	10.3322
SRPPNN	31.0001	0.9119	42.3682	37.5427	0.9494	6.4896
本文方法	34.0009	0.9852	2.8504	42.0446	0.9914	2.7870

方法	WV3			IKONOS
方法	D_λ↓	D_S↓	Q↑	D_λ↓	D_S↓	Q↑
PSCF	0.1073	0.0683	0.8317	0.1867	0.1236	0.7137
PSCINN	0.1214	0.0925	0.7967	0.4231	0.1696	0.4816
DISPNet	0.0423	0.0500	0.9099	0.0869	0.0708	0.8493
PNN	0.0519	0.0688	0.8829	0.1199	0.1081	0.7858
PanNet	0.0421	0.0841	0.8777	0.0827	0.1109	0.8161
LAGConv	0.0605	0.0722	0.8717	0.0916	0.0749	0.8410
MSDCNN	0.0329	0.0669	0.9030	0.1213	0.1023	0.7898
GPPNN	0.0732	0.1249	0.8107	0.3066	0.1491	0.5913
SRPPNN	0.0455	0.0661	0.8919	0.0773	0.0626	0.8656
本文方法	0.0336	0.0426	0.9252	0.0611	0.0430	0.8988

[1]	Xuyang LI, Yan GAO, Hongyan QUAN. Road extraction algorithm for remote sensing images guided by deep semantics [J]. J* E* C* N* U* N* S*, 2026, 2026(4): 73-82.
[2]	Zefeng HE, Fuke SHEN, Tongquan WEI. Dual decision adaptive freezing for fast and accurate transfer learning [J]. J* E* C* N* U* N* S*, 2025, 2025(6): 29-38.
[3]	Sijia ZHAO, Fanyu HAN, Wei WANG. Research on the GitHub developer geographic location prediction method based on multi-dimensional feature fusion [J]. J* E* C* N* U* N* S*, 2025, 2025(5): 1-13.
[4]	Luping CAO, Yong XIA. Detection of high-resolution high-bandwidth fractional vortex beams under atmospheric turbulence [J]. J* E* C* N* U* N* S*, 2025, 2025(3): 51-60.
[5]	Yiping LIANG, Luwei XIAO, Linlin WANG. Label-perception augmented causal analysis of mental health over social media [J]. J* E* C* N* U* N* S*, 2025, 2025(1): 124-137.
[6]	Lulu JIANG, Siqi SUN, Haidong ZOU, Lina LU, Rui FENG. Diabetic retinopathy grading based on dual-view image feature fusion [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 39-48.
[7]	Caidie HUANG, Xinping WANG, Liangyu CHEN, Yong LIU. Research on a knowledge tracking model based on the stacked gated recurrent unit residual network [J]. Journal of East China Normal University(Natural Science), 2022, 2022(6): 68-78.
[8]	Yilin MA, Huiling TAO, Qiwen DONG, Ye WANG. Prediction of remaining useful life of aeroengines based on the Transformer with multi-feature fusion [J]. Journal of East China Normal University(Natural Science), 2022, 2022(5): 219-232.
[9]	Zejie WANG, Chaomin SHEN, Chun ZHAO, Xinmei LIU, Jie CHEN. Recognition of classroom learning behaviors based on the fusion of human pose estimation and object detection [J]. Journal of East China Normal University(Natural Science), 2022, 2022(2): 55-66.
[10]	Bo LIU, Xiaodong BAI, Gengxin ZHANG, Jun SHEN, Jidong XIE, Laiding ZHAO, Tao HONG. Review of deep learning in cognitive radio [J]. Journal of East China Normal University(Natural Science), 2021, 2021(1): 36-52.
[11]	ZHANG Xu, HUANG Dingjiang. Defect detection on aluminum surfaces based on deep learning [J]. Journal of East China Normal University(Natural Science), 2020, 2020(6): 105-114.
[12]	HAN Chengcheng, LI Lei, LIU Tingting, GAO Ming. Approaches for semantic textual similarity [J]. Journal of East China Normal University(Natural Science), 2020, 2020(5): 95-112.
[13]	YANG Kang, HANG Ding-jiang, GAO Ming. A review of machine reading comprehension for automatic QA [J]. Journal of East China Normal University(Natural Sc, 2019, 2019(5): 36-52.
[14]	CHEN Yuan-zhe, KUANG Jun, LIU Ting-ting, GAO Ming, ZHOU Ao-ying. A survey on coreference resolution [J]. Journal of East China Normal University(Natural Sc, 2019, 2019(5): 16-35.
[15]	LIU Heng-yu, ZHANG Tian-cheng, WU Pei-wen, YU Ge. A review of knowledge tracking [J]. Journal of East China Normal University(Natural Sc, 2019, 2019(5): 1-15.