基于帧注意力的视频去模糊模型

doi:10.3969/j.issn.1000-5641.2026.04.004

摘要/Abstract

摘要：

现有的视频去模糊模型分为基于滑动窗口和基于循环结构两种: 一种是基于滑动窗口的方法, 这种方法将连续的若干帧输入模型, 这样的设计使得中心帧能够利用到的清晰信息十分有限; 另一种是基于循环结构的方法, 这种方法将视频中的每一帧以循环神经网络的方式输入模型, 但包含清晰信息的隐式特征会随着传播距离的增加而逐渐衰弱. 基于上述问题, 本文提出了基于帧注意力的方法. 这是一种不同于上述两种方法的全新的视频去模糊框架, 会为中心帧选取最适合的辅助帧集合. 为了解决帧间位移较大的问题, 本文设计了轻量的光流模型, 并采用知识蒸馏的方式进行微调, 提升了对齐模块处理模糊图像的能力. 在两个常用的视频去模糊数据集上进行了实验, 结果表明, 该方法的峰值信噪比达到31.91 dB, 具有很好的性能.

关键词: 视频去模糊, 光流估计, 图像复原

Abstract:

Existing video deblurring models can be categorized into two types: sliding window-based and loop-based methods. Sliding window-based methods process consecutive frames as input, limiting the clear information available to the central frame. Loop-based methods use a recurrent neural network to process each video frame sequentially, but the implicit features carrying clear information gradually weaken as the propagation distance increases. To address these issues, this article proposes a frame attention-based method—a novel video deblurring framework that differs from the two aforementioned methods by selecting the most suitable set of auxiliary frames for the central frame. To mitigate the challenge of large displacement between frames, we designed a lightweight optical flow model and fine-tuned it using distillation to enhance the alignment module’s ability to process blurred images. Experiments on two commonly used video deblurring datasets demonstrate that our method achieves a peak signal-to-noise ratio of 31.91 dB, indicating strong performance.

Key words: video deblurring, optical flow estimation, image restoration

中图分类号:

TP389.1

鲁梦洁, 陈蕾. 基于帧注意力的视频去模糊模型[J]. 华东师范大学学报（自然科学版）, 2026, 2026(4): 29-41.

Mengjie LU, Lei CHEN. Video deblurring model based on frame attention[J]. J* E* C* N* U* N* S*, 2026, 2026(4): 29-41.

图/表 15

图1

图2

图3

图4

图5

图6

表1

表2

图7

表3

表4

图8

图9

表5

图10

参考文献 31

1	Lee H S, Lee K M. Dense 3D reconstruction from severely blurred images using a single moving camera [C]// 2013 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2013: 273-280.
2	Mei C, Reid I. Modeling and generating complex motion blur for real-time tracking [C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2008: 1-8.
3	Jin H, Favaro P, Cipolla R. Visual tracking in the presence of motion blur [C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). IEEE, 2005: 18-25.
4	Wang Y S, Lu Y F, Gao Y, et al. Efficient video deblurring guided by motion magnitude [C]// Computer Vision – ECCV 2022. 2022: 413-429.
5	Nah S, Son S, Lee K M. Recurrent neural networks with intra-frame iterations for video deblurring [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019: 8102-8111.
6	Kim T H, Lee K M, Schölkopf B, et al. Online video deblurring via dynamic temporal blending network [C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 4038-4047.
7	Nah S, Kim T H, Lee K M. Deep multi-scale convolutional neural network for dynamic scene deblurring [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 3883-3891.
8	蒲泽栋, 马伟, 米庆.. 面向视频去模糊的高效时空特征提取循环神经网络. 计算机辅助设计与图形学学报, 2023, 35 (11): 1720- 1730.
9	欧阳文森. 基于多尺度循环网络的运动模糊图像复原方法研究 [D]. 天津: 天津大学, 2022.
10	Wang X T, Chan K C K, Yu K, et al. EDVR: video restoration with enhanced deformable convolutional networks [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2019: 1-10.
11	Su S C, Delbracio M, Wang J, et al. Deep video deblurring for hand-held cameras [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 237-246.
12	Pan J S, Xu B M, Bai H R, et al.. Cascaded deep video deblurring using temporal sharpness prior and non-local spatial-temporal similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (8): 9411- 9425.
13	Pan J S, Bai H R, Tang J H. Cascaded deep video deblurring using temporal sharpness prior [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 3043-3051.
14	Li D X, Xu C C, Zhang K H, et al. ARVo: learning all-range volumetric correspondence for video deblurring [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 7721-7731.
15	邹世奇, 刘洪.. 基于金字塔池化与注意力机制的视频去模糊算法. 智能计算机与应用, 2023, 13 (12): 75- 79.
16	徐浙峰. 暗光场景下视频去模糊研究 [D]. 杭州: 杭州电子科技大学, 2023.
17	段风志. 基于可变形卷积网络的视频去模糊算法 [D]. 哈尔滨: 哈尔滨工业大学, 2022.
18	Zhong Z H, Gao Y, Zheng Y Q, et al. Efficient spatio-temporal recurrent neural network for video deblurring [C]// Computer Vision – ECCV 2020. 2020: 191-207.
19	Kim T H, Lee K M. Generalized video deblurring for dynamic scenes [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015: 5426-5434.
20	Bar L, Berkels B, Rumpf M, et al. A variational framework for simultaneous motion estimation and restoration of motion-blurred video [C]// 2007 IEEE 11th International Conference on Computer Vision. IEEE, 2007: 1-8.
21	Cho S, Wang J, Lee S.. Video deblurring for hand-held cameras using patch-based synthesis. ACM Transactions on Graphics, 2012, 31 (4): 1- 9.
22	Kim T H, Lee K M. Segmentation-free dynamic scene deblurring [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2014: 2766-2773.
23	Dai S Y, Wu Y. Motion from blur [C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2008: 1-8.
24	Huang Z Y, Shi X Y, Zhang C, et al. FlowFormer: a transformer architecture for optical flow [C]// Computer Vision – ECCV 2022. 2022: 668-685.
25	Hore A, Ziou D. Image quality metrics: PSNR vs. SSIM [C]// 2010 20th International Conference on Pattern Recognition. IEEE, 2010: 2366-2369.
26	Tao X, Gao H Y, Shen X Y, et al. Scale-recurrent network for deep image deblurring [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 8174-8182.
27	Zhou S C, Zhang J W, Pan J S, et al. Spatio-temporal filter adaptive network for video deblurring [C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2019: 2482-2491.
28	Xiang X G, Wei H, Pan J S. Deep video deblurring using sharpness features from exemplars [J]. IEEE Transactions on Image Processing, 2020, 29: 8976-8987.
29	Zhang X Q, Wang T, Jiang R H, et al.. Multi-attention convolutional neural network for video deblurring. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32 (4): 1986- 1997.
30	Son H, Lee J Y, Lee J, et al.. Recurrent video deblurring with blur-invariant motion estimation and pixel volumes. ACM Transactions on Graphics, 2021, 40 (5): 1- 18.
31	Li C, Song L, Xie R, et al.. Local bidirection recurrent network for efficient video deblurring with the fused temporal merge module. ACM Transactions on Multimedia Computing, Communications, and Applications, 2023, 19 (5s): 1- 18.

模型	GoPro上的实验结果		BSD上的实验结果		参数量/(×10⁶)	计算量(GMACs)
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	参数量/(×10⁶)	计算量(GMACs)
SRN^[26]	29.94	0.8953			10.25	1527.01
DBN^[11]	28.55	0.8595	31.84	0.9170	15.31	784.75
ESTRNN^[18]	31.02	0.9109	33.36	0.9370	2.38	215.26
IFIRNN^[5]	29.69	0.8867	33.00	0.9330	1.64	217.89
STFAN^[27]	28.59	0.8608			5.37
DVDSEF^[28]	31.01	0.9130	29.95	0.8692	16.25
MACNN^[29]	30.64	0.9400			7.92
PVDNet^[30]	30.02	0.8980	32.88	0.9300	10.50	1004.12
LBRNN^[31]	31.72	0.9290	33.50	0.9420	1.86	216.14
FANet(L3B3E4)	30.55	0.9161	32.55	0.9289	1.28	111.09
FANet(L4B5E8)	31.38	0.9255	33.02	0.9311	1.62	172.68
FANet(L5B7E12)	31.91	0.9292	33.59	0.9473	2.15	235.52

模块			评价指标
质量评估	对齐	蒸馏	PSNR/dB	SSIM
-	-	-	28.38	0.8551
√	-	-	28.15	0.8442
-	√	-	28.49	0.8667
-	√	√	28.79	0.8716
√	√	-	28.83	0.8734
√	√	√	29.19	0.8846

参数m的值	PSNR/dB	计算量(GMACs)
1	28.95	78.81
2	29.19	111.09
3	29.23	143.50

参数$ {\mu } $的值	PSNR/dB	SSIM
0.4	29.14	0.8843
0.5	29.19	0.8846
0.6	29.16	0.8845
0.7	29.11	0.8840

输入CLIP的文本	SSIM
输入CLIP的文本	图1	图2	图3	图4
a clear photo	0.19680	0.10520	0.01826	0.54700
a non-blur photo	0.21470	0.15400	0.02263	0.50000
a high-quality photo	0.05032	0.04272	0.15200	0.06280
a good photo	0.05260	0.03906	0.11280	0.12420