基于图增强和注意力机制的时间序列不确定性预测

doi:10.3969/j.issn.1000-5641.2025.01.007

摘要/Abstract

摘要：

为提升对未来事件的预判能力并有效应对不确定性, 提出了一种基于图增强和注意力机制的网络架构, 用于多元时间序列的不确定性预测. 通过引入隐含式图结构并结合图神经网络技术, 捕捉各序列间相互依赖关系, 从而建模时间序列之间的相互影响; 运用注意力机制捕捉同一序列内的时序变化模式, 以建模时间序列的动态演变规律; 采用蒙特卡洛随机失活 (Monte Carlo dropout) 方法近似模型参数, 并将预测序列建模为随机分布, 以实现精确的时间序列不确定性预测. 实验证明, 该方法在保持较高预测精度的同时, 还能进行可靠的不确定性估计, 可以为决策任务提供置信度信息.

关键词: 不确定性, 图增强, 时间序列, 注意力机制

Abstract:

To improve the ability to predict future events and effectively address uncertainty, we propose a network architecture based on graph augmentation and attention mechanisms for uncertainty forecasting in multivariate time series. By introducing an implicit graph structure and integrating graph neural network techniques, we capture the mutual dependencies among sequences to model the interactions between time series. We utilize attention mechanisms to capture temporal patterns within the same sequence for modeling the dynamic evolution patterns of time series. We utilize the Monte Carlo dropout method to approximate model parameters and model the predicted sequences as a stochastic distribution, thus achieving accurate uncertainty forecasting in time series. The experimental results indicate that this approach maintains a high level of prediction precision while providing reliable uncertainty estimation, thus providing confidence for use in decision-making tasks.

Key words: uncertainty, graph augmentation, time series, attention mechanism

中图分类号:

TP391

门超杰, 赵静, 张楠. 基于图增强和注意力机制的时间序列不确定性预测[J]. 华东师范大学学报（自然科学版）, 2025, 2025(1): 82-96.

Chaojie MEN, Jing ZHAO, Nan ZHANG. Time series uncertainty forecasting based on graph augmentation and attention mechanism[J]. J* E* C* N* U* N* S*, 2025, 2025(1): 82-96.

图/表 8

图1

表1

图2

图3

图4

表2

表3

表4

参考文献 36

1	LIU C H, HOI S C H, ZHAO P L, et al. Online ARIMA algorithms for time series prediction [C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence. AAAI, 2016: 1867-1873.
2	SHUMWAY R H, STOFFER D S. Time Series Analysis and Its Applications: With R Examples [M]. Berlin: Springer, 2017: 75-146.
3	CLEVELAND R B, CLEVELAND W S, MCRAE J E, et al.. STL: A seasonal-trend decomposition procedure based on loess. Journal of Office Statistics, 1990, 6 (1): 3- 73.
4	HAN Z Y, LIU Y, ZHAO J, et al.. Real time prediction for converter gas tank levels based on multi-output least square support vector regressor. Control Engineering Practice, 2012, 20 (12): 1400- 1409.
5	GIRARD A, RASMUSSEN C E, CANDELA J Q, et al. Gaussian process priors with uncertain inputs application to multiple-step ahead time series forecasting [C]// Proceedings of the 15th International Conference on Neural Information Processing Systems. Cambridge, MA, United States: MIT Press, 2002: 545-552.
6	HAN Z Y, ZHAO J, LEUNG H, et al.. A review of deep learning models for time series prediction. IEEE Sensors Journal, 2021, 21 (6): 7833- 7848.
7	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6000-6010.
8	WEN Q S, ZHOU T, ZHANG C, et al. Transformers in time series: A survey [C]// Proceedings of the 32nd International Joint Conference on Artificial Intelligence. 2023: 6778-6786.
9	LI S Y, JIN X Y, XUAN Y, et al. Enhancing the locality and breaking the memory bottleneck of Transformer on time series forecasting [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2019: 5243-5253.
10	WU S F, XIAO X, DING Q G, et al. Adversarial sparse transformer for time series forecasting [C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2020: 17105-17115.
11	ZHOU H Y, ZHANG S H, PENG J Q, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C]// Proceedings of the AAAI Conference on Artificial Intelligence, 35, No. 12: AAAI-21 Technical Tracks 12. 2021: 11106-11115.
12	WU H X, XU J H, WANG J M, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting [C]// Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2021: 22419-22430.
13	LIU S Z, YU H, LIAO C, et al. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting [C]// International Conference on Learning Representations (ICLR 2022). 2022. https://openreview.net/pdf?id=0EXmFzUn5I.
14	ZHOU T, MA Z Q, WEN Q S, et al. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting [C]// Proceedings of the 39th International Conference on Machine Learning. 2022: 27268-27286.
15	ZHANG Y H, YAN J C. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting [C]// The 11th International Conference on Learning Representations (ICLR 2023 Conference). 2023. https://openreview.net/forum?id=vSVLM2j9eie.
16	ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting? [C]// Proceedings of the 37th AAAI Conference on Artificial Intelligence and 35th Conference on Innovative Applications of Artificial Intelligence and 13th Symposium on Educational Advances in Artificial Intelligence. AAAI, 2023: 11121-11128.
17	CHALLU C, OLIVARES K G, ORESHKIN B N, et al. N-HiTS: Neural hierarchical interpolation for time series forecasting [C]// Proceedings of the 37th AAAI Conference on Artificial Intelligence and 35th Conference on Innovative Applications of Artificial Intelligence and 13th Symposium on Educational Advances in Artificial Intelligence. AAAI, 2023: 6989-6997.
18	VIJAY E, JATI A, NGUYEN N, et al. TSMixer: Lightweight MLP-mixer model for multivariate time series forecasting [C]// Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2023: 459-469.
19	LAI G K, CHANG W C, YANG Y M, et al. Modeling long-and short-term temporal patterns with deep neural networks [C]// The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 2018: 95-104.
20	WU Z H, PAN S R, LONG G D, et al. Connecting the dots: multivariate time series forecasting with graph neural networks [C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2020: 753-763.
21	LIU M H, ZENG A L, CHEN M X, et al. SCINet: Time series modeling and forecasting with sample convolution and interaction [C]// Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022). 2022. https://openreview.net/pdf?id=AyajSjTAzmg.
22	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks [C]// Conference paper at ICLR (International Conference on Learning Representations) 2018. 2018. https://openreview.net/pdf?id=rJXMpikCZ.
23	KIUREGHIAN A D, DITLEVSEN O.. Aleatory or epistemic? Does it matter?. Structural Safety, 2009, 31 (2): 105- 112.
24	LIU J Z, PADHY S, REN J, et al.. A simple approach to improve single-model deep uncertainty via distance-awareness. Journal of Machine Learning Research, 2023, 24 (42): 1667- 1729.
25	BLUNDELL C, CORNEBISE J, KAVUKCUOGLU K, et al. Weight uncertainty in neural network [C]// Proceedings of the 32nd International Conference on Machine Learning- Volume 37. 2015: 1613-1622.
26	GAL Y, GHAHRAMANI Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning [C]// Proceedings of the 33rd International Conference on Machine Learning. 2016: 1050-1059.
27	KENDALL A, GAL Y. What uncertainties do we need in Bayesian deep learning for computer vision? [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 5580-5590.
28	LAKSHMINARAYANAN B, PRITZEL A, BLUNDELL C. Simple and scalable predictive uncertainty estimation using deep ensembles [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6405-6416.
29	PENG Z L, GUO Z H, HUANG W, et al.. Conformer: Local features coupling global representations for recognition and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (8): 9454- 9468.
30	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 94-90.
31	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2021. 2021. https://openreview.net/forum?id=YicbFdNTTy.
32	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). ACM, 2019: 4171-4186.
33	GUO C, PLEISS G, SUN Y, et al. On calibration of modern neural networks [C]// Proceedings of the 34th International Conference on Machine Learning - Volume 70. 2017: 1321-1330.
34	KULESHOV V, FENNER N, ERMON S. Accurate uncertainties for deep learning using calibrated regression [C]// Proceedings of the 35th International Conference on Machine Learning. 2018: 2796-2804.
35	BAHDANAU D, CHO K H, BENGIO Y. Neural machine translation by jointly learning to align and translate [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2015. 2015. https://iclr.cc/archive/www/lib/exe/fetch.php%3Fmedia=iclr2015:bahdanau-iclr2015.pdf.
36	WANG H Q, PENG J, HUANG F H, et al. MICN: Multi-scale local and global context modeling for long-term series forecasting [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2023. 2023. https://openreview.net/pdf?id=zt53IDUR1U.

数据集	采集率	维度	数据量	训练集	验证集	测试集	预测步长
ETTh1	次/h	7	13819	8105	2857	2857	{24,48,168,336,720}
ETTm1	次/15 min	7	56403	33937	11233	11233	{24,48,168,288,672}
WTH	次/h	12	34587	20655	6967	6965	{24,48,168,336,720}
ECL	次/h	321	25899	18053	2609	5237	{24,48,168,336,720}
ILI	次/周	7	845	601	74	170	{24,36,48,60}
Traffic	次/h	862	17139	11921	1733	3485	{24,48,168,336,720}

数据集	MSE						MAE
数据集	Informer	Autoformer	Pyraformer	FEDformer	Crossformer	TSUF	Informer	Autoformer	Pyraformer	FEDformer	Crossformer	TSUF
ETTh1	0.907	0.482	0.747	0.410	0.405	0.387	0.739	0.478	0.653	0.444	0.437	0.411
ETTm1	0.749	0.522	0.601	0.370	0.361	0.315	0.640	0.484	0.535	0.414	0.395	0.354
WTH	0.574	0.516	0.456	0.489	0.432	0.475	0.552	0.509	0.479	0.501	0.461	0.484
ECL	0.392	0.332	0.485	0.312	0.309	0.168	0.448	0.405	0.466	0.394	0.359	0.259
ILI	4.878	3.116	4.591	2.795	3.387	1.861	1.513	1.228	1.460	1.156	1.236	0.924
Traffic	0.690	0.618	0.634	0.597	0.525	0.383	0.384	0.390	0.348	0.380	0.294	0.263

数据集	MSE						MAE
数据集	LSTMa	LSTNet	MTGNN	DLinear	MICN	TSUF	LSTMa	LSTNet	MTGNN	DLinear	MICN	TSUF
ETTh1	1.193	1.909	0.568	0.403	0.504	0.387	0.896	1.165	0.538	0.424	0.497	0.411
ETTm1	1.566	1.981	0.433	0.318	0.346	0.315	1.032	1.789	0.447	0.355	0.386	0.354
WTH	1.121	0.731	0.442	0.483	0.462	0.475	0.850	0.644	0.468	0.492	0.474	0.484
ECL	1.043	0.469	0.326	0.232	0.170	0.168	0.838	0.512	0.367	0.315	0.279	0.259
ILI	4.778	5.300	4.861	2.864	2.664	1.861	1.432	1.657	1.507	1.197	1.086	0.924
Traffic	0.961	0.716	0.527	0.397	0.520	0.383	0.537	0.438	0.315	0.282	0.304	0.263

预测步长/个	MSE				MAE
预测步长/个	LSTNet	Att	Graph	Att+Graph	LSTNet	Att	Graph	Att+Graph
48	1.456	0.403	0.377	0.344	0.960	0.430	0.406	0.380
168	1.997	0.427	0.426	0.401	1.214	0.444	0.433	0.414
336	2.655	0.486	0.455	0.432	1.369	0.486	0.465	0.435

[1]	胡雯婧, 蒋龙泉, 余俊龙, 徐伊茜, 刘奇鹏, 梁雷, 李嘉豪. 基于知识蒸馏的轻量化农作物病害识别算法[J]. 华东师范大学学报（自然科学版）, 2025, 2025(1): 59-71.
[2]	王畅, 马丹, 许华容, 陈攀峰, 陈梅, 李晖. SA-MGKT: 基于自注意力融合的多图知识追踪方法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 20-31.
[3]	刘家伟, 林欣. 基于解耦常识性关联的图像描述生成算法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 131-142.
[4]	冯路平, 施力业, 吴雯, 郑骏, 胡文心, 郑巍. 基于偏好级别陌生人信息辅助的推荐系统[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 53-64.
[5]	姜璐璐, 孙司琦, 邹海东, 陆丽娜, 冯瑞. 基于双视图特征融合的糖尿病视网膜病变分级[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 39-48.
[6]	陈道佳, 陈志云. 分层级描述感知的个性化推荐系统[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 73-84.
[7]	段志尚, 冉懿, 吕笃良, 祁杰, 钟佳晨, 袁培森. 基于残差网络和深度可分离卷积增强自注意力机制的窃电识别[J]. 华东师范大学学报（自然科学版）, 2023, 2023(5): 193-204.
[8]	葛进, 陆雪松. 基于UI图像的Web前端代码自动生成[J]. 华东师范大学学报（自然科学版）, 2023, 2023(5): 100-109.
[9]	邓未, 周昉. 基于多模态的急性肾衰竭预测模型[J]. 华东师范大学学报（自然科学版）, 2023, 2023(4): 52-64.
[10]	武朝阳, 毛嘉莉. 基于神经网络的行驶时长预测[J]. 华东师范大学学报（自然科学版）, 2023, 2023(2): 106-118.
[11]	童根梅, 朱敏. 基于双向长短记忆网络和门控注意力的文本分类网络[J]. 华东师范大学学报（自然科学版）, 2022, 2022(2): 67-75.
[12]	杨梦晨, 陈旭栋, 蔡鹏, 倪葎. 早期时间序列分类方法研究综述[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 115-133.
[13]	马晓琴, 薛晓慧, 罗红郊, 刘通宇, 袁培森. 基于t-LeNet与时间序列分类的窃电行为检测[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 104-114.
[14]	梁艳春, 房爱莲. 基于多通道卷积神经网络的中文文本关系抽取[J]. 华东师范大学学报（自然科学版）, 2021, 2021(3): 96-104.
[15]	李小昌, 陈贝, 董启文, 陆雪松. 基于自编码器的旅行同伴挖掘[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 179-188.