Journal of East China Normal University(Natural Science) >

2025 , Vol. 2025 >Issue 1: 82 - 96

DOI: https://doi.org/10.3969/j.issn.1000-5641.2025.01.007

Computer Science

Time series uncertainty forecasting based on graph augmentation and attention mechanism

  • Chaojie MEN ,
  • Jing ZHAO ,
  • Nan ZHANG
Expand
  • 1. School of Computer Science and Technology, East China Normal University, Shanghai　200062, China
    2. College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, Zhejiang　325035, China

Received date: 2023-12-27

  Online published: 2025-01-20

Copyright

, 2025, Copyright reserved © 2025.
Fold

Abstract

To improve the ability to predict future events and effectively address uncertainty, we propose a network architecture based on graph augmentation and attention mechanisms for uncertainty forecasting in multivariate time series. By introducing an implicit graph structure and integrating graph neural network techniques, we capture the mutual dependencies among sequences to model the interactions between time series. We utilize attention mechanisms to capture temporal patterns within the same sequence for modeling the dynamic evolution patterns of time series. We utilize the Monte Carlo dropout method to approximate model parameters and model the predicted sequences as a stochastic distribution, thus achieving accurate uncertainty forecasting in time series. The experimental results indicate that this approach maintains a high level of prediction precision while providing reliable uncertainty estimation, thus providing confidence for use in decision-making tasks.

Key words： uncertainty; graph augmentation; time series; attention mechanism

Cite this article

Chaojie MEN , Jing ZHAO , Nan ZHANG . Time series uncertainty forecasting based on graph augmentation and attention mechanism[J]. Journal of East China Normal University(Natural Science), 2025 , 2025(1) : 82 -96 . DOI: 10.3969/j.issn.1000-5641.2025.01.007

References

1 LIU C H, HOI S C H, ZHAO P L, et al. Online ARIMA algorithms for time series prediction [C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence. AAAI, 2016: 1867-1873.
2 SHUMWAY R H, STOFFER D S. Time Series Analysis and Its Applications: With R Examples [M]. Berlin: Springer, 2017: 75-146.
3 CLEVELAND R B, CLEVELAND W S, MCRAE J E, et al.. STL: A seasonal-trend decomposition procedure based on loess. Journal of Office Statistics, 1990, 6 (1): 3- 73.
4 HAN Z Y, LIU Y, ZHAO J, et al.. Real time prediction for converter gas tank levels based on multi-output least square support vector regressor. Control Engineering Practice, 2012, 20 (12): 1400- 1409.
5 GIRARD A, RASMUSSEN C E, CANDELA J Q, et al. Gaussian process priors with uncertain inputs application to multiple-step ahead time series forecasting [C]// Proceedings of the 15th International Conference on Neural Information Processing Systems. Cambridge, MA, United States: MIT Press, 2002: 545-552.
6 HAN Z Y, ZHAO J, LEUNG H, et al.. A review of deep learning models for time series prediction. IEEE Sensors Journal, 2021, 21 (6): 7833- 7848.
7 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6000-6010.
8 WEN Q S, ZHOU T, ZHANG C, et al. Transformers in time series: A survey [C]// Proceedings of the 32nd International Joint Conference on Artificial Intelligence. 2023: 6778-6786.
9 LI S Y, JIN X Y, XUAN Y, et al. Enhancing the locality and breaking the memory bottleneck of Transformer on time series forecasting [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2019: 5243-5253.
10 WU S F, XIAO X, DING Q G, et al. Adversarial sparse transformer for time series forecasting [C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2020: 17105-17115.
11 ZHOU H Y, ZHANG S H, PENG J Q, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C]// Proceedings of the AAAI Conference on Artificial Intelligence, 35, No. 12: AAAI-21 Technical Tracks 12. 2021: 11106-11115.
12 WU H X, XU J H, WANG J M, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting [C]// Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2021: 22419-22430.
13 LIU S Z, YU H, LIAO C, et al. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting [C]// International Conference on Learning Representations (ICLR 2022). 2022. https://openreview.net/pdf?id=0EXmFzUn5I.
14 ZHOU T, MA Z Q, WEN Q S, et al. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting [C]// Proceedings of the 39th International Conference on Machine Learning. 2022: 27268-27286.
15 ZHANG Y H, YAN J C. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting [C]// The 11th International Conference on Learning Representations (ICLR 2023 Conference). 2023. https://openreview.net/forum?id=vSVLM2j9eie.
16 ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting? [C]// Proceedings of the 37th AAAI Conference on Artificial Intelligence and 35th Conference on Innovative Applications of Artificial Intelligence and 13th Symposium on Educational Advances in Artificial Intelligence. AAAI, 2023: 11121-11128.
17 CHALLU C, OLIVARES K G, ORESHKIN B N, et al. N-HiTS: Neural hierarchical interpolation for time series forecasting [C]// Proceedings of the 37th AAAI Conference on Artificial Intelligence and 35th Conference on Innovative Applications of Artificial Intelligence and 13th Symposium on Educational Advances in Artificial Intelligence. AAAI, 2023: 6989-6997.
18 VIJAY E, JATI A, NGUYEN N, et al. TSMixer: Lightweight MLP-mixer model for multivariate time series forecasting [C]// Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2023: 459-469.
19 LAI G K, CHANG W C, YANG Y M, et al. Modeling long-and short-term temporal patterns with deep neural networks [C]// The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 2018: 95-104.
20 WU Z H, PAN S R, LONG G D, et al. Connecting the dots: multivariate time series forecasting with graph neural networks [C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2020: 753-763.
21 LIU M H, ZENG A L, CHEN M X, et al. SCINet: Time series modeling and forecasting with sample convolution and interaction [C]// Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022). 2022. https://openreview.net/pdf?id=AyajSjTAzmg.
22 VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks [C]// Conference paper at ICLR (International Conference on Learning Representations) 2018. 2018. https://openreview.net/pdf?id=rJXMpikCZ.
23 KIUREGHIAN A D, DITLEVSEN O.. Aleatory or epistemic? Does it matter?. Structural Safety, 2009, 31 (2): 105- 112.
24 LIU J Z, PADHY S, REN J, et al.. A simple approach to improve single-model deep uncertainty via distance-awareness. Journal of Machine Learning Research, 2023, 24 (42): 1667- 1729.
25 BLUNDELL C, CORNEBISE J, KAVUKCUOGLU K, et al. Weight uncertainty in neural network [C]// Proceedings of the 32nd International Conference on Machine Learning- Volume 37. 2015: 1613-1622.
26 GAL Y, GHAHRAMANI Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning [C]// Proceedings of the 33rd International Conference on Machine Learning. 2016: 1050-1059.
27 KENDALL A, GAL Y. What uncertainties do we need in Bayesian deep learning for computer vision? [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 5580-5590.
28 LAKSHMINARAYANAN B, PRITZEL A, BLUNDELL C. Simple and scalable predictive uncertainty estimation using deep ensembles [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 6405-6416.
29 PENG Z L, GUO Z H, HUANG W, et al.. Conformer: Local features coupling global representations for recognition and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (8): 9454- 9468.
30 KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 94-90.
31 DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2021. 2021. https://openreview.net/forum?id=YicbFdNTTy.
32 DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). ACM, 2019: 4171-4186.
33 GUO C, PLEISS G, SUN Y, et al. On calibration of modern neural networks [C]// Proceedings of the 34th International Conference on Machine Learning - Volume 70. 2017: 1321-1330.
34 KULESHOV V, FENNER N, ERMON S. Accurate uncertainties for deep learning using calibrated regression [C]// Proceedings of the 35th International Conference on Machine Learning. 2018: 2796-2804.
35 BAHDANAU D, CHO K H, BENGIO Y. Neural machine translation by jointly learning to align and translate [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2015. 2015. https://iclr.cc/archive/www/lib/exe/fetch.php%3Fmedia=iclr2015:bahdanau-iclr2015.pdf.
36 WANG H Q, PENG J, HUANG F H, et al. MICN: Multi-scale local and global context modeling for long-term series forecasting [C]// Conference Paper at ICLR (International Conference on Learning Representations) 2023. 2023. https://openreview.net/pdf?id=zt53IDUR1U.
Options
Outlines

/

〈 〉