The explosion of time series (TS) data, driven by advancements in technology, necessitates sophisticated analytical methods. Modern management systems increasingly rely on analyzing this data, highlighting the importance of efficient processing techniques. State-of-the-art machine learning (ML) approaches for TS analysis and forecasting are becoming prevalent. In this paper, we provide an overview of suitable algorithms for TS regression tasks, comparing their performance with each other and with the traditional autoregressive integrated moving average (ARIMA) method across diverse datasets–including synthetic data, long-term recorded data, data containing outliers, and data with missing values. Our focus is on forecasting accuracy, especially for long-term predictions. A key strength of our work is the comprehensive collection of various ML methods tailored for TS data, along with their evaluation across different datasets that include various challenging scenarios. The results show that tree-based ensemble methods outperform other algorithms in most cases. This study aims to assist researchers and practitioners in selecting the most appropriate algorithm based on specific forecasting requirements and data characteristics.

References

• Ab Aziz, M. F., Mostafa, S. A., Foozy, C. F. M., Mohammed, M. A., Elhoseny, M., & Abualkishik, A. Z. (2021). Integrating Elman recurrent neural network with particle swarm optimization algorithms for an improved hybrid training of multidisciplinary datasets. Expert Systems with Applications, 183, 115441. https://doi.org/10.1016/j.eswa.2021.115441
• Ahn, H. K., & Park, N. (2021). Deep RNN-based photovoltaic power short-term forecast using power IoT sensors. Energies, 14(2), 436. https://doi.org/10.3390/en14020436
• Bengio, Y., Paolo, F., & Marco, G. (1970). Recurrent neural networks for adaptive temporal processing. In Proceedings of the 6th Italian Workshop on Parallel Architectures and Neural Networks WIRN93 (pp. 85–117). World Scientific Publishing Co.
• Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
• Brockwell, P. J., & Davis, R. A. (2002). Introduction to time series and forecasting. Springer.
• Bui, K. H. N., Cho, J., & Yi, H. (2022). Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues. Applied Intelligence, 52(3), 2763–2774. https://doi.org/10.1007/s10489-021-02587-w
• Cao, Q., Wu, Y., Yang, J., & Yin, J. (2023). Greenhouse temperature prediction based on time-series features and LightGBM. Applied Sciences, 13(3), 1610. https://doi.org/10.3390/app13031610
• Casado-Vara, R., Martin del Rey, A., Pérez-Palau, D., & Corchado, J. M. (2021). Web traffic time series forecasting using LSTM neural networks with distributed asynchronous training. Mathematics, 9(4), 421. https://doi.org/10.3390/math9040421
• Challu, C., Olivares, K. G., Oreshkin, B. N., Ramirez, F. G., Canseco, M. M., & Dubrawski, A. (2023). N-HiTS: Neural hierarchical interpolation for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, pp. 6989–6997). Association for the Advancement of Artificial Intelligence.
• Chen, T. (2015). XGBoost: Extreme gradient boosting. R Package Version 0.4-2 1 (4) 1–4.
• Chen, Y., Ren, K., Wang, Y., Fang, Y., Sun, W., & Li, D. (2023). Contiformer: Continuous-time transformer for irregular time series modeling. In Advances in Neural Information Processing Systems 36-37th Conference on Neural Information Processing Systems, NeurIPS 36 (pp. 47143–47175). Curran Associates, Inc.
• Chiarot, G., & Claudio, S. (2023). Time series compression survey. ACM Computing Surveys, 55(10), 1–32. https://doi.org/10.1145/3560814
• Chung, J., Caglar, G., KyungHyun, C., & Yoshua, B. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555.
• Cristian, C., Kin G, O., Boris N, O., Federico, G., Max, M. C., & Artur, D. (2022). N-HiTS: Neural hierarchical interpolation for time series forecasting. arXiv:2201.12886.
• Daraghmeh, M., Agarwal, A., Manzano, R., & Zaman, M. (2021). Time series forecasting using facebook prophet for cloud resource management. In 2021 IEEE International Conference on Communications Workshops (ICC Workshops) (pp. 1–6). IEEE (Institute of Electrical and Electronics Engineers).
• Dash, S., Yale, A., Guyon, I., & Bennett, K. P. (2020). Medical time-series data generation using generative adversarial networks. In Artificial Intelligence in Medicine: 18th International Conference on Artificial Intelligence in Medicine, AIME 2020, Minneapolis, MN, USA, August 25–28, 2020, Proceedings 18 (pp. 382–391). Springer Nature.
• Ernst, D., & Louette, A. (2024). Introduction to reinforcement learning. University of Liège (ULiège).
• Falchi, F., Girardi, M., Gurioli, G., Messina, N., Padovani, C., & Pellegrini, D. (2023). Deep learning and structural health monitoring: A TFT-based approach for anomaly detection in masonry towers. SSRN: 4679906.
• Faouzi, J., & Janati, H. (2020). PyTS: A Python package for time series classification. Journal of Machine Learning Research, 21(46), 1–6.
• Hartanto, A. D., Kholik, Y. N., & Pristyanto, Y. (2023). Stock price time series data forecasting using the light gradient boosting machine (LightGBM) model. JOIV: International Journal on Informatics Visualization, 7(4), 2270–2279. https://doi.org/10.62527/joiv.7.4.1740
• Hindarto, D. (2023). Comparison of RNN architectures and non-RNN architectures in sentiment analysis. Sinkron: Jurnal Dan Penelitian Teknik Informatika, 7(4), 2537–2546. https://doi.org/10.33395/sinkron.v8i4
• Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
• Jang, J. G., Dongjin, C., Jinhong, J., & Kang, U. (2018). Zoom-SVD: Fast and memory efficient method for extracting key patterns in an arbitrary time range. In In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 1083–1092). Association for Computing Machinery (ACM).
• Jeffrey, N., Gunawan, A. A. S., & Kurniawan, A. (2023). Development of multivariate stock prediction system using N-Hits and N-BEATS. In Proceedings of the Computational Methods in Systems and Software (pp. 50–63). Springer.
• Joseph, M. (2022). Modern time series forecasting with Python: Explore Industry-ready time series forecasting using modern mchine learning and deep learning. Packt Publishing, Limited.
• Ke, G., Qi, M., Thomas, F., Taifeng, W., Wei, C., Weidong, M., & Tie-Yan, L. (2017). LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems 30 – Proceedings of the 2017 Conference 2017. Curran Associates, Inc.
• Liang, F., Qian, C., Yu, W., Griffith, D., & Golmie, N. (2022). Survey of graph neural networks and applications. Wireless Communications and Mobile Computing, 2022(1), 9261537. https://doi.org/10.1155/wcm.v2022.1
• Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021). Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4), 1748–1764. https://doi.org/10.1016/j.ijforecast.2021.03.012
• Lim, B., Sercan Omer, A., Nicolas, L., & Tomas, P. (2019). Temporal fusion transformers for interpretable multi-horizon time series forecasting. arXiv:1912.09363.
• Lim, B., & Zohren, S. (2021). Time-series forecasting with deep learning: A survey. Philosophical Transactions of the Royal Society A, 379(2194), 20200209. https://doi.org/10.1098/rsta.2020.0209
• Lin, C., Weng, K., Lin, Y., Zhang, T., He, Q., & Su, Y. (2022). Time series prediction of dam deformation using a hybrid STL–CNN–GRU model based on sparrow search algorithm optimization. Applied Sciences, 12(23), 11951. https://doi.org/10.3390/app122311951
• Lin, S., Clark, R., Birke, R., Schönborn, S., Trigoni, N., & Roberts, S. (2020). Anomaly detection for time series using VAE-LSTM hybrid model. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (Vol. 2025, pp. 4322–4326). IEEE (Institute of Electrical and Electronics Engineers).
• Lindemann, B., Müller, T., Vietz, H., Jazdi, N., & Weyrich, M. (2021). A survey on long short-term memory networks for time series prediction. Procedia Cirp, 99, 650–655. https://doi.org/10.1016/j.procir.2021.03.088
• Liu, Y., Zhang, H., Chenyu, L., Xiangdong, H., Jianmin, W., & Mingsheng, L. (2024). Timer: Generative pre-trained transformers are large time series models. arXiv:2402.02368.
• Liu, Z., Zhengtong, Z., Jing, G., & Xu, C. (2021). Forecast methods for time series data: A survey. IEEE Access, 9, 91896–91912. https://doi.org/10.1109/ACCESS.2021.3091162
• López Santos, M., García-Santiago, X., Echevarría Camarero, F., Blázquez Gil, G., & Carrasco Ortega, P. (2022). Application of temporal fusion transformer for day-ahead PV power forecasting. Energies, 15(14), 5232. https://doi.org/10.3390/en15145232
• Maiti, R., Balagopal G, M., & Anand, A. (2024). Ensemble empirical mode decomposition based deep learning models for forecasting river flow time series. Expert Systems with Applications, 255, 124550. https://doi.org/10.1016/j.eswa.2024.124550
• Mateus, B. C., Mendes, M., Farinha, J. T., Assis, R., & Cardoso, A. M. (2021). Comparing LSTM and GRU models to predict the condition of a pulp paper press. Energies, 14(21), 6958. https://doi.org/10.3390/en14216958
• Mir, A. A., Çelebi, F. V., Alsolai, H., Qureshi, S. A., Rafique, M., Alzahrani, J. S., & Hamza, M. A. (2022). Anomalies prediction in radon time series for earthquake likelihood using machine learning-based ensemble model. IEEE Access, 10, 37984–37999. https://doi.org/10.1109/ACCESS.2022.3163291
• Mortezanejad, S. A. F., Wang, R., & Mohammad-Djafari, A. (2025). Physics-informed neural networks with unknown partial differential equations: An application in multivariate time series. Entropy, 27(7), 682. https://doi.org/10.3390/e27070682
• Noh, S. H. (2021). Analysis of gradient vanishing of RNNs and performance comparison. Information, 12(11), 442. https://doi.org/10.3390/info12110442
• Numenta (2021). Numenta Anomaly Benchmark (NAB). Retrieved November 18, 2024, from https://github.com/numenta/NAB?ref=hackernoon.com(open in a new window).
• Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2019). N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. arXiv:1905.10437.
• Paul, A., Mukherjee, D. P., Das, P., Gangopadhyay, A., Chintha, A. R., & Kundu, S. (2018). Improved random forest for classification. IEEE Transactions on Image Processing, 27(8), 4012–4024. https://doi.org/10.1109/TIP.2018.2834830
• Pavlyshenko, B. M. (2019). Machine-learning models for sales time series forecasting. Data, 4(1), 15. https://doi.org/10.3390/data4010015
• Qin, R., & Wang, Y. (2023). ImputeGAN: Generative adversarial network for multivariate time series imputation. Entropy, 25(1), 137. https://doi.org/10.3390/e25010137
• Rady, E., Fawzy, H., & Fattah, A. M. A. (2021). Time series forecasting using tree based methods. Journal of Statistics Applications & Probability, 10(1), 229–244. https://doi.org/10.18576/jsap
• Repository, U. M. L. (2019). Air quality. Retrieved November 18, 2024, from https://archive.ics.uci.edu/dataset/360/air+quality(open in a new window).
• Rozenberg, R., Gesnouin, J., & Moutarde, F. (2021). Asymmetrical Bi-RNN for pedestrian trajectory encoding. arXiv:2106.04419.
• Salinas, D., Flunkert, V., Gasthaus, J., & Januschowski, T. (2020). DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3), 1181–1191. https://doi.org/10.1016/j.ijforecast.2019.07.001
• Schirmer, P. A., & Mporas, I. (2024). PyDTS: A Python toolkit for deep learning time series modelling. Entropy, 26(4), 311. https://doi.org/10.3390/e26040311
• Sheridan, R. P., Wang, W. M., Liaw, A., Ma, J., & Gifford, E. M. (2016). Extreme gradient boosting as a method for quantitative structure–activity relationships. Journal of Chemical Information and Modeling, 56(12), 2353–2360. https://doi.org/10.1021/acs.jcim.6b00591
• Tan, M., Merrill, M., Gupta, V., Althoff, T., & Hartvigsen, T. (2024). Are language models actually useful for time series forecasting? In Advances in Neural Information Processing Systems 37-38th Conference on Neural Information Processing Systems 37 (pp. 60162–60191). Curran Associates, Inc.
• Tang, C., Xu, L., Yang, B., Tang, Y., & Zhao, D. (2023). GRU-based interpretable multivariate time series anomaly detection in industrial control system. Computers & Security, 127, 103094. https://doi.org/10.1016/j.cose.2023.103094
• Tavenard, R., Faouzi, J., Vandewiele, G., Divo, F., Androz, G., & Holtz, C. (2020). Tslearn, a machine learning toolkit for time series data. Journal of Machine Learning Research, 21(118), 1–6.
• Taylor, S. J., & Benjamin, L. (2017). Prophet: Forecasting at scale. PeerJ Preprints, 5, e3190v2.
• Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021). Deep learning for time series forecasting: A survey. Big Data, 9(1), 3–21. https://doi.org/10.1089/big.2020.0159
• Tsantekidis, A., Passalis, N., & Tefas, A (2022). Recurrent neural networks. In Deep Learning for Robot Perception and Cognition (pp. 101–115). Elsevier.
• Ullah, W., Ullah, A., Haq, I. U., Muhammad, K., Sajjad, M., & Baik, S. W. (2021). CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks. Multimedia Tools and Applications, 80(11), 16979–16995. https://doi.org/10.1007/s11042-020-09406-3
• Valt, R. (2021). Sunspots Dataset. Retrieved November 18, 2024, https://www.kaggle.com/datasets/robervalt/sunspots?ref=hackernoon.com(open in a new window).
• Yang, X., Yu, S., & Xinyang, C. (2024). Frequency-aware generative models for multivariate time series imputation. Advances in Neural Information Processing Systems, 37, 52595–52623.
• Zhang, C., Hu, D., & Yang, T. (2022). Anomaly detection and diagnosis for wind turbines using long short-term memory-based stacked denoising autoencoders and XGBoost. Reliability Engineering & System Safety, 222, 108445. https://doi.org/10.1016/j.ress.2022.108445
• Zhang, C., Wang, D., Wang, L., Guan, L., Yang, H., Zhang, Z., & Zhang, M. (2021). Cause-aware failure detection using an interpretable XGBoost for optical networks. Optics Express, 29(20), 31974–31992. https://doi.org/10.1364/OE.436293
• Zhang, L., Liu, P., Zhao, L., Wang, G., Zhang, W., & Liu, J. (2021). Air quality predictions with a semi-supervised bidirectional LSTM neural network. Atmospheric Pollution Research, 12(1), 328–339. https://doi.org/10.1016/j.apr.2020.09.003
• Zhang, W., Yin, C., Liu, H., Zhou, X., & Xiong, H. (2024). Irregular multivariate time series forecasting: A transformable patching graph neural networks approach. In R. Salakhutdinov et al. (Eds.), Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 60179–60196). PMLR.
• Zhang, X., Chowdhury, R. R., Gupta, R. K., & Shang, J. (2024). Large language models for time series: A survey. arXiv:2402.01801.
• Zhang, Y., Suzuki, G., & Shioya, H. (2022). Prediction and detection of sewage treatment process using N-BEATS autoencoder network. IEEE Access, 10, 112594–112608. https://doi.org/10.1109/ACCESS.2022.3216924
• Zhao, C., Huang, X., Li, Y., & Yousaf Iqbal, M. (2020). A double-channel hybrid deep neural network based on CNN and BiLSTM for remaining useful life prediction. Sensors, 20(24), 7109. https://doi.org/10.3390/s20247109

To cite this article: Seyedeh Azadeh Fallah Mortezanejad & Ruochen Wang (2026) Addressing challenges in time series forecasting: a comprehensive comparison of machine learning techniques, Statistical Theory and Related Fields, 10:2, 184-231, DOI: 10.1080/24754269.2026.2633813 To link to this article: https://doi.org/10.1080/24754269.2026.2633813