×

Bayesian optimization based dynamic ensemble for time series forecasting. (English) Zbl 07798534

Summary: Among various time series (TS) forecasting methods, ensemble forecast is extensively acknowledged as a promising ensemble approach achieving great success in research and industry. Due to the high diversification of individual model assumptions, heterogeneous information fusion contributes to generating effective and robust forecasts for Economics, Meteorology, and Transportation. This paper proposes a Bayesian optimization-based dynamic ensemble (BODE) that overcomes the single model-based methods limitation and provides a dynamic ensemble forecast combination for TS with time-varying underlying patterns. The proposed BODE method combines ten disparate model candidates, including statistical methods, machine learning (ML)-based models, and the latest deep neural networks (DNN). We take into consideration their prediction performance for the recent past to adjust their weights for combination and apply the model-based Bayesian optimization algorithm (BOA) for the combination hyperparameter (HP) tuning to endow our method with higher adaptability and better generalization performance. Besides, the frequency impact of TS data on the ensemble forecast methods is under-researched in the current literature. Therefore, four groups of distinct seasonal TS datasets are investigated in this paper. The empirical result demonstrates that our method performs robustly better performance with the main reasons analyzed in a detailed ablation study.

MSC:

62-XX Statistics
90-XX Operations research, mathematical programming
Full Text: DOI

References:

[1] Abbasimehr, H.; Paki, R., Prediction of covid-19 confirmed cases combining deep learning methods and bayesian optimization, Chaos Solitons Fractals, 142, Article 110511 pp. (2021)
[2] Abdar, M.; Pourpanah, F.; Hussain, S.; Rezazadegan, D.; Liu, L.; Ghavamzadeh, M.; Fieguth, P.; Cao, X.; Khosravi, A.; Acharya, U. R., A review of uncertainty quantification in deep learning: Techniques, applications and challenges, Inf. Fusion (2021)
[3] M. Abdar, M. Samami, S.D. Mahmoodabad, T. Doan, B. Mazoure, R. Hashemifesharaki, L. Liu, A. Khosravi, U.R. Acharya, V. Makarenkov, et al., Uncertainty quantification in skin cancer classification using three-way decision-based bayesian deep learning, Comput. Biol. Med. (2021) 104418.
[4] Abdar, M.; Zomorodi-Moghadam, M.; Zhou, X.; Gururajan, R.; Tao, X.; Barua, P. D.; Gururajan, R., A new nested ensemble technique for automated diagnosis of breast cancer, Pattern Recogn. Lett., 132, 123-131 (2020)
[5] Adhikari, R.; Agrawal, R., Performance evaluation of weights selection schemes for linear combination of multiple forecasts, Artif. Intell. Rev., 42, 529-548 (2014)
[6] M. Aiolfi, C. Capistran, A. Timmermann, Forecast combinations. CREATES research paper, 2010.
[7] Alizadeh, B.; Bafti, A. G.; Kamangir, H.; Zhang, Y.; Wright, D. B.; Franz, K. J., A novel attention-based lstm cell post-processor coupled with bayesian optimization for streamflow prediction, J. Hydrol., 601, Article 126526 pp. (2021)
[8] Australian Energy Market Operator: AEMO, Australian energy price and demand. URL:https://aemo.com.au/.
[9] Barnard, G. A.; Jenkins, G. M.; Winsten, C., Likelihood inference and time series, J. R. Stat. Soc.: Ser. A (General), 125, 321-352 (1962)
[10] Brown, R. G.; Meyer, R. F., The fundamental theorem of exponential smoothing, Oper. Res., 9, 673-685 (1961) · Zbl 0102.35702
[11] Cao, L. J.; Tay, F. E.H., Support vector machine with adaptive parameters in financial time series forecasting, IEEE Trans. Neural Networks, 14, 1506-1518 (2003)
[12] Chen, T.; Guestrin, C., Xgboost: A scalable tree boosting system, (Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (2016)), 785-794
[13] Cirstea, R. G.; Kieu, T.; Guo, C.; Yang, B.; Pan, S. J., Enhancenet: Plugin neural networks for enhancing correlated time series forecasting, (2021 IEEE 37th International Conference on Data Engineering (ICDE), IEEE (2021)), 1739-1750
[14] Claeskens, G.; Magnus, J. R.; Vasnev, A. L.; Wang, W., The forecast combination puzzle: A simple theoretical explanation, Int. J. Forecast., 32, 754-762 (2016)
[15] Clemen, R. T., Combining forecasts: A review and annotated bibliography, Int. J. Forecasting, 5, 559-583 (1989)
[16] Fan, J.; Wang, X.; Wu, L.; Zhou, H.; Zhang, F.; Yu, X.; Lu, X.; Xiang, Y., Comparison of support vector machine and extreme gradient boosting for predicting daily global solar radiation using temperature and precipitation in humid subtropical climates: A case study in china, Energy Convers. Manage., 164, 102-111 (2018)
[17] Freitas, P. S.; Rodrigues, A. J., Model combination in neural-based forecasting, Eur. J. Oper. Res., 173, 801-814 (2006) · Zbl 1120.90342
[18] Gao, R.; Du, L.; Yuen, K. F., Robust empirical wavelet fuzzy cognitive map for time series forecasting, Eng. Appl. Artif. Intell., 96, Article 103978 pp. (2020)
[19] Hochreiter, S.; Schmidhuber, J., Long short-term memory, Neural Comput., 9, 1735-1780 (1997)
[20] Holt, C., Forecasting seasonals and trends by exponentially weighted averages (onr memorandum no. 52), 10 (1957), Carnegie Institute of Technology: Carnegie Institute of Technology Pittsburgh USA
[21] International Institute of Forecasters, M4 competition. URL:https://mofc.unic.ac.cy/m4/.
[22] Jaeger, H., The “echo state” approach to analysing and training recurrent neural networks-with an erratum note, Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 148, 13 (2001)
[23] Karmy, J. P.; Maldonado, S., Hierarchical time series forecasting via support vector regression in the european travel retail industry, Expert Syst. Appl., 137, 59-73 (2019)
[24] Lea, C.; Flynn, M. D.; Vidal, R.; Reiter, A.; Hager, G. D., Temporal convolutional networks for action segmentation and detection, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)), 156-165
[25] Li, J.; Hao, J.; Feng, Q.; Sun, X.; Liu, M., Optimal selection of heterogeneous ensemble strategies of time series forecasting with multi-objective programming, Expert Syst. Appl., 166, Article 114091 pp. (2021)
[26] Liu, H.; Duan, Z.; Chen, C., A hybrid multi-resolution multi-objective ensemble model and its application for forecasting of daily pm2. 5 concentrations, Inf. Sci., 516, 266-292 (2020)
[27] Ma, J.; Ding, Y.; Cheng, J. C.; Jiang, F.; Gan, V. J.; Xu, Z., A lag-flstm deep learning network based on bayesian optimization for multi-sequential-variant pm2. 5 prediction, Sustain. Cities Soc., 60, Article 102237 pp. (2020)
[28] Makridakis, S.; Spiliotis, E.; Assimakopoulos, V., The m4 competition: 100,000 time series and 61 forecasting methods, Int. J. Forecast., 36, 54-74 (2020)
[29] P.S. de Mattos Neto, J.F. de Oliveira, S.d.O. Domingos, H.V. Siqueira, F. Madeiro, An adaptive hybrid system using deep learning for wind speed forecasting, Inf. Sci. (2021).
[30] Montgomery, D. C.; Jennings, C. L.; Kulahci, M., Introduction to time series analysis and forecasting (2015), John Wiley & Sons · Zbl 1348.91001
[31] B.N. Oreshkin, D. Carpov, N. Chapados, Y. Bengio, N-beats: Neural basis expansion analysis for interpretable time series forecasting, in: International Conference on Learning Representations, 2021.
[32] W. Pedrycz, S. Chen, Time series analysis, modeling and applications. A Computational Intelligence Perspective (e-book Google), 2013. · Zbl 1254.62095
[33] Pham, B. T.; Le, L. M.; Le, T. T.; Bui, K. T.T.; Le, V. M.; Ly, H. B.; Prakash, I., Development of advanced artificial intelligence models for daily rainfall prediction, Atmos. Res., 237, Article 104845 pp. (2020)
[34] J. Platt, Sequential minimal optimization: A fast algorithm for training support vector machines, 1998.
[35] Qiu, X.; Ren, Y.; Suganthan, P. N.; Amaratunga, G. A., Empirical mode decomposition based ensemble deep learning for load demand time series forecasting, Appl. Soft Comput., 54, 246-255 (2017)
[36] R. Rahaman, A.H. Thiery, Uncertainty quantification and deep ensembles. arXiv preprint arXiv:2007.08792, 2020.
[37] H.J. Sadaei, P.C.d.L. e Silva, F.G. Guimarães, M.H. Lee, Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series, Energy 175 (2019) 365-377.
[38] Salinas, D.; Flunkert, V.; Gasthaus, J.; Januschowski, T., Deepar: Probabilistic forecasting with autoregressive recurrent networks, Int. J. Forecast., 36, 1181-1191 (2020)
[39] J. Snoek, H. Larochelle, R.P. Adams, Practical bayesian optimization of machine learning algorithms. arXiv preprint arXiv:1206.2944, 2012. · Zbl 1433.68379
[40] Tak, N., Forecast combination with meta possibilistic fuzzy functions, Inf. Sci., 560, 168-182 (2021)
[41] F.E. Tay, L. Cao, Application of support vector machines in financial time series forecasting, Omega 29 (2001) 309-317.
[42] Thakkar, A.; Chaudhari, K., Fusion in stock market prediction: A decade survey on the necessity, recent developments, and potential future directions, Inf. Fusion, 65, 95-107 (2021)
[43] The World Bank, Commodity markets. url:https://www.worldbank.org/en/research/commodity-markets/.
[44] Trigg, D.; Leach, A., Exponential smoothing with an adaptive response rate, J. Oper. Res. Soc., 18, 53-59 (1967)
[45] Tsay, R. S., Analysis of financial time series, vol. 543 (2005), John wiley & sons · Zbl 1086.91054
[46] Wang, L.; Wang, Z.; Qu, H.; Liu, S., Optimal forecast combination based on neural networks for time series forecasting, Appl. Soft Comput., 66, 1-17 (2018)
[47] Winters, P. R., Forecasting sales by exponentially weighted moving averages, Managem. Sci., 6, 324-342 (1960) · Zbl 0995.90562
[48] Xie, P.; Li, T.; Liu, J.; Du, S.; Yang, X.; Zhang, J., Urban flow prediction from spatiotemporal data using machine learning: A survey, Inf. Fusion, 59, 1-12 (2020)
[49] Zhang, S.; Chen, Y.; Zhang, W.; Feng, R., A novel ensemble deep learning model with dynamic error correction and multi-objective ensemble pruning for time series forecasting, Inf. Sci., 544, 427-445 (2021)
[50] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, W. Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting, in: Proceedings of AAAI, 2021.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.