A Comparative Study of Deep Learning Architectures for Multivariate Financial Time Series Forecasting
Keywords:
multivariate time series, financial forecasting, deep learning, Transformer, hybrid attention-gated moduleAbstract
Accurate forecasting of multivariate financial time series remains a critical challenge due to high volatility, non-stationarity, and complex cross-variable dependencies. Although deep learning models such as LSTM, GRU, TCN, and Transformer have shown notable progress, existing research often evaluates these architectures in isolation, lacks interpretability, and provides limited analysis of robustness across different markets. These limitations impede the deployment of reliable forecasting systems in practical financial settings. This study presents a comprehensive comparative analysis of representative deep learning architectures for financial forecasting and introduces a novel Hybrid Attention-Gated Module (HAGM). HAGM combines convolutional feature extraction, gated fusion, and multi-head self-attention mechanisms to efficiently capture both local and global dependencies. Experiments were conducted on stock indices, foreign exchange, and cryptocurrency datasets, assessing model performance across multiple forecast horizons. The results demonstrate that HAGM consistently outperforms baseline models, achieving lower RMSE and MAPE while exhibiting faster convergence. Ablation studies confirm the complementary contributions of convolution, gating, and attention components, and interpretability analyses identify critical variables such as trading volume and volatility. Robustness evaluations further reveal superior cross-market generalization and resilience under noisy conditions. Overall, this work advances the methodological understanding of deep learning approaches for financial forecasting and provides actionable insights for practitioners aiming to develop accurate, efficient, and interpretable predictive systems.
References
1. H. V. Bitencourt, O. Orang, L. A. F. de Souza, P. C. Silva, and F. G. Guimaraes, "An embedding-based non-stationary fuzzy time series method for multiple output high-dimensional multivariate time series forecasting in IoT applications," Neural Computing and Applications, vol. 35, no. 13, pp. 9407-9420, 2023.
2. J. Kim, H. Kim, H. Kim, D. Lee, and S. Yoon, "A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges," Artificial Intelligence Review, vol. 58, no. 7, pp. 1-95, 2025. doi: 10.1007/s10462-025-11223-9
3. C. M. Liapis, and S. Kotsiantis, "Temporal convolutional networks and BERT-based multi-label emotion analysis for financial forecasting," Information, vol. 14, no. 11, p. 596, 2023. doi: 10.3390/info14110596
4. I. D. Mienye, T. G. Swart, and G. Obaido, "Recurrent neural networks: A comprehensive review of architectures, variants, and applications," Information, vol. 15, no. 9, p. 517, 2024. doi: 10.20944/preprints202408.0748.v1
5. M. A. Al-qaness, A. Dahou, N. T. Trouba, M. Abd Elaziz, and A. M. Helmi, "TCN-inception: temporal convolutional network and inception modules for sensor-based human activity recognition," Future Generation Computer Systems, vol. 160, pp. 375-388, 2024. doi: 10.1016/j.future.2024.06.016
6. L. Su, X. Zuo, R. Li, X. Wang, H. Zhao, and B. Huang, "A systematic review for transformer-based long-term series forecasting," Artificial Intelligence Review, vol. 58, no. 3, p. 80, 2025. doi: 10.1007/s10462-024-11044-2
7. B. Liu, and M. Lai, "Advanced machine learning for financial markets: A PCA-GRU-LSTM approach," Journal of the Knowledge Economy, vol. 16, no. 1, pp. 3140-3174, 2025. doi: 10.1007/s13132-024-02108-3
8. S. Saleti, L. Y. Panchumarthi, Y. R. Kallam, L. Parchuri, and S. Jitte, "Enhancing forecasting accuracy with a moving average-integrated hybrid ARIMA-LSTM model," SN Computer Science, vol. 5, no. 6, p. 704, 2024. doi: 10.1007/s42979-024-03060-4
9. N. H. A. Rahman, M. Z. Hussin, S. I. Sulaiman, M. A. Hairuddin, and E. H. M. Saat, "Univariate and multivariate short-term solar power forecasting of 25MWac Pasir Gudang utility-scale photovoltaic system using LSTM approach," Energy Reports, vol. 9, pp. 387-393, 2023. doi: 10.1016/j.egyr.2023.09.018
10. A. Younesi, M. Ansari, M. Fazli, A. Ejlali, M. Shafique, and J. Henkel, "A comprehensive survey of convolutions in deep learning: Applications, challenges, and future trends," IEEE Access, vol. 12, pp. 41180-41218, 2024. doi: 10.1109/access.2024.3376441
11. Z. Xu, Z. Yu, H. Zhang, J. Chen, J. Gu, T. Lukasiewicz, and V. C. Leung, "PhaCIA-TCNs: Short-term load forecasting using temporal convolutional networks with parallel hybrid activated convolution and input attention," IEEE Transactions on Network Science and Engineering, vol. 11, no. 1, pp. 427-438, 2023.
12. E. Nsengiyumva, J. K. Mung'atu, and C. Ruranga, "A comparative study of multivariate CNN, BiLSTM and hybrid CNN-BiLSTM models for forecasting foreign exchange rate using deep learning," Cogent Economics & Finance, vol. 13, no. 1, p. 2526148, 2025.
13. Q. Zhu, J. Han, K. Chai, and C. Zhao, "Time series analysis based on informer algorithms: A survey," Symmetry, vol. 15, no. 4, p. 951, 2023. doi: 10.3390/sym15040951
14. A. Wiemerslage, K. Gorman, and K. von der Wense, "Quantifying the hyperparameter sensitivity of neural networks for character-level sequence-to-sequence tasks," In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), March, 2024, pp. 674-689. doi: 10.18653/v1/2024.eacl-long.40
15. G. Chen, J. Yuan, Y. Zhang, H. Zhu, R. Huang, F. Wang, and W. Li, "Enhancing reliability through interpretability: A comprehensive survey of interpretable intelligent fault diagnosis in rotating machinery," IEEE Access, 2024. doi: 10.1109/access.2024.3430010
