Transformer-Based Transfer Learning Method for Financial Time-Series Forecasting

Article Sidebar

pdf
Published: Apr 1, 2026

Main Article Content

Lysander Booth
Adrian Veldhoorn

Abstract

Financial market forecasting is widely regarded as one of the most challenging tasks in data mining. Recently, Transformer models have achieved remarkable success in improving the accuracy of Financial Time-Series Forecasting (FTSF). However, due to the implicit complexity of financial information and the limited availability of labeled data, existing approaches often exhibit insufficient generalization capability in this domain. To alleviate the overfitting problem caused by data scarcity, this paper proposes a Transformer-based deep transfer learning framework integrated with domain-adversarial adaptation, termed TADA-FTSF. The proposed framework is designed for financial time-series forecasting tasks and aims to enhance the reliability and predictive accuracy of deep forecasting models in the financial domain.



  • Janacek, “Time series analysis, forecasting and control,” J. Time Ser. Anal., 2010.

  • S. Lee and L. I. Tong, “Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming,” Knowl.-Based Syst., vol. 24, no. 1, pp. 66-72, 2011.

  • Corberán-Vallet, J. D. Bermúdez, and E. Vercher, “Forecasting correlated time series with exponential smoothing models,” Int. J. Forecast., vol. 27, no. 2, pp. 252-265, 2011.

  • LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, 2015.

  • Fischer and C. Krauss, “Deep learning with long short-term memory networks for financial market predictions,” Eur. J. Oper. Res., vol. 270, no. 2, pp. 654-669, 2017.

  • Aryal, D. Nadarajah, P. L. Rupasinghe, et al., “Comparative analysis of deep learning models for multi-step prediction of financial time series,” J. Comput. Sci., vol. 16, no. 10, pp. 1401-1416, 2020.

  • B. Sezer, M. Gudelek, and A. M. Ozbayoglu, “Financial time series forecasting with deep learning: A systematic literature review: 2005-2019,” Appl. Soft Comput., vol. 90, p. 106181, 2020.

  • W. Li and G. S. Bastos, “Stock market forecasting using deep learning and technical analysis: A systematic review,” IEEE Access, vol. 8, pp. 185232-185242, 2022.

  • Niu, J. Z. Wang, H. Y. Lu, et al., “Developing a deep learning framework with two-stage feature selection for multivariate financial time series forecasting,” Expert Syst. Appl., vol. 148, p. 113237, 2020.

  • Li, A. Zhang, Q. Zhang, et al., “Pearson correlation coefficient-based performance enhancement of broad learning system for stock price prediction,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 69, no. 5, pp. 2413-2417, 2022.

  • Fischer and C. Krauss, "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, vol. 270, no. 2, pp. 654-669, 2018.

  • M. Q. Nelson, A. C. M. Pereira and R. A. de Oliveira, "Stock market’s price movement prediction with LSTM neural networks," 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1419-1426, 2017.

  • Lim, S. Ö. Arık, N. Loeff and T. Pfister, "Temporal fusion transformers for interpretable multi-horizon time series forecasting," International Journal of Forecasting, vol. 37, no. 4, pp. 1748-1764, 2021.

  • Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong and W. Zhang, "Informer: Beyond efficient transformer for long sequence time-series forecasting," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11106-11115, 2021.

  • Wu, J. Xu, J. Wang and M. Long, "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting," Advances in Neural Information Processing Systems, vol. 34, pp. 22419-22430, 2021.

  • Nie, N. H. Nguyen, P. Sinthong and J. Kalagnanam, "A time series is worth 64 words: Long-term forecasting with transformers," arXiv preprint arXiv:2211.14730, 2022.

  • Feng, "Hybrid BiLSTM-transformer model for identifying fraudulent transactions in financial systems," Journal of Computer Science and Software Applications, vol. 5, no. 3, 2025.

  • Fang, "A machine learning framework for enterprise risk prediction: Unified feature embedding and lightweight attention," 2026.

  • Chiang, D. Li, R. Ying, Y. Wang, Q. Gan and J. Li, "Deep learning-based dynamic graph framework for robust corporate financial health risk prediction," Proceedings of the 2025 3rd International Conference on Mathematics and Machine Learning, pp. 98-105, 2025.

  • Wang, "Multi-agent collaborative modeling for systemic risk propagation in financial markets: A game-theoretic framework," 2026.

  • Ying, Q. Liu, Y. Wang and Y. Xiao, "AI-based causal reasoning over knowledge graphs for data-driven and intervention-oriented enterprise performance analysis," 2025.

  • J. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345-1359, 2009.

  • Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation," Proceedings of the International Conference on Machine Learning, pp. 1180-1189, 2015.

  • Long, Y. Cao, J. Wang and M. Jordan, "Learning transferable features with deep adaptation networks," Proceedings of the International Conference on Machine Learning, pp. 97-105, 2015.

  • Tzeng, J. Hoffman, K. Saenko and T. Darrell, "Adversarial discriminative domain adaptation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.

  • Huang, Y. Zheng, Y. Zhao, R. Ying, K. Cao and X. Liang, "A unified meta learning and domain adaptation framework for credit fraud detection in dynamic environments," 2026.

  • Yan, Y. Ou, S. Sun, N. Chen, K. Zhou and Y. Shu, "DualShiftNet: Joint class-imbalance and distribution-shift aware learning for business risk prediction," 2026.

  • Gan, "Large language model framework for multi-document financial anomaly detection in intelligent auditing via semantic mapping and risk reasoning," Transactions on Computational and Scientific Methods, vol. 4, no. 12, 2024.

  • Cao, Y. Zhao, H. Chen, X. Liang, Y. Zheng and S. Huang, "Multi-hop relational modeling for credit fraud detection via graph neural networks," 2025.

  • Li, Q. Gan, R. Wu, C. Chen, R. Fang and J. Lai, "Causal representation learning for robust and interpretable audit risk identification in financial systems," 2025.

  • Ying, Q. Liu, Y. Wang and Y. Xiao, "AI-based causal reasoning over knowledge graphs for data-driven and intervention-oriented enterprise performance analysis," 2025.

  • Chen, R. Wu, C. Chen, H. Feng, Y. Nie and Y. Lu, "Anomaly ranking for enterprise finance using latent structural deviations and reconstruction consistency," 2026.

  • Feng, Y. Wang, R. Fang, A. Xie and Y. Wang, "Federated risk discrimination with siamese networks for financial transaction anomaly detection," Proceedings of the 2025 2nd International Conference on Digital Economy and Computer Science, pp. 231-236, 2025.

Article Details

How to Cite
Booth, L., & Veldhoorn, A. (2026). Transformer-Based Transfer Learning Method for Financial Time-Series Forecasting. Journal of Computer Science and Software Applications, 6(4). Retrieved from https://www.mfacademia.org/index.php/jcssa/article/view/268
Issue
Vol. 6 No. 4 (2026)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Mind forge Academia also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.

References

G. Janacek, “Time series analysis, forecasting and control,” J. Time Ser. Anal., 2010.

Y. S. Lee and L. I. Tong, “Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming,” Knowl.-Based Syst., vol. 24, no. 1, pp. 66-72, 2011.

A. Corberán-Vallet, J. D. Bermúdez, and E. Vercher, “Forecasting correlated time series with exponential smoothing models,” Int. J. Forecast., vol. 27, no. 2, pp. 252-265, 2011.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, 2015.

T. Fischer and C. Krauss, “Deep learning with long short-term memory networks for financial market predictions,” Eur. J. Oper. Res., vol. 270, no. 2, pp. 654-669, 2017.

S. Aryal, D. Nadarajah, P. L. Rupasinghe, et al., “Comparative analysis of deep learning models for multi-step prediction of financial time series,” J. Comput. Sci., vol. 16, no. 10, pp. 1401-1416, 2020.

O. B. Sezer, M. Gudelek, and A. M. Ozbayoglu, “Financial time series forecasting with deep learning: A systematic literature review: 2005-2019,” Appl. Soft Comput., vol. 90, p. 106181, 2020.

A. W. Li and G. S. Bastos, “Stock market forecasting using deep learning and technical analysis: A systematic review,” IEEE Access, vol. 8, pp. 185232-185242, 2022.

T. Niu, J. Z. Wang, H. Y. Lu, et al., “Developing a deep learning framework with two-stage feature selection for multivariate financial time series forecasting,” Expert Syst. Appl., vol. 148, p. 113237, 2020.

G. Li, A. Zhang, Q. Zhang, et al., “Pearson correlation coefficient-based performance enhancement of broad learning system for stock price prediction,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 69, no. 5, pp. 2413-2417, 2022.

T. Fischer and C. Krauss, "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, vol. 270, no. 2, pp. 654-669, 2018.

D. M. Q. Nelson, A. C. M. Pereira and R. A. de Oliveira, "Stock market’s price movement prediction with LSTM neural networks," 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1419-1426, 2017.

B. Lim, S. Ö. Arık, N. Loeff and T. Pfister, "Temporal fusion transformers for interpretable multi-horizon time series forecasting," International Journal of Forecasting, vol. 37, no. 4, pp. 1748-1764, 2021.

H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong and W. Zhang, "Informer: Beyond efficient transformer for long sequence time-series forecasting," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11106-11115, 2021.

H. Wu, J. Xu, J. Wang and M. Long, "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting," Advances in Neural Information Processing Systems, vol. 34, pp. 22419-22430, 2021.

Y. Nie, N. H. Nguyen, P. Sinthong and J. Kalagnanam, "A time series is worth 64 words: Long-term forecasting with transformers," arXiv preprint arXiv:2211.14730, 2022.

P. Feng, "Hybrid BiLSTM-transformer model for identifying fraudulent transactions in financial systems," Journal of Computer Science and Software Applications, vol. 5, no. 3, 2025.

R. Fang, "A machine learning framework for enterprise risk prediction: Unified feature embedding and lightweight attention," 2026.

C. Chiang, D. Li, R. Ying, Y. Wang, Q. Gan and J. Li, "Deep learning-based dynamic graph framework for robust corporate financial health risk prediction," Proceedings of the 2025 3rd International Conference on Mathematics and Machine Learning, pp. 98-105, 2025.

Y. Wang, "Multi-agent collaborative modeling for systemic risk propagation in financial markets: A game-theoretic framework," 2026.

R. Ying, Q. Liu, Y. Wang and Y. Xiao, "AI-based causal reasoning over knowledge graphs for data-driven and intervention-oriented enterprise performance analysis," 2025.

S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345-1359, 2009.

Y. Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation," Proceedings of the International Conference on Machine Learning, pp. 1180-1189, 2015.

M. Long, Y. Cao, J. Wang and M. Jordan, "Learning transferable features with deep adaptation networks," Proceedings of the International Conference on Machine Learning, pp. 97-105, 2015.

E. Tzeng, J. Hoffman, K. Saenko and T. Darrell, "Adversarial discriminative domain adaptation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.

S. Huang, Y. Zheng, Y. Zhao, R. Ying, K. Cao and X. Liang, "A unified meta learning and domain adaptation framework for credit fraud detection in dynamic environments," 2026.

R. Yan, Y. Ou, S. Sun, N. Chen, K. Zhou and Y. Shu, "DualShiftNet: Joint class-imbalance and distribution-shift aware learning for business risk prediction," 2026.

Q. Gan, "Large language model framework for multi-document financial anomaly detection in intelligent auditing via semantic mapping and risk reasoning," Transactions on Computational and Scientific Methods, vol. 4, no. 12, 2024.

K. Cao, Y. Zhao, H. Chen, X. Liang, Y. Zheng and S. Huang, "Multi-hop relational modeling for credit fraud detection via graph neural networks," 2025.

J. Li, Q. Gan, R. Wu, C. Chen, R. Fang and J. Lai, "Causal representation learning for robust and interpretable audit risk identification in financial systems," 2025.

R. Ying, Q. Liu, Y. Wang and Y. Xiao, "AI-based causal reasoning over knowledge graphs for data-driven and intervention-oriented enterprise performance analysis," 2025.

H. Chen, R. Wu, C. Chen, H. Feng, Y. Nie and Y. Lu, "Anomaly ranking for enterprise finance using latent structural deviations and reconstruction consistency," 2026.

H. Feng, Y. Wang, R. Fang, A. Xie and Y. Wang, "Federated risk discrimination with siamese networks for financial transaction anomaly detection," Proceedings of the 2025 2nd International Conference on Digital Economy and Computer Science, pp. 231-236, 2025.