Deep Learning Framework for Nonlinear Time Series Regression
Article Sidebar
Main Article Content
Abstract
This paper addresses the challenges of inadequate long-term dependency modeling and limited latent structure representation in nonlinear time series regression. It proposes a unified framework that integrates the Transformer architecture with a variational autoencoding mechanism. The method combines multi-layer self-attention with latent variable modeling to extract global dependency features in the encoder and capture hidden dynamic structures through variational inference. Specifically, the model first uses the Transformer to model the global context of the time series. Then, it constructs a probabilistic representation of latent variables based on the encoder output and applies a reparameterization mechanism to sample latent variables for regression. In the decoding stage, the model fuses the Transformer encoding and latent variable information through a feedforward network to generate high-precision predictions. Experiments conducted on public datasets include comparative tests and sensitivity analyses under varying window lengths, noise levels, and input scales. The results show that the proposed method outperforms mainstream approaches in prediction accuracy, model stability, and uncertainty representation, validating the effectiveness and practicality of the hybrid architecture in multi-scale nonlinear time series modeling.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Mind forge Academia also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.
References
A. Jadon, A. Patil, and S. Jadon, "A comprehensive survey of regression-based loss functions for time series forecasting," *International Conference on Data Management, Analytics & Innovation*, Singapore: Springer Nature Singapore, pp. 117–147, 2024.
Z. Yan, X. Zhou, J. Ren, et al., "Identifying underlying influential factors in information diffusion process on social media platform: A hybrid approach of data mining and time series regression," *Information Processing & Management*, vol. 60, no. 5, Art. no. 103438, 2023.
F. Martínez, F. Charte, M. P. Frías, et al., "Strategies for time series forecasting with generalized regression neural networks," *Neurocomputing*, vol. 491, pp. 509–521, 2022.
S. Cantero-Chinchilla, C. A. Simpson, A. Ballisat, et al., "Convolutional neural networks for ultrasound corrosion profile time series regression," *NDT & E International*, vol. 133, Art. no. 102756, 2023.
S. Shaikh, J. Gala, A. Jain, et al., "Analysis and prediction of COVID-19 using regression models and time series forecasting," *2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence)*, pp. 989–995, 2021.
A. Garioud, S. Valero, S. Giordano, et al., "Recurrent-based regression of Sentinel time series for continuous vegetation monitoring," *Remote Sensing of Environment*, vol. 263, Art. no. 112419, 2021.
C. W. Tan, C. Bergmeir, F. Petitjean, et al., "Time series extrinsic regression: Predicting numeric values from time series data," *Data Mining and Knowledge Discovery*, vol. 35, no. 3, pp. 1032–1060, 2021.
J. Hu, J. Tang, and Z. Liu, "A novel time series probabilistic prediction approach based on the monotone quantile regression neural network," *Information Sciences*, vol. 654, Art. no. 119844, 2024.
X. Wen and W. Li, "Time series prediction based on LSTM-attention-LSTM model," *IEEE Access*, vol. 11, pp. 48322–48331, 2023.
Z. Zihao, J. Geng, and W. Jiang, "A time series classification method based on 1DCNN-FNN," *2021 33rd Chinese Control and Decision Conference (CCDC)*, pp. 1566–1571, 2021.
G. Zerveas, S. Jayaraman, D. Patel, et al., "A transformer-based framework for multivariate time series representation learning," *Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining*, pp. 2114–2124, 2021.
H. Zhou, S. Zhang, J. Peng, et al., "Informer: Beyond efficient transformer for long sequence time-series forecasting," *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 35, no. 12, pp. 11106–11115, 2021.
Z. Qing and F. Huang, "Attention-enhanced network traffic prediction for intelligent load balancing optimization," *Transactions on Computational and Scientific Methods*, vol. 5, no. 8, 2025.
T. Ying and C. Ding, "Self-supervised unified representation learning with structural dependency modeling for early risk detection in distributed systems," *Artificial Intelligence and Computing Innovations*, vol. 4, no. 2, 2024.
T. Ying, L. Wang, P. Hu, and T. Ding, "Persistent memory modeling and policy optimization for large language model agents in long-horizon tasks," 2026.
P. Hu, Z. Qing, T. Ding, T. Ying, F. Xue, and L. Wang, "Energy-efficient distributed inference across the edge-cloud continuum using adaptive model compression," 2026.
N. Lyu, F. Chen, C. Zhang, C. Shao, and J. Jiang, "Deep temporal convolutional neural networks with attention mechanisms for resource contention classification in cloud computing," *Proceedings of the 2025 5th International Conference on Big Data, Artificial Intelligence and Risk Management*, pp. 62–68, Dec. 2025.
C. Zhang, C. Shao, J. Jiang, Y. Ni, and X. Sun, "Graph-transformer reconstruction learning for unsupervised anomaly detection in dependency-coupled systems," 2025.
C. Zhang, "Adaptive multi-tenant resource scheduling in cloud computing via reinforcement learning," *Transactions on Computational and Scientific Methods*, vol. 4, no. 3, 2024.
J. Qiu, J. Sun, Y. Xue, Y. Wang, Z. Yang, and C. Zhang, "Attention-enhanced long sequence temporal modeling method for backend CPU load prediction tasks," 2026, doi: 10.13140/RG.2.2.18397.78562.
Y. Xue, Y. Wang, A. Zhu, X. Sun, and C. Zhang, "Joint temporal-structural representation learning for distributed fault discrimination in microservice architectures," *2026 IEEE 8th International Conference on Communications, Information System and Computer Engineering (CISCE)*, pp. 1581–1586, Mar. 2026.
S. Pi, S. Zhang, and S. Shawulieti, "CacheRoute: KV-cache-aware routing for LLM backend services on Kubernetes," 2026.
S. Wang, B. Su, Y. Liu, S. Pi, and A. Zhu, "Safe adaptive resilience configuration for microservice backends."
Z. Jiang, S. Zhang, W. Huang, S. Pi, S. Shawulieti, and S. Xu, "Semantic-aware intelligent request routing in multi-model LLM backend gateways," 2026.
B. Zhang, Y. Zhan, Q. Wang, J. Ding, L. Yan, and W. Liu, "Hierarchical resource-aware federated edge learning in heterogeneous distributed systems," research report, 2026, doi: 10.13140/RG.2.2.30573.14562.
P. Hu, G. Uziel, and X. Ma, "Risk-constrained reinforcement learning for SLO-driven cloud-native elastic scaling," *Journal of Computer Technology and Software*, vol. 3, no. 1, 2024.
W. Huang, S. Xu, and Z. Jiang, "Privacy-aware distributed RAG with confidence-aware retrieval routing," *2026 3rd International Conference on Image Processing and Artificial Intelligence (ICIPAI)*, pp. 327–332, May 2026.
B. Chen, F. Chen, Y. Zhan, L. Ren, and Y. Tang, "Can we trust the metrics? Safe remediation under adversarial and noisy observability," *2026 3rd International Conference on Image Processing and Artificial Intelligence (ICIPAI)*, pp. 339–344, May 2026.
Z. Gu and L. Nie, "Knowledge graph-enhanced large language models for opinion target sentiment classification," *Journal of Computer Technology and Software*, vol. 4, no. 7, 2025.
L. Wang and C. Y. Chang, "Structure-aware root cause localization for microservice backends via joint graph representation learning and causal inference," *Journal of Computer Technology and Software*, vol. 3, no. 1, 2024.
J. Zhang, B. Chen, Y. Tang, F. Chen, Y. Zhan, and Z. Hu, "Drift-aware straggler mitigation for heterogeneous split federated learning via proactive cut-layer and asynchronous server updates," 2026.
X. Meng, L. Zheng, Y. Duan, S. Hu, and J. Chen, "Dynamic enterprise audit risk identification through graph-temporal transformer representation learning," ResearchGate preprint, 2026.
Z. Peng, S. Liu, X. Meng, Z. Su, and N. Sang, "A schema-aware large language model for XBRL tagging and cross-statement consistency checking," 2026.
Y. Wang, C.-C. Peng, S. Shu, Z. Wang, and I.-H. Lai, "Conflict-aware self-evolving memory for long-horizon agents," Jul. 2026, doi: 10.13140/RG.2.2.33763.72483.
J. Y. Su and X. Yan, "Process-aware multimodal drawing analytics for AI-assisted expressive arts support in educational settings," 2026.
S. Liu, S. Lu, Y. Liu, S. Y. Huang, and L. Chen, "Uncertainty-aware multimodal computer-use agent for GUI tasks," 2026.
K. Zhang, Y. Jiang, S. Liu, Z. Xiao, and Z. Yang, "Cost-constrained automatic workflow search for small LLM agents," 2026.
L. Ren, Y. Liu, and Y. Zhao, "Learning-based intelligent agents for backend resource scheduling and operational decision making," *Transactions on Computational and Scientific Methods*, vol. 5, no. 3, 2025.
T. Xia, X. Huang, Y. Zhou, F. Chang, and L. Ren, "Coordinating training and inference in heterogeneous cloud GPU clusters with learned interference models," *2026 8th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI)*, pp. 104–110, May 2026.
R. Fang, J. Zhou, Z. Zhu, R. Hu, and Y. Zheng, "Confidence-conflict routing between parametric memory, RAG, and long-context LLMs," 2026.
X. Chai, S. Li, Y. Zhao, B. Zhang, and J. Ding, "Causal generative re-ranking for multi-objective advertising," 2026.
Y. Ke, Q. Guo, D. Huang, and S. Li, "Explanation-stable anomaly detection for imbalanced heterogeneous tabular data under distribution shift," *2026 5th International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC)*, pp. 372–378, Jul. 2026.
N. Sang and C. Huang, "Risk-adaptive deposit-refund pricing for sustainable commercial orbit sharing," 2026.
C. Huang, "Integrating attention mapping and legal prior knowledge for interpretable legal reasoning with transformer-based models," 2025.
B. Zhang, S. Wang, C. He, and Y. Gong, "Pipelined KV cache migration for disaggregated LLM serving on heterogeneous edge–cloud testbeds," 2026.
Z. Luo, J. Chen, G. Feng, Y. Wang, and Q. Yang, "Generate-and-verify AI policy design for distributed serverless scheduling," *2026 6th International Conference on Advanced Algorithms and Neural Networks (AANN)*, pp. 715–720, Aug. 2026.
Q. Yang, G. Feng, Y. Wang, J. Chen, and Z. Luo, "BudgetSRE: Budget-aware multi-agent incident response for backend operations," *2026 3rd International Conference on Image Processing, Multimedia Technology and Machine Learning (IPMML)*, pp. 221–226, Aug. 2026.
L. Zheng, J. Chen, S. Hu, Y. Jin, and Y. Duan, "Evidence-grounded generation of audit procedures for financial statement risk assessment," 2026.
G. Feng, J. Chen, Z. Luo, Q. Yang, and Y. Wang, "AgentShield: A lightweight sentinel framework for mitigating hallucination propagation in multi-agent LLM systems," 2026.