DEVELOPMENT AND OPTIMIZATION OF NEURAL NETWORK MODELS WITH ATTENTION MECHANISMS FOR INTRADAY PRICE FORECASTING FOR EUR/USD

The study examines the problem of intraday forecasting of the EUR/USD currency pair using various neural network architectures, in particular models integrating attention mechanisms. Three neural network architectures were studied: the basic LSTM model, the LSTM model with the Bahdanau attention mechanism, and the Transformer model with the self-attention mechanism. The experiment was conducted on historical minute data for the period from January 2020 to December 2022. The results showed that attentional models are significantly superior to the basic LSTM architecture. The best results were obtained by the Transformer model (MSE=0.185, MAE=0.297, RMSE=0.431, MAPE=7.3%). A detailed analysis confirmed the stability and accuracy of the Transformer model. The identified advantages of attention models justify their prospects for use in algorithmic trading and require further research to optimize and adapt to real trading conditions. In particular, further research may be aimed at integrating attention models with trading strategies and risk management systems, as well as studying their behavior in the face of sudden changes in market volatility. In addition, it is proposed to explore the possibilities of combining attention architectures with other forecasting methods to increase the overall stability and reliability of forecasts in practical trading.

1. Dakalbab, F., Kumar, A., Abu Talib, M., and Nasir, Q. Advancing Forex prediction through multimodal text-driven model and attention mechanisms. Intelligent Systems with Applications, 23, 200518 (2025). https://doi.org/10.1016/j.iswa.2025.200518

2. Yildirim, D.C., Toroslu, I.H., and Fiore, U. Forecasting directional movement of Forex data using LSTM with technical and macroeconomic indicators. Financial Innovation, 7 (1), 1–21 (2021). https://doi.org/10.1186/s40854-020-00220-2

3. Saghafi, A., Bagherian, M., and Shokoohi, F. Forecasting Forex EUR/USD closing prices using a dualinput deep learning model with technical and fundamental indicators. Mathematics, 13 (9), 1472 (2025).

4. García, F., Martínez-César, S., Pérez-Rosés, H., et al. Foreign exchange forecasting models: LSTM and BiLSTM comparison. Engineering Proceedings, 68 (1), 19 (2024).

5. Islam, M.S., and Hossain, E. Foreign exchange currency rate prediction using a GRU–LSTM hybrid network. Soft Computing Letters, 3, 100009 (2021).

6. Ayitey Junior, M., Appiahene, P., and Appiah, O. Forex market forecasting with two-layer stacked long short-term memory neural network (LSTM) and correlation analysis. Journal of Electrical Systems and Information Technology, 9 (1), 14 (2022).

7. Bormpotsis, C., Sedky, M., and Patel, A. Predicting Forex currency fluctuations using a novel bioinspired modular neural network. Big Data and Cognitive Computing, 7 (3), 152 (2023).

8. Markova, M. Forex time series forecasting using hybrid convolutional neural network/long shortterm memory network model. In: International Conference on New Trends in the Applications of Differential Equations in Sciences. Cham: Springer International Publishing, 2022, pp. 295–305.

9. Shi, H., Li, Y., Wang, Z., et al. A CNN–LSTM based deep learning model with high accuracy and robustness for carbon price forecasting: A case of Shenzhen’s carbon market in China. Journal of Environmental Management, 352, 120131 (2024).

10. Pornwattanavichai, A., Maneeroj, S., and Boonsiri, S. BERTFOREX: Cascading model for Forex market forecasting using fundamental and technical indicator data based on BERT. IEEE Access, 10, 23425– 23437 (2022).

11. Ito, K., Iima, H., and Kitamura, Y. LSTM forecasting foreign exchange rates using limit order book. Finance Research Letters, 47, 102517 (2022).

12. Zafeiriou, T., and Kalles, D. Comparative analysis of neural network architectures for short-term FOREX forecasting. Intelligent Decision Technologies, 18 (3), 2177–2190 (2024).

13. Gradzki, P., and Wojcik, P. Is attention all you need for intraday Forex trading? Expert Systems, 41 (2), e13317 (2024).

14. Kantoutsis, K.T., Mavrogianni, A.N., and Theodorakatos, N.P. Transformers in high-frequency trading. Journal of Physics: Conference Series, 2701 (1), 012134 (2024).

15. Fischer, T., Sterling, M., and Lessmann, S. FX-spot predictions with state-of-the-art transformer and time embeddings. Expert Systems with Applications, 249, 123538 (2024).

16. Peng, P., Liu, X., Zhang, Y., et al. Attention-based CNN–LSTM for high-frequency multiple cryptocurrency trend prediction. Expert Systems with Applications, 237, 121520 (2024).

17. Zhang, J., and Yan, L. GRU-enhanced attention mechanism for LSTM in hybrid CNN–LSTM models for stock prediction. Journal of Global Trends in Social Science, 2 (3), 10–17 (2025). https://doi.org/10.70731/rzvs8j53

18. Zhao, L., and Yan, W.Q. Prediction of currency exchange rate based on transformers. Journal of Risk and Financial Management, 17 (8), 332 (2024).

19. Hadizadeh, A., Tarokh, M.J., and Ghazani, M.M. A novel transformer-based dual attention architecture for the prediction of financial time series. Journal of King Saud University – Computer and Information Sciences, 37, 72 (2025). https://doi.org/10.1007/s44443-025-00045-y

20. Liu, Q., Hu, Y., and Liu, H. Enhanced stock price prediction with optimized ensemble modeling using multi-source heterogeneous data: Integrating LSTM attention mechanism and multidimensional gray model. Journal of Industrial Information Integration, 42, 100711 (2024).