kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2025-22-4-97-106kaz29-2283Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEРАЗРАБОТКА И ОПТИМИЗАЦИЯ НЕЙРОННЫХ СЕТЕВЫХ МОДЕЛЕЙ С МЕХАНИЗМАМИ ВНИМАНИЯ ДЛЯ ВНУТРИДНЕВНОГО ПРОГНОЗИРОВАНИЯ ЦЕНЫ ПАРЫ EUR/USDDEVELOPMENT AND OPTIMIZATION OF NEURAL NETWORK MODELS WITH ATTENTION MECHANISMS FOR INTRADAY PRICE FORECASTING FOR EUR/USDhttps://orcid.org/0000-0002-6381-9350АбдилдаеваА. А.AbdildaevaA. A.

PhD, и.о. профессора

г. Алматы

PhD, acting Professor

Almaty

assel.abdildaeva@kaznu.edu.kzhttps://orcid.org/0000-0001-8824-9502НуртуганГ. Б.NurtuganG. B.

докторант

г. Алматы

PhD student

Almaty

g.nurtugun@gmail.comhttps://orcid.org/0000-0002-7851-4330ВойткевичК.WojtkiewiczK.

PhD, доцент

krystian.wojtkiewicz@pwr.edu.pl

PhD, assistant Professor

Wroclaw

krystian.wojtkiewicz@pwr.edu.plКазахский национальный университет им. аль-ФарабиКазахстанAl-Farabi Kazakh National UniversityKazakhstanВроцлавский университет науки и технологий (WUST)ПольшаWroclaw University of Science and Technology (WUST)Poland20252312202522497106Copyright © Абдилдаева А.А., Нуртуган Г.Б., Войткевич К., 20252025Абдилдаева А.А., Нуртуган Г.Б., Войткевич К.Abdildaeva A.A., Nurtugan G.B., Wojtkiewicz K.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/2283

В исследовании рассматривается проблема внутридневного прогнозирования курса валютной пары EUR/USD с использованием различных нейросетевых архитектур, в частности моделей, интегрирующих механизмы внимания (attention). Были исследованы три нейросетевые архитектуры: базовая модель LSTM, модель LSTM с механизмом внимания Bahdanau и модель Transformer с механизмом самовнимания (selfattention). Эксперимент проводился на исторических минутных данных за период с января 2020 по декабрь 2022 гг. Результаты показали, что модели с механизмом внимания значительно превосходят базовую архитектуру LSTM. Наилучшие результаты были получены моделью Transformer (MSE=0.185, MAE=0.297, RMSE=0.431, MAPE=7.3%). Подробный анализ подтвердил стабильность и точность модели Transformer. Выявленные преимущества attention-моделей обосновывают их перспективность для применения в алгоритмической торговле и требуют дальнейших исследований для оптимизации и адаптации к реальным торговым условиям. В частности, дальнейшие исследования могут быть направлены на интеграцию attentionмоделей с торговыми стратегиями и системами управления рисками, а также изучение их поведения в условиях резких изменений рыночной волатильности. Кроме того, предлагается исследовать возможности комбинирования attention-архитектур с другими методами прогнозирования для повышения общей устойчивости и надежности прогнозов в практическом трейдинге.

The study examines the problem of intraday forecasting of the EUR/USD currency pair using various neural network architectures, in particular models integrating attention mechanisms. Three neural network architectures were studied: the basic LSTM model, the LSTM model with the Bahdanau attention mechanism, and the Transformer model with the self-attention mechanism. The experiment was conducted on historical minute data for the period from January 2020 to December 2022. The results showed that attentional models are significantly superior to the basic LSTM architecture. The best results were obtained by the Transformer model (MSE=0.185, MAE=0.297, RMSE=0.431, MAPE=7.3%). A detailed analysis confirmed the stability and accuracy of the Transformer model. The identified advantages of attention models justify their prospects for use in algorithmic trading and require further research to optimize and adapt to real trading conditions. In particular, further research may be aimed at integrating attention models with trading strategies and risk management systems, as well as studying their behavior in the face of sudden changes in market volatility. In addition, it is proposed to explore the possibilities of combining attention architectures with other forecasting methods to increase the overall stability and reliability of forecasts in practical trading.

внутридневное прогнозированиеForexвалютные курсыEUR/USDнейронные сетимеханизм вниманияTransformerLSTMсамовниманиеintraday forecastingForexexchange ratesEUR/USDneural networksattention mechanismTransformerLSTMself-attentionReferencesDakalbab, F., Kumar, A., Abu Talib, M., and Nasir, Q. Advancing Forex prediction through multimodal text-driven model and attention mechanisms. Intelligent Systems with Applications, 23, 200518 (2025). https://doi.org/10.1016/j.iswa.2025.200518Dakalbab, F., Kumar, A., Abu Talib, M., and Nasir, Q. Advancing Forex prediction through multimodal text-driven model and attention mechanisms. Intelligent Systems with Applications, 23, 200518 (2025). https://doi.org/10.1016/j.iswa.2025.200518Yildirim, D.C., Toroslu, I.H., and Fiore, U. Forecasting directional movement of Forex data using LSTM with technical and macroeconomic indicators. Financial Innovation, 7 (1), 1–21 (2021). https://doi.org/10.1186/s40854-020-00220-2Yildirim, D.C., Toroslu, I.H., and Fiore, U. Forecasting directional movement of Forex data using LSTM with technical and macroeconomic indicators. Financial Innovation, 7 (1), 1–21 (2021). https://doi.org/10.1186/s40854-020-00220-2Saghafi, A., Bagherian, M., and Shokoohi, F. Forecasting Forex EUR/USD closing prices using a dualinput deep learning model with technical and fundamental indicators. Mathematics, 13 (9), 1472 (2025).Saghafi, A., Bagherian, M., and Shokoohi, F. Forecasting Forex EUR/USD closing prices using a dualinput deep learning model with technical and fundamental indicators. Mathematics, 13 (9), 1472 (2025).García, F., Martínez-César, S., Pérez-Rosés, H., et al. Foreign exchange forecasting models: LSTM and BiLSTM comparison. Engineering Proceedings, 68 (1), 19 (2024).García, F., Martínez-César, S., Pérez-Rosés, H., et al. Foreign exchange forecasting models: LSTM and BiLSTM comparison. Engineering Proceedings, 68 (1), 19 (2024).Islam, M.S., and Hossain, E. Foreign exchange currency rate prediction using a GRU–LSTM hybrid network. Soft Computing Letters, 3, 100009 (2021).Islam, M.S., and Hossain, E. Foreign exchange currency rate prediction using a GRU–LSTM hybrid network. Soft Computing Letters, 3, 100009 (2021).Ayitey Junior, M., Appiahene, P., and Appiah, O. Forex market forecasting with two-layer stacked long short-term memory neural network (LSTM) and correlation analysis. Journal of Electrical Systems and Information Technology, 9 (1), 14 (2022).Ayitey Junior, M., Appiahene, P., and Appiah, O. Forex market forecasting with two-layer stacked long short-term memory neural network (LSTM) and correlation analysis. Journal of Electrical Systems and Information Technology, 9 (1), 14 (2022).Bormpotsis, C., Sedky, M., and Patel, A. Predicting Forex currency fluctuations using a novel bioinspired modular neural network. Big Data and Cognitive Computing, 7 (3), 152 (2023).Bormpotsis, C., Sedky, M., and Patel, A. Predicting Forex currency fluctuations using a novel bioinspired modular neural network. Big Data and Cognitive Computing, 7 (3), 152 (2023).Markova, M. Forex time series forecasting using hybrid convolutional neural network/long shortterm memory network model. In: International Conference on New Trends in the Applications of Differential Equations in Sciences. Cham: Springer International Publishing, 2022, pp. 295–305.Markova, M. Forex time series forecasting using hybrid convolutional neural network/long shortterm memory network model. In: International Conference on New Trends in the Applications of Differential Equations in Sciences. Cham: Springer International Publishing, 2022, pp. 295–305.Shi, H., Li, Y., Wang, Z., et al. A CNN–LSTM based deep learning model with high accuracy and robustness for carbon price forecasting: A case of Shenzhen’s carbon market in China. Journal of Environmental Management, 352, 120131 (2024).Shi, H., Li, Y., Wang, Z., et al. A CNN–LSTM based deep learning model with high accuracy and robustness for carbon price forecasting: A case of Shenzhen’s carbon market in China. Journal of Environmental Management, 352, 120131 (2024).Pornwattanavichai, A., Maneeroj, S., and Boonsiri, S. BERTFOREX: Cascading model for Forex market forecasting using fundamental and technical indicator data based on BERT. IEEE Access, 10, 23425– 23437 (2022).Pornwattanavichai, A., Maneeroj, S., and Boonsiri, S. BERTFOREX: Cascading model for Forex market forecasting using fundamental and technical indicator data based on BERT. IEEE Access, 10, 23425– 23437 (2022).Ito, K., Iima, H., and Kitamura, Y. LSTM forecasting foreign exchange rates using limit order book. Finance Research Letters, 47, 102517 (2022).Ito, K., Iima, H., and Kitamura, Y. LSTM forecasting foreign exchange rates using limit order book. Finance Research Letters, 47, 102517 (2022).Zafeiriou, T., and Kalles, D. Comparative analysis of neural network architectures for short-term FOREX forecasting. Intelligent Decision Technologies, 18 (3), 2177–2190 (2024).Zafeiriou, T., and Kalles, D. Comparative analysis of neural network architectures for short-term FOREX forecasting. Intelligent Decision Technologies, 18 (3), 2177–2190 (2024).Gradzki, P., and Wojcik, P. Is attention all you need for intraday Forex trading? Expert Systems, 41 (2), e13317 (2024).Gradzki, P., and Wojcik, P. Is attention all you need for intraday Forex trading? Expert Systems, 41 (2), e13317 (2024).Kantoutsis, K.T., Mavrogianni, A.N., and Theodorakatos, N.P. Transformers in high-frequency trading. Journal of Physics: Conference Series, 2701 (1), 012134 (2024).Kantoutsis, K.T., Mavrogianni, A.N., and Theodorakatos, N.P. Transformers in high-frequency trading. Journal of Physics: Conference Series, 2701 (1), 012134 (2024).Fischer, T., Sterling, M., and Lessmann, S. FX-spot predictions with state-of-the-art transformer and time embeddings. Expert Systems with Applications, 249, 123538 (2024).Fischer, T., Sterling, M., and Lessmann, S. FX-spot predictions with state-of-the-art transformer and time embeddings. Expert Systems with Applications, 249, 123538 (2024).Peng, P., Liu, X., Zhang, Y., et al. Attention-based CNN–LSTM for high-frequency multiple cryptocurrency trend prediction. Expert Systems with Applications, 237, 121520 (2024).Peng, P., Liu, X., Zhang, Y., et al. Attention-based CNN–LSTM for high-frequency multiple cryptocurrency trend prediction. Expert Systems with Applications, 237, 121520 (2024).Zhang, J., and Yan, L. GRU-enhanced attention mechanism for LSTM in hybrid CNN–LSTM models for stock prediction. Journal of Global Trends in Social Science, 2 (3), 10–17 (2025). https://doi.org/10.70731/rzvs8j53Zhang, J., and Yan, L. GRU-enhanced attention mechanism for LSTM in hybrid CNN–LSTM models for stock prediction. Journal of Global Trends in Social Science, 2 (3), 10–17 (2025). https://doi.org/10.70731/rzvs8j53Zhao, L., and Yan, W.Q. Prediction of currency exchange rate based on transformers. Journal of Risk and Financial Management, 17 (8), 332 (2024).Zhao, L., and Yan, W.Q. Prediction of currency exchange rate based on transformers. Journal of Risk and Financial Management, 17 (8), 332 (2024).Hadizadeh, A., Tarokh, M.J., and Ghazani, M.M. A novel transformer-based dual attention architecture for the prediction of financial time series. Journal of King Saud University – Computer and Information Sciences, 37, 72 (2025). https://doi.org/10.1007/s44443-025-00045-yHadizadeh, A., Tarokh, M.J., and Ghazani, M.M. A novel transformer-based dual attention architecture for the prediction of financial time series. Journal of King Saud University – Computer and Information Sciences, 37, 72 (2025). https://doi.org/10.1007/s44443-025-00045-yLiu, Q., Hu, Y., and Liu, H. Enhanced stock price prediction with optimized ensemble modeling using multi-source heterogeneous data: Integrating LSTM attention mechanism and multidimensional gray model. Journal of Industrial Information Integration, 42, 100711 (2024).Liu, Q., Hu, Y., and Liu, H. Enhanced stock price prediction with optimized ensemble modeling using multi-source heterogeneous data: Integrating LSTM attention mechanism and multidimensional gray model. Journal of Industrial Information Integration, 42, 100711 (2024).

The authors declare that there are no conflicts of interest present.