КҮШЕЙТУ АРҚЫЛЫ ОҚЫТУДЫ ПАЙДАЛАНА ОТЫРЫП, ЗИЯТКЕРЛІК ӨНДІРІСТІ БАСҚАРУ ӘДІСТЕРІ

Бүгінде ақылды өндіріс жүйелерін құру – өзекті міндет. Күрделі өндірістік мәселелерді шешуге мүмкіндік беретін нейрондық желілер кеңінен қолданылады. Мақала GEMMA моделінің бөлігі ретінде өнеркәсіптік жабдықтардың күйін анықтауға арналған DQN, PPO сияқты нейрондық желілер негізінде құрылған арматурамен оқыту алгоритмдеріне арналған. Француздық GEMMA моделі SFC (Sequential Function Charts) тіліне негізделген және процестерді басқару стандарттарын қамтиды. GEMMA моделінің D аймағына нейрондық желілерді енгізу ұсынылады. Модельдеу және эксперимент нәтижелері екі түрлі деректер базасы негізінде жүзеге асырылды: біреуі жасанды түрде жасалды, екіншісі өнеркәсіптік өндірістен алынды. Қарастырылған архитектураларды қолдану өнеркәсіптік деректермен жұмыс істеу үшін жақсы нәтижелерге қол жеткізуге мүмкіндік береді.

1. Jayakumar, S., and Nandakumar, S. Distributed resource optimization using the Q-learning algorithm in device-to-device communication: A reinforcement learning paradigm. Results in Engineering, 23, 102462 (2024). https://doi.org/10.1016/j.rineng.2024.102462

2. Xia, K., Sacco, C., Kirkpatrick, M., et al. A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence. Journal of Manufacturing Systems, 58, 210–230 (2021). https://doi.org/10.1016/j.jmsy.2020.06.012

3. Park, K.T., Son, Y.H., Ko, S.W., and Noh, S.D. Digital twin and reinforcement learning-based resilient production control for micro smart factory. Applied Sciences, 11 (7) (2021). https://doi.org/10.3390/app11072977

4. Ullah, I., Hassan, U.U., and Ali, M.I. Multi-level federated learning for Industry 4.0: A crowdsourcing approach. Procedia Computer Science, 217, 423–435 (2022). https://doi.org/10.1016/j.procs.2022.12.238

5. Mezair, T., Djenouri, Y., and Belhadi, A. A sustainable deep learning framework for fault detection in 6G Industry 4.0 heterogeneous data environments. Computer Communications, 187, 164–171 (2022). https://doi.org/10.1016/j.comcom.2022.02.010

6. Neupane, D., and Seok, J. Bearing fault detection and diagnosis using Case Western Reserve University dataset with deep learning approaches: A review. IEEE Access, 8, 93155–93178 (2020). https://doi.org/10.1109/ACCESS.2020.2990528

7. Mattera, G., Mattera, R., Vespoli, S., and Salatiello, E. Anomaly detection in manufacturing systems with temporal networks and unsupervised machine learning. Computers & Industrial Engineering, 203, 111023 (2025). https://doi.org/10.1016/j.cie.2025.111023

8. Prabu, S., Senthilraja, R., Ali, A.M., Jayapoorani, S., and Arun, M. AI-driven predictive maintenance for smart manufacturing systems using digital twin technology. International Journal of Computational and Experimental Science and Engineering, 11 (1), 1350–1355 (2025). https://doi.org/10.22399/ijcesen.1099

9. Asmat, H., Din, I., Almogren, A., and Khan, M. Digital twin with soft actor-critic reinforcement learning for transitioning from Industry 4.0 to 5.0. IEEE Access (2025). https://doi.org/10.1109/ACCESS.2025.3546085

10. Ruiz Rodríguez, M.L., Kubler, S., de Giorgio, A., Cordy, M., Robert, J., and Le Traon, Y. Multi-agent deep reinforcement learning based predictive maintenance on parallel machines. Robotics and Computer-Integrated Manufacturing, 78, 102406 (2022). https://doi.org/10.1016/j.rcim.2022.102406

11. Pliego Marugán, A., Pinar-Pérez, J.M., and García Márquez, F.P. A reinforcement learning agent for maintenance of deteriorating systems with increasingly imperfect repairs. Reliability Engineering & System Safety, 252, 110466 (2024). https://doi.org/10.1016/j.ress.2024.110466

12. Serrano-Ruiz, J.C., Mula, J., and Poler, R. Development of a multidimensional conceptual model for job shop smart manufacturing scheduling from the Industry 4.0 perspective. Journal of Manufacturing Systems, 63, 185–202 (2022). https://doi.org/10.1016/j.jmsy.2022.03.011

13. Zhou, L., Zhang, L., and Horn, B. Deep reinforcement learning-based dynamic scheduling in smart manufacturing. Procedia CIRP, 93, 383–388 (2020). https://doi.org/10.1016/j.procir.2020.05.163

14. Zhang, M., Lu, Y., Hu, Y., et al. Dynamic scheduling method for job-shop manufacturing systems by deep reinforcement learning with proximal policy optimization. Sustainability, 14 (9) (2022). https://doi.org/10.3390/su14095177

15. Yang, S., and Xu, Z. Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing. International Journal of Production Research, 60 (16), 4936–4953 (2022). https://doi.org/10.1080/00207543.2021.1943037

16. Ho, G., Tang, Y., Leung, E., and Tong, P. Integrated reinforcement learning of automated guided vehicles dynamic path planning for smart logistics and operations. Transportation Research Part E: Logistics and Transportation Review, 196 (2025). https://doi.org/10.1016/j.tre.2025.104008

17. Zhou, T., Tang, D., and Wang, L. Reinforcement learning with composite rewards for production scheduling in a smart factory. IEEEAccess, 9, 752–766 (2021). https://doi.org/10.1109/ACCESS.2020.3046784

18. Bin Shaikat, F., and Ahmed Faysal, S. Optimization of production scheduling in smart manufacturing environments using machine learning algorithms (2025).

19. Kim, Y., Kim, H., Ha, B., and Kim, W. Federated digital twins: A scheduling approach based on temporal graph neural network and deep reinforcement learning. IEEE Access (2025). https://doi.org/10.1109/ACCESS.2025.3530558

20. Chen, Y., Zhang, J., and Rauf, M. Dynamic scheduling of hybrid flow shop problem with uncertain process time and flexible maintenance using NeuroEvolution of Augmenting Topologies. IET Collaborative Intelligent Manufacturing, 6 (3) (2024). https://doi.org/10.1049/cim2.12119

21. Schoepp, S., Taghian, M., and Miwa, S. Enhancing hardware fault tolerance in machines with reinforcement learning policy gradient algorithms (2024). https://doi.org/10.48550/arXiv.2407.15283

22. Modirrousta, M.H., Aliyari Shoorehdeli, M., and Yari, M. Imbalanced classification in faulty turbine data: New proximal policy optimisation. IET Collaborative Intelligent Manufacturing, 6 (3) (2024). https://doi.org/10.1049/cim2.12114

23. Yang, Y., Juntao, L., and Lingling, P. Multi-robot path planning based on a deep reinforcement learning DQN algorithm. CAAI Transactions on Intelligence Technology, 5 (3), 177–183 (2020). https://doi.org/10.1049/trit.2020.0024

24. Zhu, Q., Han, J., Chai, K., and Zhao, C. Time series analysis based on Informer algorithms: A survey. Symmetry, 15 (4) (2023). https://doi.org/10.3390/sym15040951

25. Zhang, X., Yang, K., and Zheng, L. Transformer fault diagnosis method based on TimesNet and Informer. Actuators, 13 (2) (2024). https://doi.org/10.3390/act13020074

26. Pan, Y., et al. Understanding and mitigating the limitations of prioritized experience replay (2023). https://doi.org/10.48550/arXiv.2007.09569

27. Bu, F., and Chang, D.E. Double prioritized state recycled experience replay. In: Proceedings of the IEEE International Conference on Consumer Electronics Asia (2020). https://doi.org/10.1109/ICCEAsia49877.2020.9276975

28. Chandrinos, N., Loi, I., Zachos, P., et al. Effectiveness of L2 regularization in privacy-preserving machine learning (2024). https://doi.org/10.48550/arXiv.2412.01541

29. Alinda Rahmi, N., and Defit, S. The use of hyperparameter tuning in model classification: A scientific work area identification. International Journal of Applied Sciences and Smart Technologies, 8 (4) (2025). http://dx.doi.org/10.62527/joiv.8.4.3092

30. Shekhar, S., Bansode, A., and Salim, A. A comparative study of hyper-parameter optimization tools. In: 2021 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) (2021). https://doi.org/10.1109/CSDE53843.2021.9718485

31. Jordon, J., Szpruch, L., Houssiau, F., et al. Synthetic data – what, why and how? (2022). https://doi.org/10.48550/arXiv.2205.03257

32. Fan, C., Chen, M., Wang, X., et al. A review on data preprocessing techniques toward efficient and reliable knowledge discovery from building operational data. Frontiers in Energy Research, 9 (2021). https://doi.org/10.3389/fenrg.2021.652801

33. Bekar, E.T., Nyqvist, P., and Skoogh, A. An intelligent approach for data pre-processing and analysis in predictive maintenance with an industrial case study. Advances in Mechanical Engineering, 12 (5) (2020). https://doi.org/10.1177/168781402091

34. Ou, F., Wang, H., Zhang, C., et al. Industrial data-driven machine learning soft sensing for optimal operation of etching tools. Digital Chemical Engineering, 13 (2024). https://doi.org/10.1016/j.dche.2024.100195

35. Terven, J., Cordova-Esparza, D., Ramirez-Pedraza, A., et al. Loss functions and metrics in deep learning (2023). https://doi.org/10.48550/arXiv.2307.02694

36. Le, P.B., and Nguyen, Z.T. ROC curves, loss functions, and distorted probabilities in binary classification. Mathematics, 10 (9) (2022). https://doi.org/10.3390/math10091410

37. Samigulina, G.A., Samigulina, Z.I., Bekeshev, D., and Butakova, D. Data-driven machinery faults detection techniques using artificial intelligence in Industry 4.0 concept. Procedia Computer Science, 257, 404–411 (2025). https://doi.org/10.1016/j.procs.2025.03.053