kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2025-22-1-10-24kaz29-1726Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEРАЗРАБОТКА МОДЕЛИ РАСПОЗНАВАНИЯ КАЗАХСКОГО ЯЗЫКА ЖЕСТОВ НА ОСНОВЕ YOLO NASDEVELOPMENT OF A KAZAKH SIGN LANGUAGE RECOGNITION MODEL BASED ON YOLO-NAShttps://orcid.org/0000-0002-5124-5759ОтманМ.OthmanM.

профессор 

 г. Куала-Лумпур 

 Professor 

 Kuala Lumpur 

mothmanupm@gmail.comОралбековаД.OralbekovaD.

сениор-лектор 

 г. Алматы 

 Senior Lecturer 

 Almaty 

dinaoral@mail.ruhttps://orcid.org/0009-0000-2467-5721БержановаУ. Г.BerzhanovaU. G.

 докторант 

 г. Алматы 

 Doctoral student 

 Almaty 

berzhanovaulmekenn@gmail.comУниверситет Путра МалайзияМалайзияUniversiti Putra MalaysiaMalaysiaКазахский национальный университет им. аль-ФарабиКазахстанSatbayev UniversityKazakhstan2025220320252211024Copyright © Отман М., Оралбекова Д., Бержанова У.Г., 20252025Отман М., Оралбекова Д., Бержанова У.Г.Othman M., Oralbekova D., Berzhanova U.G.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/1726

Разработка надежной модели распознавания казахского жестового языка является важным шагом на пути к развитию инклюзивной коммуникации и помощи людям с нарушениями слуха. В данной работе подробно описывается процесс сбора и аннотирования данных, в которых использовались изображения жестов. Особое внимание уделяется подготовке и предварительной обработке данных для обеспечения их совместимости с моделью. Процесс обучения модели включает оптимизацию гиперпараметров и использование различных методов для повышения точности распознавания. Мы также провели комплексную оценку производительности модели на основе тестовых данных, чтобы убедиться в ее эффективности в реальных условиях. Помимо основного этапа разработки мы рассматриваем возможность тестирования модели YOLO-NAS на том же наборе данных для изучения потенциальных улучшений точности и производительности. В заключение следует отметить, что результаты нашего исследования могут быть использованы для дальнейшей разработки технологий, способствующих интеграции людей с нарушениями слуха в общество, а также для создания образовательных и коммуникационных платформ на основе казахского жестового языка.

The development of a reliable model of recognition of Kazakh sign language is an important step towards the development of inclusive communication and assistance to people with hearing impairments. This paper describes in detail the process of collecting and annotating data in which gesture images were used. Special attention is paid to the preparation and preprocessing of data to ensure their compatibility with the model. The process of learning the model involves optimizing hyperparameters and using various techniques to improve recognition accuracy. We also conducted a comprehensive performance assessment of the model based on test data to ensure its effectiveness in real-world conditions. In addition to the main development phase, we are considering testing the YOLO-NAS model on the same dataset to explore potential improvements in accuracy and performance. In conclusion, the results of our research can be used to further develop technologies that facilitate the integration of people with hearing impairments into society, as well as to create educational and communication platforms based on the Kazakh sign language.

You Only Look Once (YOLO)YOLO-NASглубокое обучениесверточная нейронная сеть (CNN)искусственный интеллектказахский язык жестовYou Only Look Once (YOLO)YOLO-NASDeep LearningConvolutional Neural Network (CNN)Artificial IntelligenceKazakh sign languageReferencesDaniels S., Suciati N., Fathichah C. Indonesian sign language recognition using YOLO method. IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2021, vol. 1077, no. 1, p. 012029. https://doi.org/10.1088/1757-899x/1077/1/012029.Daniels S., Suciati N., Fathichah C. Indonesian sign language recognition using YOLO method. IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2021, vol. 1077, no. 1, p. 012029. https://doi.org/10.1088/1757-899x/1077/1/012029.Al Ahmadi S., Mohammad F., Al Dawsari H. Efficient YOLO Based Deep Learning Model for Arabic Sign Language Recognition, 2024. https://doi.org/10.21203/rs.3.rs-4006855/v1.Al Ahmadi S., Mohammad F., Al Dawsari H. Efficient YOLO Based Deep Learning Model for Arabic Sign Language Recognition, 2024. https://doi.org/10.21203/rs.3.rs-4006855/v1.Mesbahi S. C. et al. Hand gesture recognition based on various Deep Learning YOLO models. International Journal of Advanced Computer Science and Applications, 2023, vol. 14, no. 4.Mesbahi S. C. et al. Hand gesture recognition based on various Deep Learning YOLO models. International Journal of Advanced Computer Science and Applications, 2023, vol. 14, no. 4.Doždor Z. et al. TY-Net: Transforming YOLO for hand gesture recognition. IEEE access., 2023. https://doi.org/10.14569/ijacsa.2023.0140435.Doždor Z. et al. TY-Net: Transforming YOLO for hand gesture recognition. IEEE access., 2023. https://doi.org/10.14569/ijacsa.2023.0140435.Yerraboina S. Real-Time Hand Gesture Recognition System, 2024.Yerraboina S. Real-Time Hand Gesture Recognition System, 2024.Mallikarjuna Swamy N. et al. Indian sign language detection using YOLOv3. High Performance Computing and Networking: Select Proceedings of CHSN 2021. Singapore: Springer Singapore, 2022, pp. 157–168. https://doi.org/10.1007/978-981-16-9885-9_13.Mallikarjuna Swamy N. et al. Indian sign language detection using YOLOv3. High Performance Computing and Networking: Select Proceedings of CHSN 2021. Singapore: Springer Singapore, 2022, pp. 157–168. https://doi.org/10.1007/978-981-16-9885-9_13.Asri M. et al. A real time Malaysian sign language detection algorithm based on YOLOv3. International Journal of Recent Technology and Engineering, 2019, vol. 8, no. 2, pp. 651–656. https://doi.org/10.35940/ijrte.b1102.0982s1119.Asri M. et al. A real time Malaysian sign language detection algorithm based on YOLOv3. International Journal of Recent Technology and Engineering, 2019, vol. 8, no. 2, pp. 651–656. https://doi.org/10.35940/ijrte.b1102.0982s1119.Khaliluzzaman M., Kobra K., Liaqat S. Comparative analysis on real-time hand gesture and sign language recognition using convexity defects and YOLOv3. Sigma Journal of Engineering and Natural Sciences, 2024, vol. 42, no. 1, pp. 99–115. https://doi.org/10.14744/sigma.2024.00012.Khaliluzzaman M., Kobra K., Liaqat S. Comparative analysis on real-time hand gesture and sign language recognition using convexity defects and YOLOv3. Sigma Journal of Engineering and Natural Sciences, 2024, vol. 42, no. 1, pp. 99–115. https://doi.org/10.14744/sigma.2024.00012.Mujahid A. et al. Real-time hand gesture recognition based on Deep Learning YOLOv3 model. Applied Sciences, 2021, vol. 11, no. 9, p. 4164. https://doi.org/10.3390/app11094164.Mujahid A. et al. Real-time hand gesture recognition based on Deep Learning YOLOv3 model. Applied Sciences, 2021, vol. 11, no. 9, p. 4164. https://doi.org/10.3390/app11094164.Lawand S. J. et al. Sign Language Hand Gesture Identification Using YOLOv3. Available at SSRN 4385690. http://dx.doi.org/10.2139/ssrn.4385690.Lawand S. J. et al. Sign Language Hand Gesture Identification Using YOLOv3. Available at SSRN 4385690. http://dx.doi.org/10.2139/ssrn.4385690.Alaftekin M., Pacal I., Cicek K. Real-time sign language recognition based on YOLO algorithm. Neural Computing and Applications, 2024, vol. 36, no. 14, pp. 7609–7624. https://doi.org/10.1007/s00521-024-09503-6.Alaftekin M., Pacal I., Cicek K. Real-time sign language recognition based on YOLO algorithm. Neural Computing and Applications, 2024, vol. 36, no. 14, pp. 7609–7624. https://doi.org/10.1007/s00521-024-09503-6.Al-shaheen A., Çevik M., Alqaraghulı A. American sign language recognition using YOLOv4 method. International Journal of Multidisciplinary Studies and Innovative Technologies, 2022, vol. 6, no. 1, pp. 61–65, https://doi.org/ 10.36287/ijmsit.6.1.61.Al-shaheen A., Çevik M., Alqaraghulı A. American sign language recognition using YOLOv4 method. International Journal of Multidisciplinary Studies and Innovative Technologies, 2022, vol. 6, no. 1, pp. 61–65, https://doi.org/ 10.36287/ijmsit.6.1.61.Alaftekin M., Pacal I. & Cicek K. Real-time sign language recognition based on YOLO algorithm. Neural Comput & Applic, 2024, vol. 36, pp. 7609–7624. https://doi.org/10.1007/s00521-024-09503-6.Alaftekin M., Pacal I. & Cicek K. Real-time sign language recognition based on YOLO algorithm. Neural Comput & Applic, 2024, vol. 36, pp. 7609–7624. https://doi.org/10.1007/s00521-024-09503-6.Sreemathy R. et al. Continuous word level sign language recognition using an expert system based on machine learning //International Journal of Cognitive Computing in Engineering, 2023, vol. 4, pp. 170–178. https://doi.org/10.1016/j.ijcce.2023.04.002.Sreemathy R. et al. Continuous word level sign language recognition using an expert system based on machine learning //International Journal of Cognitive Computing in Engineering, 2023, vol. 4, pp. 170–178. https://doi.org/10.1016/j.ijcce.2023.04.002.Begum N. et al. Borno-net: a real-time Bengali sign-character detection and sentence generation system using quantized YOLOv4-Tiny and LSTMs //Applied Sciences, 2023, vol. 13, no. 9, p. 5219. https://doi.org/10.3390/app13095219.Begum N. et al. Borno-net: a real-time Bengali sign-character detection and sentence generation system using quantized YOLOv4-Tiny and LSTMs //Applied Sciences, 2023, vol. 13, no. 9, p. 5219. https://doi.org/10.3390/app13095219.Bankar S. et al. Real time sign language recognition using Deep Learning. International Research Journal of Engineering and Technology, 2022, vol. 9, no. 4, pp. 955–959. https://doi.org/10.22214/ijraset.2023.55621.Bankar S. et al. Real time sign language recognition using Deep Learning. International Research Journal of Engineering and Technology, 2022, vol. 9, no. 4, pp. 955–959. https://doi.org/10.22214/ijraset.2023.55621.Aiouez S. et al. Real-time Arabic Sign Language Recognition based on YOLOv5. IMPROVE, 2022, pp. 17–25. https://doi.org/10.5220/0010979300003209.Aiouez S. et al. Real-time Arabic Sign Language Recognition based on YOLOv5. IMPROVE, 2022, pp. 17–25. https://doi.org/10.5220/0010979300003209.Reddy P.V. et al. Sign Language Recognition based on YOLOv5 Algorithm for the Telugu Sign Language. arXiv e-prints, 2024. С. arXiv: 2406.10231, https://doi.org/10.48550/arXiv.2406.10231.Reddy P.V. et al. Sign Language Recognition based on YOLOv5 Algorithm for the Telugu Sign Language. arXiv e-prints, 2024. С. arXiv: 2406.10231, https://doi.org/10.48550/arXiv.2406.10231.Venkatraja V.M.C. et al. Sign language to speech converter for indian languages, 2023.Venkatraja V.M.C. et al. Sign language to speech converter for indian languages, 2023.Attia N.F., Ahmed M.T.F.S., Alshewimy M.A.M. Efficient Deep Learning models based on tension techniques for sign language recognition. Intelligent systems with applications, 2023, vol. 20, p. 200284. https://doi.org/10.1016/j.iswa.2023.200284.Attia N.F., Ahmed M.T.F.S., Alshewimy M.A.M. Efficient Deep Learning models based on tension techniques for sign language recognition. Intelligent systems with applications, 2023, vol. 20, p. 200284. https://doi.org/10.1016/j.iswa.2023.200284.Buttar A.M. et al. Deep Learning in sign language recognition: a hybrid approach for the recognition of static and dynamic signs. Mathematics, 2023, vol. 11, no. 17, p. 3729. https://doi.org/10.3390/math11173729.Buttar A.M. et al. Deep Learning in sign language recognition: a hybrid approach for the recognition of static and dynamic signs. Mathematics, 2023, vol. 11, no. 17, p. 3729. https://doi.org/10.3390/math11173729.Siddique S. et al. Deep Learning-based bangla sign language detection with an edge device. Intelligent Systems with Applications, 2023, vol. 18, p. 200224. https://doi.org/10.1016/j.iswa.2023.200224.Siddique S. et al. Deep Learning-based bangla sign language detection with an edge device. Intelligent Systems with Applications, 2023, vol. 18, p. 200224. https://doi.org/10.1016/j.iswa.2023.200224.Nair A. B. et al. Malayalam Sign Language Identification using Finetuned YOLOv8 and Computer Vision Techniques. arXiv preprint arXiv:2405.06702, 2024. https://doi.org/10.48550/arXiv.2405.06702.Nair A. B. et al. Malayalam Sign Language Identification using Finetuned YOLOv8 and Computer Vision Techniques. arXiv preprint arXiv:2405.06702, 2024. https://doi.org/10.48550/arXiv.2405.06702.Kalimuthu S. Video Captioning Based on Sign Language Using YOLOV8 Model, 2023. https://doi.org/10.1007/978-3-031-45878-1_21.Kalimuthu S. Video Captioning Based on Sign Language Using YOLOV8 Model, 2023. https://doi.org/10.1007/978-3-031-45878-1_21.Hinge R. et al. Improving Indian Sign Language Interpretation with Deep Learning-Based Translation System. Journal of technical education, p. 44.Hinge R. et al. Improving Indian Sign Language Interpretation with Deep Learning-Based Translation System. Journal of technical education, p. 44.ZainEldin H. et al. Active convolutional neural networks sign language (ActiveCNN-SL) framework: a paradigm shift in deaf-mute communication. Artificial Intelligence Review, 2024, vol. 57, no. 6, p. 162. https://doi.org/10.1007/s10462-024-10792-5.ZainEldin H. et al. Active convolutional neural networks sign language (ActiveCNN-SL) framework: a paradigm shift in deaf-mute communication. Artificial Intelligence Review, 2024, vol. 57, no. 6, p. 162. https://doi.org/10.1007/s10462-024-10792-5.Purnomo H. et al. Utilizing the YOLOv8 Model for Accurate Hand Gesture Recognition with Complex Background. Available at SSRN 4777516. http://dx.doi.org/10.2139/ssrn.4777516.Purnomo H. et al. Utilizing the YOLOv8 Model for Accurate Hand Gesture Recognition with Complex Background. Available at SSRN 4777516. http://dx.doi.org/10.2139/ssrn.4777516.Mukhanov S. et al. Gesture recognition of machine learning and convolutional neural network methods for kazakh sign language. Scientific Journal of Astana IT University, 2023, pp. 85–100. https://doi.org/10.37943/15lpcu4095.Mukhanov S. et al. Gesture recognition of machine learning and convolutional neural network methods for kazakh sign language. Scientific Journal of Astana IT University, 2023, pp. 85–100. https://doi.org/10.37943/15lpcu4095.Tang Y., Wang Y., Qian Y. Real-time railroad track components inspection framework based on YOLONAS and edge computing. IOP Conference Series: Earth and Environmental Science. – IOP Publishing, 2024, vol. 1337, no. 1, p. 012017. https://doi.org/10.1088/1755-1315/1337/1/012017.Tang Y., Wang Y., Qian Y. Real-time railroad track components inspection framework based on YOLONAS and edge computing. IOP Conference Series: Earth and Environmental Science. – IOP Publishing, 2024, vol. 1337, no. 1, p. 012017. https://doi.org/10.1088/1755-1315/1337/1/012017.

The authors declare that there are no conflicts of interest present.