kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2025-22-1-94-102kaz29-1735Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEРОЛЬ ИСКУССТВЕННОГО ИНТЕЛЛЕКТА В РАСПОЗНАВАНИИ ЯЗЫКА ЖЕСТОВROLE OF ARTIFICIAL INTELLIGENCE IN SIGN LANGUAGE RECOGNITIONhttps://orcid.org/0009-0008-2976-0771ТурсынМ. Ш.TursynM. S.

 магистр 

 г. Алматы 

 Master 

 Almaty 

me_tursyn@kbtu.kzКазахстанско-Британский технический университетКазахстанKazakh British Technical UniversityKazakhstan20252303202522194102Copyright © Турсын М.Ш., 20252025Турсын М.Ш.Tursyn M.S.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/1735

На протяжении десятилетий растущие вычислительные возможности и развитие новых технологий в области искусственного интеллекта дали нам возможность переводить язык жестов в режиме реального времени. Существует два основных подхода к распознаванию языка жестов: аппаратный и программный. Аппаратный подход основан на использовании специальных перчаток, устройств на базе Kinect и датчиков различного уровня. С другой стороны, одним из подходов к работе с языком жестов является использование нейронных сетей, то есть подход, основанный на программном обеспечении. В этой работе я изучил существующие подходы и поэкспериментировал с машинным обучением и моделями нейронных сетей для распознавания языка жестов. Я получил набор данных по азербайджанскому языку жестов, затем обучил свои модели на основе этого набора данных и получил результаты и показатели. Набор данных содержал более тринадцати тысяч образцов жестов, которые могут быть использованы в казахском жестовом языке. В конце я обсудил вероятную возможность использования разработанных моделей.

Over the decades the increasing computational capability and development of new technologies in the field of artificial intelligence have given us the ability to translate sign language in real time. There exist two main approaches to sign language recognition, the hardware-based approach and the software-based approach. The hardware-based approach relies on using special gloves, Kinect-based devices, and different levels of sensors. On the other hand, one of the approaches to working with sign language is using neural networks, which is the softwarebased approach. In this work, I observed existing approaches and experimented with machine learning and neural network models for sign language recognition. I got the dataset of Azerbaijani Sign Language, then trained my models based on that dataset, and got the results and metrics. The dataset contained over thirteen thousand samples of signs, which can be used in Kazakh Sign Language. In the end, I discussed the probable opportunity of using the developed models.

распознавание языка жестовискусственный интеллектмашинное обучениенейронная сетьязык жестовsign language recognitionartificial intelligencemachine learningneural networksign languageReferencesDubey S., Suryawanshi S., Rachamalla A., and K. Madhu Babu. Sign language recognition. International Journal for Research in Applied Science & Engineering Technology (IJRASET) https://doi.org/10.22214/ijraset.2023.48586Dubey S., Suryawanshi S., Rachamalla A., and K. Madhu Babu. Sign language recognition. International Journal for Research in Applied Science & Engineering Technology (IJRASET) https://doi.org/10.22214/ijraset.2023.48586Farooq U., Rahim M.S.M., Sabir,N. et al. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Comput & Applic, 2021, vol. 33, pp. 14357–14399. https://doi.org/10.1007/s00521-021-06079-3Farooq U., Rahim M.S.M., Sabir,N. et al. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Comput & Applic, 2021, vol. 33, pp. 14357–14399. https://doi.org/10.1007/s00521-021-06079-3Amrutha K. and P. Prabu. ML Based Sign Language Recognition System, 2021 International Conference on Innovative Trends in Information Technology (ICITIIT), Kottayam, India, 2021, pp. 1–6, https://doi.org/10.1109/ICITIIT51526.2021.9399594.Amrutha K. and P. Prabu. ML Based Sign Language Recognition System, 2021 International Conference on Innovative Trends in Information Technology (ICITIIT), Kottayam, India, 2021, pp. 1–6, https://doi.org/10.1109/ICITIIT51526.2021.9399594.Dr. Sabeenian R.S., Bharathwaj S., Aadhil M. Sign Language Recognition Using Deep Learning and Computer Vision. Journal of Advanced Research in Dynamical and Control Systems, 202, vol. 12, pp. 964–968. https://doi.org/10.5373/JARDCS/V12SP5/20201842.Dr. Sabeenian R.S., Bharathwaj S., Aadhil M. Sign Language Recognition Using Deep Learning and Computer Vision. Journal of Advanced Research in Dynamical and Control Systems, 202, vol. 12, pp. 964–968. https://doi.org/10.5373/JARDCS/V12SP5/20201842.Adithya V., Rajesh R. A Deep Convolutional Neural Network Approach for Static Hand Gesture Recognition, Procedia Computer Science, 2020, vol. 171, pp. 2353–2361. https://doi.org/10.1016/j.procs.2020.04.255.Adithya V., Rajesh R. A Deep Convolutional Neural Network Approach for Static Hand Gesture Recognition, Procedia Computer Science, 2020, vol. 171, pp. 2353–2361. https://doi.org/10.1016/j.procs.2020.04.255.Hasanov Ja., Alishzade N., Nazimzade A., Dadashzade S., Tahirov T. Development of a hybrid word recognition system and dataset for the Azerbaijani Sign Language dactyl alphabet. Speech Communication, 2023, vol. 153, p. 102960. https://doi.org/10.1016/j.specom.2023.102960.Hasanov Ja., Alishzade N., Nazimzade A., Dadashzade S., Tahirov T. Development of a hybrid word recognition system and dataset for the Azerbaijani Sign Language dactyl alphabet. Speech Communication, 2023, vol. 153, p. 102960. https://doi.org/10.1016/j.specom.2023.102960.Imashev A., Mukushev M., Kimmelman V., Sandygulova A. A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL, 2023, pp. 631–640. https://doi.org/10.18653/v1/2020.conll-1.51.Imashev A., Mukushev M., Kimmelman V., Sandygulova A. A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL, 2023, pp. 631–640. https://doi.org/10.18653/v1/2020.conll-1.51.Source code of experiments in GitHub. https://github.com/MeTuA/Sign-LanguageRecognition-with-Mediapipe/tree/main.Source code of experiments in GitHub. https://github.com/MeTuA/Sign-LanguageRecognition-with-Mediapipe/tree/main.Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.Whitfield B., Feedforward Neural Networks: A Quick Primer for Deep Learning, August 2022, builtin.com, https://builtin.com/data-science/feedforward-neural-network-intro.Whitfield B., Feedforward Neural Networks: A Quick Primer for Deep Learning, August 2022, builtin.com, https://builtin.com/data-science/feedforward-neural-network-intro.Al-Hammadi M. et al. Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation, in IEEE Access, 2020, vol. 8, pp. 192527–192542. https://doi.org/10.1109/ACCESS.2020.3032140.Al-Hammadi M. et al. Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation, in IEEE Access, 2020, vol. 8, pp. 192527–192542. https://doi.org/10.1109/ACCESS.2020.3032140.Bragg D., Verhoef T., Vogler Ch., Morris M., Koller O., Bellard M., Berke L., Boudreault P., Braffort A., Caselli N., Huenerfauth M., Kacorri H. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective, 2019, pp. 16–31. https://doi.org/10.1145/3308561.3353774.Bragg D., Verhoef T., Vogler Ch., Morris M., Koller O., Bellard M., Berke L., Boudreault P., Braffort A., Caselli N., Huenerfauth M., Kacorri H. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective, 2019, pp. 16–31. https://doi.org/10.1145/3308561.3353774.Kothadiya D., Bhatt Ch., Saba T., Rehman A. SIGNFORMER: DeepVision Transformer for Sign Language Recognition. IEEE Access., 2023, pp. 1–1. https://doi.org/10.1109/ACCESS.2022.3231130.Kothadiya D., Bhatt Ch., Saba T., Rehman A. SIGNFORMER: DeepVision Transformer for Sign Language Recognition. IEEE Access., 2023, pp. 1–1. https://doi.org/10.1109/ACCESS.2022.3231130.Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit Ja., Houlsby N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. https://doi.org/10.48550/arXiv.2010.11929.Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit Ja., Houlsby N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. https://doi.org/10.48550/arXiv.2010.11929.Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.Roy P., Han J., Chouhan S., Thumu B. American Sign Language Video to Text Translation, 2024. https://doi.org/10.48550/arXiv.2402.07255.Roy P., Han J., Chouhan S., Thumu B. American Sign Language Video to Text Translation, 2024. https://doi.org/10.48550/arXiv.2402.07255.Zhengsheng G., Zhiwei H., Wenxiang J., Xing W., Rui W., Kehai C., Zhaopeng T., Yong X., Min Z. https://doi.org/10.48550/arXiv.2402.07726.Zhengsheng G., Zhiwei H., Wenxiang J., Xing W., Rui W., Kehai C., Zhaopeng T., Yong X., Min Z. https://doi.org/10.48550/arXiv.2402.07726.Rust Ph., Shi B., Wang S., Camgöz N.C., Maillard Je. Towards Privacy-Aware Sign Language Translation at Scale https://doi.org/10.48550/arXiv.2402.09611.Rust Ph., Shi B., Wang S., Camgöz N.C., Maillard Je. Towards Privacy-Aware Sign Language Translation at Scale https://doi.org/10.48550/arXiv.2402.09611.Lee C.C., Rahiman M.H.F, Rahim R.A. and Saad F.S.A. A Deep Feedforward Neural Network Model for Image Prediction. Journal of Physics: Conference Series 1878, 2021, p. 012062. https://doi.org/10.1088/1742-6596/1878/1/012062.Lee C.C., Rahiman M.H.F, Rahim R.A. and Saad F.S.A. A Deep Feedforward Neural Network Model for Image Prediction. Journal of Physics: Conference Series 1878, 2021, p. 012062. https://doi.org/10.1088/1742-6596/1878/1/012062.Bartłomiej M., Kamil M. and Paweł C. Symposium for Young Scientists in Technology, Engineering and Mathematics, https://api.semanticscholar.org/CorpusID:235693217.Bartłomiej M., Kamil M. and Paweł C. Symposium for Young Scientists in Technology, Engineering and Mathematics, https://api.semanticscholar.org/CorpusID:235693217.Teja Gontumukkala S.S., Varun Godavarthi Y. S., Ravi Teja Gonugunta B.R., Subramani R. and K. Murali. 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), https://doi: 10.1109/ICCCNT51525.2021.9579803.Teja Gontumukkala S.S., Varun Godavarthi Y. S., Ravi Teja Gonugunta B.R., Subramani R. and K. Murali. 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), https://doi: 10.1109/ICCCNT51525.2021.9579803.

The authors declare that there are no conflicts of interest present.