ROLE OF ARTIFICIAL INTELLIGENCE IN SIGN LANGUAGE RECOGNITION

Over the decades the increasing computational capability and development of new technologies in the field of artificial intelligence have given us the ability to translate sign language in real time. There exist two main approaches to sign language recognition, the hardware-based approach and the software-based approach. The hardware-based approach relies on using special gloves, Kinect-based devices, and different levels of sensors. On the other hand, one of the approaches to working with sign language is using neural networks, which is the softwarebased approach. In this work, I observed existing approaches and experimented with machine learning and neural network models for sign language recognition. I got the dataset of Azerbaijani Sign Language, then trained my models based on that dataset, and got the results and metrics. The dataset contained over thirteen thousand samples of signs, which can be used in Kazakh Sign Language. In the end, I discussed the probable opportunity of using the developed models.

1. Dubey S., Suryawanshi S., Rachamalla A., and K. Madhu Babu. Sign language recognition. International Journal for Research in Applied Science & Engineering Technology (IJRASET) https://doi.org/10.22214/ijraset.2023.48586

2. Farooq U., Rahim M.S.M., Sabir,N. et al. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Comput & Applic, 2021, vol. 33, pp. 14357–14399. https://doi.org/10.1007/s00521-021-06079-3

3. Amrutha K. and P. Prabu. ML Based Sign Language Recognition System, 2021 International Conference on Innovative Trends in Information Technology (ICITIIT), Kottayam, India, 2021, pp. 1–6, https://doi.org/10.1109/ICITIIT51526.2021.9399594.

4. Dr. Sabeenian R.S., Bharathwaj S., Aadhil M. Sign Language Recognition Using Deep Learning and Computer Vision. Journal of Advanced Research in Dynamical and Control Systems, 202, vol. 12, pp. 964–968. https://doi.org/10.5373/JARDCS/V12SP5/20201842.

5. Adithya V., Rajesh R. A Deep Convolutional Neural Network Approach for Static Hand Gesture Recognition, Procedia Computer Science, 2020, vol. 171, pp. 2353–2361. https://doi.org/10.1016/j.procs.2020.04.255.

6. Hasanov Ja., Alishzade N., Nazimzade A., Dadashzade S., Tahirov T. Development of a hybrid word recognition system and dataset for the Azerbaijani Sign Language dactyl alphabet. Speech Communication, 2023, vol. 153, p. 102960. https://doi.org/10.1016/j.specom.2023.102960.

7. Imashev A., Mukushev M., Kimmelman V., Sandygulova A. A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL, 2023, pp. 631–640. https://doi.org/10.18653/v1/2020.conll-1.51.

8. Source code of experiments in GitHub. https://github.com/MeTuA/Sign-LanguageRecognition-with-Mediapipe/tree/main.

9. Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.

10. Whitfield B., Feedforward Neural Networks: A Quick Primer for Deep Learning, August 2022, builtin.com, https://builtin.com/data-science/feedforward-neural-network-intro.

11. Al-Hammadi M. et al. Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation, in IEEE Access, 2020, vol. 8, pp. 192527–192542. https://doi.org/10.1109/ACCESS.2020.3032140.

12. Bragg D., Verhoef T., Vogler Ch., Morris M., Koller O., Bellard M., Berke L., Boudreault P., Braffort A., Caselli N., Huenerfauth M., Kacorri H. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective, 2019, pp. 16–31. https://doi.org/10.1145/3308561.3353774.

13. Kothadiya D., Bhatt Ch., Saba T., Rehman A. SIGNFORMER: DeepVision Transformer for Sign Language Recognition. IEEE Access., 2023, pp. 1–1. https://doi.org/10.1109/ACCESS.2022.3231130.

14. Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit Ja., Houlsby N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. https://doi.org/10.48550/arXiv.2010.11929.

15. Tabsharani F., techtarget.com, August 2023, Accessed 23 February 2024, https://www.techtarget.com/whatis/definition/support-vector-machine-SVM.

16. Roy P., Han J., Chouhan S., Thumu B. American Sign Language Video to Text Translation, 2024. https://doi.org/10.48550/arXiv.2402.07255.

17. Zhengsheng G., Zhiwei H., Wenxiang J., Xing W., Rui W., Kehai C., Zhaopeng T., Yong X., Min Z. https://doi.org/10.48550/arXiv.2402.07726.

18. Rust Ph., Shi B., Wang S., Camgöz N.C., Maillard Je. Towards Privacy-Aware Sign Language Translation at Scale https://doi.org/10.48550/arXiv.2402.09611.

19. Lee C.C., Rahiman M.H.F, Rahim R.A. and Saad F.S.A. A Deep Feedforward Neural Network Model for Image Prediction. Journal of Physics: Conference Series 1878, 2021, p. 012062. https://doi.org/10.1088/1742-6596/1878/1/012062.

20. Bartłomiej M., Kamil M. and Paweł C. Symposium for Young Scientists in Technology, Engineering and Mathematics, https://api.semanticscholar.org/CorpusID:235693217.

21. Teja Gontumukkala S.S., Varun Godavarthi Y. S., Ravi Teja Gonugunta B.R., Subramani R. and K. Murali. 12th International Conference on Computing Communication and Networking Technologies (ICCCNT), https://doi: 10.1109/ICCCNT51525.2021.9579803.