kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2024-21-4-81-90kaz29-1541Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEАВТОМАТИЧЕСКОЕ ОБНАРУЖЕНИЕ И РАСПОЗНАВАНИЕ ДОРОЖНЫХ ЗНАКОВ С ПОМОЩЬЮ СВЕРТОЧНЫХ НЕЙРОННЫХ СЕТЕЙAUTOMATIC DETECTION AND RECOGNITION OF ROAD SIGNS USING CONVOLUTIONAL NEURAL NETWORKShttps://orcid.org/0000-0002-8341-7113ОмаровБ. С.OmarovB. S.

PhD, доцент

г. Алматы

PhD, Associate Professor

Almaty

batyahan@gmail.comhttps://orcid.org/0000-0002-9290-6074ЗиятбековаГ. З.ZiyatbekovaG. Z.

PhD, и.о. доцента, с.н.с.

г. Алматы

PhD, Acting Associate Professor, senior researcher

Almaty

ziyatbekova1@gmail.comhttps://orcid.org/0009-0006-5384-1497БатырЖ. А.BatyrZh. A.

магистр

г. Алматы

Master

Almaty

zhan.batyr01@gmail.comhttps://orcid.org/0000-0003-0598-4806МайлыбаеваА. Д.MailybayevaA. D.

к.ф.-м.н., ассоц. профессор

г. Атырау

Candidate of Physical and Mathematical Sciences; Associate Professor

Atyrau

mjkka@mail.ruhttps://orcid.org/0000-0002-7667-5464БидахметЖ.BydakhmetZh.

PhD, ассоц, профессор

г. Алматы

PhD, Associate Professor

Almaty

zhanar.18.05@gmail.comhttps://orcid.org/0009-0001-4099-0112ШаметоваГ. К.ShametovaG. K.

докторант, ст. преподаватель

г. Алматы

Doctoral student, Senior Lecturer

Almaty

gauharshametova@gmail.comВойчикВ.WójcikW.

профессор

г. Люблин

Professor

Lublin

waldemar.wojcik@pollub.plКазахский национальный университет имени аль-ФарабиКазахстанAl-Farabi Kazakh National UniversityKazakhstanКазахский национальный университет имени аль-Фараби; Институт информационных и вычислительных технологий КН МНВО РККазахстанAl-Farabi Kazakh National University; Institute of Information and Computational Technologies SC MSHE RKKazakhstanАтырауский университет имени Х. ДосмухамедоваКазахстанKhalel Dosmukhamedov Atyrau UniversityKazakhstanАлматинский университет энергетики и связи имени Г. ДаукееваКазахстанAlmaty University of Power Engineering and Telecommunications after Gumarbek DaukeevKazakhstanКазахский национальный университет имени аль-Фараби; Алматинский университет энергетики и связи имени Г. ДаукееваКазахстанAl-Farabi Kazakh National University; Almaty University of Power Engineering and Telecommunications after Gumarbek DaukeevKazakhstanЛюблинский технический университетПольшаLublin Technical UniversityPoland2024231220242148190Copyright © Омаров Б.С., Зиятбекова Г.З., Батыр Ж.А., Майлыбаева А.Д., Бидахмет Ж., Шаметова Г.К., Войчик В., 20242024Омаров Б.С., Зиятбекова Г.З., Батыр Ж.А., Майлыбаева А.Д., Бидахмет Ж., Шаметова Г.К., Войчик В.Omarov B.S., Ziyatbekova G.Z., Batyr Z.A., Mailybayeva A.D., Bydakhmet Z., Shametova G.K., Wójcik W.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/1541

В работе рассматривается использование сверточных нейронных сетей (CNN) для улучшения систем распознавания дорожных знаков именно в непогодных условиях. Также используется расширенный набор данных немецкого теста распознавания дорожных знаков (GTSRB), основанного на новой модели CNN, который содержит более пятидесяти тысяч изображений с надписями, охватывающих более сорока категорий. В модели представлены адаптивные слои выделения объектов, предназначенные для устранения проблем с видимостью, вызванных такими погодными факторами, как дождь, туман и снег. Для моделирования различных погодных сценариев применяются передовые методы увеличения объема данных, что увеличивает разнообразие обучающего набора данных. Это исследование не только рассматривает теоретические и практические усовершенствования, предоставленные CNNs для обнаружения дорожных знаков в неблагоприятных условиях, но и проверяет эффективность модели с помощью таких показателей, как точность, отзывчивость и показатель F1. Результаты подтверждают эффективность модели в минимизации ложных срабатываний и точной идентификации дорожных знаков. В статье подчеркивается важность тщательной подготовки набора данных, оптимизации моделей и усовершенствования обучения для повышения производительности системы обнаружения. Это положительно сказывается на интеллектуальных транспортных системах, автономном вождении и безопасности дорожного движения, что свидетельствует о будущем прогрессе в области надежных технологий распознавания дорожных знаков.

This paper examines the use of convolutional neural networks (CNNs) to improve traffic sign recognition systems, precisely in non-weather conditions. An extended dataset of the German Traffic Sign Recognition Test (GTSRB), based on a new CNN model, is also used, which contains more than fifty thousand labeled images covering more than forty categories. The model presents adaptive object selection layers designed to eliminate visibility problems caused by weather factors such as rain, fog, and snow. Advanced data augmentation techniques are applied to model different weather scenarios, which increases the diversity of the training dataset. Through an analysis of theoretical and practical aspects, the study demonstrates how CNNs enhance the accuracy and efficiency of road sign detection systems in a different weather condition. This study not only examines the theoretical and practical improvements provided by CNNs for traffic sign detection in unfavorable conditions, but also verifies the effectiveness of the model through metrics such as accuracy, responsiveness, and F1 score. The results confirm the effectiveness of the model in minimizing false positives and accurately identifying traffic signs. The paper emphasizes the importance of careful dataset preparation, model optimization and improved training to enhance the performance of the detection system. This has positive implications for intelligent transportation systems, autonomous driving and road safety, indicating future progress in robust traffic sign recognition technologies.

CNNраспознавание дорожных знаковискусственный интеллектанализ изображенийклассификацияCNNtraffic sign detectionartificial intelligenceimage analysisclassificationReferencesSun Y., Ge P., Liu D. Traffic sign detection and recognition based on convolutional neural network. 2019 Chinese automation congress (CAC), IEEE, 2019, pp. 2851–2854.Sun Y., Ge P., Liu D. Traffic sign detection and recognition based on convolutional neural network. 2019 Chinese automation congress (CAC), IEEE, 2019, pp. 2851–2854.Ahmed S., Kamal U., & Hasan M.K. DFR-TSD: A deep learning based framework for robust traffic sign detection under challenging weather conditions. IEEE Transactions on Intelligent Transportation Systems, 2022, vol. 23, no. 6, pр. 5150–5162. https://doi.org/10.1109/TITS.2020.3048878/Ahmed S., Kamal U., & Hasan M.K. DFR-TSD: A deep learning based framework for robust traffic sign detection under challenging weather conditions. IEEE Transactions on Intelligent Transportation Systems, 2022, vol. 23, no. 6, pр. 5150–5162. https://doi.org/10.1109/TITS.2020.3048878/Liu Z. et al. MR-CNN: A multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access., 2019, vol. 7, pp. 57120–57128.Liu Z. et al. MR-CNN: A multi-scale region-based convolutional neural network for small traffic sign recognition. IEEE Access., 2019, vol. 7, pp. 57120–57128.Lin C. et al. Transfer learning based traffic sign recognition using inception-v3 model. Periodica Polytechnica Transportation Engineering, 2019, vol. 47, no. 3, pp. 242–250.Lin C. et al. Transfer learning based traffic sign recognition using inception-v3 model. Periodica Polytechnica Transportation Engineering, 2019, vol. 47, no. 3, pp. 242–250.Eskandarian A., Wu C., Sun C. Research advances and challenges of autonomous and connected ground vehicles. IEEE Transactions on Intelligent Transportation Systems, 2019, vol. 22, no. 2, pp. 683–711.Eskandarian A., Wu C., Sun C. Research advances and challenges of autonomous and connected ground vehicles. IEEE Transactions on Intelligent Transportation Systems, 2019, vol. 22, no. 2, pp. 683–711.Janai J. et al. Computer vision for autonomous vehicles: Problems, datasets and state of the art // Foundations and Trends® in Computer Graphics and Vision, 2020, vol. 12, no. 1–3, pp. 1–308.Janai J. et al. Computer vision for autonomous vehicles: Problems, datasets and state of the art // Foundations and Trends® in Computer Graphics and Vision, 2020, vol. 12, no. 1–3, pp. 1–308.Singh I. et al. Dropout-VGG based convolutional neural network for traffic sign categorization. Congress on Intelligent Systems: Proceedings of CIS 2021, Singapore Springer Nature Singapore, 2022, vol 1, pp. 247–261.Singh I. et al. Dropout-VGG based convolutional neural network for traffic sign categorization. Congress on Intelligent Systems: Proceedings of CIS 2021, Singapore Springer Nature Singapore, 2022, vol 1, pp. 247–261.Dang, T.P., Tran N.T., To V. H., & Tran Thi M.K. Improved YOLOv5 for real-time traffic signs recognition in bad weather conditions. The Journal of supercomputing, 2023, vol. 79, Issue. 10, pp. 10706–10724. https://link.springer.com/article/10.1007/s11227-023-05097-3.Dang, T.P., Tran N.T., To V. H., & Tran Thi M.K. Improved YOLOv5 for real-time traffic signs recognition in bad weather conditions. The Journal of supercomputing, 2023, vol. 79, Issue. 10, pp. 10706–10724. https://link.springer.com/article/10.1007/s11227-023-05097-3.Puli M. S., Sunitha M., Aluri O. S. B., Jain D.R., Rayabharapu M., & Venkatesh M. Deep Learning-Based Framework For Robust Traffic Sign Detection Under Challenging Weather Conditions. Journal of Survey in Fisheries Sciences, 2023, vol. 23, Issue. 6, pp. 5150–5162. https://doi.org/10.1109/TITS.2020.3048878.Puli M. S., Sunitha M., Aluri O. S. B., Jain D.R., Rayabharapu M., & Venkatesh M. Deep Learning-Based Framework For Robust Traffic Sign Detection Under Challenging Weather Conditions. Journal of Survey in Fisheries Sciences, 2023, vol. 23, Issue. 6, pp. 5150–5162. https://doi.org/10.1109/TITS.2020.3048878.Qian R., Yue Y., Coenen F., & Zhang B. Traffic sign recognition with convolutional neural network based on max pooling positions. In 2016 12th International conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), 2016, pp. 578–582. IEEE. https://doi.org/10.1109/FSKD.2016.7603237.Qian R., Yue Y., Coenen F., & Zhang B. Traffic sign recognition with convolutional neural network based on max pooling positions. In 2016 12th International conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), 2016, pp. 578–582. IEEE. https://doi.org/10.1109/FSKD.2016.7603237.Xu M. et al. A comprehensive survey of image augmentation techniques for deep learning. Pattern Recognition, 2023, vol. 137, p. 109347.Xu M. et al. A comprehensive survey of image augmentation techniques for deep learning. Pattern Recognition, 2023, vol. 137, p. 109347.Cong S., Zhou Y. A review of convolutional neural network architectures and their optimizations. Artificial Intelligence Review, 2023, vol. 56, no. 3, pp. 1905–1969.Cong S., Zhou Y. A review of convolutional neural network architectures and their optimizations. Artificial Intelligence Review, 2023, vol. 56, no. 3, pp. 1905–1969.Tian Y. Artificial intelligence image recognition method based on convolutional neural network algorithm. Ieee Access., 2020, vol. 8, pp. 125731–125744.Tian Y. Artificial intelligence image recognition method based on convolutional neural network algorithm. Ieee Access., 2020, vol. 8, pp. 125731–125744.Yacouby R., & Axman,D. Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification models. In Proceedings of the first workshop on evaluation and comparison of NLP systems, 2020, pp. 79–91. https://doi.org/10.18653/v1/2020.eval4nlp-1.9.Yacouby R., & Axman,D. Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification models. In Proceedings of the first workshop on evaluation and comparison of NLP systems, 2020, pp. 79–91. https://doi.org/10.18653/v1/2020.eval4nlp-1.9.Yao Z., Song X., Zhao L., & Yin Y. Real-time method for traffic sign detection and recognition based on YOLOv3-tiny with multiscale feature extraction. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2021, vol. 235, Issue. 7, pp. 1978–1991. https://doi.org/10.1177/0954407020980559.Yao Z., Song X., Zhao L., & Yin Y. Real-time method for traffic sign detection and recognition based on YOLOv3-tiny with multiscale feature extraction. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2021, vol. 235, Issue. 7, pp. 1978–1991. https://doi.org/10.1177/0954407020980559.Yucong S., & Shuqing, G. Traffic sign recognition based on HOG feature extraction. Journal of Measurements in Engineering, 2021, vol. 9, Issue. 3, pр. 142–155. https://doi.org/10.21595/jme.2021.22022.Yucong S., & Shuqing, G. Traffic sign recognition based on HOG feature extraction. Journal of Measurements in Engineering, 2021, vol. 9, Issue. 3, pр. 142–155. https://doi.org/10.21595/jme.2021.22022.Feng L., & Jia Y. Traffic sign recognition based on YOLOX in extreme weather. In 2022 Global Conference on Robotics, Artificial Intelligence and Information Technology (GCRAIT), 2022, pр. 299–303. https://doi.org/10.1109/GCRAIT55928.2022.00070.Feng L., & Jia Y. Traffic sign recognition based on YOLOX in extreme weather. In 2022 Global Conference on Robotics, Artificial Intelligence and Information Technology (GCRAIT), 2022, pр. 299–303. https://doi.org/10.1109/GCRAIT55928.2022.00070.Feng D. et al. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems, 2020, vol. 22, no. 3, pp. 1341–1360.Feng D. et al. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems, 2020, vol. 22, no. 3, pp. 1341–1360.Lin Z., Yih M., Ota J.M., Owens J., & Muyan-Ozcelik P. Benchmarking Deep Learning Frameworks and Investigating FPGA Deployment for Traffic Sign Classification and Detection. IEEE Transactions on Intelligent Vehicles, 2019, vol. 4, Issue. 3, pp. 385–395. https://doi.org/10.1109/TIV.2019.2919458.Lin Z., Yih M., Ota J.M., Owens J., & Muyan-Ozcelik P. Benchmarking Deep Learning Frameworks and Investigating FPGA Deployment for Traffic Sign Classification and Detection. IEEE Transactions on Intelligent Vehicles, 2019, vol. 4, Issue. 3, pp. 385–395. https://doi.org/10.1109/TIV.2019.2919458.Sun P., Zhang R.Y., Jiang T., Kong C., Xu W., Zhan M., Tomizuka L., Li Z., Yuan C., Wang & Luo, P. Sparse R-CNN: End-to-end object detection with learnable proposals, 2020. arXiv:2011.12450 https://doi.org/10.48550/arXiv.2011.12450.Sun P., Zhang R.Y., Jiang T., Kong C., Xu W., Zhan M., Tomizuka L., Li Z., Yuan C., Wang & Luo, P. Sparse R-CNN: End-to-end object detection with learnable proposals, 2020. arXiv:2011.12450 https://doi.org/10.48550/arXiv.2011.12450.

The authors declare that there are no conflicts of interest present.