kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2025-22-2-110-126kaz29-1992Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEКЛАССИФИКАЦИЯ ЭМОЦИЙ С ИСПОЛЬЗОВАНИЕМ СВЕРТОЧНЫХ НЕЙРОННЫХ СЕТЕЙ С РАЗЛИЧНЫМИ АРХИТЕКТУРАМИEMOTION CLASSIFICATION USING CONVOLUTIONAL NEURAL NETWORKS WITH DIFFERENT ARCHITECTUREShttps://orcid.org/0009-0006-2267-0365ЕршовЭ.YershovE.

 студент 

 г. Алматы 

 Bachelor’s student 

 Almaty 

yershov_evan@kaznu.edu.kzhttps://orcid.org/0009-0001-9124-2560ОрынбасарС.OrynbassarS.

 PhD студент 

 г. Алматы 

 PhD student 

 Almaty 

sayat.orynbassar@kaznu.edu.kzhttps://orcid.org/0000-0001-8206-7425ЖоламановБ.ZholamanovB.

 PhD студент 

 г. Алматы 

 PhD student 

 Almaty 

zholamanov.batyrbek@kaznu.kzhttps://orcid.org/0000-0002-6795-5384НұрғалиевМ.NurgaliyevM.

 PhD 

 г. Алматы 

 PhD 

 Almaty 

madiyar.nurgaliyev@kaznu.edu.kzhttps://orcid.org/0000-0002-3935-7213ДосымбетоваГ.DosymbetovaG.

 PhD 

 г. Алматы 

 PhD 

 Almaty 

gulbakhar.dossymbetova@kaznu.edu.kzhttps://orcid.org/0009-0005-4945-6273ХумарбекқызыТ.KhumarbekkyzyT.

 магистрант 

 г. Алматы 

 Master’s student 

 Almaty 

khumarbekkyzy_t@kaznu.edu.kzКазахский национальный университет им. аль-ФарабиКазахстанAl-Farabi Kazakh National UniversityKazakhstan202506072025222110126Copyright © Ершов Э., Орынбасар С., Жоламанов Б., Нұрғалиев М., Досымбетова Г., Хумарбекқызы Т., 20252025Ершов Э., Орынбасар С., Жоламанов Б., Нұрғалиев М., Досымбетова Г., Хумарбекқызы Т.Yershov E., Orynbassar S., Zholamanov B., Nurgaliyev M., Dosymbetova G., Khumarbekkyzy T.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/1992

Тепловизионная съемка представляет собой неинвазивный и надежный подход к распознаванию эмоций, фиксируя температурные изменения на лице, связанные с психофизиологическим состоянием человека. В настоящем исследовании рассматривается применение глубоких нейронных сетей для классификации шести базовых эмоций – радость, грусть, страх, отвращение, гнев и удивление – по термограммам лица. Сбалансированный набор данных был собран в контролируемых экспериментальных условиях, и были оценены четыре архитектуры глубокого обучения: сверточная нейронная сеть (CNN), полностью сверточная сеть (FCN), EfficientNet и MobileNet. Модели обучались и тестировались на предварительно обработанных термографических изображениях лица. Среди исследуемых архитектур наивысшую точность – 90.04% – показала FCN. Результаты демонстрируют, что модели глубокого обучения, особенно FCN, хорошо подходят для задач распознавания эмоций по тепловизионным данным и могут быть использованы в психофизиологическом мониторинге, здравоохранении и системах взаимодействия человек – машина в реальном времени.

Thermal imaging offers a non-invasive and robust approach to emotion recognition by capturing facial temperature patterns that correlate with psychophysiological states. This study investigates the application of deep neural networks to classify six basic human emotions – happiness, sadness, fear, disgust, anger, and surprise – using facial thermograms. A balanced dataset was collected under controlled experimental conditions, and four deep learning architectures were evaluated: Convolutional Neural Network (CNN), Fully Convolutional Network (FCN), EfficientNet, and MobileNet. The models were trained and tested on a curated set of preprocessed thermal facial images. Among the evaluated architectures, FCN achieved the highest classification accuracy of 90.04%. The results demonstrate that deep learning models, particularly FCNs, are well-suited for emotion recognition from thermal data, with potential applications in psychophysiological monitoring, healthcare, and real-time humancomputer interaction systems.

CNNEfficient NetMobile NetFully Convolution Networkтермограмманейронные сетиCNNEfficient NetMobile NetFully Convolution Networkthermogramsneural networksReferencesSzeliski R. Computer vision: algorithms and applications. – Springer Nature. – 2022. https://doi.org/10.1007/978-3-030-34372-91 Szeliski R. Computer vision: algorithms and applications. – Springer Nature. – 2022. https://doi.org/10.1007/978-3-030-34372-9Huang Zi-Yu, et al. A study on computer vision for facial emotion recognition // Scientific reports. – 2023. – Vol. 13. – No. 1. – P. 8425. https://doi.org/10.1038/s41598-023-35446-4Huang Zi-Yu, et al. A study on computer vision for facial emotion recognition // Scientific reports. – 2023. – Vol. 13. – No. 1. – P. 8425. https://doi.org/10.1038/s41598-023-35446-4Issa D., Demirci MF, Yazici A. Speech emotion recognition with deep convolutional neural networks // Biomedical Signal Processing and Control. – 2020. – Vol. 59. – P. 101894. https://doi.org/10.1016/j.bspc.2020.101894Issa D., Demirci MF, Yazici A. Speech emotion recognition with deep convolutional neural networks // Biomedical Signal Processing and Control. – 2020. – Vol. 59. – P. 101894. https://doi.org/10.1016/j.bspc.2020.101894Gautam, Chahak, and K.R. Seeja. Facial emotion recognition using Handcrafted features and CNN // Procedia Computer Science. – 2023. – Vol. 218. – P. 1295–1303. https://doi.org/10.1016/j.procs.2023.01.108Gautam, Chahak, and K.R. Seeja. Facial emotion recognition using Handcrafted features and CNN // Procedia Computer Science. – 2023. – Vol. 218. – P. 1295–1303. https://doi.org/10.1016/j.procs.2023.01.108Prasad, Babu Rajendra, and B. Sai Chandana. Human face emotions recognition from thermal images using DenseNet // International journal of electrical and computer engineering systems. – 2023. – Vol. 14. – P. 155–167. https://doi.org/10.32985/ijeces.14.2.5Prasad, Babu Rajendra, and B. Sai Chandana. Human face emotions recognition from thermal images using DenseNet // International journal of electrical and computer engineering systems. – 2023. – Vol. 14. – P. 155–167. https://doi.org/10.32985/ijeces.14.2.5Assiri, Basem, and Mohammad Alamgir Hossain. Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism // Mathematical Biosciences and Engineering. – 2023. – Vol. 20. – P. 913–929. https://doi.org/10.3934/mbe.2023042Assiri, Basem, and Mohammad Alamgir Hossain. Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism // Mathematical Biosciences and Engineering. – 2023. – Vol. 20. – P. 913–929. https://doi.org/10.3934/mbe.2023042Zuo W. et al. Gradient histogram estimation and preservation for texture enhanced image denoising // IEEE transactions on image processing. – 2014. – Vol. 23. – No. 6. – P. 2459–2472. https://doi.org/10.1109/TIP.2014.2316423Zuo W. et al. Gradient histogram estimation and preservation for texture enhanced image denoising // IEEE transactions on image processing. – 2014. – Vol. 23. – No. 6. – P. 2459–2472. https://doi.org/10.1109/TIP.2014.2316423Bouchene, Mohammed Mehdi. Bayesian optimization of histogram of oriented gradients (HOG) parameters for facial recognition // The Journal of Supercomputing. – 2024. – Vol. 80. – No. 14. – P. 20118–20149. https://doi.org/10.1007/s11227-024-06259-7Bouchene, Mohammed Mehdi. Bayesian optimization of histogram of oriented gradients (HOG) parameters for facial recognition // The Journal of Supercomputing. – 2024. – Vol. 80. – No. 14. – P. 20118–20149. https://doi.org/10.1007/s11227-024-06259-7Ma J. Based on the fourier transform and the wavelet transformation of the digital image processing // 2012 International Conference on Computer Science and Information Processing (CSIP). – IEEE, 2012. – P. 1232–1234. https://doi.org/10.1109/CSIP.2012.6309081Ma J. Based on the fourier transform and the wavelet transformation of the digital image processing // 2012 International Conference on Computer Science and Information Processing (CSIP). – IEEE, 2012. – P. 1232–1234. https://doi.org/10.1109/CSIP.2012.6309081Lindeberg T. Scale invariant feature transform. – 2012. https://doi.org/10.4249/scholarpedia.10491Lindeberg T. Scale invariant feature transform. – 2012. https://doi.org/10.4249/scholarpedia.10491Thai LH, Hai TS, Thuy NT Image classification using support vector machine and artificial neural network // International Journal of Information Technology and Computer Science. – 2012. – Vol. 4. – No. 5. – P. 32–38. https://doi.org/10.5815/ijitcs.2012.05.05Thai LH, Hai TS, Thuy NT Image classification using support vector machine and artificial neural network // International Journal of Information Technology and Computer Science. – 2012. – Vol. 4. – No. 5. – P. 32–38. https://doi.org/10.5815/ijitcs.2012.05.05Ma L., Crawford MM, Tian J. Local manifold learning-based $k$-nearest-neighbor for hyperspectral image classification // IEEE Transactions on Geoscience and Remote Sensing. – 2010. – Vol. 48. – No. 11. – P. 4099–4109. https://doi.org/10.1109/TGRS.2010.2055876Ma L., Crawford MM, Tian J. Local manifold learning-based $k$-nearest-neighbor for hyperspectral image classification // IEEE Transactions on Geoscience and Remote Sensing. – 2010. – Vol. 48. – No. 11. – P. 4099–4109. https://doi.org/10.1109/TGRS.2010.2055876Dong Q., Zhu X., Gong S. Single-label multi-class image classification by deep logistic regression // Proceedings of the AAAI conference on artificial intelligence. – 2019. – Vol. 33. – No. 01. – P. 3486–3493. https://doi.org/10.1609/aaai.v33i01.33013486Dong Q., Zhu X., Gong S. Single-label multi-class image classification by deep logistic regression // Proceedings of the AAAI conference on artificial intelligence. – 2019. – Vol. 33. – No. 01. – P. 3486–3493. https://doi.org/10.1609/aaai.v33i01.33013486Timofte R., Tuytelaars T., Van Gool L. Naive bayes image classification: beyond nearest neighbors // Asian Conference on Computer Vision. – Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. – P. 689–703. https://doi.org/10.1007/978-3-642-37331-2_52Timofte R., Tuytelaars T., Van Gool L. Naive bayes image classification: beyond nearest neighbors // Asian Conference on Computer Vision. – Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. – P. 689–703. https://doi.org/10.1007/978-3-642-37331-2_52Wang H. et al. Image recognition of plant diseases based on principal component analysis and neural networks // 2012 8th International Conference on Natural Computation. – IEEE, 2012. – P. 246–251. https://doi.org/10.1109/ICNC.2012.6234701Wang H. et al. Image recognition of plant diseases based on principal component analysis and neural networks // 2012 8th International Conference on Natural Computation. – IEEE, 2012. – P. 246–251. https://doi.org/10.1109/ICNC.2012.6234701Tharwat A. et al. Linear discriminant analysis: A detailed tutorial // AI communications. – 2017. – Vol. 30. – No. 2. – P. 169–190. https://doi.org/10.3233/AIC-170729Tharwat A. et al. Linear discriminant analysis: A detailed tutorial // AI communications. – 2017. – Vol. 30. – No. 2. – P. 169–190. https://doi.org/10.3233/AIC-170729Zhao, Shuping, et al. Linear discriminant analysis. Nature Reviews Methods Primers. – 2024. – Vol. 4. – No. 1. – P. 70. https://doi.org/10.1038/s43586-024-00346-yZhao, Shuping, et al. Linear discriminant analysis. Nature Reviews Methods Primers. – 2024. – Vol. 4. – No. 1. – P. 70. https://doi.org/10.1038/s43586-024-00346-yXu B., Ye Y., Nie L. An improved random forest classifier for image classification //2012 IEEE international conference on information and automation. – IEEE, 2012. – P. 795–800. https://doi.org/10.1109/ICInfA.2012.6246927Xu B., Ye Y., Nie L. An improved random forest classifier for image classification //2012 IEEE international conference on information and automation. – IEEE, 2012. – P. 795–800. https://doi.org/10.1109/ICInfA.2012.6246927Badrulhisham NAS, Mangshor NNA Emotion recognition using convolutional neural network (CNN) // Journal of Physics: Conference Series. – IOP Publishing, 2021. – Vol. 1962. – No. 1. – P. 012040. http://dx.doi.org/10.1088/1742-6596/1962/1/012040Badrulhisham NAS, Mangshor NNA Emotion recognition using convolutional neural network (CNN) // Journal of Physics: Conference Series. – IOP Publishing, 2021. – Vol. 1962. – No. 1. – P. 012040. http://dx.doi.org/10.1088/1742-6596/1962/1/012040Maggiori E. et al. Fully convolutional neural networks for remote sensing image classification // 2016 IEEE international geoscience and remote sensing symposium (IGARSS). – IEEE, 2016. – P. 5071–5074. https://doi.org/10.1109/IGARSS.2016.7730322Maggiori E. et al. Fully convolutional neural networks for remote sensing image classification // 2016 IEEE international geoscience and remote sensing symposium (IGARSS). – IEEE, 2016. – P. 5071–5074. https://doi.org/10.1109/IGARSS.2016.7730322Espejo-Garcia B. et al. Using EfficientNet and transfer learning for image-based diagnosis of nutrient deficiencies //Computers and Electronics in Agriculture. – 2022. – Vol. 196. – P. 106868. https://doi.org/10.1016/j.compag.2022.106868Espejo-Garcia B. et al. Using EfficientNet and transfer learning for image-based diagnosis of nutrient deficiencies //Computers and Electronics in Agriculture. – 2022. – Vol. 196. – P. 106868. https://doi.org/10.1016/j.compag.2022.106868Wang W. et al. A novel image classification approach via dense- MobileNet models //Mobile Information Systems. – 2020. – Vol. 2020. – No. 1. – P. 7602384. https://doi.org/10.1155/2020/7602384Wang W. et al. A novel image classification approach via dense- MobileNet models //Mobile Information Systems. – 2020. – Vol. 2020. – No. 1. – P. 7602384. https://doi.org/10.1155/2020/7602384

The authors declare that there are no conflicts of interest present.