EMOTION CLASSIFICATION USING CONVOLUTIONAL NEURAL NETWORKS WITH DIFFERENT ARCHITECTURES

Thermal imaging offers a non-invasive and robust approach to emotion recognition by capturing facial temperature patterns that correlate with psychophysiological states. This study investigates the application of deep neural networks to classify six basic human emotions – happiness, sadness, fear, disgust, anger, and surprise – using facial thermograms. A balanced dataset was collected under controlled experimental conditions, and four deep learning architectures were evaluated: Convolutional Neural Network (CNN), Fully Convolutional Network (FCN), EfficientNet, and MobileNet. The models were trained and tested on a curated set of preprocessed thermal facial images. Among the evaluated architectures, FCN achieved the highest classification accuracy of 90.04%. The results demonstrate that deep learning models, particularly FCNs, are well-suited for emotion recognition from thermal data, with potential applications in psychophysiological monitoring, healthcare, and real-time humancomputer interaction systems.

1. 1 Szeliski R. Computer vision: algorithms and applications. – Springer Nature. – 2022. https://doi.org/10.1007/978-3-030-34372-9

2. Huang Zi-Yu, et al. A study on computer vision for facial emotion recognition // Scientific reports. – 2023. – Vol. 13. – No. 1. – P. 8425. https://doi.org/10.1038/s41598-023-35446-4

3. Issa D., Demirci MF, Yazici A. Speech emotion recognition with deep convolutional neural networks // Biomedical Signal Processing and Control. – 2020. – Vol. 59. – P. 101894. https://doi.org/10.1016/j.bspc.2020.101894

4. Gautam, Chahak, and K.R. Seeja. Facial emotion recognition using Handcrafted features and CNN // Procedia Computer Science. – 2023. – Vol. 218. – P. 1295–1303. https://doi.org/10.1016/j.procs.2023.01.108

5. Prasad, Babu Rajendra, and B. Sai Chandana. Human face emotions recognition from thermal images using DenseNet // International journal of electrical and computer engineering systems. – 2023. – Vol. 14. – P. 155–167. https://doi.org/10.32985/ijeces.14.2.5

6. Assiri, Basem, and Mohammad Alamgir Hossain. Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism // Mathematical Biosciences and Engineering. – 2023. – Vol. 20. – P. 913–929. https://doi.org/10.3934/mbe.2023042

7. Zuo W. et al. Gradient histogram estimation and preservation for texture enhanced image denoising // IEEE transactions on image processing. – 2014. – Vol. 23. – No. 6. – P. 2459–2472. https://doi.org/10.1109/TIP.2014.2316423

8. Bouchene, Mohammed Mehdi. Bayesian optimization of histogram of oriented gradients (HOG) parameters for facial recognition // The Journal of Supercomputing. – 2024. – Vol. 80. – No. 14. – P. 20118–20149. https://doi.org/10.1007/s11227-024-06259-7

9. Ma J. Based on the fourier transform and the wavelet transformation of the digital image processing // 2012 International Conference on Computer Science and Information Processing (CSIP). – IEEE, 2012. – P. 1232–1234. https://doi.org/10.1109/CSIP.2012.6309081

10. Lindeberg T. Scale invariant feature transform. – 2012. https://doi.org/10.4249/scholarpedia.10491

11. Thai LH, Hai TS, Thuy NT Image classification using support vector machine and artificial neural network // International Journal of Information Technology and Computer Science. – 2012. – Vol. 4. – No. 5. – P. 32–38. https://doi.org/10.5815/ijitcs.2012.05.05

12. Ma L., Crawford MM, Tian J. Local manifold learning-based $k$-nearest-neighbor for hyperspectral image classification // IEEE Transactions on Geoscience and Remote Sensing. – 2010. – Vol. 48. – No. 11. – P. 4099–4109. https://doi.org/10.1109/TGRS.2010.2055876

13. Dong Q., Zhu X., Gong S. Single-label multi-class image classification by deep logistic regression // Proceedings of the AAAI conference on artificial intelligence. – 2019. – Vol. 33. – No. 01. – P. 3486–3493. https://doi.org/10.1609/aaai.v33i01.33013486

14. Timofte R., Tuytelaars T., Van Gool L. Naive bayes image classification: beyond nearest neighbors // Asian Conference on Computer Vision. – Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. – P. 689–703. https://doi.org/10.1007/978-3-642-37331-2_52

15. Wang H. et al. Image recognition of plant diseases based on principal component analysis and neural networks // 2012 8th International Conference on Natural Computation. – IEEE, 2012. – P. 246–251. https://doi.org/10.1109/ICNC.2012.6234701

16. Tharwat A. et al. Linear discriminant analysis: A detailed tutorial // AI communications. – 2017. – Vol. 30. – No. 2. – P. 169–190. https://doi.org/10.3233/AIC-170729

17. Zhao, Shuping, et al. Linear discriminant analysis. Nature Reviews Methods Primers. – 2024. – Vol. 4. – No. 1. – P. 70. https://doi.org/10.1038/s43586-024-00346-y

18. Xu B., Ye Y., Nie L. An improved random forest classifier for image classification //2012 IEEE international conference on information and automation. – IEEE, 2012. – P. 795–800. https://doi.org/10.1109/ICInfA.2012.6246927

19. Badrulhisham NAS, Mangshor NNA Emotion recognition using convolutional neural network (CNN) // Journal of Physics: Conference Series. – IOP Publishing, 2021. – Vol. 1962. – No. 1. – P. 012040. http://dx.doi.org/10.1088/1742-6596/1962/1/012040

20. Maggiori E. et al. Fully convolutional neural networks for remote sensing image classification // 2016 IEEE international geoscience and remote sensing symposium (IGARSS). – IEEE, 2016. – P. 5071–5074. https://doi.org/10.1109/IGARSS.2016.7730322

21. Espejo-Garcia B. et al. Using EfficientNet and transfer learning for image-based diagnosis of nutrient deficiencies //Computers and Electronics in Agriculture. – 2022. – Vol. 196. – P. 106868. https://doi.org/10.1016/j.compag.2022.106868

22. Wang W. et al. A novel image classification approach via dense- MobileNet models //Mobile Information Systems. – 2020. – Vol. 2020. – No. 1. – P. 7602384. https://doi.org/10.1155/2020/7602384