The detection of emotional states from human facial expressions is a crucial research area in artificial intelligence and computer vision. This paper explores real-time emotion recognition using the EfficientNet-B0 model on the FaceEmocDS dataset. The FaceEmocDS dataset comprises 72,412 high-quality images, encompassing eight emotion classes: anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. Derived from multiple open-source datasets, it ensures diversity and balance after rigorous cleaning to remove low-quality or redundant samples. The model was trained via transfer learning, leveraging pre-trained weights from ImageNet, and fine-tuned over 30 epochs on the PyTorch framework. It achieved 74.32% accuracy on the validation set. To enhance generalization, data augmentation techniques were applied, including random rotations (up to 10 degrees), color jittering (adjusting brightness, contrast, and saturation), random resized cropping, horizontal flipping, and random erasing (with probability 0.7). Class weights were incorporated into the CrossEntropyLoss function to address imbalances, particularly in underrepresented classes like contempt and disgust. For real-time implementation, MediaPipe Face Detection was integrated for efficient face localization, followed by emotion classification with the EfficientNet-B0 model. The system operates at 20-30 frames per second (FPS) on standard hardware, making it suitable for dynamic environments. Test results yielded an accuracy of 73.18% and a weighted F1-score of 0.7327, with highest performance on happiness (93.08%) and lowest on contempt (63.05%). Confusion matrix analysis revealed common misclassifications, such as contempt with anger or disgust, highlighting areas for improvement. Compared to baseline models such as VGG and ResNet, this approach provides superior efficiency in terms of parameter count and inference speed while maintaining competitive accuracy.
[1]. H. Ursula and T. Pasca, "Darwin and Emotion Expression.," The American psychologist, vol. 120, no. 8, p. 64, 2009.
[2]. P. Ekman and W. V. Friesen., "Facial action coding systems," Consulting Psychologists Press, 1978.
[3]. P. Ekman, "Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique.," Psychological Bulletin,, pp. 268-287, 1994.
[4]. E. a. M. National Academies of Sciences, Facial Recognition Technology: Current Capabilities, Future Prospects, and Governance, Washington, DC: The National Academies Press, 2024.
[5]. Y. Wang, W. Song, W. Tao, A. Liotta, D. Yang, X. Li and W. … Zhang, "A systematic review on affective computing: emotion models, databases, and recent advances.," Information Fusion, vol. 83, no. 84, pp. 19-52, 2022.
[6]. A. Kurbanov, "Facial Emotion Recognition by CNN Combined Ensemble Model," EAI Endorsed Trans AI Robotics, vol. 4, 2025.
[7]. A. Poongodai, V. Bhavitha, D. Hemanth, G. Jahnavi and S. Huzma, "Human Emotion Detection Through Real-Time Facial Expressions Using Deep Learning," in 6th International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2025.
[8]. A. Akbar, S. Saifullah, H. Prapcoyo, H. Rustamadji and N. Cahyana, "EfficientNet B0 Feature Extraction with L2-SVM Classification for Robust Facial Expression Recognition.," Journal of Information Systems and Informatics, pp. 1106-1129, 2025.
[9]. A. Farkhod, A. Abdusalomov, M. Mukhiddinov and Y.-I. Cho, "Development of Real-Time Landmark-Based Emotion Recognition CNN for Masked Faces.," Sensors , vol. 22, p. 8704, 2022.
[10]. A. Rehman, M. Mujahid, A. Elyassih, B. AlGhofaily and S. Bahaj, "Comprehensive Review and Analysis on Facial Emotion Recognition: Performance Insights into Deep and Traditional Learning with Current Updates and Challenges.," Computers, Materials & Continua, vol. 82, pp. 41-72, 2024.
[11]. Y. Wang, S. Yan, Y. Liu, W. Song, J. Liu, Y. Chang, X. Mai, W. Z. Xiping Hu and Z. Gan, "A Survey on Facial Expression Recognition of Static and Dynamic Emotions," arXiv, 28 08 2024.
[Online]. Available: https://arxiv.org/html/2408.15777v1.
[Accessed 2024].
[12]. R. Agarwal, N. Mittal and H. Madasu, "Convolutional Neural Network Based Facial Expression Recognition Using Image Filtering Techniques," International Journal of Intelligent Engineering and Systems, 2021.
[13]. S. Aggarwal, M. K. Sharma, H. O. K. Jha, A. P. Singh, R. Sharma and G. Mitra, "Comparing pre-trained algorithms for facial emotion recognition: An analysis," International Journal of Engineering in Computer Science, vol. 5, no. 1, pp. 54-60, 2023.
[14]. Y. Liu, "The Study of Performance Related to Classical Convolutional Neural Networks in the Field of Facial Emotion Recognition," Applied and Computational Engineering, vol. 8, pp. 470-474, 2023.
[15]. A. Akhand M., R. S, Siddique N, A. S. Kamal M. and Shimamura T., "Facial Emotion Recognition Using Transfer Learning in the Deep CNN," Electronics, vol. 10, no. 9, p. 1036, 2021.
[16]. P. Utami, R. Hartanto and I. Soesanti, "The EfficientNet Performance for Facial Expressions Recognition," in 5th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Yogyakarta, Indonesia, 2022.
[17]. T. Kusunose, X. Kang, K. Kiuchi, R. Nishimura, M. Sasayama and K. Matsumoto, "Facial Expression Emotion Recognition Based on Transfer Learning and Generative Model," in 8th International Conference on Systems and Informatics (ICSAI), Kunming, China, 2022.
[18]. E. Agung, A. Rifai and T. Wijayanto, "Image-based facial emotion recognition using convolutional neural network on emognition dataset.," Scientific Reports, vol. 14, p. 14429, 2024.
[19]. U. Rawat and C. S. Rai, "mproving Facial Emotion Recognition Through Transfer Learning with Deep Convolutional Neural Network (DCNN) Models,," in 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS), Tashkent, Uzbekistan, 2023.
[20]. W. A. Khan, A. U. Rahman, A. Zafar, A. Ashfaq and F. Karamat, "Facial Emotion Recognition: A Comparison of Classic and Novel Convolutional Neural Networks through Transfer Learning," in 1st International Conference on Innovative Engineering Sciences and Technological Research (ICIESTR), Muscat, Oman, 2024.
[21]. K. Desai, H. N. Patel, D. Dashlaniya, P. Desai, M. Shah and P. Goel, "A Transfer Learning Framework for Facial Emotion Recognition: Leveraging Pre-Trained Convolutional Neural Networks," in 2nd International Conference on Advancement in Computation & Computer Technologies (InCACCT), , Gharuan, India,, 2024.
[22]. A. A. Istiqomah, C. A. Sari, A. Susanto and E. H. Rachmawanto, "Facial Expression Recognition using Convolutional Neural Networks with Transfer Learning Resnet-50," Journal of Applied Informatics and Computing, vol. 8, no. 2, p. 257–264., 2024.