i-manager Publications

Transfer Learning Based Effective Emotional Face Recognition using DCNN via Cropping Techniques

Anjani Suputri Devi D.*, Suneetha Eluri**

*-** Department of Computer Science and Engineering, Jawaharlal Nehru Technological University, Kakinada, Andhra Pradesh, India.

Periodicity:June - August'2022
DOI : https://doi.org/10.26634/jcom.10.2.19059

Abstract

Facial Expression Recognition (FER) has grown in popularity as a result of the recent advancement and use of humancomputer interface technologies. Because the images can vary in brightness, backdrop, position, etc. it is challenging for current machine learning and deep learning models to identify facial expression. If the database is small, it doesn't operate well. Feature extraction is crucial for FER, and if the derived characteristics can be separated, even a straightforward approach can help tremendously. Deep learning techniques and automated feature extraction, allow some irrelevant features to conflict with important features. In this paper, we deal with limited data and simply extract useful features from images. To make data more numerous and allow for the extraction of just important facial features, we suggest innovative face cropping, rotation, and simplification procedures and advocate using the Transfer Learning technique to construct DCNN for building a very accurate FER system. By replacing the dense top layer(s) with FER, a pretrained DCNN model is adopted, and the model is then modified with facial expression data. The training of the dense layer(s) is followed by adjusting each of the pre-trained DCNN blocks in turn. This new pipeline technique has gradually increased the accuracy of FER to a higher degree. On the CK+ and JAFFE datasets, experiments were run to assess the suggested methodology. For 7-class studies on the CK+ and JAFFE databases, high average accuracy in recognition of 99.49% and 98.58% were acquired.

Keywords

Convolutional Neural Network (CNN), Deep CNN (DCNN), Transfer Learning, CK+, JAFFE.

How to Cite this Article?

Devi, D. A. S., and Eluri, S. (2022). Transfer Learning Based Effective Emotional Face Recognition using DCNN via Cropping Techniques. i-manager’s Journal on Computer Science, 10(2), 8-19. https://doi.org/10.26634/jcom.10.2.19059

References

[1]. Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., ...& Asari, V. K. (2019). A state-ofthe- art survey on deep learning theory and architectures. Electronics, 8(3), 292. https://doi.org/10.3390/electronics8030292

[2]. Autism and Developmental Disabilities Monitoring Network Surveillance Year 2010 Principal Investigators. (2014). Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morbidity and Mortality Weekly Report: Surveillance Summaries, 63(2), 1-21.

[3]. Boutorh, A., & Guessoum, A. (2016). Complex diseases SNP selection and classification by hybrid association rule mining and artificial neural networkbased evolutionary algorithms. Engineering Applications of Artificial Intelligence, 51, 58-70. https://doi.org/10.1016/j.engappai.2016.01.004

[4]. Chang, C. Y., & Huang, Y. C. (2010, July). Personalized facial expression recognition in indoor environments. In the 2010, International Joint Conference on Neural Networks (IJCNN), (pp. 1-8). IEEE. https://doi.org/10.1109/IJCNN.2010.5596316

[5]. Feng, X., Pietikainen, M., & Hadid, A. (2007). Facial expression recognition based on local binary patterns. Pattern Recognition and Image Analysis. 17, 592–598. https://doi.org/10.1134/S1054661807040190

[6]. Fernández-Caballero, A., Martínez-Rodrigo, A., Pastor, J. M., Castillo, J. C., Lozano-Monasor, E., López, M. T., ...& Fernández-Sotos, A. (2016). Smart environment architecture for emotion detection and regulation. Journal of Biomedical Informatics, 64, 55-73. https://doi.org/10.1016/j.jbi.2016.09.015

[7]. Gogić, I., Manhart, M., Pandžić, I. S., &Ahlberg, J. (2020). Fast facial expression recognition using local binary features and shallow neural networks. The Visual Computer, 36(1), 97-112. https://doi.org/10.1007/s00371-018-1585-8

[8]. Goh, K. M., Ng, C. H., Lim, L. L., & Sheikh, U. U. (2020). Micro-expression recognition: an updated review of current trends, challenges and solutions. The Visual Computer, 36(3), 445-468. https://doi.org/10.1007/s00371-018-1607-6

[9]. Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020). A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 53(8), 5455-5516. https://doi.org/10.1007/s10462-020-09825-6

[10]. Ko, B. C. (2018). A brief review of facial emotion recognition based on visual information. Sensors, 18(2), 401. https://doi.org/10.3390/s18020401

[11]. Lee, C. C., Shih, C. Y., Lai, W. P., & Lin, P. C. (2012). An improved boosting algorithm and its application to facial emotion recognition. Journal of Ambient Intelligence and Humanized Computing, 3(1), 11-17. https://doi.org/10.1007/s12652-011-0085-8

[12]. Liew, C. F., & Yairi, T. (2015). Facial expression recognition and analysis: a comparison study of feature descriptors. IPSJ Transactions on Computer Vision and Applications, 7, 104-120. https://doi.org/10.2197/ipsjtcva.7.104

[13]. Lin, C. H. (2016). Novel application of continuously variable transmission system using composite recurrent Laguerre orthogonal polynomials modified PSO NN control system. ISA Transactions, 64, 405-417. https://doi.org/10.1016/j.isatra.2016.05.013

[14]. Liu, M., Li, S., Shan, S., & Chen, X. (2015). Au-inspired deep networks for facial expression feature learning. Neurocomputing, 159, 126-136. https://doi.org/10.1016/j.neucom.2015.02.011

[15]. Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010, June). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, (pp. 94-101). IEEE. https://doi.org/10.1109/CVPRW.2010.5543262

[16]. Lyons, M. J., Budynek, J., & Akamatsu, S. (1999). Automatic classification of single facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12), 1357-1362. https://doi.org/10.1109/34.817413

[17]. Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135-164. https://doi.org/10.1023/B:VISI.0000029666.37597.d3

[18]. Mayya, V., Pai, R. M., & Pai, M. M. (2016). Automatic facial expression recognition using DCNN. Procedia Computer Science, 93, 453-461. https://doi.org/10.1016/j.procs.2016.07.233

[19]. Mehrabian, A. (2008). Communication without words. In Communication Theory, (pp. 193-200). Routledge.

[20]. Mohammadi, M. R., Fatemizadeh, E., &Mahoor, M. H. (2014). PCA-based dictionary building for accurate facial expression recognition via sparse representation. Journal of Visual Communication and Image Representation, 25(5), 1082-1092. https://doi.org/10.1016/j.jvcir.2014.03.006

[21]. Mollahosseini, A., Chan, D., &Mahoor, M. H. (2016, March). Going deeper in facial expression recognition using deep neural networks. In 2016, IEEE Winter Conference on Applications of Computer Vision (WACV), (pp. 1-10). IEEE. https://doi.org/10.1109/WACV.2016.7477450

[22]. O'Toole, A. J., Harms, J., Snow, S. L., Hurst, D. R., Pappas, M. R., Ayyad, J. H., & Abdi, H. (2005). A video database of moving faces and people. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 812-816. https://doi.org/10.1109/TPAMI.2005.90

[23]. Sahu, M., & Dash, R. (2021). A survey on deep learning: Convolution neural network (CNN). In Intelligent and Cloud Computing, (pp. 317-325). Springer, Singapore. https://doi.org/10.1007/978-981-15-6202-0_32

[24]. Sánchez, D., Melin, P., & Castillo, O. (2017). Optimization of modular granular neural networks using a firefly algorithm for human recognition. Engineering Applications of Artificial Intelligence, 64, 172-186. https://doi.org/10.1016/j.engappai.2017.06.007

[25]. Shan, C., Gong, S., & McOwan, P. W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6), 803-816. https://doi.org/10.1016/j.imavis.2008.08.005

[26]. Shih, F. Y., Chuang, C. F., & Wang, P. S. (2008). Per formance comparisons of facial expression recognition in JAFFE database. International Journal of Pattern Recognition and Artificial Intelligence, 22(3), 445-459. https://doi.org/10.1142/S0218001408006284

[27]. Sreedharan, N. P. N., Ganesan, B., Raveendran, R., Sarala, P., Dennis, B., &Boothalingam R, R. (2018). Grey wolf optimisation based feature selection and classification for facial emotion recognition. IET Biometrics, 7(5), 490-499. https://doi.org/10.1049/ietbmt.2017.0160

[28]. Suja, P. S., & Tripathi, S. (2016, February). Real-time emotion recognition from facial images using Raspberry Pi II. In 2016, 3rd International Conference on Signal Processing and Integrated Networks (SPIN), (pp. 666-670). IEEE. https://doi.org/10.1109/SPIN.2016.7566780

[29]. Thonse, U., Behere, R. V., Praharaj, S. K., & Sharma, P. S. V. N. (2018). Facial emotion recognition, sociooccupational functioning and expressed emotions in schizophrenia versus bipolar disorder. Psychiatry Research, 264, 354-360. https://doi.org/10.1016/j.psychres.2018.03.027

[30]. Wen, G., Hou, Z., Li, H., Li, D., Jiang, L., & Xun, E. (2017). Ensemble of deep neural networks with probability-based fusion for facial expression recognition. Cognitive Computation, 9(5), 597-610. https://doi.org/10.1007/s12559-017-9472-6

[31]. Xiao-xu, Q. I., & Wei, J. (2007, April). Application of wavelet energy feature in facial expression recognition. In 2007 International Workshop on Anti-Counterfeiting, Security and Identification (ASID), (pp. 169-174). IEEE. https://doi.org/10.1109/IWASID.2007.373720

[32]. Xie, S., & Hu, H. (2018). Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks. IEEE Transactions on Multimedia, 21(1), 211-220. https://doi.org/10.1109/TMM.2018.2844085

[33]. Yaddaden, Y., Bouzouane, A., Adda, M., & Bouchard, B. (2016, June). A new approach of facial expression recognition for ambient assisted living. In Proceedings of the 9th ACM International Conference on Pervasive Technologies Related to Assistive Environments, (pp. 1-8). https://doi.org/10.1145/2910674.2910703

[34]. Yang, M., Liu, Y., & You, Z. (2017). The Euclidean embedding learning based on convolutional neural network for stereo matching. Neurocomputing, 267, 195-200. https://doi.org/10.1016/j.neucom.2017.06.007

[35]. Zhang, K., Huang, Y., Wu, H., & Wang, L. (2015, November). Facial smile detection based on deep learning features. In 2015, 3rd IAPR Asian Conference on Pattern Recognition (ACPR), (pp. 534-538). IEEE. https://doi.org/10.1109/ACPR.2015.7486560

[36]. Zhao, L., Zhuang, G., & Xu, X. (2008, June). Facial expression recognition based on PCA and NMF. In 2008 7th World Congress on Intelligent Control and Automation, (pp. 6826-6829). IEEE. https://doi.org/10.1109/WCICA.2008.4593968

[37]. Zhao, X., Shi, X., & Zhang, S. (2015). Facial expression recognition via deep learning. IETE Technical Review, 32(5), 347-355. https://doi.org/10.1080/02564602.2015.1017542

[38]. Zhi, R., & Ruan, Q. (2008). Facial expression recognition based on two-dimensional discriminant locality preserving projections. Neurocomputing, 71(7-9), 1730-1734. https://doi.org/10.1016/j.neucom.2007.12.002

Transfer Learning Based Effective Emotional Face Recognition using DCNN via Cropping Techniques

Abstract

Keywords

How to Cite this Article?

References

If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Options for accessing this content:

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Pdf	35	35	200	20
Online	35	35	200	15
Pdf & Online	35	35	400	25