Facial expression recognition under difficult conditions: A comprehensive study on edge directional texture patterns

• Autorzy: Ahmed F. , Kabir M. H.

Identyfikatory

DOI

10.2478/amcs-2018-0030

• Języki publikacji: EN

Abstrakty

EN

In recent years, research in automated facial expression recognition has attained significant attention for its potential applicability in human–computer interaction, surveillance systems, animation, and consumer electronics. However, recognition in uncontrolled environments under the presence of illumination and pose variations, low-resolution video, occlusion, and random noise is still a challenging research problem. In this paper, we investigate recognition of facial expression in difficult conditions by means of an effective facial feature descriptor, namely the directional ternary pattern (DTP). Given a face image, the DTP operator describes the facial feature by quantizing the eight-directional edge response values, capturing essential texture properties, such as presence of edges, corners, points, lines, etc. We also present an enhancement of the basic DTP encoding method, namely the compressed DTP (cDTP) that can describe the local texture more effectively with fewer features. The recognition performances of the proposed DTP and cDTP descriptors are evaluated using the Cohn–Kanade (CK) and the Japanese female facial expression (JAFFE) database. In our experiments, we simulate difficult conditions using original database images with lighting variations, low-resolution images obtained by down-sampling the original, and images corrupted with Gaussian noise. In all cases, the proposed method outperforms some of the well-known face feature descriptors.

Słowa kluczowe

EN

directional ternary pattern; compressed DTP; facial feature descriptor; texture encoding; support vector machine

PL

deskryptor cech; kodowanie struktury; maszyna wektorów wsparcia

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2018
• Tom: Vol. 28, no. 2
• Strony: 399--409
• Opis fizyczny: Bibliogr. 34 poz., rys., tab.

Twórcy

autor

Ahmed F.

• Department of Computer Science, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada

autor

Kabir M. H.

• Department of Computer Science and Engineering, Islamic University of Technology, Board Bazar, Gazipur 1704, Bangladesh

Bibliografia

• [1] Ahmed, F. (2012). Gradient directional pattern: A robust feature descriptor for facial expression recognition, IET Electronics Letters 48(19): 1203–1204.
• [2] Ahmed, F. and Kabir, M.H. (2012a). Directional ternary pattern (DTP) for facial expression recognition, IEEE International Conference on Consumer Electronics, Berlin, Germany, pp. 265–266.
• [3] Ahmed, F. and Kabir, M.H. (2012b). Facial feature representation with directional ternary pattern (DTP): Application to gender classification, Proceedings of the IEEE International Conference on Information Reuse and Integration, Las Vegas, NV, USA, pp. 159–164.
• [4] Ahonen, T., Hadid, A. and Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12): 2037–2041.
• [5] Chen, H., Belhumeur, P. and Jacobs, D. (2000). In search of illumination invariants, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA, Vol. 1, pp. 254–261.
• [6] Donato, G., Bartlett, M.S., Hagar, J.C., Ekman, P. and Sejnowski, T.J. (1999). Classifying facial actions, IEEE Transactions on Pattern Analysis and Machince Intelligence 21(10): 974–989.
• [7] Ekman, P. and Friesen, W. (1978). Facial Action Coding System: A Technique for Measurement of Facial Movement, Consulting Psychologists Press, Palo Alto, CA.
• [8] Fa, C.C. and Shin, F.Y. (2006). Recognizing facial action units using independent component analysis and support vector machine, Pattern Recognition 39(9): 1795–1798.
• [9] Gundimada, S. and Asari, V.K. (2009). Facial recognition using multisensor images based on localized kernel eigen spaces, IEEE Transactions on Image Processing 18(6): 1314–1325.
• [10] Guo, G.D. and Dyer, C.R. (2003). Simultaneous feature selection and classifier training via linear programming: A case study for face expression recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp. 346–352.
• [11] Guo, Z., Zhang, L. and Zhang, D. (2010). Rotation invariant texture classification using LBP variance (LBPV) with global matching, Pattern Recognition 43(3): 706–719.
• [12] Hsu, C.W. and Lin, C.J. (2002). A comparison on methods for multiclass support vector machines, IEEE Transactions on Neural Networks 13(2): 415–425.
• [13] Jabid, T., Kabir, M.H. and Chae, O. (2010). Robust facial expression recognition based on local directional pattern, ETRI Journal 32(5): 784–794.
• [14] Jabid, T., Kabir, M.H. and Chae, O. (2012). Local directional pattern (LDP) for face recognition, International Journal of Innovative Computing, Information and Control 8(4): 2423–2437.
• [15] Kabir, H., Jabid, T. and Chae, O. (2012). Local directional pattern variance (LDPV): A robust feature descriptor for facial expression recognition, International Arab Journal of Information Technology 9(4): 382–391.
• [16] Kanade, T., Cohn, J. and Tian, Y. (2000). Comprehensive database for facial expression analysis, Proceedings of the IEEE International Conference on Automated Face and Gesture Recognition, Grenoble, France, pp. 46–53.
• [17] Ling, H., Soatto, S., Ramanathan, N. and Jacobs, D. (2007). A study of face recognition as people age, Proceedings of the IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, pp. 1–8.
• [18] Lyons, M.J., Budynek, J. and Akamatsu, S. (1999). Automatic classification of single facial images, IEEE Transactions on Pattern Analysis and Machine Intelligence 21(12): 1357–1362.
• [19] Ojala, T., Pietikainen, M. and Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7): 971–987.
• [20] Padgett, C. and Cottrell, G. (1997). Representing face images for emotion classification, in M. Mozer et al. (Eds.), Advances in Neural Information Processing Systems, Vol. 9, MIT Press, Cambridge, MA, pp. 894–900.
• [21] Rivera, A.R., Castillo, J.R. and Chae, O. (2013). Local directional number pattern for face analysis: Face and expression recognition, IEEE Transactions on Image Processing 22(5): 1740–1752.
• [22] Shan, C., Gong, S. and McOwan, P.W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study, Image and Vision Computing 27(6): 803–816.
• [23] Sniezynski, B. (2015). A strategy learning model for autonomous agents based on classification, International Journal of Applied Mathematics and Computer Science 25(3): 471–482, DOI: 10.1515/amcs-2015-0035.
• [24] Tan, X. and Triggs, B. (2007). Enhanced local texture feature sets for face recognition under difficult lighting conditions, Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Rio de Janeiro, Brazil, pp. 168–182.
• [25] Tenne, Y. (2017). Machine-learning in optimization of expensive black-box functions, International Journal of Applied Mathematics and Computer Science 27(1): 105–118, DOI: 10.1515/amcs-2017-0008.
• [26] Uddin, M.Z., Lee, J.J. and Kim, T.S. (2009). An enhanced independent component-based human expression recognition from video, IEEE Transactions on Consumer Electronics 55(4): 2216–2224.
• [27] Umbaugh, S.E. (2011). Digital Image Processing and Analysis, CRC Press, Boca Raton, FL.
• [28] Valstar, M., Patras, I. and Pantic, M. (2005). Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data, Proceedings of the IEEE CVPR Workshop, San Diego, CA, USA, Vol. 3, pp. 76–84.
• [29] Viola, P. and Jones, M. (2004). Robust real-time face detection, International Journal of Computer Vision 57(2): 137–154.
• [30] Yang, J., Zhang, D., Frangi, A. and Yang, J. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 26(1): 131–137.
• [31] Yao, B., Hu, P., Zhang, M. and Jin, M. (2014). A support vector machine with the tabu search algorithm for freeway incident detection, International Journal of Applied Mathematics and Computer Science 24(2): 397–404, DOI: 10.2478/amcs-2014-0030.
• [32] Zhang, Z. (1999). Feature-based facial expression recognition: Sensitivity analysis and experiment with a multi-layer perceptron, International Journal of Pattern Recognition and Artificial Intelligence 13(6): 893–911.
• [33] Zhao, G. and Pietikainen, M. (2009). Boosted multi-resolution spatiotemporal descriptors for facial expression recognition, Pattern Recognition Letters 30(12): 1117–1127.
• [34] Zhao, S., Gao, Y. and Zhang, B. (2008). Sobel-LBP, Proceedings of the IEEE International Conference on Image Processing, San Diego, CA, USA, pp. 2144–2147.

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).