Facial Emotion Classification Using Active Appearance Model and Support Vector Machine Classifier

• Autorzy: Beszédeš M. , Culverhouse P. , Oravec M.
• Języki publikacji: EN

Abstrakty

EN

Automatic analysis of human face expression is an interesting and non-trivial problem. In the last decade, many approaches have been described for emotion recognition based on analysis of facial expression. However, little has been done in the sub-area of the recognition of facial emotion intensity levels. This paper proposes the analysis of the use of Active Appearance Models (AAMs) and Support Vector Machine (SVM) classifiers in the recognition of human facial emotion and emotion intensity levels. AAMs are known as a tool for statistical modeling of object shape/appearance or for precise object feature detection. In our case, we examine their properties as a technique for feature extraction. We analyze the influence of various facial feature data types (shape / texture / combined AAM parameter vectors) and the size of facial images on the final classification accuracy. Then, approaches to proper C-SVM classifiers (RBF kernel) training parameter adjustment are described. Moreover, an alternative way of classification accuracy evaluation using the human visual system as a reference point is discussed. Unlike the usual to the approach evaluation of recognition algorithms (based on comparison of final classification accuracies), the proposed evaluation schema is independent of the testing set parameters, such as number, age and gender of subjects or the intensity of their emotions. Finally, we show that our automatic system gives emotion categories for images more consistent labels than human subjects, while humans are more consistent in identifying emotion intensity level compared to our system.

Słowa kluczowe

EN

automatic facial emotion recognition; active appearance model; support vector machine (SVM); psychological experiment; cross-validation

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2009
• Tom: Vol. 18, No. 1
• Strony: 21--46
• Opis fizyczny: Bibliogr. 33 poz., il., wykr.

Twórcy

autor

Beszédeš M.

autor

Culverhouse P.

autor

Oravec M.

• Dept. of Telecommunications, FEI STU Bratislava, Ilkovicova 3, 812 19 Bratislava, Slovakia

Bibliografia

• [1] Ekman P., Friesen W. V.: Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, Vol. 17, 1971.
• [2] Bassili J. N.: Facial Motion in the Perception of Faces and of Emotional Expression. Journal of Experimental Psychology Human Perception and Performance, Vol. 4, 373-379, 1978.
• [3] Ekman P., Friesen W. V.: Facial Action Coding System. Palo Alto: Consulting Psychologist Press, 1978.
• [4] Russell J. A.: Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, Vol. 115(1), 102-141, 1994.
• [5] Vapnik V.: The Nature of Statistical Learning Theory, Springer, New York, 1995.
• [6] Padgett C., Cottrell G.: Representing face images for emotion classification. Advances in Neural Information Processing Systems, Vol. 9, 894-900, 1997.
• [7] Burges, C. J. C.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, Vol. 2(2), Springer, 121-167, 1998.
• [8] Edwards G. J., Cootes T. F., Taylor C. J.: Face recognition using active appearance models. Proceedings of the European Conference on Computer Vision, Vol. 2, 581-695, 1998.
• [9] Vapnik V.: Statistical Learning Theory, John Wiley, New York, 1998.
• [10] Chapelle O., Haffner P., Vapnik V.: SVMs for Histogram-Based Image Classification. IEEE Transactions on Neural Networks, Vol. 10(5), 1055-1064, 1999.
• [11] Cootes T. F., Edwards G. J., Taylor C. J.: Comparing active shape models with active appearance models. Proceedings of the British Machine Vision Conference 1999, Vol. 1, 173-182
• [12] Lyons M. J., Budynek J., Akamatsu S.: Automatic classification of single facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21(12), 1357-1362, 1999.
• [13] Kanade T., Cohn J. F., Tian Y.: Comprehensive database for facial expression analysis, Proceedings of the fourth IEEE International conference on automatic face and gesture recognition, 46-53, 2000.
• [14] Pantic M., Rothkrantz L. J. M.: Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, 2000.
• [15] Pantic M., Rothkrantz L. J. M.: Expert system for automatic analysis of facial expressions. Image and Vision Computing, Vol. 18(11), 881-905, 2000.
• [16] Chih-Chung C., Chih-Jen L.: LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
• [17] Cootes T. F., Edwards G. J., Taylor C. J.: Active Appearance Models. IEEE Trans. Pattern Analysisand Machine Intelligence, Vol. 6(23), 681-685, 2001.
• [18] Hou X., Li S., Zhang H., Cheng Q.: Direct Appearance Models. Computer Vision and Pattern Recognition, Vol. 1, 828-833, 2001.
• [19] Muller K. R., Mika S., Ratsch C., Tsuda K., Scholkopf B.: An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, Vol. 12(2), 181-201, 2001.
• [20] Viola P., Jones M.: Rapid object detection using a boosted cascade of simple features. Proc. CVPR, Vol. 1, 2001.
• [21] Cootes T. F., Kittipanya-ngam P.: Comparing variations on the active appearance model algorithm. British Machine Vision Conference, 837-846, 2002.
• [22] Hsu C. W., Lin C. J.: A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks, Vol. 13(2), 415-425, 2002.
• [23] Yan S. C., Liu C., Li S. Z., Zhang H., Shum H. Y., Cheng Q.: Texture-Constrained Active Shape Models, International Workshop on Generative-Model-Based Vision, Denmark, 2002.
• [24] Fasel B., Luettin J.: Automatic facial expression analysis: A survey. Pattern Recognition, Vol. 36(1), 259-275, 2003.
• [25] Michel P. and El Kaliouby R.: Facial Expression Recognition using Support Vector Machines. The 10th International Conference on Human-Computer Interaction, Crete, Greece, 2003.
• [26] Matthews I., Baker S.: Active Appearance Models Revisited. International Journal of Computer Vision, Springer, Vol. 60(2), 135-164, 2004.
• [27] Beszedes M., Oravec M.: Adaboost Algorithm Used for Skin Color Detection. 29th International Conference Telecommunications and Signal Processing TSP-2006 Brno, Czech Republic, 96-99, 2005.
• [28] Dormer R., Reiter M., Langs G., Peloschek P., Bischof H.: Fast Active Appearance Model Search Using Canonical Correlation Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, 2006.
• [29] Liebelt J., Xiao J., Yang J.: Robust AAM Fitting by Fusion of Images and Disparity Data. Proc. of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 2483-2490, 2006.
• [30] Wallhoff F.: Facial Expressions and Emotion Database. Technische Universitat Munchen, http://www.mmk.ei.tum.de/~waf/fgnet/feedtum.htm, 2006.
• [31] Wang J., Yin L., Wei X., Sun Y.: 3D Facial Expression Recognition Based on Primitive Surface Feature Distribution. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol 2, 1399-1406, 2006.
• [32] Yin L., Wei X., Sun Y., Wang J., Matthew J.: A 3D facial expression database for facial behavior research. 7th International Conference on Automatic Face and Gesture Recognition, 211-216, 2007.
• [33] Mpiperis I., Malasiotis S., Petridis V., Strintzis M.G.: 3D Facial Expression Recognition Using Swarm Intelligence. IEEE International conference on Accoustics, Speech and Signal Processing (ICASSP 2008), Las Vegas, Nevada, USA, accepted for publication, 2008.