Convolutional and Recurrent Neural Networks for Face Image Analysis

• Autorzy: Yüksel Kıvanç , Skarbek Władysław

Identyfikatory

DOI

10.2478/fcds-2019-0017

• Języki publikacji: EN

Abstrakty

EN

In the presented research two Deep Neural Network (DNN) models for face image analysis were developed. The first one detects eyes, nose and mouth and it is based on a moderate size Convolutional Neural Network (CNN) while the second one identifies 68 landmarks resulting in a novel Face Alignment Network composed of a CNN and a recurrent neural network. The Face Parts Detector inputs face image and outputs the pixel coordinates of bounding boxes for detected facial parts. The Face Alignment Network extracts deep features in CNN module while in the recurrent module it generates 68 facial landmarks using not only this deep features, but also the geometry of facial parts. Both methods are robust to varying head poses and changing light conditions.

Słowa kluczowe

EN

deep learning; convolutional neural networks; recurrent neural network; facial landmark localization; facial parts detection; computer vision; image processing

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2019
• Tom: Vol. 44, No. 3
• Strony: 331--347
• Opis fizyczny: Bibliogr. 24 poz., rys., tab.

Twórcy

autor

Yüksel Kıvanç

• Promity, Warsaw, Poland, formerly MSc student of Warsaw University of Technology

autor

Skarbek Władysław

• Institute of Radioelectronics and Multimedia Technology, Warsaw University of Technology

Bibliografia

• [1] Bahdanau, D., Cho, K. and Bengio, Y. 2014, ‘Neural machine translation by jointly learning to align and translate', arXiv preprint arXiv:1409.0473 .
• [2] Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J. and Kumar, N., 2013, ‘Localizing parts of faces using a consensus of exemplars', IEEE transactions on pattern analysis and machine intelligence 35(12), 2930-2940.
• [3] Cao, X., Wei, Y., Wen, F. and Sun, J. 2014, ‘Face alignment by explicit shape regression’, International Journal of Computer Vision 107(2), 177-190.
• [4] Cortes, C. and Vapnik, V., 1995. Support-vector networks. Machine learning, 20(3), pp.273-297.
• [5] Dalal, N. and Triggs, B., 2005, June. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on (Vol. 1, pp. 886-893). IEEE.
• [6] Karpathy, A. and Fei-Fei, L. 2015, Deep visual-semantic alignments for generating image descriptions, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 3128-3137.
• [7] King, D.E., 2009. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10(Jul), pp. 1755-1758.
• [8] Kingma, D.P. and Ba, J., 2014, ‘Adam: A method for stochastic optimization’, arXiv preprint arXiv:1412.6980.
• [9] Kowalski, M., Naruniec, J. and Trzcinski, T., 2017, ‘Deep alignment network: A convolutional neural network for robust face alignment’, CoRR abs/1706.01789. URL: http://arxiv.org/abs 1706.01789
• [10] Kowalski, M. and Naruniec, J., 2016, ‘Face alignment using k-cluster regression forests with weighted splitting’, IEEE Signal Processing Letters 23(11), 1567-1571.
• [11] Lee, D., Park, H. and Yoo, C.D., 2015, Face alignment using cascade gaussian process regression trees, in ‘Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition’, pp. 4204-4212.
• [12] Le, V., Brandt, J., Lin, Z., Bourdev, L. and Huang, T.S., 2012, Interactive facial feature localization, in ‘European Conference on Computer Vision’, Springer, pp. 679-692.
• [13] Ren, S., Cao, X., Wei, Y. and Sun, J., 2014, Face alignment at 3000 fps via regressing local binary features, in ‘Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition’, pp. 1685-1692.
• [14] Sagonas, C., Tzimiropoulos, G., Zafeiriou, S. and Pantic, M., 2013, 300 faces in-the-wild challenge: The first facial landmark localization challenge, in ‘Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on’, IEEE, pp. 397-403.
• [15] Sebe, N. and Lew, M.S., 2013. Robust computer vision: Theory and applications (Vol. 26). Springer Science & Business Media.
• [16] Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A. and Potts, C., 2013, Recursive deep models for semantic compositionality over a sentiment treebank, in ‘Proceedings of the 2013 conference on empirical methods in natural language processing’, pp. 1631-1642.
• [17] Trigeorgis, G., Snape, P., Nicolaou, M. A., Antonakos, E. and Zafeiriou, S., 2016, Mnemonic descent method: A recurrent process applied for end-to-end face align-ment, in ‘Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition’, pp. 4177-4187.
• [18] Wen, Z. and Huang, T.S., 2006. 3D Face Processing: Modeling, Analysis and Synthesis (Vol. 8). Springer Science & Business Media.
• [19] Xiao, S., Feng, J., Xing, J., Lai, H., Yan, S. and Kassim, A., 2016, Robust facial landmark detection via recurrent attentive-refinement networks, in ‘European conference on computer vision’, Springer, pp. 57-72.
• [20] Xiong, X. and De la Torre, F., 2013, Supervised descent method and its applications to face alignment, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 532-539.
• [21] Yue-Hei Ng, J., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R. and Toderici, G., 2015, Beyond short snippets: Deep networks for video classification, in ‘Proceedings of the IEEE conference on computer vision and pattern recognition’, pp. 4694-4702.
• [22] Yuksel, K., Chang, X. and Skarbek, W., 2017, August. Smile detectors correlation. In Photonics Applications in Astronomy, Communications, Industry, and High Energy Physics Experiments 2017 (Vol. 10445, p. 104451L). International Society for Optics and Photonics.
• [23] Zhu, S., Li, C., Change Loy, C. and Tang, X., 2015, Face alignment by coarse-to-fine shape searching, in ‘Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition’, pp. 4998-5006.
• [24] Zhu, X. and Ramanan, D., 2012, June. Face detection, pose estimation, and landmark localization in the wild. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on (pp. 2879-2886). IEEE.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).