Hybrid end-to-end approach integrating online learning with face-identification system

• Autorzy: Nguyene Dat Van , Nguyen Son Trung , Pham Thi Hong Anh , Pham Van Toan , Hoang Thao Thu , Thanh Ta Minh

Identyfikatory

DOI

10.7494/csci.2023.24.2.4840

• Języki publikacji: EN

Abstrakty

EN

Facial recognition has been one of the most intriguing and exciting research topics over the last few years. It involves multiple face-based algorithms such asfacial detection, facial alignment, facial representation, and facial recognition. However, all of these algorithms are derived from large deep-learning architectures, leading to limitations in development, scalability, accuracy, and deployment for public use with mere CPU servers. Also, large data sets that contain hundreds of thousands of records are often required for training purposes. In this paper, we propose a complete pipeline for an effective face-recognition application that requires only a small data set of Vietnamese celebrities and a CPU for training, solving the problem of data leakage, and the need for GPU devices. The pipeline is based on the combination of a conversion algorithm from face vectors to string tokens and the indexing & retrieval process by Elasticsearch, thereby tackling the problem of online learning in facial recognition. Compared with other popular algorithms on the same data set, our proposed pipeline not only outperforms the counterpart in terms of accuracy but also delivers faster inference, which is essential to real-time applications.

Słowa kluczowe

EN

facial recognition; visual search engine; end-to-end applications; online learning; ElasticSearch (ES); ES

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2023
• Tom: T. 24 (2)
• Strony: 141--161
• Opis fizyczny: Bibliogr. 51 poz., rys., tab., wykr.

Twórcy

autor

Nguyene Dat Van

autor

Nguyen Son Trung

autor

Pham Thi Hong Anh

autor

Pham Van Toan

autor

Hoang Thao Thu

autor

Thanh Ta Minh

Bibliografia

• [1] Akter S., Sima R.A., Ullah M.S., Hossain S.A.: Smart Security Surveillance using IoT. In: 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), pp. 659–663, 2018. doi: 10.1109/ICRITO.2018.8748703.
• [2] Almabdy S., Elrefaei L.: Deep Convolutional Neural Network-Based Approaches for Face Recognition, Applied Sciences, vol. 9, 2019. doi: 10.3390/app9204397.
• [3] Amos B., Ludwiczuk B., Satyanarayanan M.: OpenFace: A general-purpose face recognition library with mobile applications. Technical report, CMU-CS-16-118, CMU School of Computer Science, 2016.
• [4] Bradski G.: The OpenCV Library, Dr Dobb’s Journal of Software Tools, 2000.
• [5] Cao Q., Shen L., Xie W., Parkhi O.M., Zisserman A.: VGGFace2: A dataset for recognising faces across pose and age, arXiv: 171008092, 2018. doi: 10.48550/arXiv.1710.08092.
• [6] Chowdhry D.A., Hussain A., Ur Rehman M.Z., Ahmad F., Ahmad A., Pervaiz M.:Smart security system for sensitive area using face recognition. In:2013 IEEEConference on Sustainable Utilization and Development in Engineering and Tech-nology (CSUDET), pp. 11–14, 2013. doi: 10.1109/CSUDET.2013.6670976.
• [7] Dalal N., Triggs B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 886–893, 2005. doi: 10.1109/CVPR.2005.177.
• [8] Deb D., Nain N., Jain A.K.: Longitudinal Study of Child Face Recognition, arXiv:171103990, 2017. doi: 10.48550/arXiv.1711.03990.
• [9] Deng J., Guo J., Xue N., Zafeiriou S.: ArcFace: Additive Angular Margin Loss for Deep Face Recognition, arXiv: 180107698, 2019. doi: 10.48550/arXiv.1801.07698.
• [10] Deng J., Guo J., Zhou Y., Yu J., Kotsia I., Zafeiriou S.: RetinaFace: Single-stageDense Face Localisation in the Wild, arXiv: 190500641, 2019. doi: 10.48550/arXiv.1905.00641.
• [11] Django 3.1, 2020. https://www.djangoproject.com/.
• [12] Elasticsearch: Function Score query 6.8, 2019. https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-function-score-query.html.
• [13] Elasticsearch: Rescoring 6.8, 2019.https : / / www.elastic.co / guide / en /elasticsearch/reference/6.8/search-request-rescore.html.
• [14] Girshick R., Donahue J., Darrell T., Malik J.: Rich feature hierarchies foraccurate object detection and semantic segmentation, 2014. doi: 10.48550/arXiv.1311.2524.
• [15] Goodfellow I., Bengio Y., Courville A.: Deep Learning, MIT Press, 2016. http://www.deeplearningbook.org.
• [16] He K., Zhang X., Ren S., Sun J.: Deep Residual Learning for Image Recognition, arXiv: 151203385, 2015. doi: 10.48550/arXiv.1512.03385.
• [17] Hearst M.A., Dumais S.T., Osuna E., Platt J., Scholkopf B.: Support vector machines, IEEE Intelligent Systems and their Applications, vol. 13(4), pp. 18–28,1998. doi: 10.1109/5254.708428.
• [18] Howard A.G., Zhu M., Chen B., Kalenichenko D., Wang W., Weyand T., Andreetto M., Adam H.: MobileNets: Efficient Convolutional Neural Networks for MobileVision Applications, arXiv: 170404861, 2017. doi: 10.48550/arXiv.1704.04861.
• [19] Huang G.B., Learned-Miller E.: Labeled Faces in the Wild: Updates and New Reporting Procedures, Technical report, UM-CS-2014-003, University of Massachusetts, Amherst, 2014. https://vis-www.cs.umass.edu/lfw/lfw_update.pdf.
• [20] Iterative, DVC: Data Version Control – Git for Data & Models, 2020. https://github.com/iterative/dvc.
• [21] King D.E.: Dlib-ml: A Machine Learning Toolkit,The Journal of Machine Learning Research, vol. 10, pp. 1755–1758, 2009.
• [22] Komulainen J., Hadid A., Pietikäinen M.: Context based face anti-spoofing. In:2013 IEEE Sixth International Conference on Biometrics: Theory, Applicationsand Systems (BTAS), pp. 1–8, 2013. doi: 10.1109/BTAS.2013.6712690.
• [23] Li H., Lin Z., Shen X., Brandt J., Hua G.: A convolutional neural network cascade for face detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5325–5334, 2015.
• [24] Lienhart R., Maydt J.: An extended set of Haar-like features for rapid object detection. In: Proceedings of International Conference on Image Processing, vol. 1,pp. I–I, 2002. doi: 10.1109/ICIP.2002.1038171.
• [25] Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.Y., Berg A.: SSD: Single Shot MultiBox Detector. In: Computer Vision – ECCV 2016, pp. 21–37,2016. doi: 10.1007/978-3-319-46448-0_2.
• [26] Mu C., Zhao J., Yang G., Zhang J., Yan Z.: Towards Practical Visual Search Engine within Elasticsearch, arXiv: 180608896, 2019. doi: 10.48550/arXiv.1806.08896.
• [27] Ng H.W., Winkler S.: A data-driven approach to cleaning large face datasets, 2014 IEEE International Conference on Image Processing, ICIP 2014, pp. 343–347,2015. doi: 10.1109/ICIP.2014.7025068.
• [28] Owayjan M., Dergham A., Haber G., Fakih N., Hamoush A., Abdo E.: Face Recognition Security System. In: International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering (CISSE 2013), 2013.
• [29] Parmar D.N., Mehta B.B.: Face Recognition Methods & Applications, International Journal of Computer Technology and Applications, vol. 4, pp. 84–86, 2014.
• [30] Pearson K.F.R.S.: LIII. On lines and planes of closest fit to systems of points inspace,The London, Edinburgh, and Dublin Philosophical Magazine and Journalof Science, vol. 2(11), pp. 559–572, 1901. doi: 10.1080/14786440109462720.
• [31] Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay E.: Scikit-learn: Machine Learningin Python, Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
• [32] Sahoo D., Pham Q., Lu J., Hoi S.C.H.: Online Deep Learning: Learning Deep Neural Networks on the Fly, arXiv: 1711.03705, 2017. doi: 10.48550/arXiv.1711.03705.
• [33] Satish A., Devarajan N.: Preprocessing technique for face recognition applications under varying illumination conditions, Global Journal of Computer Science and Technology Graphics and Vision, vol. 12, pp. 13–18, 2012.
• [34] Schroff F., Kalenichenko D., Philbin J.: FaceNet: A unified embedding for face recognition and clustering, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. doi: 10.1109/cvpr.2015.7298682.
• [35] Shi Y., Otto C., Jain A.K.: Face Clustering: Representation and Pairwise Constraints, IEEE Transactions on Information Forensics and Security, vol. 13(7), pp. 1626–1640, 2018. doi: 10.1109/tifs.2018.2796999.
• [36] Taigman Y., Yang M., Ranzato M., Wolf L.: DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014.doi: 10.1109/CVPR.2014.220.
• [37] Tang X., Du D.K., He Z., Liu J.: PyramidBox: A Context-assisted Single Shot Face Detector, arXiv:180307737, 2018. doi: 10.48550/arXiv.1803.07737.
• [38] Tensorflow Serving 6.8, 2019. https://github.com/tensorflow/serving.
• [39] Tolba A., El-Baz A., El-Harby A.: Face Recognition: A Literature Review, International Journal of Signal Processing, vol. 2, pp. 88–103, 2005.
• [40] Van T.P., Nguyen T.M., Tran N.N., Nguyen H.V., Doan L.B., Dao H.Q.,Minh T.T.: Interpreting the Latent Space of Generative Adversarial Networks using Supervised Learning. In: 2020 International Conference on Advanced Computing and Applications (ACOMP), pp. 49–54, 2020. doi: 10.1109/ACOMP50827.2020.00015.
• [41] Vikram K., Padmavathi S.: Facial parts detection using Viola Jones algorithm. In: 2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS), pp. 1–4, 2017.
• [42] Viola P., Jones M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, 2001. doi: 10.1109/CVPR.2001.990517.
• [43] Wang H., Wang Y., Zhou Z., Ji X., Gong D., Zhou J., Li Z., Liu W.: CosFace:Large Margin Cosine Loss for Deep Face Recognition, 2018.
• [44] Wang M., Deng W.: Deep face recognition: A survey, Neurocomputing, vol. 429, pp. 215–244, 2021. doi: 10.1016/j.neucom.2020.10.081.
• [45] Wang Y., Yao Q.: Few-shot Learning: A Survey, arXiv: 190405046v1, 2019. http://arxiv.org/pdf/1904.05046v1.
• [46] Wold S., Esbensen K., Geladi P.: Principal component analysis, Chemometrics and Intelligent Laboratory Systems, vol. 2(1), pp. 37–52, 1987. doi: 10.1016/0169-7439(87)80084-9.
• [47] Wright J., Yang A.Y., Ganesh A., Sastry S.S., Ma Y.: Robust Face Recognition via Sparse Representation, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 31, pp. 210–227, 2009. doi: 10.1109/TPAMI.2008.79.
• [48] Yang H., Jia X., Loy C.C., Robinson P.: An Empirical Study of Recent Face Alignment Methods, 2015. doi: 10.13140/RG.2.1.4603.8484.
• [49] Yang J., Luo L., Qian J., Tai Y., Zhang F., Xu Y.: Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39(1), pp. 156–171, 2017.
• [50] Yi D., Lei Z., Liao S., Li S.Z.: Learning Face Representation from Scratch, arXiv:1411.7923, 2014. doi: 10.48550/arXiv.1411.7923.
• [51] Zhang K., Zhang Z., Li Z., Qiao Y.: Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks, IEEE Signal Processing Letters,vol. 23(10), pp. 1499–1503, 2016. doi: 10.1109/LSP.2016.2603342.