Detection of driver fatigue symptoms using transfer learning

• Autorzy: Jakubowski J. , Chmielińska J.

Identyfikatory

DOI

10.24425/bpas.2018.125934

• Języki publikacji: EN

Abstrakty

EN

This paper presents the results of the scientific investigations which aimed at developing the detectors of the selected driver fatigue symptoms based on face images. The presented approach assumed using convolutional neural networks and transfer learning technique. In the conducted research the pretrained model of AlexNet was used. The net underwent slight modification of the structure and then the fine-tuning procedure was applied with the use of an appropriate dataset. In this way all detectors of the selected fatigue symptoms were created. The results of conducted computations indicate that it is potentially possible to apply such an approach to the problem of fatigue symptom detection. The values of the overall misclassification rates for the most troublesome symptom are less than 5.5%, which seems to be a quite satisfactory result.

Słowa kluczowe

EN

driver fatigue; convolutional neural networks; transfer learning; AlexNet

PL

zmęczenie kierowcy; splotowe sieci neuronowe; AlexNet

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2018
• Tom: Vol. 66, nr 6
• Strony: 869--874
• Opis fizyczny: Bibliogr. 27 poz., rys., wykr., tab.

Twórcy

autor

Jakubowski J.

• Faculty of Electronics, Military University of Technology, 2 Witolda Urbanowicza St., 00-908 Warsaw, Poland

autor

Chmielińska J.

• Faculty of Electronics, Military University of Technology, 2 Witolda Urbanowicza St., 00-908 Warsaw, Poland

Bibliografia

• [1] P. Razin, M. Kruszewski, M. Niezgoda, and T. Kamiński, “Selected methods of driver fatigue detection”, Warsaw University of Technology Research Papers – Transport, vol. 114, pp. 291‒302 (2016) [in Polish].
• [2] H.J. Eoh, M.K. Chung, and S.H. Kim, ”Electroencephalographic study of drowsiness in simulated driving with sleep deprivation”, International Journal of Industrial Ergonomics 35 (4), 307‒320 (2005).
• [3] X. Zhu, W.L. Zheng, B.L. Lu, X. Chen, S. Chen, and Ch. Wang, “EOG-based drowsiness detection using convolutional neural network”, International Joint Conference on Neural Networks, Beijing, China, 2014.
• [4] S.Y. Shi, W.Z. Tang, and Y.Y. Wang, “A review on fatigue driving detection”, ITM Web of Conferences, vol. 12 (2017).
• [5] M.H. Sigari, M.R. Pourshahabi, M. Soryani, and M. Fathy, “A review on driver face monitoring systems for fatigue and distraction detection”, International Journal of Advanced Science and Technology 64, 73‒100 (2014).
• [6] R. Grace, V.E. Byrne, D.M. Bierman, J.-M. Legrand, D. Gricourt, B.K. Davis, J.J. Staszewski, and B. Carnahan, “A drowsy driver detection system for heavy vehicles”, Proceedings of the 17th Digital Avionics Systems Conference, I36 1‒8, 1998.
• [7] I.H. Choi and Y.G. Kim, “Head pose and gaze direction tracking for detecting a drowsy driver”, Applied Mathematics and Information Sciences 9 (2L), 505‒512 (2015).
• [8] Y. Lu and Z. Wang, “Detecting driver yawning in succesive images”, The 1st International Conference on Bioinformatics and Biomedical Engineering, USA, 2007.
• [9] N. Alioua, A. Amine, and M. Rziza, “Driver’s fatigue detection based on yawning extraction”, International Journal of Vehicular Technology, vol. 2014 (2014).
• [10] T. Nakamura, A. Maejima, and S. Morishima, “Detection of driver’s drowsy facial expresion”, Second IAPR Asian Conference on Pattern Recognition, 2013.
• [11] Y. Zhang and C. Hua, “Driver fatigue recognition based on facial expression analysis using local binary patterns”, Optik – international Journal for Light and Electron Optics 126 (23), 4501‒4505 (2015).
• [12] X. Fun, B.C. Yin, and Y.F. Sun, “Dynamic human fatigue detection using feature-level fusion”, in Image and Signal Processing. ICISP 2008. Lecture Notes in Computer Science, vol. 5099, eds. A. Elmoataz, O. Lezoray, F. Nouboud, D. Mammass, Springer, Berlin, Heidelberg, 2008.
• [13] K. Yuen, S. Martin, and M.M. Trivedi, “On looking at faces in an automobile: issues, algorithms and evaluation on naturalistic driving dataset”, 23rd International Conference on Pattern Recognition, Mexico, 2016.
• [14] S. Ribarić, J. Lovrenčić, and N. Pavešić, “A neural-network-based system for monitoring driver fatigue”, 15th IEEE Mediterranean Electrotechnical Conference – MELECON 2010, Malta, 2010.
• [15] Ch. Yan, H. Jiang, B. Zhang, and F. Coenen, “Recognizing driver inattention by convolutional neural networks”, 8th International Congress on Image and Signal Processing, China, 2015.
• [16] K. Dwivedi, K. Biswaranjan, and A. Sethi, Drowsy driver detection using representation learning, IEEE International Advance Computing Conference, India, 2014.
• [17] X.P. Huynh, S.M. Park, and Y.G. Kim, “Detection of driver drowsiness using 3D deep neural network and semisupervised gradient boosting machine”, in Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science, vol. 10118, eds. C.S. Chen, J. Lu, K.K. Ma, Springer, Cham, 2017.
• [18] J. Chmielińska and J. Jakubowski, “Application of convolutional neural network to the problem of detecting selected symptoms of driver fatigue, Electric Review 93 (10), 6‒10 (2017) [in Polish].
• [19] A. Krizhevsky, I. Sutskever, and G.E. Hinton, “Imagenet classification with deep convolutional neural networks”, Neural Information Processing Systems, 2012.
• [20] Ch. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions”, Computer Vision and Pattern Recognition (2015).
• [21] Ch.K. Shie, Ch.H. Chuang, Ch.N. Chou, M.H. Wu, and E.Y. Chang, “Transfer representation learning for medical image analysis”, 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2015.
• [22] A.B. Sargano, X. Wang, P. Angelov, and Z. Habib, “Human action recognition using transfer learning with deep representations”, International Joint Conference on Neural Networks, 2017.
• [23] H.W. Ng, V.D. Nguyen, V. Vonikakis, and S. Winkler, “Deep Learning for Emotion Recognition on Small Datasets Using Transfer Learning”, International Conference on Multimodal Interaction, 2015.
• [24] S. Vora, A. Rangesh, and M.M. Trivedi, “On generalizing driver gaze zone estimation using convolutional neural networks”, IEEE Intelligent Vehicles Symposium, USA, 2017.
• [25] S. Park, F. Pan, S. Kang, and C.D. Yoo, “Driver drowsiness detection system based on feature representation learning using various deep networks”, in Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science, vol. 10118, eds. C.S. Chen, J. Lu, K.K. Ma,Springer, Cham, 2017.
• [26] Matlab R2017a documentation – Neural Network Toolbox.
• [27] X. Glorot, A. Bordes, Y. Bengio, “Deep Sparse Rectifer Neural Networks”, Proceedings of the 14th International Conference on Artifcial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, pp. 315‒323 (2011).

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).