Non-contact video-based remote photoplethysmography for human stress detection

• Autorzy: Nikolaiev Sergii , Telenyk Sergii , Tymoshenko Yury

Identyfikatory

DOI

10.14313/JAMRIS/2-2020/21

• Języki publikacji: EN

Abstrakty

EN

This paper presents the experimental results for stress index calculation using developed by the authors information technology for non-contact remote human heart rate variability (HRV) retrieval in various conditions from video stream using common wide spread web cameras with minimal frame resolution of 640x480 pixels at average frame rate of 25 frames per second. The developed system architecture based on remote photoplethysmography (rPPG) technology is overviewed including description of all its main components and processes involved in converting video stream of frames into valuable rPPG signal. Also, algorithm of RR-peaks detection and RR-intervals retrieval is described. It is capable to detect 99.3% of heart contractions from raw rPPG signal. The usecases of measuring stress index in a wide variety of situations starting with car and tractor drivers at work research and finishing with students passing exams are presented and analyzed in detail. The results of the experiments have shown that the rPPG system is capable of retrieving stress level that is in accordance with the feelings of experiments’ participants.

Słowa kluczowe

EN

video processing; web cameras; stress index; remote photoplethysmography; rPPG; heart rate; heart rate variability; Predictive; Preventive; Personalized Medicine; Participatory Medicine

• Wydawca: Łukasiewicz Industrial Research Institute for Automation and Measurements PIAP
• Czasopismo: Journal of Automation Mobile Robotics and Intelligent Systems
• Rocznik: 2020
• Tom: Vol. 14, No. 2
• Strony: 63--73
• Opis fizyczny: Bibliogr. 30 poz., rys.

Twórcy

autor

Nikolaiev Sergii

• Institute for Applied System Analysis, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine

autor

Telenyk Sergii

• Department of Electrical Engineering and Computer Science, Cracow University of Technology, Krakow, Poland

autor

Tymoshenko Yury

• Institute for Applied System Analysis, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine

Bibliografia

• [1] O. Golubnitschaja, J. Kinkorova and V. Costigliola, “Predictive, Preventive and Personalised Medicine as the hardcore of ‘Horizon 2020’”, EPMA Journal, vol. 5, no. 1, 2014 DOI: 10.1186/1878-5085-5-6.
• [2] S. Nikolaiev and Y. Timoshenko, “Reinvention of the cardiovascular diseases prevention and prediction due to ubiquitous convergence of mobile apps and machine learning”. In: 2015 Information Technologies in Innovation Business Conference (ITIB), 2015, 23–26 DOI: 10.1109/ITIB.2015.7355066.
• [3] “Stress and Heart Disease”. The American Institute of Stress, www.stress.org/stress-and-heart-disease. Accessed on: 2020-08-11.
• [4] M. Malik, “Heart Rate Variability. Standards of Measurement, Physiological Interpretation, and Clinical Use”, Annals of Noninvasive Electrocardiology, vol. 1, no. 2, 1996, 151–181 DOI: 10.1111/j.1542-474X.1996.tb00275.x.
• [5] X. Li, J. Chen, G. Zhao and M. Pietikainen, “Remote Heart Rate Measurement from Face Videos under Realistic Situations”. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, 4264–4271 DOI: 10.1109/CVPR.2014.543.
• [6] W. Verkruysse, L. O. Svaasand and J. S. Nelson, “Remote plethysmographic imaging using ambient light”, Optics Express, vol. 16, no. 26, 2008 DOI: 10.1364/OE.16.021434.
• [7] M.-Z. Poh, D. J. McDuff and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation”, Optics Express, vol. 18, no. 10, 2010 DOI: 10.1364/OE.18.010762.
• [8] M. van Gastel, S. Stuijk and G. de Haan, “Motion Robust Remote-PPG in Infrared”, IEEE Transactions on Biomedical Engineering, vol. 62, no. 5, 2015, 1425–1433 DOI: 10.1109/TBME.2015.2390261.
• [9] K. Humphreys, C. Markham and T. E. Ward, “A CMOS camera-based system for clinical photoplethysmographic applications”. In: F. D. Murtagh (eds.), Proceedings Volume 5823, Opto-Ireland 2005: Imaging and Vision, 2005 DOI: 10.1117/12.604822.
• [10] W. Verkruysse, L. O. Svaasand and J. S. Nelson, “Remote plethysmographic imaging using ambient light”, Optics Express, vol. 16, no. 26, 2008 OI: 10.1364/OE.16.021434.
• [11] M. Lewandowska, J. Rumiński, T. Kocejko and J. Nowak, “Measuring pulse rate with a webcam — A non-contact method for evaluating cardiac activity”. In: 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), 2011, 405–410.
• [12] M.-Z. Poh, D. J. McDuff and R. W. Picard, “Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam”, IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, 2011 DOI: 10.1109/TBME.2010.2086456.
• [13] G. de Haan and V. Jeanne, “Robust Pulse Rate From Chrominance-Based rPPG”, IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, 2013, 2878–2886 DOI: 10.1109/TBME.2013.2266196.
• [14] G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature”, Physiological Measurement, vol. 35, no. 9, 2014, 1913–1926 DOI: 10.1088/0967-3334/35/9/1913.
• [15] W. Wang, S. Stuijk and G. de Haan, “A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation”, IEEE Transactions on Biomedical Engineering, vol. 63, no. 9, 2016,1974–1984 DOI: 10.1109/TBME.2015.2508602.
• [16] X. Li, J. Chen, G. Zhao and M. Pietikainen, “Remote Heart Rate Measurement from Face Videos under Realistic Situations”. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, 4264–4271 DOI: 10.1109/CVPR.2014.543.
• [17] L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. A. Clifton and C. Pugh, “Non-contact video- -based vital sign monitoring using ambient light and auto-regressive models”, Physiological Measurement, vol. 35, no. 5, 2014, 807–831, 10.1088/0967-3334/35/5/807.
• [18] W. Wang, S. Stuijk and G. de Haan, “Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG”, IEEE Transactions on Biomedical Engineering, vol. 62, no. 2, 2015, 415–425 DOI: 10.1109/TBME.2014.2356291.
• [19] M. Kumar, A. Veeraraghavan and A. Sabharwal, “DistancePPG: Robust non-contact vital signs onitoring using a camera”, Biomedical Optics Express, vol. 6, no. 5, 2015 DOI: 10.1364/BOE.6.001565.
• [20] S. Tulyakov, X. Alameda-Pineda, E. Ricci, L. Yin, J. F. Cohn and N. Sebe, “Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions”.In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 2396–2404 OI: 10.1109/CVPR.2016.263.
• [21] A. R. Guazzi, M. Villarroel, J. Jorge, J. Daly, M. C. Frise, P. A. Robbins and L. Tarassenko, Non-contact measurement of oxygen saturation with an RGB camera”, Biomedical Optics xpress, vol. 6, no. 9, 2015 DOI: 10.1364/BOE.6.003320.
• [22] I. C. Jeong and J. Finkelstein, “Introducing Contactless Blood Pressure Assessment Using a High Speed Video Camera”, Journal of Medical Systems, vol. 40, no. 4, 2016 DOI: 10.1007/s10916-016-0439-z.
• [23] L. K. Mestha, S. Kyal, Beilei Xu, L. E. Lewis and V. Kumar, “Towards continuous monitoring of ulse rate in neonatal intensive care unit with a webcam”. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014, 3817–3820 DOI: 10.1109/EMBC.2014.6944455.
• [24] S. Fernando, W. Wang, I. Kirenko, G. de Haan, S. Bambang Oetomo, H. Corporaal and J. van alfsen, “Feasibility of Contactless Pulse Rate Monitoring of Neonates using Google Glass”. In: Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare – “Transforming healthcare through innovations in mobile and wireless technologies”, 2015 DOI: 10.4108/eai.14-10-2015.2261589.
• [25] J.-P. Couderc, S. Kyal, L. K. Mestha, B. Xu, D. R. Peterson, X. Xia and B. Hall, “Detection of atrial fibrillation using contactless facial video monitoring”, Heart Rhythm, vol. 12, no. 1, 2015, 195–201 DOI: 10.1016/j.hrthm.2014.08.035.
• [26] B. Kaur, S. Moses, M. Luthra and V. N. Ikonomidou, “Remote stress detection using a visible spectrum camera”. In: H. H. Szu, L. Dai and Y. Zheng (eds.), Proceedings Volume 9496, Independent Component Analyses, Compressive Sampling, Large Data Analyses (LDA), Neural Networks, Biosystems, and Nanoengineering XIII, 2015 DOI: 10.1117/12.2177159.
• [27] S. S. Nikolaiev, Y. O. Tymoshenko and K. Y. Matviiv, “Haar Cascade Face Detector Quality Dependence on Training Dataset Variablity”, Research Bulletin of the National Technical University of Ukraine “Kyiv Politechnic Institute”, no. 6, 2017, 38–46 DOI: 10.20535/1810-0546.2017.6.115181.
• [28] S. Nikolaiev and H. Chereda, “Sampling Rate Independent Filtration Approach for Automatic ECG Delineation”, International Scientific Journal “Internauka”, vol. 27, no. 5, 2017.
• [29] S. Nikolaiev, “Metric and algorithm for similarity between two temporal event sequences calculation”, System research and information technologies, no. 3, 2017, 127–135 DOI: 10.20535/SRIT.2308-8893.2017.3.12.
• [30] R. M. Bayevsky and G. G. Ivanov, “Cardiac Rhythm Variability: the Theoretical Aspects and the Opportunities of Clinical Application”, Ultrazvukovaya i funktsionalnaya diagnostika, no. 3, 2001, 108–127 (in Russian).

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).