Pose and Optical Flow Fusion (POFF) for accurate tremor detection and quantification

• Autorzy: Alper Mehmet Akif , Goudreau John , Daniel Morris

Identyfikatory

DOI

10.1016/j.bbe.2020.01.009

• Języki publikacji: EN

Abstrakty

EN

Limb tremor measurements are one factor used to characterize and quantify the severity of neurodegenerative disorders. These tremor measurements can also provide dosage-response feedback to guide medication treatments. Here, we propose a system to automatically measure limb tremors in home or clinic settings. The key feature of proposed method is that it is contactless; not requiring a user to wear or hold a device or marker. Our sensor is a Kinect 2, which measures color and depth and estimates rough limb motion. We show that its pose accuracy is poor for small limb tremors below 10 mm amplitude, and so we propose an additional level of tremor tracking that recovers limb motion at a higher precision. Our method upgrades the sensitivity to achieve detection and analysis for tremors down to 2 mm amplitude. We include empirical experiments and measurements showing improved tremor amplitude and frequency estimation using our proposed Pose and Optical Flow Fusion (POFF) algorithm.

Słowa kluczowe

EN

limb tracking; contactless; tremor amplitude; tremor frequency; temporal alignment; accelerometer; Kinect 2; marker; Parkinson's disease

PL

śledzenie kończyn; amplituda drżenia; częstotliwość drżenia; akcelerometr; choroba Parkinsona

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2020
• Tom: Vol. 40, no. 1
• Strony: 468--481
• Opis fizyczny: Bibliogr. 69 poz., rys., tab., wykr.

Twórcy

autor

Alper Mehmet Akif

• Electrical and Computer Engineering, Michigan State University, East Lansing, MI, USA

autor

Goudreau John

• Movement Disorders Sub-Specialty Clinic, Neurology, Michigan State University, East Lansing, MI, USA

autor

Daniel Morris

• Electrical and Computer Engineering, Michigan State University, East Lansing, MI, USA

Bibliografia

• [1] Deuschl G, Raethjen J, Lindemann M, Krack P. The pathophysiology of tremor. Muscle Nerve 2001.
• [2] Deuschl G, Bain P, Brin M. Consensus Statement of the Movement Disorder Society on Tremor. Mov Disord 2010.
• [3] Rovini E, Maremmani C, Cavallo F. How wearable sensors can support parkinson's disease diagnosis and treatment: a systematic review. Front Neurosci 2017.
• [4] Mao ZL, Modi NB. Dose-response analysis of the effect of carbidopa-levodopa extended-release capsules (IPX066) in levodopa-naive patients with parkinson disease. J Clin Pharmacol 2016.
• [5] Wolpe N, et al. Sensory attenuation in Parkinson's disease is related to disease severity and dopamine dose. Sci Rep 2018.
• [6] Zhu R, Zhou Z. A real-time articulated human motion tracking using tri-axis inertial/magnetic sensors package. IEEE Trans Neural Syst Rehabil Eng 2004.
• [7] Dai H, Zhang P, Lueth TC. Quantitative assessment of parkinsonian tremor based on an inertial measurement unit. Sensors (Switzerland) 2015.
• [8] Dutta D, Modak S, Kumar A, Roychowdhury J, Mandal S. Bayesian network aided grasp and grip efficiency estimation using a smart data glove for post-stroke diagnosis. Biocybern Biomed Eng 2017.
• [9] Asuroglu T, Açici K, Berke Erdas Ç, Kilinç Toprak M, Erdem H, Ogul H. Parkinson's disease monitoring from gait analysis via foot-worn sensors. Biocybern Biomed Eng 2018.
• [10] Obdrzalek S, et al. ‘‘Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population,’’ in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; 2012.
• [11] Beuter A, Titcombe MS, Richer F, Gross C, Guehl D. Effect of deep brain stimulation on amplitude and frequency characteristics of rest tremor in Parkinson's disease. Thalamus Relat Syst 2001;1(3):203–11.
• [12] Meshack RP, Norman KE. A randomized controlled trial of the effects of weights on amplitude and frequency of postural hand tremor in people with Parkinson's disease. Clin Rehabil 2002.
• [13] Shotton J, et al. Real-time human pose recognition in parts from single depth images. Stud Comput Intell 2013.
• [14] Williem, Tai Y-W, Park IK. Accurate and real-time depth video acquisition using Kinect–stereo camera fusion. Opt Eng 2014;53(4). 043110.
• [15] Niazmand K, Tonn K, Kalaras A, Fietzek UM, Mehrkens JH, Lueth TC. ‘‘Quantitative evaluation of Parkinson's disease using sensor based smart glove,’’ in Proceedings - IEEE Symposium on Computer-Based Medical Systems; 2011.
• [16] Szumilas M, Lewenstein K, ͆lubowska E. Verification of the functionality of device for monitoring human tremor. Biocybern Biomed Eng 2015.
• [17] Camara C, et al. Resting tremor classification and detection in Parkinson's disease patients. Biomed Signal Process Control 2015.
• [18] Salarian A, Russmann H, Wider C, Burkhard PR, Vingerhoets FJG, Aminian K. Quantification of tremor and bradykinesia in Parkinson's disease using a novel ambulatory monitoring system. IEEE Trans Biomed Eng 2007.
• [19] Ji L, Wang H, Zheng T, Qi X. Motion trajectory of human arms based on the dual quaternion with motion tracker. Multimed Tools Appl 2017.
• [20] del Castillo MD, Lambrecht S, Serrano JI, Benito-León J, Rocon E, Romero JP. Identification of activities of daily living in tremorous patients using inertial sensors. Expert Syst Appl 2017;83:40–8.
• [21] Gallego JA, Rocon E, Roa JO, Moreno JC, Pons JL. Real-time estimation of pathological tremor parameters from gyroscope data. Sensors 2010;10(3):2129–49.
• [22] Olivares A, Olivares G, Mula F, Górriz JM, Ramírez J. Wagyromag: wireless sensor network for monitoring and processing human body movement in healthcare applications. J Syst Archit 2011.
• [23] Bakstein E, Burgess J, Warwick K, Ruiz V, Aziz T, Stein J. Parkinsonian tremor identification with multiple local field potential feature classification. J Neurosci Methods 2012.
• [24] Joshi D, Khajuria A, Joshi P. An automatic non-invasive method for Parkinson's disease classification. Comput Methods Programs Biomed 2017.
• [25] Schaffer L, Kincses Z, Pletl S. ‘‘A real-time pose estimation algorithm based on FPGA and sensor fusion,’’ SISY 2018 - IEEE 16th int. Symp. Intell. Syst. informatics, proc; 2018;149–53.
• [26] Chang RS, Chiu JH, Chen FP, Chen JC, Yang JL. A Parkinson's disease measurement system using laser lines and a CMOS image sensor. Sensors 2011.
• [27] Yang JL, Chang RS, Chen FP, Chern CM, Chiu JH. Detection of hand tremor in patients with Parkinson's disease using a non-invasive laser line triangulation measurement method. Meas J Int Meas Confed 2016.
• [28] Grooten WJA, Sandberg L, Ressman J, Diamantoglou N, Johansson E, Rasmussen-Barr E. Reliability and validity of a novel Kinect-based software program for measuring posture, balance and side-bending. BMC Musculoskelet Disord 2018.
• [29] Diaz-Monterrosas PR, Posada-Gomez R, Martinez-Sibaja A, Aguilar-Lasserre AA, Juarez-Martinez U, Trujillo-Caballero JC. Brief review on the validity and reliability of microsoft kinect sensors for functional assessment applications. Adv Electr Comput Eng 2018.
• [30] Casacanditella L, Cosoli G, Ceravolo MG, Tomasini EP. Non-contact measurement of tremor for the characterisation of Parkinsonian individuals: comparison between Kinect and Laser Doppler vibrometer. J Phys Conf Ser 2017.
• [31] Heinrich F, Schmitz-Hübsch T, Ellermeyer T, Mansow- Model S, Lipp A. Video-based tremor analysis via Kinect® System in comparison to accelerometric and electromyographical tremor detection [abstract]. Mov Disord 2016.
• [32] Pöhlmann STL, Harkness EF, Taylor CJ, Astley SM. Evaluation of kinect 3D sensor for healthcare imaging. J Med Biol Eng 2016.
• [33] Torres R, et al. Diagnosis of the corporal movement in Parkinson's disease using kinect sensors. IFMBE Proc 2015.
• [34] Xiao D, et al. A Kinect TM camera based navigation system for percutaneous abdominal puncture. Phys Med Biol 2016.
• [35] Bieryla KA. Xbox Kinect training to improve clinical measures of balance in older adults: a pilot study. Aging Clin Exp Res 2016.
• [36] Bonnechère B, Sholukha V, Omelina L, Jan SVS, Jansen B. 3D analysis of upper limbs motion during rehabilitation exercises using the kinectTM sensor: Development, laboratory validation and clinical application. Sensors (Switzerland) 2018.
• [37] Xu X, McGorry RW, Chou LS, Hua Lin J, chi Chang C. Accuracy of the Microsoft KinectTM for measuring gait parameters during treadmill walking. Gait Posture 2015.
• [38] Wochatz M, et al. Reliability and validity of the Kinect V2 for the assessment of lower extremity rehabilitation exercises. Gait Posture 2019.
• [39] Huber ME, Seitz AL, Leeser M, Sternad D. ‘‘Validity and reliability of Kinect skeleton for measuring shoulder joint angles: a feasibility study,’’ Physiother. (United Kingdom); 2015.
• [40] Puh U, Hoehlein B, Deutsch JE. Validity and reliability of the kinect for assessment of standardized transitional movements and balance: systematic review and translation into practice. Phys Med Rehabil Clin N Am 2019.
• [41] Auvinet E, Multon F, Manning V, Meunier J, Cobb JP. Validity and sensitivity of the longitudinal asymmetry index to detect gait asymmetry using Microsoft Kinect data. Gait Posture 2017.
• [42] Reither LR, Foreman MH, Migotsky N, Haddix C, Engsberg JR. Upper extremity movement reliability and validity of the Kinect version 2. Disabil Rehabil Assist Technol 2018.
• [43] Pagliari D, Pinto L. Calibration of Kinect for Xbox one and comparison between the two generations of microsoft sensors. Sensors (Switzerland) 2015.
• [44] Sooklal S, Mohan P, Teelucksingh S. ‘‘Using the Kinect for detecting tremors: challenges and opportunities,’’ in 2014 IEEE-EMBS International Conference on Biomedical and Health Informatics, BHI 2014; 2014.
• [45] Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L. Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. Gait Posture 2014.
• [46] Ishii I. Color-histogram-based tracking at 2000 fps. J Electron Imaging 2012;21(1):013010.
• [47] Pressigout M, et al. Hybrid tracking approach using optical flow and pose estimation to cite this version : HAL Id : inria-00351874; 2009.
• [48] Krupicka R, Szabo Z, Viteckova S, Ruzicka E. Motion capture system for finger movement measurement in Parkinson disease. Radioengineering 2014.
• [49] Wei X, Zhang P, Chai J. Accurate realtime full-body motion capture using a single depth camera. ACM Trans Graph 2012.
• [50] Soran B, Lee JHS, Shapiro L. Tremor detection using motion filtering and SVM. Proc IAPR Int Conf Pattern Recogn 2012.
• [51] Chen KH, Lin PC, Chen YJ, Yang BS, Lin CH. Development of method for quantifying essential tremor using a small optical device. J Neurosci Methods 2016.
• [52] Blumrosen G, Uziel M, Rubinsky B, Porrat D. Noncontact tremor characterization using low-power wideband radar technology. IEEE Trans Biomed Eng 2012.
• [53] Kondori FA, Yousefi S, Li H. Direct three-dimensional head pose estimation from Kinect-type sensors. Electron Lett 2014.
• [54] Hafeez A. Object Recognition Through Kinect Using Harris Transform 2014;2(June):420–4.
• [55] Fabian J, Young T, Jones JCP, Clayton GM. Integrating the microsoft kinect with simulink: real-time object tracking example. IEEE ASME Trans Mechatron 2014.
• [56] Stone EE, Skubic M. Fall detection in homes of older adults using the microsoft kinect. IEEE J Biomed Heal Informatics 2015.
• [57] Henry P, Krainin M, Herbst E, Ren X, Fox D. RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Rob Res 2012;31(5):647–63.
• [58] Kotovsky J, Rosen M. A wearable tremor-suppression orthosis. J Rehabil Res Dev 1998;35(4):373–87.
• [59] Liu C, Freeman WT, Adelson EH, Weiss Y. ‘‘Human-assisted motion annotation,’’ in 26th IEEE conference on Computer Vision and Pattern Recognition, CVPR; 2008.
• [60] Mcauley JH, Marsden CD. Physiological and pathological tremors and rhythmic central motor control. Brain 2000.
• [61] Burkhard PR, Shale H, Langston JW, Tetrud JW. Quantification of dyskinesia in Parkinson's disease: validation of a novel instrumental method. Mov Disord 1999;14(5):754–63.
• [62] Burg JP. ‘‘Maximum entropy spectral analysis,’’ proc. 37th meet. Soc. Explor. Geophys., vol. Oklahoma C; 1967.
• [63] Nichols JK, Sena MP, Hu JL, O'Reilly OM, Feeley BT, Lotz JC. A Kinect-based movement assessment system: marker position comparison to Vicon. Comput Methods Biomech Biomed Engin 2017.
• [64] Fischer RA, Fisher RA. The use of multiple measurements in taxonomic problems. Ann Eugen 1936.
• [65] Gonzalez-Jorge H, et al. Metrological comparison between Kinect i and Kinect II sensors. Meas J Int Meas Confed 2015;70:21–6.
• [66] Xu X, McGorry RW. The validity of the first and second generation Microsoft Kinect for identifying joint center locations during static postures. Appl Ergon 2015.
• [67] Calzetti S, Baratti M, Gresty M, Findley L. Frequency/ amplitude characteristics of postural tremor of the hands in a population of patients with bilateral essential tremor: implications for the classification and mechanism of essential tremor. J Neurol Neurosurg Psychiatry 1987;50 (5):561–7.
• [68] Yang K, et al. Objective and quantitative assessment of motor function in Parkinson's disease—from the perspective of practical applications. Ann Transl Med 2016;4(5):90.
• [69] Debella-Gilo M, Kääb A. Sub-pixel precision image matching for measuring surface displacements on mass movements using normalized cross-correlation. Remote Sens Environ 2011;115(1):130–42.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).