Data size considerations and hyperparameter choices in case-based reasoning approach to glucose prediction

• Autorzy: Zulj Sara , Carvalho Paulo , Ribeiro Rogério T. , Andrade Rita , Magjarevic Ratko

Identyfikatory

DOI

10.1016/j.bbe.2021.04.013

• Języki publikacji: EN

Abstrakty

EN

Machine learning algorithms have become popular in diabetes research, especially within the scope of glucose prediction from continuous glucose monitoring (CGM) data. We investigated the design choices in case-based reasoning (CBR) approach to glucose prediction from the CGM data. Design choices were made with regards to the distance function (city-block, Euclidean, cosine, Pearson’s correlation), number of observations, and adaptation of the solution (average, weighted average, linear regression) used in the model, and were evaluated using five-fold cross-validation to establish the impact of each choice to the prediction error. Our best models showed mean absolute error of 13.35 ± 3.04 mg/dL for prediction horizon PH = 30 min, and 30.23 ± 6.50 mg/dL for PH = 60 min. The experiments were performed using the data of 20 subjects recorded in free-living conditions. The problem of using small datasets to test blood glucose prediction models and assess the prediction error of the model was also addressed in this paper. We proposed for the first time the methodology for estimation of the impact of the number of subjects (i.e., dataset size) on the distribution of the prediction error of the model. The proposed methodology is based on Monte Carlo cross-validation with the systematic reduction of subjects in the dataset. The implementation of the methodology was used to gauge the change in the prediction error when the number of subjects in the dataset increases, and as such allows the projection on the prediction error in case the dataset is extended with new subjects.

Słowa kluczowe

EN

glucose prediction; case based reasoning; instance based learning; dataset size; Monte Carlo cross validation

PL

stężenie glukozy; zbiór danych; walidacja krzyżowa

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2021
• Tom: Vol. 41, no. 2
• Strony: 733--745
• Opis fizyczny: Bibliogr. 39 poz., tab., wykr.

Twórcy

autor

Zulj Sara

• University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia

autor

Carvalho Paulo

• University of Coimbra, Faculty of Science and Technology, Coimbra, Portugal

autor

Ribeiro Rogério T.

• University of Aveiro, Department of Medical Sciences, Aveiro, Portugal

autor

Andrade Rita

• APDP – Diabetes Portugal, Education and Research Centre (APDP-ERC), Lisbon, Portugal

autor

Magjarevic Ratko

• University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia

Bibliografia

• [1] Daneman D. Type 1 diabetes. Lancet 2006;367(9513):847–58. https://doi.org/10.1016/S0140-6736(06)68341-4.
• [2] Klonoff DC, Ahn D, Drincic A. Continuous glucose monitoring: A review of the technology and clinical use. Diabetes Res Clin Pract 2017;133:178–92. https://doi.org/ 10.1016/j.diabres.2017.08.005.
• [3] Woldaregay AZ, Ðrsand E, Walderhaug S, Albers D, Mamykina L, Botsis T, et al. Data-driven modeling and prediction of blood glucose dynamics: Machine learning applications in type 1 diabetes. Artif Intell Med 2019;98:109–34. https://doi. org/10.1016/j.artmed.2019.07.007.
• [4] Gadaleta M, Facchinetti A, Grisan E, Rossi M. Prediction of adverse glycemic events from continuous glucose monitoring signal. IEEE J Biomed Heal Informatics 2019;23(2):650–9. https://doi.org/10.1109/JBHI.622102010.1109/ JBHI.2018.2823763.
• [5] Alfian G, Syafrudin M, Anshari M, Benes F, Atmaji FTD, Fahrurrozi I, et al. Blood glucose prediction model for type 1 diabetes based on artificial neural network with time-domain features. Biocybern Biomed Eng 2020;40(4):1586–99. https:// doi.org/10.1016/j.bbe.2020.10.004.
• [6] Li K, Liu C, Zhu T, Herrero P, Georgiou P. GluNet: A deep learning framework for accurate glucose forecasting. IEEE J Biomed Heal Informatics 2020;24(2):414–23. https://doi.org/ 10.1109/JBHI.622102010.1109/JBHI.2019.2931842.
• [7] Allam F, Nossai Z, Gomma H, Ibrahim I, Abdelsalam M. A recurrent neural network approach for predicting glucose concentration in type-1 diabetic patients. In: IFIP Adv. Inf. Commun. Technol.. AICT, Springer New York LLC; 2011. p. 254–9. 10.1007/978-3-642-23957-1_29.
• [8] Li K, Daniels J, Liu C, Herrero P, Georgiou P. Convolutional recurrent neural networks for glucose prediction. IEEE J Biomed Heal Informatics 2020;24(2):603–13. https://doi.org/ 10.1109/JBHI.2019.2908488.
• [9] Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine learning-based prediction models in healthcare. WIREs Data Min Knowl Discov 2020;10(5). https://doi.org/10.1002/widm.1379.
• [10] Towards trustable machine learning. Nat Biomed Eng 2018;2:709–10. 10.1038/s41551-018-0315-x.
• [11] Aamodt A, Plaza E. Case-based reasoning: foundational issues, methodological variations, and system approaches, vol. 7. AI Communications. IOS Press; 1994.
• [12] Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak 2020;20:310. https://doi.org/10.1186/s12911-020-01332-6.
• [13] Hidalgo JI, Colmenar JM, Kronberger G, Winkler SM, Garnica O, Lanchares J. Data based prediction of blood glucose concentrations using evolutionary methods. J Med Syst 2017;41:1–20. https://doi.org/10.1007/s10916-017-0788-2.
• [14] Man CD, Micheletto F, Lv D, Breton M, Kovatchev B, Cobelli C. The UVA/PADOVA type 1 diabetes simulator. J Diabetes Sci Technol 2014;8(1):26–34. https://doi.org/10.1177/ 1932296813514502.
• [15] Agar BU, Eren M, Cinar A. GLUCOSIM: Educational software for virtual experiments with patients with type 1 diabetes. Annu Int Conf IEEE Eng Med Biol - Proc 2005;vol. 7 VOLS:845–8. https://doi.org/10.1109/iembs.2005.1616547.
• [16] Lehmann ED, Deutsch T, Carson ER, So¨nksen PH. AIDA: an interactive diabetes advisor. Comput Methods Programs Biomed 1994;41(3-4):183–203. https://doi.org/10.1016/0169- 2607(94)90054-X.
• [17] Zecchin C, Facchinetti A, Sparacino G, Cobelli C. How much is short-term glucose prediction in type 1 diabetes improved by adding insulin delivery and meal content information to CGM data? A proof-of-concept study. J Diabetes Sci Technol 2016;10(5):1149–60. https://doi.org/10.1177/ 1932296816654161.
• [18] Böhm A-K, Jensen ML, Sørensen MR, Stargardt T. Real-world evidence of user engagement with mobile health for diabetes management: Longitudinal observational study. JMIR MHealth UHealth 2020;8(11):e22212. https://doi.org/10.2196/ 22212.
• [19] Garg Y. Multi-variate time series similarity measures and their robustness. Against Temporal Asynchrony 2015.
• [20] Dankers FJWM, Traverso A, Wee L, van Kuijk SMJ. Prediction modeling methodology. In: Fundam. Clin. Data Sci.. Springer International Publishing; 2019. p. 101–20. 10.1007/978-3-319-99713-1_8.
• [21] Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Int Jt Conf Artif Intell 1995.
• [22] Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: A comparison of resampling methods. Bioinformatics 2005;21(15):3301–7. https://doi.org/10.1093/ bioinformatics/bti499.
• [23] Kim J-H. Estimating classification error rate: Repeated crossvalidation, repeated hold-out and bootstrap. Comput Stat Data Anal 2009;53(11):3735–45. https://doi.org/10.1016/j. Csda.2009.04.009.
• [24] Lendasse A, Wertz V, Verleysen M. Model selection with cross-validations and bootstraps - Application to time series prediction with RBFN models. Lect Notes Comput Sci (Including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 2003;2714:573–80. https://doi.org/10.1007/3- 540-44989-2_68.
• [25] Saeb S, Lonini L, Jayaraman A, Mohr DC, Kording KP. The need to approximate the use-case in clinical machine learning. GigaScience 2017;6:1–9. https://doi.org/10.1093/ gigascience/gix019.
• [26] Vanwinckelen G, Blockeel H, Hämäläinen W, Petitjean F, Webb G. Look before you leap: Some insights into learner evaluation with cross-validation. Vol. 1. 2014.
• [27] Arlot S, Celisse A. A survey of cross-validation procedures for model selection. Stat Surv 2010;4:40–79. https://doi.org/ 10.1214/09-SS054.
• [28] Zulj S, Carvalho P, Ribeiro R, Magjarevic R. Handling Missing Data in CGM Records. In: Lin K-P, Magjarevic R, de Carvalho P, editors. Futur. Trends Biomed. Heal. Informatics Cybersecurity Med. Devices, Cham: Springer International Publishing; 2020, p. 420–7. 10.1007/978-3-030-30636-6_57.
• [29] Battelino T, Danne T, Bergenstal RM, Amiel SA, Beck R, Biester T, et al. Clinical targets for continuous glucose monitoring data interpretation: Recommendations from the international consensus on time in range. Diabetes Care 2019;42:1593–603. https://doi.org/10.2337/dci19-0028.
• [30] Reifman J, Rajaraman S, Gribok A, Ward WK. Predictive monitoring for improved management of glucose levels. J Diabetes Sci Technol 2007;1(4):478–86. https://doi.org/ 10.1177/193229680700100405.
• [31] Georga EI, Prı´ncipe JC, Fotiadis DI. Short-term prediction of glucose in type 1 diabetes using kernel adaptive filters. Med Biol Eng Comput 2019;57(1):27–46. https://doi.org/10.1007/ s11517-018-1859-3.
• [32] Eren-Oruklu M, Cinar A, Quinn L, Smith D. Estimation of future glucose concentrations with subject-specific recursive linear models. Diabetes Technol Ther 2009;11(4):243–53. https://doi.org/10.1089/dia.2008.0065.
• [33] MATLAB. version 9.7.0 (R2019b). Natick, Massachusetts: The MathWorks Inc.; 2020.
• [34] Zanderigo F, Sparacino G, Kovatchev B, Cobelli C. Glucose prediction algorithms from continuous monitoring data: assessment of accuracy via continuous glucose error-grid analysis. J Diabetes Sci Technol 2007;1(5):645–51. https://doi. Org/10.1177/193229680700100508.
• [35] Facchinetti A, Sparacino G, Guerra S, Luijf YM, DeVries JH, Mader JK, et al. Real-time improvement of continuous glucose monitoring accuracy: The smart sensor concept. Diabetes Care 2013;36(4):793–800. https://doi.org/10.2337/ dc12-0736.
• [36] Facchinetti A, Sparacino G, Trifoglio E, Cobelli C. A new index to optimally design and compare continuous glucose monitoring glucose prediction algorithms. Diabetes Technol Ther 2011;13(2):111–9. https://doi.org/10.1089/dia.2010.0151.
• [37] Clarke WL. The original clarke error grid analysis (EGA). Diabetes Technol Ther 2005;7(5):776–9. https://doi.org/ 10.1089/dia.2005.7.776.
• [38] Zecchin C, Facchinetti A, Sparacino G, Cobelli C. Jump neural network for online short-time prediction of blood glucose from continuous monitoring sensors and meal information. Comput Methods Programs Biomed 2014;113(1):144–52. https://doi.org/10.1016/j.cmpb.2013.09.016.
• [39] Rodríguez-Rodríguez I, Chatzigiannakis I, Rodríguez J-V, Maranghi M, Gentili M, Zamora-Izquierdo M-Á . Utility of big data in predicting short-term blood glucose levels in type 1 diabetes mellitus through machine learning techniques. Sensors (Switzerland) 2019;19.;19(20):4482. https://doi.org/ 10.3390/s19204482.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).