Advancing blood glucose prediction with neural architecture search and deep reinforcement learning for type 1 diabetics

• Autorzy: Domanski Peter , Ray Aritra , Lafata Kyle , Firouzi Farshad , Chakrabarty Krishnendu , Pflüger Dirk

Identyfikatory

DOI

10.1016/j.bbe.2024.07.006

• Języki publikacji: EN

Abstrakty

EN

For individuals with Type-1 diabetes mellitus, accurate prediction of future blood glucose values is crucial to aid its regulation with insulin administration, tailored to the individual’s specific needs. The authors propose a novel approach for the integration of a neural architecture search framework with deep reinforcement learning to autonomously generate and train architectures, optimized for each subject over model size and analytical prediction performance, for the blood glucose prediction task in individuals with Type-1 diabetes. The authors evaluate the proposed approach on the OhioT1DM dataset, which includes blood glucose monitoring records at 5-min intervals over 8 weeks for 12 patients with Type-1 diabetes mellitus. Prior work focused on predicting blood glucose levels in 30 and 45-min prediction horizons, equivalent to 6 and 9 data points, respectively. Compared to the previously achieved best error, the proposed method demonstrates improvements of 18.4 % and 22.5 % on average for mean absolute error in the 30-min and 45-min prediction horizons, respectively, through the proposed deep reinforcement learning framework. Using the deep reinforcement learning framework, the best-case and worst-case analytical performance measured over root mean square error and mean absolute error was obtained for subject ID 570 and subject ID 584, respectively. Models optimized for performance on the prediction task and model size were obtained after implementing neural architecture search in conjunction with deep reinforcement learning on these two extreme cases. The authors demonstrate improvements of 4.8 % using Long Short Term Memory-based architectures and 5.7 % with Gated Recurrent Units-based architectures for patient ID 570 on the analytical prediction performance by integrating neural architecture search with deep reinforcement learning framework. The patient with the lowest performance (ID 584) on the deep reinforcement learning method had an even greater performance boost, with improvements of 10.0 % and 12.6 % observed for the Long Short-Term Memory and Gated Recurrent Units, respectively. The subject-specific optimized models over performance and model size from the neural architecture search in conjunction with deep reinforcement learning had a reduction in model size which ranged from 20 to 150 times compared to the model obtained using only the deep reinforcement learning method. The smaller size, indicating a reduction in model complexity in terms of the number of trainable network parameters, was achieved without a loss in the prediction performance.

Słowa kluczowe

EN

deep reinforcement learning; neural architecture search; blood glucose prediction; type 1 diabetes

PL

uczenie przez wzmacnianie; przewidywanie poziomu glukozy; cukrzyca typu 1

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2024
• Tom: Vol. 44, no. 3
• Strony: 481--500
• Opis fizyczny: Bibliogr. 66 poz., rys., tab., wykr.

Twórcy

autor

Domanski Peter

• University of Stuttgart, Universitätsstraße 38, Stuttgart, 70569, Baden-Württemberg, Germany

autor

Ray Aritra

• Duke University, 534 Research Drive, Durham, 27705, NC, United States

autor

Lafata Kyle

• Duke University, 534 Research Drive, Durham, 27705, NC, United States

autor

Firouzi Farshad

• Duke University, 534 Research Drive, Durham, 27705, NC, United States

autor

Chakrabarty Krishnendu

• Arizona State University, 699 S Mill Avenue, Tempe, 85281, AZ, United States

autor

Pflüger Dirk

• University of Stuttgart, Universitätsstraße 38, Stuttgart, 70569, Baden-Württemberg, Germany

Bibliografia

• [1] American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care 2014;37(Supplement_1):S81-90.
• [2] Deshpande AD, Harris-Hayes M, Schootman M. Epidemiology of diabetes and diabetes-related complications. Phys Ther 2008;88(11):1254-64.
• [3] Ogurtsova K, Guariguata L, Barengo NC, Ruiz PL, Sacre JW, Karuranga S, et al. IDF diabetes atlas: Global estimates of undiagnosed diabetes in adults for 2021. Diabetes Res Clin Pract 2022;183:109118.
• [4] Elsayed N, ElSayed Z, Ozer M. Early stage diabetes prediction via extreme learning machine. In: SoutheastCon 2022. IEEE; 2022, p. 374-9.
• [5] Dudukcu HV, Taskiran M, Yildirim T. Blood glucose prediction with deep neural networks using weighted decision level fusion. Biocybern Biomed Eng 2021;41(3):1208-23.
• [6] Bhimireddy A, Sinha P, Oluwalade B, Gichoya JW, Purkayastha S. Blood glucose level prediction as time-series modeling using sequence-to-sequence neural networks. In: CEUR workshop proceedings.
• [7] Zhu T, Li K, Chen J, Herrero P, Georgiou P. Dilated recurrent neural networks for glucose forecasting in type 1 diabetes. J Healthc Inform Res 2020;4:308-24.
• [8] Yang T, Wu R, Tao R, Wen S, Ma N, Zhao Y, et al. Multi-scale long short-term memory network with multi-lag structure for blood glucose prediction. In: KDH@ECAI, vol. 45, 2020, p. 136-40.
• [9] Freiburghaus J, Rizzotti A, Albertetti F. A deep learning approach for blood glucose prediction of type 1 diabetes. In: Proceedings of the proceedings of the 5th international workshop on knowledge discovery in healthcare data co-located with 24th European conference on artificial intelligence. 2020.
• [10] Bevan R, Coenen F. Experiments in non-personalized future blood glucose level prediction. In: CEUR workshop proceedings, vol. 2675, 2020, p. 100-4.
• [11] Gu K, Dang R, Prioleau T. Neural physiological model: A simple module for blood glucose prediction. In: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society. IEEE; 2020, p. 5476-81.
• [12] Hameed H, Kleinberg S. Investigating potentials and pitfalls of knowledge distillation across datasets for blood glucose forecasting. In: Proceedings of the 5th annual workshop on knowledge discovery in healthcare data. 2020.
• [13] Cui R, Hettiarachchi C, Nolan CJ, Daskalaki E, Suominen H. Personalised short-term glucose prediction via recurrent self-attention network. In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems. IEEE; 2021, p. 154-9.
• [14] Nemat H, Khadem H, Eissa MR, Elliott J, Benaissa M. Blood glucose level prediction: advanced deep-ensemble learning approach. IEEE J Biomed Health Inform 2022;26(6):2758-69.
• [15] Shuvo MM, Islam SK. Deep multitask learning by stacked long short-term memory for predicting personalized blood glucose concentration. IEEE J Biomed Health Inform 2023;27(3):1612-23.
• [16] Marling C, Bunescu R. The OhioT1DM dataset for blood glucose level prediction: update 2020. In: CEUR workshop proceedings, vol. 2675, NIH Public Access; 2020, p. 71.
• [17] Benidis K, Rangapuram SS, Flunkert V, Wang Y, Maddix D, Turkmen C, et al. Deep learning for time series forecasting: Tutorial and literature survey. ACM Comput Surv 2022;55(6):1-36.
• [18] Tang S, Pan Y. Feature extraction via recurrent random deep ensembles and its application in gruop-level happiness estimation. 2017, arXiv preprint arXiv:1707.09871.
• [19] Keren G, Schuller B. Convolutional RNN: An enhanced model for extracting features from sequential data. In: 2016 International joint conference on neural networks. (IJCNN), IEEE; 2016, p. 3412-9.
• [20] Fu Q, Han Z, Chen J, Lu Y, Wu H, Wang Y. Applications of reinforcement learning for building energy efficiency control: A review. J Build Eng 2022;50:104165.
• [21] Domanski P, Ray A, Firouzi F, Lafata K, Chakrabarty K, Pflüger D. Blood glucose prediction for type-1 diabetics using deep reinforcement learning. In: 2023 IEEE international conference on digital health. IEEE; 2023, p. 339-47.
• [22] Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019, p. 2623-31.
• [23] Klonoff DC, Lias C, Vigersky R, Clarke W, Parkes JL, Sacks DB, et al. The surveillance error grid. J Diabetes Sci Technol 2014;8(4):658-72.
• [24] Dudukcu HV, Taskiran M, Yildirim T. Consolidated or individual training: which one is better for blood glucose prediction?. In: 2021 International conference on innovations in intelligent systems and applications. IEEE; 2021, p. 1-6.
• [25] Francescato MP, Geat M, Fusi S, Stupar G, Noacco C, Cattin L. Carbohydrate requirement and insulin concentration during moderate exercise in type 1 diabetic patients. Metabolism 2004;53(9):1126-30.
• [26] Zhu T, Li K, Kuang L, Herrero P, Georgiou P. An insulin bolus advisor for type 1 diabetes using deep reinforcement learning. Sensors 2020;20(18):5058.
• [27] Zhu T, Li K, Herrero P, Chen J, Georgiou P. A deep learning algorithm for personalized blood glucose prediction. In: KHD@ IJCAI. 2018, p. 64-78.
• [28] Ubl M, Koutny T, Della Cioppa A, De Falco I, Tarantino E, Scafuri U. Distributed assessment of virtual insulin-pump settings using smartcgms and dmms. r for diabetes treatment. Sensors 2022;22(23):9445.
• [29] Man CD, Micheletto F, Lv D, Breton M, Kovatchev B, Cobelli C. The UVA/PADOVA type 1 diabetes simulator: new features. J Diabetes Sci Technol 2014;8(1):26-34.
• [30] Kovatchev BP, Breton M, Man CDalla, Cobelli C. In silico preclinical trials: A proof of concept in closed-loop control of type 1 diabetes.
• [31] Colmegna P, Bisio A, McFadden R, Wakeman C, Oliveri MC, Nass R, et al. Evaluation of a web-based simulation tool for self-management support in type 1 diabetes: A pilot study. IEEE J Biomed Health Inform 2022;27(1):515-25.
• [32] Xie J. [dataset] jxx123/simglucose. GitHub. 2021. Available from: https://github.com/jxx123/simglucose.
• [33] Hettiarachchi C. [dataset] chirathyh/GluCoEnv. GitHub. 2024 [cited 2024 Feb1]. Available from: https://github.com/chirathyh/GluCoEnv.
• [34] [dataset] What is OpenAPS? - OpenAPS.org. Openaps.org. 2018. Available from: https://openaps.org/what-is-openaps/.
• [35] [dataset] Tidepool. www.tidepool.org. Available from: https://www.tidepool.org/.
• [36] Broll S, Urbanek J, Buchanan D, Chun E, Muschelli J, Punjabi NM, et al. Interpreting blood glucose data with R package iglu. PLoS One 2021;16(4):e0248560.
• [37] Colás A, Vigil L, Vargas B, Cuesta-Frau D, Varela M. Detrended fluctuation analysis in the prediction of type 2 diabetes mellitus in patients at risk: Model optimization and comparison with other metrics. PLoS One 2019;14(12):e0225817.
• [38] Dubosson F, Ranvier JE, Bromuri S, Calbimonte JP, Ruiz J, Schumacher M. The open D1NAMO dataset: A multi-modal dataset for research on non-invasive type 1 diabetes management. Inform Med Unlocked 2018;13:92-100.
• [39] Hall H, Perelman D, Breschi A, Limcaoco P, Kellogg R, McLaughlin T, et al. Glucotypes reveal new patterns of glucose dysregulation. PLoS Biol 2018;16(7):e2005143.
• [40] Weinstock RS, DuBose SN, Bergenstal RM, Chaytor NS, Peterson C, Olson BA, et al. Risk factors associated with severe hypoglycemia in older adults with type 1 diabetes. Diabetes Care 2016;39(4):603-10.
• [41] Neinstein A, Wong J, Look H, Arbiter B, Quirk K, McCanne S, et al. A case study in open source innovation: developing the tidepool platform for interoperability in type 1 diabetes management. J Am Med Inform Assoc 2016;23(2):324-32.
• [42] Rubin-Falcone H, Fox I, Wiens J. Deep residual time-series forecasting: application to blood glucose prediction. In: KDH@ ECAI. 2020, p. 105-9.
• [43] Ma N, Zhao Y, Wen S, Yang T, Wu R, Tao R, et al. Online blood glucose prediction using autoregressive moving average model with residual compensation network. In: KDH@ ECAI. 2020, p. 151-5.
• [44] Daniels J, Herrero P, Georgiou P. Personalised glucose prediction via deep multitask networks. In: KDH@ ECAI. 2020, p. 110-4.
• [45] Sun X, Rashid MM, Sevil M, Hobbs N, Brandt R, Askari MR, et al. Prediction of blood glucose levels for people with type 1 diabetes using latent-variable-based model. In: KDH@ ECAI. 2020, p. 115-9.
• [46] Pavan J, Prendin F, Meneghetti L, Cappon G, Sparacino G, Facchinetti A, et al. Personalized machine learning algorithm based on shallow network and error imputation module for an improved blood glucose prediction. In: KDH@ ECAI. 2020, p. 95-9.
• [47] Zhu T, Yao X, Li K, Herrero P, Georgiou P. Blood glucose prediction for type 1 diabetes using generative adversarial networks. In: CEUR workshop proceedings. vol. 2675, 2020, p. 90-4.
• [48] Clarke WL. The original clarke error grid analysis (EGA). Diabetes Technol Therapeutics 2005;7(5):776-9.
• [49] Zhou T, Ye X, Lu H, Zheng X, Qiu S, Liu Y. Dense convolutional network and its application in medical image analysis. BioMed Res Int 2022;2022.
• [50] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735-80.
• [51] Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.
• [52] Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. 2014, arXiv preprint arXiv:1406.1078.
• [53] Chung J, Gulcehre C, Cho K, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. 2014, arXiv preprint arXiv:1412.3555.
• [54] Sutton RS, Barto AG. Reinforcement learning: An introduction. MIT Press; 2018.
• [55] Liu T, Tan Z, Xu C, Chen H, Li Z. Study on deep reinforcement learning techniques for building energy consumption forecasting. Energy Build 2020;208:109675.
• [56] Haarnoja T, Zhou A, Abbeel P, Levine S. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International conference on machine learning. PMLR; 2018, p. 1861-70.
• [57] Konda V, Tsitsiklis J. Actor-critic algorithms. In: Advances in neural information processing systems, 1999, p. 12.
• [58] He X, Zhao K, Chu X. AutoML: A survey of the state-of-the-art. Knowl-Based Syst 2021;212:106622.
• [59] Waring J, Lindvall C, Umeton R. Automated machine learning: Review of the state-of-the-art and opportunities for healthcare. Artif Intell Med 2020;104:101822.
• [60] Elsken T, Metzen JH, Hutter F. Neural architecture search: A survey. J Mach Learn Res 2019;20(1):1997-2017.
• [61] Bergenstal RM, Gal RL, Connor CG, Gubitosi-Klug R, Kruger D, Olson BA, et al. Racial differences in the relationship of glucose concentrations and hemoglobin A1c levels. Ann Internal Med 2017;167(2):95-102.
• [62] Beck RW, Connor CG, Mullen DM, Wesley DM, Bergenstal RM. The fallacy of average: how using HbA1c alone to assess glycemic control can be misleading. Diabetes Care 2017;40(8):994-9.
• [63] Peters AL. Role of continuous glucose monitoring in diabetes treatment. Arlington (VA): American Diabetes Association; 2018.
• [64] D’Antoni F, Petrosino L, Marchetti A, Bacco L, Pieralice S, Vollero L, et al. Layered meta-learning algorithm for predicting adverse events in type 1 diabetes. IEEE Access 2023;11:9074-94.
• [65] Staal OM, Sælid S, Fougner A, Stavdahl Ø. Kalman smoothing for objective and automatic preprocessing of glucose data. IEEE J Biomed Health Inform 2018;23(1):218-26.
• [66] Facchinetti A, Del Favero S, Sparacino G, Cobelli C. Modeling transient disconnections and compression artifacts of continuous glucose sensors. Diabetes Technol Therapeutics 2016;18(4):264-72.