Skip to main content

Risk factors for adverse outcomes during mechanical ventilation of 1152 COVID-19 patients: a multicenter machine learning study with highly granular data from the Dutch Data Warehouse

Abstract

Background

The identification of risk factors for adverse outcomes and prolonged intensive care unit (ICU) stay in COVID-19 patients is essential for prognostication, determining treatment intensity, and resource allocation. Previous studies have determined risk factors on admission only, and included a limited number of predictors. Therefore, using data from the highly granular and multicenter Dutch Data Warehouse, we developed machine learning models to identify risk factors for ICU mortality, ventilator-free days and ICU-free days during the course of invasive mechanical ventilation (IMV) in COVID-19 patients.

Methods

The DDW is a growing electronic health record database of critically ill COVID-19 patients in the Netherlands. All adult ICU patients on IMV were eligible for inclusion. Transfers, patients admitted for less than 24 h, and patients still admitted at time of data extraction were excluded. Predictors were selected based on the literature, and included medication dosage and fluid balance. Multiple algorithms were trained and validated on up to three sets of observations per patient on day 1, 7, and 14 using fivefold nested cross-validation, keeping observations from an individual patient in the same split.

Results

A total of 1152 patients were included in the model. XGBoost models performed best for all outcomes and were used to calculate predictor importance. Using Shapley additive explanations (SHAP), age was the most important demographic risk factor for the outcomes upon start of IMV and throughout its course. The relative probability of death across age values is visualized in Partial Dependence Plots (PDPs), with an increase starting at 54 years. Besides age, acidaemia, low P/F-ratios and high driving pressures demonstrated a higher probability of death. The PDP for driving pressure showed a relative probability increase starting at 12 cmH2O.

Conclusion

Age is the most important demographic risk factor of ICU mortality, ICU-free days and ventilator-free days throughout the course of invasive mechanical ventilation in critically ill COVID-19 patients. pH, P/F ratio, and driving pressure should be monitored closely over the course of mechanical ventilation as risk factors predictive of these outcomes.

Background

Since December 2019, coronavirus disease 2019 (COVID-19) has quickly spread around the world [1]. Many countries have experienced high mortality rates and overburdened intensive care units (ICUs) [2]. Although many COVID-19 registries have improved our understanding of patient characteristics upon ICU admission [3,4,5], much remains to be elucidated about the predictors of mortality and length of stay in critically ill COVID-19 patients. In particular, a better understanding of these predictors could aid clinicians in the prognosis of critically ill patients and may aid policy-makers and medical professionals in optimizing resource allocation. This is of pivotal importance at the time of possible ICU admission, but also throughout the entire course of ICU treatment.

Currently, multicenter and ICU-tailored predictive modeling is scarce for COVID-19 patients. Prognostication in COVID-19 has largely centered around severity of disease, ICU admission, need for mechanical ventilation, length of stay and mortality in the general hospital population [6,7,8]. In addition, ICU-specific models often fail to incorporate the wide variety of dedicated ICU therapies such as mechanical ventilation or high-risk medication. Furthermore, many of these models are single center and are frequently limited to risk factors at ICU admission, while COVID-19 often requires lengthy intensive care stays. Lastly, many prognostication models lacked adherence to established documentation guidelines and principles of data sharing to improve reproducibility of predictive studies [6]. Overall, we identified a gap for reproducible, multicenter predictive models in the ICU that include ICU-specific predictors over time.

In this study, we aim to identify the risk factors for intensive care mortality, ICU-free days and ventilator-free days throughout the duration of invasive mechanical ventilation (IMV), focusing on the first, 7th and 14th day after intubation. For these analyses, it is essential to capture all available data throughout ICU admission. We therefore relied on the Dutch Data Warehouse (DDW), a large, observational, multicenter collaboration uniting 66 out of 81 intensive care units in the Netherlands [9]. Our hypothesis is that ICU treatment characteristics become more important as predictors of outcome throughout the course of IMV.

Methods

The Medical Ethics Committee at Amsterdam UMC, location Vrije Universiteit medical center (VUmc) waived the need for patient informed consent and approved an opt-out procedure for the collection of COVID-19 patient data during the COVID-19 crisis. This report follows the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guideline [10].

Source of data

The DDW is coordinated by Amsterdam UMC and is supported by the Dutch Society for Intensive Care (NVIC) and the Foundation for National Intensive Care Evaluation (NICE). The highly granular data warehouse is continuously expanding and currently contains over 400 million data points combining electronic health record (EHR) data from 25 hospitals on 2382 critically ill COVID-19 patients throughout their ICU treatment. A more detailed description of the DDW, the data ingestion and the data preprocessing has been published previously [9]. In brief, data were extracted in the highest frequency available, routine hourly or bihourly measurements, or at least multiple measurements per day. Data were pseudonymized in the participating hospitals. Because of the variation in parameter names between hospitals, each parameter name from a hospital was mapped to a list of predefined parameter names. Data entry errors were filtered, and derived parameters were added. The continuous data validation process included checkpoints for correct mapping, source hospital verification and distribution plots of all used parameters. The resulting data are available for researchers and clinicians within ethical and legal boundaries [11].

Patient population

All adult intensive care patients with COVID-19 on invasive mechanical ventilation from the participating hospitals were included in this study. Additional file 1: Fig. S1 outlines the patient selection process. Patients were admitted between the beginning of the crisis in March 2020 until the 23rd of January 2021. COVID-19 was defined as a positive real-time reverse transcriptase polymerase chain reaction (RT-PCR) assay for SARS-CoV-2 or a COVID-19 Reporting and Data System (CO-RADS) score and clinical suspicion with no obvious other cause of respiratory distress [12]. Patients still admitted at data extraction as well as transfers were excluded since their outcomes are unknown. Admissions lasting less than 24 h were removed because they lack sufficient data for modeling.

Outcomes and predictors

The primary outcomes of the study were intensive care mortality, ICU-free days in 30 days and ventilator-free days in 30 days [13]. The ICU-free days describe the number of days a patient is alive and outside of the ICU in the first 30 days after prediction. Similarly, ventilator-free days describe the days patients are alive and without invasive mechanical ventilation within the first 30 days after prediction. By definition, both outcomes were set to 0 for patients who died in the ICU within the 30-day time window.

To streamline the nomenclature from the statistical and machine learning field, all independent variables, also known as features, included in the modeling are referred to as predictors. All items from the Simplified Acute Physiology Score II (SAPSII) and sequential organ failure assessment (SOFA) score were included as predictors [14, 15]. For the ventilatory predictors, all variables from the landmark paper relating driving pressure to survival by Amato et al. were included [16]. A team of 3 experienced intensive care clinicians reviewed the list and added potentially relevant predictors. These included structured comorbidity data routinely collected for the Dutch National Intensive Care Evaluation (NICE), as well as information regarding the patient positioning and ventilation characteristics. Notably, fluid balance and the total equivalent dose of vasopressors and steroids administered were included. Finally, the length of intubation for each observation, in days, was also used as a predictor. A full list of predictors, medications, comorbidities and their definitions can be found in Additional file 1: Table S1.

Modeling

Up to three observations were constructed for each patient depending on their length of stay, averaging the available predictor values in the 24 h preceding 1 day, 7 days, and 14 days of IMV. This process is illustrated in Additional file 1: Fig. S2. ICU mortality was modeled as a classification problem with a decision tree, logistic regression and XGBoost algorithm to investigate the performance of both simple and complex linear and non-linear models. Ventilator and ICU-free days were treated as regression problems with a Lasso and Ridge linear model, as well as an XGBoost regressor. For every outcome and every algorithm, a single model was fit on data points from all days.

Overall model performance was evaluated using the area under the receiver operating characteristic (AUROC), average precision, calibration loss, and Brier score. A nested cross-validation was performed for hyperparameter optimization and to assess performance on the whole dataset. This approach first splits the data into five outer holdout sets with 20% of the data each. For each holdout set, the remaining 80% of the data were used to fit and optimize a model via fivefold cross-validation and a randomized search over a predefined range of hyperparameter values. Observations belonging to the same patient were always kept in the same set to avoid leakage of information. A graphical representation of the process is shown in Additional file 1: Fig. S3.

For each outer holdout set, data imputation, standardization and automated feature selection were performed independently on each train set and then applied to the test set. Missing data were imputed using median imputation for simplicity and predictors were standardized to have a mean of 0 and a standard deviation of 1. A Lasso regression was used for automatic feature selection [17], and its L1 regularization term was optimized together with the classifiers’ hyperparameters. The best-performing estimator from each inner cross-validation was then used to predict the performance on the corresponding holdout test set. The overall performance resulted from the average performance of all outer folds.

Predictor importance and interpretation

In order to interpret the models, each algorithm was retrained on the whole dataset using the best hyperparameters found by the nested cross-validation. The importance of the individual predictors was gauged using the Shapley additive explanation (SHAP) framework [18]. SHAP values are state of the art in machine learning explainability and represent the marginal contribution of a predictor to the overall prediction. Interventional SHAP values were calculated separately for each observation in the dataset [19], and then grouped to find the mean predictor importance on the different days and the whole dataset. In addition, Partial Dependence Plots (PDPs) were created for each predictor [20]. PDPs show the average change in probability of the outcome for all values of a predictor, while keeping all other predictors constant. All analyses were performed in Python 3.8 (Python Software Foundation).

Role of the funding source

The sponsors had no role in any part of the design of the study, data collection, analysis, interpretation of data, the writing of the report nor the decision to submit.

Results

Cohort description

A total of 1152 patients were on invasive mechanical ventilation and included in the modeling. 883 of these patients were admitted before the 1st of September 2020 during the first wave in the Netherlands, 269 patients were admitted after this date during the second wave. Compared to day 1761 patients were mechanically ventilated for more than 7 days (66%), and 383 for more than 14 days (33%). Patient demographics, lab values, and respiratory characteristics are summarized in Table 1 for the different prediction timepoints throughout the course of IMV. For the total cohort on day 1, median age was 66 years (IQR 58–72 years), the majority were male (73%), and the median body mass index (BMI) was 27.8 (IQR 25.3–31.5 kg/m2).

Table 1 Patient characteristics on day 1, day 7 and day 14 of invasive mechanical ventilation

Interestingly, mortality during ICU admission occurred in 28.8% of patients that survived at least 24 h on IMV and only slightly increased throughout the course of mechanical ventilation; 32.4% of patients that survived up until day 14 on IMV still died afterwards. Median ventilator-free days on day 30 were 15 days (IQR 0–23) for the entire cohort and median ICU-free days were 7 days (IQR 0–21). The median C-reactive protein (CRP) decreased throughout these time points, whereas the leukocytes increased. Of note, the ventilatory ratio (minute volume * PCO2/(predicted body weight * 100 * 37.5)) and the D-dimer increased with longer ICU admission.

Model results

The overall model results and the results on the different days of IMV are presented in Table 2 for ICU mortality and ICU and ventilator-free days after 30 days. Additional performance metrics can be found in Additional file 1: Table S2. The XGBoost algorithm yielded the highest performance for each of the outcomes, as well as an increase in performance later into the IMV course. Given the performance and ability of the XGBoost algorithm to encode the interaction between possibly non-linear predictors, the predictor importance was produced with this model.

Table 2 Model performance for the different outcomes and day of IMV
Fig. 1
figure1

Importance of the top 10 predictors for the prediction of ICU mortality and ICU free days, as well as the difference for predictors over time. A ICU mortality. B ICU-free days

Predictor importance

The most important predictors per time point based on the SHAP values are presented in Fig. 1; for ventilatory-free days these plots can be found in Additional file 1: Fig. S4. Furthermore, an unregularized linear model was trained to identify statistically significant relationships between the predictors and each outcome, shown in Additional file 1: Table S3. These predictors largely overlapped with the predictors identified with the SHAP values. Lastly, strongly correlated predictors removed during the data preparation are listed in Additional file 1: Table S4.

Fig. 2
figure2

Partial dependence plots. PDP for age, pH, and driving pressure. The median value of all observations is indicated with a red vertical line

Overall, age was the most important demographic predictor of all three outcomes. The PDP shows the relative increase of ICU mortality probability with age, displaying an increase starting at 54 years relative to the median and the steepest increase around the median age of 64 years (Fig. 2). None of the comorbidities showed up as important predictors. Besides age, mechanical ventilation parameters were the most important predictors for all outcomes. Interestingly, fluid balance and medication were neither significantly correlated with outcome, nor among the most important predictors based on their SHAP values. However, medication dosage was unavailable for several hospitals (11 out of 25).

The pH, P/F ratio and driving pressure were the most important mechanical ventilation predictors for all outcomes; acidaemic conditions, low P/F-ratios, and high driving pressures were associated with a higher probability of mortality. The magnitude and direction of all predictors’ effect can be observed in the SHAP plots in Additional file 1: Figs. S5, S6, and S7. pH was strongly correlated with pCO2, while no correlation was found with creatinine, AKI stage, or lactate. The median pH in the PDP falls within the normal range for pH. The course of pH between survivors and non-survivors can be observed in Fig. 3 and shows higher pH in survivors, albeit close to the normal range of pH. Conversely, the average applied driving pressure increased in non-survivors compared to survivors throughout the course of IMV. The PDP shows that probability of ICU mortality increases with the mean driving pressure value in the last 24 h at 1, 7, and 14 days. The PDPs of the other predictors can be found in Additional file 1: Fig. S8.

Fig. 3
figure3

Course of pH and driving pressure. Plots show the course of pH and driving pressure only for patients intubated at least 7 (left) or 14 (right) days, respectively

Discussion

This study identifies risk factors for ICU mortality, ICU-free days and ventilator-free days throughout the course of invasive mechanical ventilation for critically ill COVID-19 patients. Even though demographics of the COVID-19 population remained similar throughout the first 14 days of IMV, age was consistently the most important demographic risk factor of outcome. pH and respiratory characteristics became increasingly important risk factors throughout the course of IMV.

Risk factors of poor ICU outcome are important to gauge prognosis for a group of patients, in order to scientifically underpin the larger debate of resource allocation in the COVID-19 crisis, and to generate hypotheses to improve our understanding of the disease. Although the exact pathophysiology remains unknown [21], previous work has linked age on admission to poor prognosis in critically ill COVID-19 patients [3, 22]. We now show that increasing age is a consistent risk factor for ICU mortality throughout the course of IMV, with an increase in the relative probability of death starting at 54 years. In addition, age is the most important demographic predictor for ventilator-free days and ICU-free days.

Besides demographics, the presented models are unique in the variety of clinical characteristics incorporated as predictors. From these parameters, pH, P/F ratio, and driving pressure demonstrate increasing importance over the course of IMV. Studies investigating the role of these predictors in critically ill COVID-19 patients are limited, but did identify pH as an important predictor [23]. No studies are available looking at the role of these predictors throughout the course of ICU admission. In the current study pH is correlated with pCO2, while no significant correlation is found with renal function or lactate. Persisting low pH may therefore reflect continued protective ventilation with permissive hypercapnia and serve as a proxy for severity of respiratory illness. Likewise, driving pressure and P/F ratio may reflect the state of the lung. Whether maintaining a lower pH or high driving pressure may have adverse effects on the body throughout the course of IMV directly remains to be investigated.

Observational data and future perspectives

While the risk factors identified in this work provide important prognostic insight for clinicians and policy-makers, relationships provide associations rather than causal effects. For causal modeling, however, a thorough understanding of causal pathways is essential. Predictors associated with outcome and intervention under study need to be understood to improve causal inference. In addition, as with any observational dataset, not all relevant predictors may be captured, potentially leading to confounding. This work sheds light on important predictors and fuels the discussion on potential confounders and standardization across EHRs. Lastly, this work generates important insight in relevant predictors that require further study. In line with the lung protective strategy, including acceptance of low pH and high pCO2, driving pressure has previously been causally related to outcome [16]. This work elaborates that such a relationship extends beyond the first 24 h of IMV, but remains to be researched further.

Model performance

Identification of risk factors depends on the goodness of fit of the model, and we show that model performance for ICU mortality [24], as well as for ventilator and ICU days [25], is consistent with the pre-COVID-19 literature. We observe an increase in model performance later throughout the course of IMV, which may indicate that the clinical characteristics better reflect the state of the patient, or the predictors more uniformly relate to the outcome. Ideally, prediction models would be integrated in the EHR and provide clinicians with a personalized mortality prediction at any given time at the bedside. Further investigations are needed to optimize individual predictions to be reliable for clinical decisions with irreversible consequences.

Strengths and limitations

This paper has several strengths. First of all, this study is unique in both the variety of predictors available per patient and the time-course data included in the models. Moreover, the multicenter data reflect practice differences between centers and improves external validity. Finally, data and code used in this study are available to clinicians and researchers within legal and ethical boundaries [11]. Data sharing is essential to replicate and verify results, compare underlying data and collaborate to foster the understanding of COVID-19.

The present study also comes with limitations. Firstly, removing transferred patients may have introduced bias in the dataset. Transferals may represent a healthier cohort of patients that are fit for transport or a sicker cohort transferred for specialized care such as extracorporeal life support (ECLS). Nonetheless, whenever data from the referring and receiving hospital were available, patient data were connected to limit the number of exclusions. In addition, previous analyses have shown that on admission, transferred patients are similar to non-transferred patients [9]. The DDW represents a relatively unselected sample of patients since all COVID-19 patients from the participating ICUs were included, limiting selection bias. Secondly, observations throughout the time course of IMV may be correlated in the same patient. To prevent leakage of correlated information, however, we keep observations of the same patient in the same split. In addition, predictors may be correlated with each other in the same observation. For this, we removed correlated predictors and we trained decision trees, which are robust to correlations. Moreover, medication dosage was still missing for certain hospitals. When unavailable, values were imputed with the median daily dosage in the training set. Nonetheless, we expect steroids to converge to similar doses in the beginning of admission due to the latest evidence.

Conclusion

This study trained a set of machine learning algorithms on a large, full-admission cohort of COVID-19 patients to identify the risk factors for ICU mortality, and ventilation- and ICU-free days throughout the course of invasive mechanical ventilation. Consistently, age was the most important demographic risk factor, with an increase in the relative probability of death starting at 54 years. Nonetheless, pH, P/F ratio, and driving pressure provided increasingly important risk factors over time. These results can be used for prognostication and to provide insight for the debate on resource allocation. In addition, the results of this research serve as a stepping stone for causal inference and individualized predictions research.

Availability of data and materials

The dataset supporting the conclusions of this article is available within restrictions imposed by privacy laws and ethics through www.amsterdammedicaldatascience.nl

Abbreviations

DDW:

Dutch data warehouse

IMV:

Invasive mechanical ventilation

SHAP:

Shapley additive explanations

PDP:

Partial dependence plots

References

  1. 1.

    Dong E, Du H, Gardner L (2020) An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 20:533–534. https://doi.org/10.1016/S1473-3099(20)30120-1

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Quah P, Li A, Phua J (2020) Mortality rates of patients with COVID-19 in the intensive care unit: a systematic review of the emerging literature. Crit Care 24:285. https://doi.org/10.1186/s13054-020-03006-1

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Grasselli G, Greco M, Zanella A et al (2020) Risk factors associated with mortality among patients with COVID-19 in intensive care units in Lombardy, Italy. JAMA Intern Med. https://doi.org/10.1001/jamainternmed.2020.3539

    Article  PubMed  Google Scholar 

  4. 4.

    Richardson S, Hirsch JS, Narasimhan M et al (2020) Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA 323:2052–2059. https://doi.org/10.1001/jama.2020.6775

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Karagiannidis C, Mostert C, Hentschker C et al (2020) Case characteristics, resource use, and outcomes of 10 021 patients with COVID-19 admitted to 920 German hospitals: an observational study. Lancet Respir Med 8:853–862. https://doi.org/10.1016/S2213-2600(20)30316-7

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Wynants L, Calster BV, Collins GS et al (2020) Prediction models for diagnosis and prognosis of COVID-19: systematic review and critical appraisal. BMJ 369:m1328. https://doi.org/10.1136/bmj.m1328

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    El-Solh AA, Lawson Y, Carter M et al (2020) Comparison of in-hospital mortality risk prediction models from COVID-19. PLoS ONE 15:e0244629. https://doi.org/10.1371/journal.pone.0244629

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Pijls BG, Jolani S, Atherley A et al (2021) Demographic risk factors for COVID-19 infection, severity, ICU admission and death: a meta-analysis of 59 studies. BMJ Open 11:e044640. https://doi.org/10.1136/bmjopen-2020-044640

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Fleuren LM, de Bruin DP, Tonutti M et al (2021) Large-scale ICU data sharing for global collaboration: the first 1633 critically ill COVID-19 patients in the Dutch Data Warehouse. Intensive Care Med. https://doi.org/10.1007/s00134-021-06361-x

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594. https://doi.org/10.1136/bmj.g7594

    Article  PubMed  Google Scholar 

  11. 11.

    Amsterdam Medical Data Science. https://www.amsterdammedicaldatascience.nl/. Accessed 20 Nov 2020

  12. 12.

    Prokop M, van Everdingen W, van Rees VT et al (2020) CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19—definition and evaluation. Radiology 296:E97–E104. https://doi.org/10.1148/radiol.2020201473

    Article  PubMed  Google Scholar 

  13. 13.

    Yehya N, Harhay MO, Curley MAQ et al (2019) Reappraisal of ventilator-free days in critical care research. Am J Respir Crit Care Med 200:828–836. https://doi.org/10.1164/rccm.201810-2050CP

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Vincent JL, Moreno R, Takala J et al (1996) The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the working group on sepsis-related problems of the European Society of Intensive Care Medicine. Intensive Care Med 22:707–710. https://doi.org/10.1007/BF01709751

    CAS  Article  Google Scholar 

  15. 15.

    Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963. https://doi.org/10.1001/jama.270.24.2957

    Article  PubMed  Google Scholar 

  16. 16.

    Amato MBP, Meade MO, Slutsky AS et al (2015) Driving pressure and survival in the acute respiratory distress syndrome. N Engl J Med 372:747–755. https://doi.org/10.1056/NEJMsa1410639

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  17. 17.

    Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol 58:267–288

    Google Scholar 

  18. 18.

    Lundberg S, Lee S-I (2017) A unified approach to interpreting model predictions. ArXiv170507874 Cs Stat

  19. 19.

    Chen H, Janizek JD, Lundberg S, Lee S-I (2020) True to the Model or True to the Data? ArXiv200616234 Cs Stat

  20. 20.

    Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232. https://doi.org/10.1214/aos/1013203451

    Article  Google Scholar 

  21. 21.

    Márquez EJ, Chung C, Marches R et al (2020) Sexual-dimorphism in human immune system aging. Nat Commun 11:751. https://doi.org/10.1038/s41467-020-14396-9

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Zhou F, Yu T, Du R et al (2020) Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 395:1054–1062. https://doi.org/10.1016/S0140-6736(20)30566-3

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Linli Z, Chen Y, Tian G et al (2020) Identifying and quantifying robust risk factors for mortality in critically ill patients with COVID-19 using quantile regression. Am J Emerg Med. https://doi.org/10.1016/j.ajem.2020.08.090

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Keuning BE, Kaufmann T, Wiersema R et al (2020) Mortality prediction models in the adult critically ill: a scoping review. Acta Anaesthesiol Scand 64:424–442. https://doi.org/10.1111/aas.13527

    Article  PubMed  Google Scholar 

  25. 25.

    Verburg IWM, Atashi A, Eslami S et al (2017) Which models can i use to predict adult ICU length of stay? A systematic review. Crit Care Med 45:e222–e231. https://doi.org/10.1097/CCM.0000000000002054

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The Dutch ICU Data Sharing Against COVID-19 Collaborators: From collaborating hospitals having shared data: Thijs C.D. Rettig, MD, PhD, Department of Intensive Care, Amphia Ziekenhuis, Breda en Oosterhout, The Netherlands. M.C. Reuland, MD, Department of Intensive Care Medicine, Amsterdam UMC, Universiteit van Amsterdam, Amsterdam, The Netherlands. Laura van Manen, MD, Department of Intensive Care, BovenIJ Ziekenhuis, Amsterdam, The Netherlands. Leon Montenij, MD, PhD, Department of Anesthesiology, Pain Management and Intensive Care, Catharina Ziekenhuis Eindhoven, Eindhoven, The Netherlands. Jasper van Bommel, MD, PhD, Department of Intensive Care, Erasmus Medical Center, Rotterdam, The Netherlands. Roy van den Berg, Department of Intensive Care, ETZ Tilburg, Tilburg, The Netherlands. Ellen van Geest, Department of ICMT, Haga Ziekenhuis, Den Haag, The Netherlands. Anisa Hana, MD, PhD, Intensive Care, Laurentius Ziekenhuis, Roermond, The Netherlands. W.G. Boersma, MD, PhD, Department of Pulmonary Medicine, Northwest Clinics, Alkmaar, The Netherlands. B. van den Bogaard, MD, PhD, ICU, OLVG, Amsterdam, The Netherlands. Prof. Peter Pickkers, Department of Intensive Care Medicine, Radboud University Medical Centre, Nijmegen, The Netherlands. Pim van der Heiden, MD, PhD, Intensive Care, Reinier de Graaf Gasthuis, Delft, The Netherlands. Claudia (C.W.) van Gemeren, MD, Intensive Care, Spaarne Gasthuis, Haarlem en Hoofddorp, The Netherlands. Arend Jan Meinders, Department of Internal Medicine and Intensive Care, St Antonius Hospital, Nieuwegein, The Netherlands. Martha de Bruin, MD, Department of Intensive Care, Franciscus Gasthuis & Vlietland, Rotterdam, the Netherlands. Emma Rademaker, MD, MSc, Department of Intensive Care, UMC Utrecht, Utrecht, The Netherlands. Frits (H.M.) van Osch, PhD, Epidemioloog, Department of Clinical Epidemiology, VieCuri Medisch Centrum, Venlo, The Netherlands. Martijn de Kruif, MD, PhD, Department of Pulmonology, Zuyderland MC, Heerlen, The Netherlands. Nicolas Schroten, MD, Intensive Care, Albert Schweitzerziekenhuis, Dordrecht, The Netherlands. Klaas Sierk Arnold, MD, Anesthesiology, Antonius Ziekenhuis Sneek, Sneek, The Netherlands. J.W. Fijen, MD PhD, Department of Intensive Care, Diakonessenhuis Hospital, Utrecht, The Netherlands. Jacomar J.M. van Koesveld, MD, ICU, IJsselland Ziekenhuis, Capelle aan den IJssel, The Netherlands. Koen S. Simons, MD, PhD, Department of Intensive Care, Jeroen Bosch Ziekenhuis, Den Bosch, The Netherlands. Joost Labout, MD, PhD, ICU, Maasstad Ziekenhuis Rotterdam, The Netherlands. Bart van de Gaauw, MD, ICU, Martiniziekenhuis, Groningen, The Netherlands. Michael Kuiper, Intensive Care, Medisch Centrum Leeuwarden, Leeuwarden, The Netherlands. Albertus Beishuizen, MD PhD, Department of Intensive Care, Medisch Spectrum Twente, Enschede, the Netherlands. Dennis Geutjes, Department of Information Technology, Slingeland Ziekenhuis, Doetinchem, The Netherlands. Johan Lutisan, MD, ICU, WZA, Assen, The Netherlands. Bart P. X. Grady, MD, PhD, department of Intensive Care, Ziekenhuisgroep Twente, Almelo, The Netherlands. Remko van den Akker, Intensive Care, Adrz, Goes, The Netherlands. From collaborating hospitals having signed the data sharing agreement: Bram Simons, MD, Intensive Care, Bravis Ziekenhuis, Bergen op Zoom en Roosendaal, The Netherlands. A.A. Rijkeboer, MD, ICU, Flevoziekenhuis, Almere, The Netherlands. Sesmu Arbous, MD, PhD, Intensivist, LUMC, Leiden, The Netherlands. Marcel Aries, MD, PhD, MUMC+, University Maastricht, Maastricht, The Netherlands. Niels C. Gritters van den Oever, MD, Intensive Care, Treant Zorggroep, Emmen, The Netherlands. Martijn van Tellingen, MD, EDIC, Department of Intensive Care Medicine, afdeling Intensive Care, ziekenhuis Tjongerschans, Heerenveen, The Netherlands. Annemieke Dijkstra, MD, Department of Intensive Care Medicine, Het Van Weel-Bethesda Ziekenhuis, Dirksland, The Netherlands. Rutger van Raalte, Department of Intensive Care, Tergooi hospital, Hilversum, The Netherlands. From the Laboratory for Critical Care Computational Intelligence: Luca Roggeveen, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Fuda van Diggelen, MSc, Quantitative Data Analytics Group, Department of Computer Sciences, Faculty of Science, VU University, Amsterdam, The Netherlands Ali el Hassouni, PhD, Quantitative Data Analytics Group, Department of Computer Sciences, Faculty of Science, VU University, Amsterdam, The Netherlands. David Romero Guzman, PhD, Quantitative Data Analytics Group, Department of Computer Sciences, Faculty of Science, VU University, Amsterdam, The Netherlands. Sandjai Bhulai, PhD, Analytics and Optimization Group, Department of Mathematics, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands. Dagmar Ouweneel, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Ronald Driessen, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Jan Peppink, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. H.J. de Grooth, MD, PhD, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. G.J. Zijlstra, MD, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. A.J. van Tienhoven, MD, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Evelien van der Heiden, MD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Jan Jaap Spijkstra, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Hans van der Spoel, MD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Angelique de Man, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Thomas Klausch, PhD, Department of Clinical Epidemiology, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Heder de Vries, MD, Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. From Pacmed: Michael de Neree tot Babberich, Pacmed, Amsterdam, The Netherlands. Olivier Thijssens, Pacmed, Amsterdam, The Netherlands. Lot Wagemakers, Pacmed, Amsterdam, The Netherlands. Hilde G.A. van der Pol, Pacmed, Amsterdam, The Netherlands. Tom Hendriks, Pacmed, Amsterdam, The Netherlands. Julie Berend, Pacmed, Amsterdam, The Netherlands. Virginia Ceni Silva, Pacmed, Amsterdam, The Netherlands. Bob Kullberg, Pacmed, Amsterdam, The Netherlands. From RCCnet: Leo Heunks, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Nicole Juffermans, MD, PhD, ICU, OLVG, Amsterdam, The Netherlands. Arjan Slooter, Intensive Care, UMC Utrecht, Utrecht, The Netherlands.

Funding

Partially funded by grants from ZonMw (project 10430012010003, file 50-55700-98-908), Zorgverzekeraars Nederland and the Corona Research Fund. The sponsors had no role in any part of the study.

Author information

Affiliations

Authors

Consortia

Contributions

LF and MT drafted the manuscript and performed the analyses. DB, RL, MF, TM, MS, SV, AB, DQ, RN, TH, TD, PT, WH and PE were involved in data processing and analytics. All authors contributed to data collection and critically reviewed the manuscript. All authors have full access to the data. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lucas M. Fleuren.

Ethics declarations

Ethics approval and consent to participate

The Medical Ethics Committee at Amsterdam UMC, location VUmc waived the need for patient informed consent and approved of an opt-out procedure for the collection of COVID-19 patient data during the COVID-19 crisis.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Figure S1.

Patient selection. Figure S2. Selection of observations throughout the course of IMV. Figure S3. Nested cross-validation. Figure S4. Importance of the top 10 predictors for the prediction of ventilator free days, as well as the difference for predictors over time. Figure S5. SHAP plot ICU mortality (XGBoost). Figure S6. SHAP plot for ICU free days (XGBoost). Figure S7. SHAP plot for ventilator free days (XGBoost). Figure S8. PDPs. Table S1. Overview of all predictors used in the model with a definition where applicable. Table S2. Overall algorithm performance for each of the different outcomes. Table S3. Statistical results for a regression model per outcome. Table S4. Predictor correlations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fleuren, L.M., Tonutti, M., de Bruin, D.P. et al. Risk factors for adverse outcomes during mechanical ventilation of 1152 COVID-19 patients: a multicenter machine learning study with highly granular data from the Dutch Data Warehouse. ICMx 9, 32 (2021). https://doi.org/10.1186/s40635-021-00397-5

Download citation

Keywords

  • COVID-19
  • Mortality prediction
  • Risk factors
  • Machine learning