Predicting Hospital Survival in Patients Admitted to ICU with Pulmonary Embolism

Objective The Pulmonary Embolism Severity Index (PESI) and simplified PESI (sPESI) predict mortality for patients with PE. We compared PESI/sPESI to the Acute Physiology and Chronic Health Evaluation IV (APACHE-IV) in predicting mortality in patients with PE admitted to the intensive care unit (ICU). Additionally, we assessed the performance of a novel ICU-sPESI score created by adding three clinical variables associated with acuity of PE presentation (intubation, confusion [altered mental status], use of vasoactive infusions) to sPESI. Materials and Methods Using the eICU Collaborative Research Database from 2014 to 2015, we conducted a large retrospective cohort study of adult patients admitted to the ICU with a primary diagnosis of PE. We calculated APACHE-IV, PESI, sPESI, and ICU-sPESI scores and compared their performance for predicting in-hospital mortality using area under the receiver operating characteristic (AUROC) curve. Score thresholds for >99% negative predictive values (NPV) were calculated for each score. Survival was estimated using the Kaplan–Meier method. Results We included 1424 PE cases. In-hospital mortality was 6.3% [95% CI: 5.1%-7.6%]. AUROC for APACHE-IV, PESI, and sPESI were 0.870, 0.848, and 0.777, respectively. APACHE-IV and PESI outperformed sPESI (P < 0.01 for both comparisons), while APACHE-IV and PESI demonstrated similar performance (P = 0.322). The ICU-sPESI performance was similar to APACHE-IV and PESI (AUROC = 0.847; AUROC comparison: APACHE-IV vs ICU-sPESI: P = 0.396; PESI vs ICU-sPESI: P = 0.945). Hospital mortality for ICU-sPESI scores 0-2 was 1.1%, and for scores 3, 4, 5, 6, and ≥7 was 8.6%, 11.7%, 29.2%, 37.5%, and 76.9%, respectively. Score thresholds for >99% NPV were ≤48 for APACHE-IV, ≤115 for PESI, and 0 points for sPESI and ICU-sPESI. Conclusions By accounting for severity of PE presentation, our newly proposed ICU-sPESI score provided improved PE mortality prediction compared to the original sPESI score and offered excellent discrimination of mortality risk.


Introduction
Pulmonary embolism (PE) is the third highest cause of cardiovascular-related death, accounting for over 100,000 annual deaths in the United States. 1 Mortality after acute PE is between 5% and 15%, 2 with mortality between 35% and 58% for patients presenting with hypotension and/or shock. 3arly establishment of mortality risk in patients with PE may play an important role in management strategies. 4,5The Acute Physiology and Chronic Health Evaluation IV (APACHE-IV) 6 is one of the most advanced scoring systems designed for estimating short-term mortality among all patients admitted to an intensive care unit (ICU) using a wide range of comorbidities and clinical variables derived within the first 24 h of ICU admission. 7,8However, the APACHE-IV's 129 variables make its calculation complex and labor-intensive. 9,10In contrast, the less complex Pulmonary Embolism Severity Index (PESI), developed as a risk stratification tool for patients with PE, utilizes 11 differentially weighted variables (Supplemental Table 1). 11wever, because PESI is considered cumbersome and is rarely calculated in clinical practice, 12 a less complex, simplified PESI score (sPESI) was introduced, including 6 equally weighted (present =1; absent =0) and easily accessible variables (Supplemental Table 1). 13Interestingly, altered mental status, a variable with the highest weight in the PESI original multivariable analysis, was not included as a component of the sPESI.5][16] Although there are abundant data on the accuracy of PESI and sPESI for patients with PE, their performance has been infrequently evaluated in PE patients who were deemed to require ICU admission. 3,17he first aim of this study was to compare the predictive abilities of PESI and sPESI scores to the all-cause mortality APACHE-IV score using a multicenter cohort of United States patients admitted to the ICU with PE.The second aim was to test the hypothesis whether the addition of 3 early and easily accessible binary clinical markers of PE severity (Intubation, Confusion [altered mental status], Use of vasoactive infusions) to sPESI, creating a new ICU-sPESI score, could improve mortality prediction.

Data Source and Ethics Statement
Data were obtained from the eICU Collaborative Research Database v2.0, a multicenter critical care database provided by Philips Healthcare in collaboration with the MIT Laboratory for Computational Physiology.This database includes a wide range of clinical data for more than 200,000 ICU admissions from 208 hospitals, including both academic and community hospitals of various sizes, across all regions of the United States from 2014 and 2015. 18,19his study was conducted in accordance with the Declaration of Helsinki and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines. 20The study did not require institutional review board approval due to its retrospective nature, lack of direct patient intervention, and security framework, ensuring re-identification risks met Safe Harbor criteria as certified by Privacert (Cambridge, MA; Health Insurance Portability and Accountability Act Certification no.1031219-2).The author Martin J. Ryll obtained access (CITI Data or Specimens Only Research Training Record ID 50703292) and was responsible for data extraction and analysis.The code for data extraction and analysis can be accessed online at github.com/RyllMartin/eICU_predicting_PE_ICU_mortality.

Patient Selection and Data Extraction
All ICU patients with a primary admission diagnosis of PE were included, except those <18 years old or those missing key data (APACHE-IV score, survival outcome, and Glasgow Coma Scale [GCS]).Data extracted included age, sex, ethnicity, body mass index (BMI), and selected clinical variables within 24 h of admission (GCS, APACHE-IV scores, vital signs, mental status, hemodynamic management, intubation status).Other variables used to calculate the PESI and sPESI included the presence of (a) chronic lung disease (history of asthma, chronic obstructive or restrictive pulmonary disease, home oxygen requirement, or lung transplant), (b) heart failure, (c) chronic cardiopulmonary disease (history of chronic lung disease [see above], heart failure, history of myocardial infarction, coronary artery bypass, pacemaker, or implantable cardioverter-defibrillator), and (d) cancer history.Variables for heart rate, body temperature, and respiratory rate were derived from the Acute Physiology Score (APS; part of APACHE) vitals.Missing values among the vitals variables were assumed as normal, in line with the original PESI publication. 11Altered mental status (AMS) or confusion was deemed present if the GCS category verbal response was <5 (i.e., not verbally responsive and oriented).The final PESI and sPESI risk scores were calculated from these components as listed in Supplemental Table 1. 11,13For the ICU-sPESI, a 9-variable equally weighted score was designed by adding 3 binary clinical variables of disease acuity to the sPESI if present within the first 24 h after admission: (a) respiratory failure requiring intubation (I), (b) confusion (C) or altered mental status; and (c) need for use (U) of vasopressor or inotropic support.These markers were chosen based on prior studies showing their independent association with PE mortality. 3,21,22

Statistical Analyses
Vital signs were assessed from medians calculated over 30 min and we used the most extreme values for the final score calculation.Continuous data are presented as median and IQR, categorical and binary variables as count and percentage.Categorical and binary variables were compared between survivors and non-survivors by Chi-Square test or Fisher-exact test as appropriate.Continuous variables were tested for normal distribution via Shapiro-Wilk test and compared by independent t-test or Mann-Whitney U test where appropriate.We derived mortality predictions for each score using a univariate logistic regression model with the score as the independent and in-hospital mortality as the dependent variable.Receiver operating characteristic (ROC) curves were calculated and plotted for the four scores.The areas under the receiver operating characteristic (AUROCs) curve of the scores were compared with a test for comparing AUROCs on the same patient cohort as described by Hanley and McNeil. 23For the main analysis, PESI, sPESI, and ICU-sPESI were calculated using data from the first 24 h following ICU admission.We performed a sensitivity analysis that examined AUROC for PESI, sPESI, and ICU-sPESI calculated at different time points following ICU admission (2, 3, 6, and 12 h after admission to the ICU).In addition, we calculated the highest score thresholds associated with negative predictive values (NPV) exceeding 99%.To visualize the fit of our models at different score ranges, the predicted and actual in-hospital mortality were plotted against deciles of the APACHE-IV and PESI scores and against direct score value for the sPESI and ICU-sPESI scores.Using the Kaplan-Meier method, survival curves were calculated for different patient groups stratified by PESI classes, sPESI scores, and ICU-sPESI scores.All data were censored at 40 days post-ICU admission, with the last event recorded on the 39 th day.Patients who survived to hospital discharge were assumed alive post-discharge.To assess potential differences among these survival curves, a multivariate log-rank test was first utilized.In the case of observed differences, pairwise log-rank tests were performed, adjusting for multiple comparisons using the Bonferroni correction.In addition, Kaplan-Meier survival curves with 95% confidence intervals were created for 4 ICU-sPESI risk classes: Class I (ICU-sPESI score ≤2 points), Class II (3-4 points), Class III (5-6 points), and Class IV (≥7 points).The class groupings were based on pairwise comparisons of individual score survival curves (unadjusted log-rank test) with groups created for scores whose survival curves did not differ significantly.All reported tests were two-sided, and P-values < 0.05% were considered statistically significant.Statistical analyses were performed with the Python v.3.9 (Python Software Foundation, Wilmington, Delaware, USA) program.

Results
Of the 1697 patients with a primary admission diagnosis of PE, 270 were excluded due to missing key data and 3 patients Compared to survivors, nonsurvivors were older (median age 70 vs 62 years, P < 0.001) and had higher rates of cancer history (34.4% vs 16.1%, P < 0.001), obstructive pulmonary disease (26.7% vs 16.1%, P = 0.014), dementia (7.8% vs 2.5%, P = 0.011) and liver cirrhosis (4.4% vs 0.8%, P = 0.012) (Table 1).Fewer nonsurvivors reported a history of previous thromboembolic events than survivors, but the difference did not reach significance (7.8% vs 15.3%, P = 0.074).The rates of thrombolytic use were low and not significantly different between survivors and nonsurvivors (3.8% vs 2.2%, respectively P = 0.770).A detailed presentation of comorbidities among the two groups are presented in Supplemental Table 2.
Supplemental Table 3 shows PESI, sPESI, and ICU-sPESI score components on admission according to outcome (survival vs no-survival).Except for sex and history of heart failure, all individual components of PESI and sPESI were significantly associated with in-hospital mortality.Regarding the three additional ICU-sPESI variables, nonsurvivors were more likely to be intubated, have altered mental status, and to be hemodynamically unstable requiring vasoactive support (all P < 0.001).Additional vital signs for survivors and nonsurvivors obtained within 24 h of ICU admission are provided in Supplemental Table 4.
ROC curves were used to assess the performance of the four scores in regard to prediction of mortality.The AUROC for APACHE-IV calculated from data available at 24 h after admission was 0.870, for PESI 0.848, and for sPESI 0.770 (Figure 1).Numerically, APACHE-IV had the highest discriminative ability among the three scoring systems; however, its AUROC was not significantly higher than that of the PESI (P = 0.322).Both the APACHE-IV and PESI outperformed sPESI, as evidenced by the differences in their AUROCs (sPESI vs APACHE-IV, P = 0.006, sPESI vs PESI, P = 0.010).The ICU-sPESI had an AUROC of 0.847, which was greater than the sPESI (P < 0.001) and comparable to APACHE-IV and PESI (APACHE-IV vs ICU-sPESI: P = 0.396; PESI vs ICU-sPESI: P = 0.945, Figure 1).The sensitivity analysis comparing AUROCs for PE risk scores at shorter time intervals (2, 3, 6, 12 h) after ICU admission demonstrated slightly lower AUROC values, but the pattern was similar for all three risk scores (Supplemental Table 5).Using "any type of ventilation" (invasive and/or noninvasive) instead of "intubation/invasive mechanical ventilation" for the ICU-sPESI's "I" variable did not alter the ICU-sPESI's predictive performance (Supplemental Figure 2).Figure 2 depicts the observed and predicted in-hospital mortality from the univariate logistic regression models for APACHE-IV, PESI, sPESI, and ICU-sPESI.All scores showed a comparable observed and predicted in-hospital mortality across their respective score ranges, demonstrating accurate calibration for the logistic regression models.To evaluate the predictive capability for low mortality risk, the highest (best) score thresholds for >99% NPV were calculated.The corresponding score thresholds were ≤48 for APACHE-IV, ≤115 for PESI, and zero (0) points for sPESI and ICU-sPESI (Figure 2 vertical dotted lines).
Figure 3 shows the Kaplan-Meier survival plots for PESI classes, and sPESI and ICU-sPESI scores.For all scores, survival differed across groups and decreased as the scores increased (log-rank test P < 0.001).Details regarding statistically significant pairwise comparisons between respective score groups are provided in the footnote of Figure 3. Figure 4 shows Kaplan-Meier curves with 95% confidence intervals for the estimated survival rate when ICU-sPESI scores were grouped into 4 risk classes (see Methods).Hospital mortality increased in each higher ICU-sPESI risk class (point estimate [95% CI]) for Class I 1.1%

Discussion
For patients with PE admitted to the ICU, the APACHE-IV and PESI provided comparable predictive power for in-hospital mortality, whereas the discriminative predictive power of sPESI was lower.Thus, by incorporating three easily obtainable markers of PE acuity into the sPESI score, we created a novel ICU-sPESI score that allowed for improved discrimination of PE mortality risk.
For patients admitted to ICU with PE the APACHE-IV AUROC was 0.870, similar to that observed in the original validation study of critically ill adults (0.880). 6The PESI and sPESI are validated assessments for mortality prediction from PE in patients. 13,24,25Although in our study the AUROC of sPESI (0.770) for prediction of in-hospital mortality was inferior to that of PESI (0.848), both tests performed better than in reports by Barnes 26 for 7-and 30-day mortality (PESI 0.652, sPESI 0.666), and I ̇din 17 (PESI 0.624) for 30-day mortality.The discrepancy in AUROCs between our study and that of Barnes 26 and I ̇din 17 is not clear.Multiple reports demonstrated that sPESI is sensitive in identifying low-risk PE patients. 12,13,15,24,25Yet, in our study of PE patients admitted to the ICU, sPESI had inferior predictive performance compared to PESI, possibly reflecting the exclusion of variable such as altered mental status which had the highest weight in PESI score.9][30] While this approach may improve the score performance, it introduces complexity to calculation.The eICU database lacks imaging data and did not report markers of myocardial injury or dysfunction for >60% of our patients, thereby limiting our ability to evaluate the impact of the ESC-recommended 27 variables.
2][33] However, defining a score that precisely identifies low-risk PE may be challenging.Based on 99% NPV calculations, the low-risk PE patients in our study were those with an APACHE-IV score ≤48, a PESI score <115, and sPESI and ICU-sPESI scores of zero points.However, some studies reported different cutoff points for identifying low-risk PE. 15 , 34 For example, Brikho 15 and Bellou 34 proposed a PESI score of <85 (PESI class ≤ II) as a cutoff point for outpatient treatment, despite associated 30-day mortality of 1.6%-3.5%.In contrast, a higher PESI score of ≤115 (Class IV, i.e., high-risk PESI score) 15 in our study was associated with very low (0.93%) mortality rate, and therefore may be considered a low PE risk.These discrepancies in cutoff points of low-risk PE may be related to study designs, definitions of timing of death (in-hospital, vs 7-day, vs 30-day death), type of cohort examined (eg, all patients vs ICU admissions) or due to reports coming from single institution with nonstandard expertise in managing PE.It is also important to develop a reliable and simple score that predicts risk of high PE mortality.Since high mortality risk was not precisely discriminated by PESI classes and sPESI score, we postulated that adding easily obtainable variables related to the severity of clinical presentation may improve the ability of the sPESI to better categorize mortality risk in PE patients admitted to the ICU.Specifically, in earlier reports the use of vasopressors and respiratory insufficiency requiring intubation/mechanical ventilation were predictors of mortality in PE patients. 3,21urthermore, altered mental status was an independent predictor [OR 9.1] of a life-threatening adverse outcomes in PE patients. 22sing this information, we proposed a new ICU-sPESI mortality prediction score by adding these clinical variables to the original sPESI score.This novel score demonstrated improved discrimination of mortality risk compared to PESI and sPESI.

Strengths and Limitations
Because our study is based on data from a large multicenter registry, it measures mortality across institutions with diverse patients and different expertise in managing PE, contributing to external validity.However, using this database leads to certain limitations.In our study, we included patients with a primary diagnosis of PE on admission to the ICU rather than diagnosis confirmed on discharge because the eICU does not report a final discharge diagnosis or imaging data.Therefore, we cannot exclude potential misdiagnosis of PE on admission to the ICU.Furthermore, we were unable to compare our findings to patients with PE admitted to other services, such as regular wards, as data on these patients are not available in the eICU.Finally, in our study, the overall in-hospital PE mortality (6.3%) was slightly lower than in a previous US-based large multicenter study on ICU admissions of patients with PE (9.2%-9.5%). 35Sauer et al. 36 examined data from four adult clinical ICU datasets which also included the eICU Collaborative Research Database.They found that the four databases differ in sample sizes, acuity of illness for admitted patients, treatment intensities, and frequency of reported parameters.Therefore, it is possible that outcomes reported from different databases may differ due to differences in patient acuity and treatment intensities.
We used vital signs and clinical events observed during 24 h after ICU admission as score components of PE severity.Practical predictive scores would be most useful if obtained at shorter timeframes after admission, therefore further studies should examine this prospectively.However, our sensitivity analysis of AUROCs for PESI, sPESI, and ICU-sPESI at various time periods during the first 24 h following ICU admission showed comparable albeit slightly diminished predictive accuracy, mainly attributable to the clinical events.As data on these clinical events is considerably less granular, this may be explained by the lag between event occurrence and actual event charting, or alternatively by the 24-h period allowing more time to establish symptomatology of PE relevant for prediction of outcome.Although our cohort consisted of PE patients admitted to the ICU, we do not have insight of true indications for ICU admission, and literature has shown that ICU admissions for PE have a huge variation between hospitals with differing acuity thresholds for ICU admission criteria. 35urthermore, as our results represent aggregates of the data of many centers, the predictive performance of the scores may not be generalizable to a center where the level of expertise and treatment modalities substantially fall outside the average standards of care for PE.Specifically, it has been shown that patients with acute PE may benefit when treated at high-volume centers by experienced physicians. 37,38In addition, the reported accuracy in predicting mortality using risk scores is associated with advances in treatments. 39Specifically, mortality in highrisk patients with PE decreased from 1999 to 2017, primarily due to improved treatment of most severe PE patients with shock and cardiac arrest 39 which underscores the need to continue reassessing the performance of these scores over time.

Conclusion
In conclusion, while APACHE-IV and PESI scores demonstrated good prediction of mortality among critically ill patients with PE, the performance of sPESI was lower.After incorporating three clinical markers of disease acuity into the sPESI calculation, the discriminative ability of the novel ICU-sPESI score improved and was not different from that of PESI.ICU-sPESI scores provide improved prediction of mortality among high-risk PE patients admitted to the ICU.Future prospective studies are needed to validate the accuracy of the ICU-sPESI score for PE patients admitted to the ICU, especially for those considered to be at high risk.

Figure 1 .
Figure 1.Receiver operating characteristic curves for the APACHE-IV, PESI, sPESI, and ICU-sPESI score for predicting in-hospital mortality.

Figure 2 .
Figure 2. (A-D) Actual versus predicted mortality across the spectrum of scores for APACHE-IV (A), PESI (B) (both scores shown in deciles), sPESI (C), and ICU-sPESI (D) (both scores shown in direct score values).Vertical dotted line indicates the score threshold for a NPV of >99% for each test for critically ill patients with PE.

Table 1 .
Demographics and Comorbidities among Patients with a Primary Diagnosis of Pulmonary Embolism Requiring Intensive Care Unit Admission.
Binary variables and categories are shown as count (percentage), continuous variables as median [25th quantile-75th quantile].Individual features had no missing values, except for body mass index with 3.7% missing values and ethnicity with <1% missing values.a Includes other major vessel atherosclerotic disease.Abbreviations: ITP, idiopathic thrombocytopenic purpura; AICD, automatic implantable cardioverter defibrillator; PE, pulmonary embolism.