Comparison of National Surgical Quality Improvement Program Surgical Risk Calculator and Trauma and Injury Severity Score Risk Assessment Tools in Predicting Outcomes in High-Risk Operative Trauma Patients

Background The Trauma and Injury Severity Score (TRISS) uses anatomic/physiologic variables to predict outcomes. The National Surgical Quality Improvement Program Surgical Risk Calculator (NSQIP-SRC) includes functional status and comorbidities. It is unclear which of these tools is superior for high-risk trauma patients (American Society of Anesthesiologists Physical Status (ASA-PS) class IV or V). This study compares risk prediction of TRISS and NSQIP-SRC for mortality, length of stay (LOS), and complications for high-risk operative trauma patients. Methods This is a prospective study of high-risk (ASA-PS IV or V) trauma patients (≥18 years-old) undergoing surgery at 4 trauma centers. We compared TRISS vs NSQIP-SRC vs NSQIP-SRC + TRISS for ability to predict mortality, LOS, and complications using linear, logistic, and negative binomial regression. Results Of 284 patients, 48 (16.9%) died. The median LOS was 16 days and number of complications was 1. TRISS + NSQIP-SRC best predicted mortality (AUROC: .877 vs .723 vs .843, P = .0018) and number of complications (pseudo-R2/median error (ME) 5.26%/1.15 vs 3.39%/1.33 vs 2.07%/1.41, P < .001) compared to NSQIP-SRC or TRISS, but there was no difference between TRISS + NSQIP-SRC and NSQIP-SRC with LOS prediction (P = .43). Discussion For high-risk operative trauma patients, TRISS + NSQIP-SRC performed better at predicting mortality and number of complications compared to NSQIP-SRC or TRISS alone but similar to NSQIP-SRC alone for LOS. Thus, future risk prediction and comparisons across trauma centers for high-risk operative trauma patients should include a combination of anatomic/physiologic data, comorbidities, and functional status.


Introduction
There are over 4 million deaths annually due to trauma with a global burden exceeding 1 trillion dollars per year. 1 Due to the great impact of trauma, outcome prediction has become a topic of interest.
The American Society of Anesthesiologists Physical Status (ASA-PS) was developed as an easy method to evaluate surgical risk based on functional status and medical comorbidities.ASA-PS has been validated in many surgical populations; however, it was not explicitly designed for use in trauma patients. 2,3The highest risk patients, categorized by ASA-PS Class IVand V, represent patients with severe systemic disease that is a constant threat to life and patients who are moribund and not expected to survive without surgery, respectively.
The Trauma and Injury Severity Score (TRISS) uses injury and physiologic variables such as injury severity score (ISS), systolic blood pressure (SBP), respiratory rate (RR), heart rate (HR), and Glasgow Coma Score (GCS) to predict mortality in trauma patients. 4It has additionally been validated to predict length of stay (LOS) and total number of complications for trauma patients. 5,6However, TRISS does not factor in comorbidities or functional status which can greatly affect clinical outcomes.The National Surgical Quality Improvement Program Surgical Risk Calculator (NSQIP-SRC) was created as a more comprehensive nontrauma, surgical risk assessment tool that includes over 20 variables and incorporates functional status to estimate the risk of unfavorable outcomes.However, there is limited data regarding its use in operative trauma patients. 7herefore, this study aims to evaluate TRISS and NSQIP-SRC risk assessment tools' ability to predict mortality, LOS, and number of complications for highrisk (ASA-PS Class IV or V) operative trauma patients.We hypothesize improved prediction of these outcomes by a model combining TRISS plus NSQIP-SRC compared to either tool alone as this will incorporate injury profile, comorbidities, and functional status.

Data Collection
Institutional review board approval was obtained, and a waiver of consent granted at each participating institution for this planned subgroup analysis of a multicenter prospective observational study (9/1/2018-2/29/2020) at 4 Level-I trauma centers.All high-risk (ASA-PS IVor V) adult (≥18 years old) trauma patients who underwent surgery within 24 hours of admission were included.Patients who were pregnant or prisoners were excluded.This study followed the guidelines outlined in the statement of Strengthening the Reporting of Observational Studies in Epidemiology (STROBE).
Three risk assessment tools were compared: TRISS vs NSQIP-SRC vs TRISS + NSQIP-SRC.The TRISS + NSQIP-SRC model was produced by using the previously described TRISS methodology formula 8 to estimate the probability of interested outcomes which was then added as a single variable to NSQIP-SRC to produce results of the TRISS + NSQIP-SRC model.The primary outcome was in-hospital morality.Secondary outcomes were hospital LOS and number of complications including shock, delirium, cardiac arrest, myocardial infarction, new onset arrhythmias, respiratory failure, acute lung injury, acute respiratory distress syndrome, tracheostomy, gastrostomy, acute renal failure, sepsis, infection (surgical site, pneumonia, urinary tract, and bacteremia), venous thromboembolism, rhabdomyolysis, and unplanned return to the operating room.Demographic data included age, race, sex, and body mass index.Additional data collected included comorbidities (systemic inflammatory response syndrome/ sepsis, ventilator dependence, diabetes, hypertension, congestive heart failure, dyspnea, smoking, severe chronic obstructive pulmonary disease, dialysis, and acute renal failure), functional status, ASA-PS score (assigned by the attending anesthesiologist), mechanism of injury, initial vital signs (HR, RR, and SBP), and GCS.

Predicting Mortality
The survivor group was compared to the non-survivor group using Fisher's exact test for categorical variables and Wilcoxon rank-sum test for continuous variables.Brier score and area under the receiver-operating characteristic curve (AUROC) were calculated for each model to assess prediction quality.The AUROC of each model was compared using chi-squared tests to assess for statistical difference.

Predicting Length of Stay
Linear regression models were used to predict hospital LOS.Pseudo R 2 , median error, and average absolute error (AAE) were calculated for all three models.Models were further compared with a likelihood ratio (LR) test to assess for statistical differences in model fit.Each model was used to predict median LOS, interquartile range (IQR), and minimum (min)/maximum (max) predicted values of LOS.Predicted values were compared to observed values for each model to assess if the distribution of predicted values mirrored the observed values, thereby assessing prediction quality.

Predicting Number of Complications
The number of complications was not normally distributed, and few patients had a large number of complications so negative binomial regression was used to estimate number of complications for each model (TRISS vs NSQIP-SRC vs TRISS + NSQIP-SRC).Pseudo R 2 , median error, and AAE were calculated for each of the three models.Nested models were further compared with a likelihood ratio (LR) test, whereas Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used for non-nested models to assess for statistical differences in model fit.All three models were used to predict the median number of complications, interquartile range (IQR), and min/max predicted values which were compared to observed values for each model.Statistical significance was set at P-value <.05.All analyses were performed by a statistician using Stata version 15.

Demographics, Comorbidities, Functional Status, and Injury Profile
There were 284 adult high-risk trauma patients who underwent an operation within 24 hours of admission.The patients were predominately male (80%) with a mean age of 43.The most common mechanisms of injury were gunshot wound (20.4%) and pedestrian struck (19.4%), and there were more ASA-PS score IV (57%) compared to ASA-PS V (43%) patients.

Mortality
A total of 48 (16.9%) high-risk trauma patients died during index hospitalization.A summary of the demographics/ characteristics in each of the cohorts (ie, survivors and nonsurvivors) can be found in Tables 1 and 2. The TRISS +

Length of Stay
The median observed LOS in the overall population was 16 days (IQR 7-31) with a range of 1-169 days.A comparison of each prediction model (TRISS, NSQIP-SRC, and TRISS + NSQIP-SRC) with the observed values can be found in Table 4. NSQIP-SRC and TRISS + NSQIP-SRC models performed better than TRISS alone based on higher pseudo R 2 (1.40% vs 1.42% vs .00%)and lower median error (13.42 vs 13.30 vs 14.01) (Table 5).However, there was no difference between NSQIP-SRC and TRISS + NSQIP-SRC when compared with LR test (χ 2 = .61,P = .435).

Discussion
Despite the enormous health, social, and economic burden of traumatic injuries, 1,9 it remains unknown which tools are most accurate in predicting in-hospital outcomes (ie, mortality, LOS, and complications) for high-risk operative trauma patients.Therefore, this multicenter analysis aimed to compare risk prediction tools that have been previously validated for operative general surgery and generalized trauma patients 7,10 in a high-risk trauma cohort.The current study found that a model using TRISS plus NSQIP-SRC together better predicted mortality and number of complications compared to either tool alone.
Regarding hospital LOS, NSQIP-SRC alone performed better than TRISS alone and just as well as TRISS + NSQIP-SRC in high-risk trauma patients undergoing surgery within 24 hours of admission.Mortality prediction is particularly important in highrisk trauma patients as this is an objective and unfortunately not uncommon occurrence with a reported incidence as high as 25% among this high-risk surgical population. 11,12Interestingly, this current study reports a mortality rate of 16.9% which is relatively low considering our cohort was all high-risk (ASA-PS IV/V) trauma patients.This may be a reflection of improved trauma care and survival rates compared to historical reports.In terms of mortality prediction, prior studies have demonstrated that TRISS and NSQIP-SRC perform well in the generalized trauma population 5,10,[13][14][15] while NSQIP-SRC is superior in the elderly trauma population. 7herefore, it is not surprising that combining NSQIP-SRC with TRISS would produce the most accurate model for mortality prediction in high surgical risk trauma patients given that it incorporates injury severity, physiologic status, comorbidities, and functional status which allows for a holistic assessment of a complex and aging trauma population.
As improvements are made to trauma care with decreasing rates of mortality in trauma, hospital systems have become more interested in quality metrics such as complications and hospital LOS.7][18][19][20] As NSQIP-SRC incorporates some aspects of these variables, which are present in many high surgical risk trauma patients, it seems logical that TRISS + NSQIP-SRC best predicted number of complications.In contrast, regarding LOS there was no difference between NSQIP-SRC alone compared to TRISS + NSQIP-SRC.This data is consistent with a prior study of 1500 patients in New Zealand which showed TRISS to be a poor prediction model for hospital LOS. 16As such, the use of TRISS alone to evaluate LOS should be discontinued and adoption of the variables included in NSQIP-SRC may prove helpful for trauma centers when evaluating their own outcomes or comparing quality metrics.
There are several limitations to this study, including those intrinsic to its multicenter design and reliance on multiple professionals for data collection.Most significant is the usage of multiple anesthesiologists to assign ASA-PS scores which may be inconsistent for trauma patients.Furthermore, ASA-PS score assignments were not explicitly stated to be pre-injury or post-injury which may result in score variability.However, this is at least partially mitigated by the performance of this study at high volume trauma centers, with comfort in this practice of assigning ASA-PS to trauma patients.Furthermore, there were undoubtably some patients included in the study who had unknown medical history and no family to provide history which may affect ASA-PS and NSQIP-SRC evaluation.
Another limitation to this study is the relatively small sample size despite its multicenter design.Additionally, we evaluated complications as a continuous variable which may have less clinical relevance rather than it being a categorical (binary) variable.Also, we defined high surgical risk trauma patients as an assigned ASA score of IV or V which may differ from other methodologies limiting direct comparison with previous literature.Nonetheless, we believe using this definition is appropriate as it is applicable to clinical practice.In addition, our results are based on trauma patients who underwent operative intervention within 24 hours of presentation; therefore, the findings may not be applicable to patients undergoing surgery later in their hospitalization.However, this study was prospective and evaluated the highest risk trauma patients who would be expected to undergo surgery within the included time period.Future studies are needed to validate these findings in a generalized trauma population.Lastly, the proposed TRISS + NSQIP-SRC model is complex.Though, in the setting of quality metrics and comparisons across medical centers, both of these tools are already being utilized, but separately.This study suggests that by combining these easily performed tools, there is an increased accuracy of estimating patient outcomes which results in a better evaluation of quality metrics.

Conclusions
This subgroup analysis of a prospective multicenter study is the first to evaluate NSQIP-SRC and TRISS for the prediction of outcomes in high surgical risk (ASA-PS IV or V) trauma patients undergoing surgery within 24 hours of admission.A model using TRISS plus NSQIP-SRC together better predicted mortality and number of complications than either tool alone.However, NSQIP-SRC alone performed as well as TRISS + NSQIP-SRC at predicting hospital LOS.Further studies are needed to validate these findings and improve upon prediction models in high-risk trauma patients.In the meantime, this data suggests that comparisons across trauma centers for high-risk operative trauma patients should include a combination of anatomic and physiologic data from TRISS plus comorbidities and functional status data from NSQIP-SRC.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Table 1 .
Demographics and Comorbidities of High-Risk Trauma Patients Who Underwent an Operation Within 24 Hours of Admission.
SIRS = systemic inflammatory response syndrome, CHF = congestive heart failure, COPD = chronic obstructive pulmonary disease, ASA-PS = American Society of Anesthesiologists physical status, RTS = revised trauma score.a Fisher's exact test.

Table 2 .
Functional Status and Injury Characteristics of High-Risk Trauma Patients Who Underwent an Operation Within 24 Hours of Admission.

Table 3 .
Brier Scores and AUROC Curve for Models to Predict Mortality in High-Risk Operative Trauma Patients.
TRISS = Trauma and Injury Severity Score, ASA-PS = American Society of Anesthesiologists physical status, NSQIP-SRC = National Surgical Quality Improvement Program Surgical Risk Calculator.

Table 4 .
Hospital Length of Stay (Days) and Number of Complications in the Observed and Predictive Models for High-Risk Operative Trauma Patients.

Table 5 .
Predictive Ability for Length of Stay and Number of Complications for High-Risk Operative Trauma Patients.Trauma and Injury Severity Score, ASA-PS = American Society of Anesthesiologists physical status, NSQIP-SRC = National Surgical Quality Improvement Program Surgical Risk Calculator, AAE = average absolute error, IQR = interquartile range.