Assessing the Accuracy of International Classification of Diseases (ICD) Coding for Delirium

Objective: We assessed the accuracy of the ICD-10 code for delirium (F05) and its relationship with delirium discharge summary documentation. Methods: We performed a retrospective chart review at three academic hospitals. The Chart-based Delirium Identification Instrument (CHART-DEL) was used to identify 108 hospitalized patients aged ≥65 years with delirium, and 758 patients without delirium as controls. We assessed the proportion of patients who received the F05 code and calculated the sensitivity and specificity. We compared the rates of F05 code received between patients with and without “delirium” documented in the discharge summary. Results: Among delirious patients, 46.3% received a F05 code, which has a sensitivity of 46.3% and specificity of 99.6% for delirium. Of charts with “delirium” in the discharge summary (n = 67), 67.2% were appropriately coded. Conclusions: Current ICD-10 data inadequately capture delirium. Delirium documentation in the discharge summary is associated with improved delirium coding.


Introduction
Delirium is an acute disturbance in attention, awareness, and cognition, (American Psychiatric Association, 2013) affecting 29-64% of older adults admitted to medical and geriatric wards (Inouye et al., 2014). The World Health Organization International Classification of Diseases, Tenth Revision (ICD-10) is used to capture diagnoses from patient hospitalizations. These data generate statistics on healthcare utilization and disease burden for administrative, research and funding purposes. Within delirium research, identifying delirium requires prospective clinical assessments using validated tools such as the Confusion Assessment Method (CAM), or the time-intensive CHART-DEL, (Inouye et al., 2005) validated to identify delirium from the chart, in retrospective studies. Using readily available ICD data is enticing but has questionable appropriateness and validity. Previous studies demonstrate exclusive use of ICD data underestimates delirium (Casey et al., 2019;Katznelson et al., 2010).
To our knowledge, there have not been studies comparing the accuracy of ICD-10 coding for delirium with the CHART-DEL. We aimed to address this gap and identify variables associated with accurate ICD-10 coding for delirium.

Setting and Patients
We performed a multi-center retrospective chart review of patients aged ≥65 years old, consecutively admitted to one of three academic tertiary acute care hospitals in Toronto, Canada between April 1 and June 30, 2016, under a medical (General Internal Medicine/Clinical Teaching Unit, Hospitalist, Neurology, Cardiology) or surgical (General Surgery, Orthopedic Surgery, Cardiac Surgery, Neurosurgery) service.
These were included based on their higher estimated prevalence of delirium and comprised both elective and nonelective admissions (Inouye et al., 2014). Our study was approved by our institutions' research ethics board prior to commencement with a waiver for individual patient consent.
We consecutively screened 1168 charts for delirium using the CHART-DEL (74% sensitivity, 83% specificity) (Inouye et al., 2005). Parallel ratings for 10 charts were completed between an expert rater (i.e., geriatrician) and each researcher. We identified 118 patients with a "Definite" or "Probable" delirium, (Xu et al., 2011) and included 108 patients with ICD-10 data available. Of 961 patients screened negative for delirium, 758 patients were included consecutively as controls, and 203 patients excluded due to incomplete ICD-10 data abstraction. We also excluded patients with missing chart components or who died in hospital, as the latter have abbreviated discharge summaries (Figure 1).

Data Collection
For delirious patients, we collected demographic data (e.g., age, baseline cognition, functional status, and others). A history of delirium was identified by reviewing the chart for documented delirium dating one year prior. Using hospital administrative ICD datasets, we assessed whether patients received an ICD-10 code for delirium (F05: "Delirium due to known physiological condition," inclusive of all subheadings within the stem). We defined delirium by the F05 code and its subheadings based on previous studies assessing for delirium through ICD-10 data (Casey et al., 2019;Clegg et al., 2011). We evaluated the proportion of delirious patients who received a code relating to symptoms of delirium or other terminology. As our primary outcome was to evaluate the accuracy of the F05 code, we did not determine a priori other codes that could be relevant to delirium. Rather, our ICD-10 dataset included titles of all ICD-10 codes, which allowed us to evaluate their relevance to delirium. In our institutions, ICD-10 codes are assigned by trained administrators, independent of the patient's clinical team and our research team. Assessment of the ICD-10 codes assigned was completed after CHART-DEL screening to minimize bias during delirium screening.
Of patients identified to have delirium, we assessed each patient's discharge summary for delirium documentation. This included the use of "delirium" specifically, and other terms, such as "confusion" or "disorientation," as outlined in Appendix II, CHART-DEL Manual (Xu et al., 2011).

Statistical Analysis
Using the CHART-DEL as our reference standard to identify delirium, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the F05 code. We calculated the proportion of patients with delirium documentation in their discharge summary who received the F05 code and used a chi-squared analysis to assess differences in F05 coding between patients with and without delirium discharge summary documentation.
A univariate and multivariable logistic regression was performed on delirious patients (n = 108) to identify factors predictive of receiving a F05 code. Variables were selected a priori, by researcher consensus (two geriatricians with clinical and/or research expertise in delirium). These included patient sex, history of dementia, Charlson Comorbidity Index (CCI), length of stay, delirium symptoms prior to hospitalization, delirium recognition by the attending team, formally diagnosed delirium, delirium type, delirium duration, discharge summary structure, documentation of delirium in the discharge summary, and author of discharge summary. A multivariate logistic regression model using an Akaike Information Criterion stepwise backwards approach determined independent predictors for the F05 code. Model discrimination was calculated with the C-statistic and calibration with the Hosmer and Lemeshow goodness of fit test.

Results
We included 108 patients with delirium identified from the chart and 758 patients as controls. Patients were on average 79.7 years old and 55.6% were male ( Table 1).

F05 Coding and Delirium Discharge Summary Documentation
Sixty-seven delirious patients (62.0%) had "delirium" documented in the discharge summary, whereas 85 delirious patients (78.7%) had more broadly, "delirium" or another acceptable term. Charts with "delirium" in the discharge summary received a F05 code significantly more than those without documentation (67.2% vs. 12.2%, p < .001). Charts with "delirium" or acceptable term in the discharge summary similarly received significantly more F05 codes compared to charts without documentation (54.1% vs. 17.4%, p = .002). Of charts without delirium discharge summary documentation (n = 21), 90.5% (n = 19) were diagnosed with delirium using the term "delirium", as documented in the chart somewhere other than the discharge summary.

Factors Affecting Delirium International Classification of Diseases Coding
The only univariate positive predictor for receiving a F05 code was having "delirium" documented in the discharge summary (OR 9.72, 95% CI (2.95-32.02]) ( Other category included complex continuing care, long-term care unit in hospital. Retirement homes are multi-residence living facilities generally for adults ≥65 years, who have their own unit; additional facilities are often provided by the home such as recreational activities or meals. Nursing home and long-term care homes are living facilities for individuals who require assistance for their basic activities of daily living and who have a higher level of medical care required. e Baseline social supports were evaluated for patients residing at home and excluded those living in institutionalized care. External supports included government and privately funded services f Charlson Comorbidity Index was calculated by the research team using chart abstraction data, rather than ICD-10 coding data. Weighting was inclusive of dementia, but not adjusted for age (Charlson et al., 1987).

Discussion
Less than half (n = 50) of delirious patients identified by the CHART-DEL received the F05 code. The sensitivity of 46.3% is improved from previously reported sensitivities of 3-12%. (Inouye et al., 2005;Katznelson et al., 2010;Pendlebury et al., 2020) We suspect differences are attributed to the reference standard used (CAM vs. CHART-DEL). Inouye et al. found absent delirium documentation increased false negative coding results (Inouye et al., 2005). Our study identified more true positives, related to improved delirium documentation, as selected for by our screening methodology; however, we likely  omitted delirious patients with poor chart documentation. Over half (53.7%) of our delirious patients did not receive a F05 code, which illustrates inaccurate coding despite adequate chart documentation. However, 16 delirious patients who did not receive a F05 code received a related ICD-10 diagnosis or symptom (Table 2). Although coders are not permitted to infer diagnoses, 98 of the 108 delirious patients received a diagnosis through the specific term "delirium" as documented in the chart somewhere other than the discharge summary. This significant discrepancy in accurate F05 coding can have implications on our local hospital funding and reported delirium statistics. The specificity of the F05 code remains high (99.6%), similar to previous (Inouye et al., 2005;Pendlebury et al., 2020). Notably, the sensitivity and specificity reported is for delirium identified by the CHART-DEL as the reference standard. As the CHART-DEL has a sensitivity of 74% for detecting delirium, our reported sensitivity of the F05 code (46.3%) is an overestimate. Therefore, our results support using the F05 code to identify delirium instead of the laborintensive CHART-DEL. However, the F05 code cannot reliably identify all delirium cases, and therefore, cannot be used to exclude delirium or estimate prevalence or incidence. If used exclusively to identify delirium, researchers must acknowledge the bias toward patients with better delirium documentation.
Although the F05 code has a high PPV (94.3%) and NPV (92.8%) for delirium, both tests are a function of disease prevalence. As delirium prevalence increases, the NPV of the F05 code is expected to decrease. This differs from sensitivity and specificity, which remain unchanged irrespective of prevalence, and therefore are of greater utility. Hence, improving the sensitivity of the F05 code is important and our results support the impact of "delirium" documentation on accurate F05 coding.
A limitation of our study is the ICD-10 outcomes were not used to determine our sample size, as this study was completed in follow-up to another investigation. We also acknowledge our sample inherently captures patients with adequate delirium chart documentation, as this was required for detection by the CHART-DEL. However, not all charts had "delirium" documented in the discharge summary, and when it was, a F05 code was assigned significantly more often. This emphasizes the importance of documenting "delirium" in the discharge summary by the attending team to improve coding accuracy. Another way of improving the accuracy of coding for delirium involves implementing multicomponent interventions as outlined by Pendlebury et al., which increased the sensitivity of the F05 coding for delirium, as identified by the gold standard of prospective clinical assessments using the CAM and DSM criteria, from 12.8% to 60.2% (Pendlebury et al., 2020).
The presence of delirium symptoms at presentation was negatively associated with accurate ICD-10 coding, but not maintained in our multivariable analysis. We emphasize the impact of detailed discharge summaries on accurate coding as our previous study showed delirium symptoms at admission was negatively associated with delirium documentation in the discharge summary (Chuen et al., 2021). Additionally, hypoactive delirium and increasing comorbidity scores were negatively associated with receiving a F05 code in our multivariable analysis. Hypoactive delirium is underrecognized and poorly documented (Voyer et al., 2008). This negatively impacts coding accuracy as coding administrators rely solely on chart information. Our results demonstrate increasing comorbidity by the CCI score was also negatively associated with receiving a F05 code. This conflicts with previous results from Pendlebury et al., where it was associated with improved delirium coding (Pendlebury et al., 2020). This may be attributed to differences in methodology, as our CCI scores were calculated from abstracted chart data rather than ICD-10 codes received, or differences in training or depth of chart review for coding administrators. Another study has also demonstrated more comorbid patients experience greater inaccuracies in coding (Boustani et al., 2010).
The American Health Information Management Association specifies the discharge summary and its listed diagnoses as key components to improving coding accuracy and specificity (Cassidy, 2012). Our results demonstrate delirium documentation in the discharge summary is associated with increased delirium ICD-10 coding accuracy. Efforts are needed to improve delirium documentation in discharge summaries.
Identifying delirium retrospectively is challenging and time consuming. The F05 code for delirium is highly specific for identifying delirium. Unfortunately, the sensitivity is subpar, which limits its utility in estimating delirium incidence or prevalence. In delirious patients, the documentation of "delirium" in the discharge summary was predictive for receiving a F05 code. Therefore, improving the delirium documentation in discharge summaries is one method of improving ICD-10 delirium coding accuracy.
financial or personal conflicts to disclose. JM has no financial or personal conflicts to disclose. SA has no financial or personal conflicts to disclose. VC2 has no financial or personal conflicts to disclose.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Savlov/Schmidt Scholars Program in Geriatrics, through the Sinai Health Systems and University Health Network and the Comprehensive Research Experience for Medical Students (CREMS) Office through the University of Toronto, Faculty of Medicine. The funders did not have any role in the study design, data analysis or interpretation, preparation, review, and approval of the final manuscript.

IRB Protocol Approval Numbers
Our study protocol was approved by the Mount Sinai Hospital Research Ethics Board (REB#16-0260-C) and the University Health Network Research Ethics Board (REB# 16-5750.5) prior to commencement, who granted permission to access and use the medical records described in this study. A waiver for individual patient consent was granted as well.

Supplemental Material
Supplemental material for this article is available online.