Cost-effectiveness of Artificial Intelligence for Proximal Caries Detection

Dental caries affects more than 3 billion individuals globally and generates significant costs and health care burden (Kassebaum et al. 2017). Building on the understanding that initial (noncavitated) caries lesions can be arrested non- or microinvasively (e.g., fluoride varnish application, sealing, caries infiltration), the traditional invasive/restorative therapy of caries is restricted to advanced lesions, mainly as the placement of restorations has been demonstrated to initiate a spiral of escalating and increasingly expensive retreatments (Frencken et al. 2016; Schwendicke, Splieth, et al. 2019).

Early management of noncavitated lesions requires their detection first. The standard diagnostic strategy, visual-tactile inspection, does usually not permit detecting early lesions on nonassessable (e.g., proximal) surfaces (Gimenez et al. 2015). A common additional method to detect early lesions on proximal surfaces and to assess their extent is bitewing radiography (Schwendicke, Stolpe, et al. 2015). While being more sensitive for detecting early lesions than visual-tactile assessment, the assessment of bitewings comes with significant variance between examiners and a considerable proportion of false-positive or false-negative detections (Schwendicke, Stolpe, et al. 2015).

Artificial intelligence (AI), specifically deep learning using convolutional neural networks (CNNs), has been suggested to help overcome the limited reliability and validity of dental image analysis. CNNs allow to map an input (image) to an output (classification), based on a set of weights, learned from data (LeCun et al. 2015). The learning process involves labeling an image and providing both the image and the label to the CNN, which iteratively adjusts itself to eventually be able to predict the presence of the labeled entity (a carious lesion) on unseen data. For detecting caries lesions, we previously trained, validated, and tested a fully convolutional neural network, U-net, yielding an accuracy superior to individual dentists (Garcia Cantu et al. 2020). Notably, the CNN showed significantly higher sensitivity than dentists, especially for detecting early lesions (more details are provided below). Dentists would use this AI technology via a software application, in our case allowing to display detected carious lesions on bitewings provided to the software via an upload function. The clinical flow would be expanded by the radiographic viewing software, for instance, allowing to assess bitewings in their native state and using an overlay of AI-detected lesions (assisting the assessment of lesion extension/depth).

In clinical care, the detection of a pathology (a caries lesion) has only limited impact on the patient (e.g., his or her health) or the health care system (e.g., the generated costs). Health outcomes and costs are instead determined by the subsequent treatment decisions (the costs for the diagnostics themselves are usually only a fraction of the treatment costs). We previously demonstrated that a systematic evaluation of diagnostic strategies should be performed in combination with the provided therapy, ideally over a long-term horizon, mainly as initial treatment decisions come with long-term consequences (e.g., retreatments required and costs generated) (Schwendicke, Paris, et al. 2015; Schwendicke, Stolpe, et al. 2015). In the present study, we aimed to assess the cost-effectiveness of employing AI for proximal caries detection on bitewing radiographs.

The number of studies using AI and, specifically, deep learning in dentistry and dental image analysis is rapidly increasing (Schwendicke, Golla, et al. 2019). So far, these studies largely have focused on evaluating the developed models for their accuracy, which, by itself, transports only limited value to patients, providers, and health care organizers. This value rather emanates from the subsequent decisions made, as these determine if, for example, a caries lesion is arrested or progressed and if a tooth is lost or retained long term, all of which come with associated costs (Baelum et al. 2012). The present study assessed the cost-effectiveness of an AI-based diagnostic assistance application for caries detection and found AI less costly and more effective over a lifetime horizon despite being more costly initially. We hence confirm our hypothesis.

A number of aspects need to be discussed. First, and as shown in other instances in dental cost-effectiveness analyses before, we found the initially more costly comparator to be cost-effective long term (Schwendicke et al. 2016). This was because costs occurring over the lifetime of a patient, mainly for restorative, prosthetic, or further care, are rather high and accumulate in an escalating fashion. The relevance of costs for tooth replacement, for example, was confirmed in a sensitivity analysis of the present study. Our findings highlight the relevance of an appropriate study horizon to yield meaningful results; a short-term follow-up study would not have been able to reflect this comprehensively. Second, this cost-effectiveness was realized largely via AI having a higher sensitivity, facilitating effective arresting therapies. Notably, the difference in cost-effectiveness between AI and no AI was not as pronounced as the difference in sensitivities (AI was by large twice as sensitive as the dentists) mainly due to reversed differences in specificity, in which the dentists showed a limited advantage. This demonstrates the effects false-positive diagnoses can have on cost-effectiveness and highlights the importance of the test specificity as a measure of accuracy. Our findings suggest the need of dentists doublechecking any AI suggestions. Third, and associated, the cost-effectiveness emanating from sensitivity and specificity stems from the deduced therapies; we have demonstrated in previous studies that combining a sensitive detection method is only cost-effective if “safe,” noninvasive, or microinvasive therapies are applied (Schwendicke, Paris, et al. 2015). Assigning invasive therapies to any (i.e., also early) detected lesion (as has been found the standard for many dentists worldwide) drastically changed the cost-effectiveness of AI: the assessment without AI showed nearly the same costs and effectiveness, mainly as the low sensitivity of dentists to detect early lesions allowed only very limited negative impacts of restoring these lesions. In contrast, AI was now by far more costly and less effective than assessment without AI. It seems recommendable that such sensitive AI applications provide information about not only the detected lesion but also its depth and recommended therapy options to mitigate the outlined detrimental effects of possibly harmful treatment decisions stemming from early lesion detection. Fourth, we found this cost-effectiveness to be modified by the risk profile of the population. In high-risk populations, it was more relevant to be sensitive (using AI) than in low-risk populations. Future AI applications should aim to reflect such risk profiles, for example, by cross-usage of meta-data of each practice (reflecting on prevalence) or risk profile data from each individual patient. Last, we found the cost-effectiveness ranking to hold true in a range of further sensitivity analyses. These reflected the uncertainties in costs but also accuracies, discount rates, and clinical pathways (via replacement probabilities for lost teeth). For costs, it was relevant to show that even for relatively high costs per AI application, cost-effectiveness was given. Notably, these costs are only assumptions right now and reflect the potential direct medical costs to providers; they are, however, grounded in credible assumptions toward the costs for developing and operating such an AI application, as detailed in the Appendix. Nonmedical or indirect costs (e.g., those for implementation, educating staff) were not considered. From a societal perspective, further costs (e.g., indirect costs to patients) may be relevant. Regarding the accuracies, we used different accuracy data for dentists and found the model to be cost-effective in both analyses. Notably, both analyses also showcased the discussed variability in dentists’ accuracy, which introduces uncertainty in cost-effectiveness (e.g., Fig. 2A). Future studies should aim to reflect these uncertainties in more depth and may be wanting to put monetary values on reducing them (e.g., through value-of-information analyses).

This study has a number of strengths and limitations. First, and as a strength, this is the first study assessing the cost-effectiveness of any “AI” or, specifically, deep learning intervention in dentistry. Our study thus provides, for the first time, a different perspective on the potential impact of this disruptive technology for dental care. It further can assist to inform decision makers in health care organizations but also research funding to prioritize (or not) further assessments of and investments into these technologies. Second, the used approach and the employed model have been validated before; they allow extrapolating short-term accuracy data into meaningful long-term outcomes (tooth retention and costs). Third, and as a limitation, the data informing our study were partially based on 1 specific diagnostic accuracy study, testing 1 deep learning model against a limited and likely nonrepresentative number of dentists. Comparing different models against different dentists may yield different outcomes. Fourth, our evaluation was focused on German health care and is not fully generalizable. This relates to both costs but also the modeled treatment decisions and further aspects such as assumptions toward tooth replacement or discounting. That said, cost estimation using fee items of the German fee item catalogues has been found to reflect the true treatment efforts to some degree and to yield estimates comparable with those from other health care settings (Schwendicke, Graetz, et al. 2014; Schwendicke et al. 2018). Moreover, they reflect the true direct medical costs to third party-payers and hence, indirectly, society and are relevant for decision makers in Germany. Also, while costs may somewhat differ in other health care systems, it is unlikely that the obtained ranking of interventions will change, but rather the magnitude of the observed differences. Last, and as mentioned, this study only partially reflects all aspects relevant to decision making; it is a simplification, and our findings should be confirmed by prospective randomized studies reflecting on the real-world decisions and cost-effectiveness emanating from the application of AI for diagnostic assistance in dentistry. For example, it is conceivable that dentists may not accept all detections and deviate in their treatment decisions, possibly to the benefit or detriment of patients’ health. Moreover, as outlined, treatment decisions will admittedly consider a range of factors (perceived lesion activity, patients’ wishes, dentists’ expertise), all of which will determine cost-effectiveness to some degree.

In conclusion, and within the limitations of this study, AI to support proximal caries detection on bitewings was cost-effective regardless of a payer’s willingness to pay. This cost-effectiveness was grounded in a higher sensitivity to detect early caries lesions, allowing to arrest them and thereby avoiding costly late retreatments. Notably, it only held true if detected early lesions were treated nonrestoratively. AI has the potential to improve care at lower health care costs.

F. Schwendicke, J. Krois, contributed to conception, design, data acquisition, analysis, and interpretation, drafted and critically revised the manuscript; J.G. Rossi, contributed to data acquisition, analysis, and interpretation, critically revised the manuscript; G. Göstemeyer, contributed to data acquisition and interpretation, critically revised the manuscript; K. Elhennawy, contributed to data analysis and interpretation, critically revised the manuscript; A.G. Cantu, S. Gehrung, contributed to data analysis, critically revised the manuscript; R. Gaudin, A. Chaurasia, contributed to data interpretation, critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.

We thank the dentists for their effort of labeling the image data.