Cost-effectiveness of Artificial Intelligence for Proximal Caries Detection

Artificial intelligence (AI) can assist dentists in image assessment, for example, caries detection. The wider health and cost impact of employing AI for dental diagnostics has not yet been evaluated. We compared the cost-effectiveness of proximal caries detection on bitewing radiographs with versus without AI. U-Net, a fully convolutional neural network, had been trained, validated, and tested on 3,293, 252, and 141 bitewing radiographs, respectively, on which 4 experienced dentists had marked carious lesions (reference test). Lesions were stratified for initial lesions (E1/E2/D1, presumed noncavitated, receiving caries infiltration if detected) and advanced lesions (D2/D3, presumed cavitated, receiving restorative care if detected). A Markov model was used to simulate the consequences of true- and false-positive and true- and false-negative detections, as well as the subsequent decisions over the lifetime of patients. A German mixed-payers perspective was adopted. Our health outcome was tooth retention years. Costs were measured in 2020 euro. Monte-Carlo microsimulations and univariate and probabilistic sensitivity analyses were conducted. The incremental cost-effectiveness ratio (ICER) and the cost-effectiveness acceptability at different willingness-to-pay thresholds were quantified. AI showed an accuracy of 0.80; dentists’ mean accuracy was significantly lower at 0.71 (minimum–maximum: 0.61–0.78, P < 0.05). AI was significantly more sensitive than dentists (0.75 vs. 0.36 [0.19–0.65]; P = 0.006), while its specificity was not significantly lower (0.83 vs. 0.91 [0.69–0.98]; P > 0.05). In the base-case scenario, AI was more effective (tooth retention for a mean 64 [2.5%–97.5%: 61–65] y) and less costly (298 [244–367] euro) than assessment without AI (62 [59–64] y; 322 [257–394] euro). The ICER was −13.9 euro/y (i.e., AI saved money at higher effectiveness). In the majority (>77%) of all cases, AI was less costly and more effective. Applying AI for caries detection is likely to be cost-effective, mainly as fewer lesions remain undetected. Notably, this cost-effectiveness requires dentists to manage detected early lesions nonrestoratively.

In the base-case, a population with low caries prevalence and low risk of lesion development and progression was simulated; in a sensitivity analysis, a population with high prevalence and risk was assessed. The rationale for not modelling this uncertainty in one joint evaluation (e.g. using uncertainty distributions) was to display risk-specific cost-effectiveness, as this has been found relevant in the context of our study (Schwendicke et al. 2015a). Surface-prevalence of proximal caries lesions was estimated based on data from Sweden (Mejàre et al. 2004) as described before (Schwendicke et al. 2015a).

Cost estimation
German dentists use fee items to claim for reimbursement for dental treatments. For most dental procedures and patients, items will be drawn from the public catalogue BEMA. For few treatments (composite restorations in posterior teeth, implants and ISC), fees are usually derived from the private catalogue; publicly insured patients pay the additional costs ouf-ofpocket or via additional private insurers For GOZ, factoring of the chargeable item points is common to determine the fees of private treatment in Germany; we used the standard multiplication factor (2.3). Using fee items allowed us to estimate costs occurring to payers, which was in line with our study perspective (Schwendicke et al. 2013a).
BEMA defines fee items within the public insurance, which covers 87% of insured Germans (GKV-Spitzenverband 2013), with only few treatments not being fully covered or reimbursed.
For these items, calculation was based on GOZ or "analogue-items". BEMA points vary between federal states, insurers and treatment groups. For our cost calculation we used mean state points for the biggest insurer (AOK) from one federal state, Bavaria. For GOZ, point values were applied, with 0.0562421 Euro/point. Factoring of GOZ item-points was usually performed via the standard multiplication factor (×2.3). Certain positions (mainly radiographic assessments) are further coded in GOÄ (Gebührenordnung für Ärzte), with 9 GOÄ points equaling 1 BEMA point. Laboratory and material costs were estimated based on Laboratory Fee Catalogues (BEL II/BEB). Costs for BELII/BEB have been transformed into monetary values for the following tables. Items were restricted in number and character to reflect cost limitations. Total costs per course of treatment were calculated based on the quantification (q) of itemized costs (c), i.e. c1×q1 + c2×q2 etc., and calculated in Euro. Details can be found further below.

Net benefit approach and cost-effectiveness acceptability
Using estimates for costs (c, in Euro) and effectiveness (e, in years), the net benefit of each strategy combination was calculated using the formula net benefit = λ × ∆e -∆c, with λ denoting the ceiling threshold of willingness to pay, i.e. the additional costs a decision maker is willing to bear for gaining an additional unit of effectiveness (Drummond et al. 2005).
If λ>∆c/∆e, an alternative intervention is considered more cost-effective than the comparator despite possibly being more costly (Briggs et al. 2002). We used the net-benefit approach to calculate the probability of a detection strategy being acceptable regarding its costeffectiveness for payers with different willingness-to-pay ceiling thresholds.

Detailed calculation of costs per course of treatment
Costs for visual-tactile detection were assumed to be based on BEMA 01. We assumed that the costs of a regular examination would be distributed over all examined teeth and expected an average of 28 teeth to be examined, i.e. BEMA 01 equaling 18.90 Euro divided by 28 = 0.68 Euro. Costs for radiographic assessment were estimated based on BEMA Ä925a. We assumed all posterior teeth to be assessed, i.e. costs were distributed over 16 teeth. Hence, GOÄ925a equaling 12.60 was divided by 16 = 0.79. The costs for the AI intervention were varied between 4.00 to 12.00 Euro per image analysis, as it is currently unclear which costs would be generated; the base-case costs were set at 8.00 Euro. AI costs were also distributed over the teeth as described. A more detailed assessment of AI costs can be found below.
Infiltration was assumed to cost 84.99 Euro as described elsewhere (Schwendicke et al. 2014b). All other costs were estimated as follows.
(1) Costs for AI We assumed the following costs to have occurred for establishing the AI intervention: -Data generation and labeling: 3,686 images, 4 annotators, each annotation per image 5 min, average salary incl tax and social security contributions approx. 39,- We further assumed costs per goods sold (COGS) to occur as follows: -Cloud infrastructure. According to our simulations, costs of 1-3 Euro per image analysis occur depending on the load and the accepted waiting time for each dentist.
-Support etc. including possible step-by-step instructions; we assumed one monthly email and one monthly phone contact, each consuming 10 min of a support employee's time, i.e. 39,-Euro per hour (TV-E13), in total 13 Euro/ month.
Assuming each dentist to use the intervention for each bitewing, at an estimated number of 20 bitewings per month, this summed up to 0.65 Euro per month.
-The total COGS hence summed up to 1.65-3.65 Euro/image. Total costs per image were hence assumed to be 4.05 Euro to 6.05 Euro per analysis. We further assumed any provider of such a tool would also need to generate additional revenue for management, sales and marketing, development and profit (overhead), and assumed this block to account for 55% of the abovementioned costs (i.e. 2.24-3.33 Euro). Hence, total costs were assumed to range between 6.29-9.38 Euro. Given that developmental costs would, however, be diluted with each use case exceeding 100,000 analyses, and given that the costs for revenue, sales and marketing, development and profit would also be reduced if the intervention was scaled up in its use, our cost estimate between 4-12 Euro seemed to realistically cover possible cost scenarios.
(2) Direct capping and direct restoration