![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Objectives
Assessing high-sensitivity tests for mortal illness is crucial in emergency and critical care medicine. Estimating the 95% confidence interval (CI) of the likelihood ratio (LR) can be challenging when sample sensitivity is 100%. We aimed to develop, compare, and automate a bootstrapping method to estimate the negative LR CI when sample sensitivity is 100%.
Methods
The lowest population sensitivity that is most likely to yield sample sensitivity 100% is located using the binomial distribution. Random binomial samples generated using this population sensitivity are then used in the LR bootstrap. A free R program, “bootLR,” automates the process. Extensive simulations were performed to determine how often the LR bootstrap and comparator method 95% CIs cover the true population negative LR value. Finally, the 95% CI was compared for theoretical sample sizes and sensitivities approaching and including 100% using: (1) a technique of individual extremes, (2) SAS software based on the technique of Gart and Nam, (3) the Score CI (as implemented in the StatXact, SAS, and R PropCI package), and (4) the bootstrapping technique.
Results
The bootstrapping approach demonstrates appropriate coverage of the nominal 95% CI over a spectrum of populations and sample sizes. Considering a study of sample size 200 with 100 patients with disease, and specificity 60%, the lowest population sensitivity with median sample sensitivity 100% is 99.31%. When all 100 patients with disease test positive, the negative LR 95% CIs are: individual extremes technique (0,0.073), StatXact (0,0.064), SAS Score method (0,0.057), R PropCI (0,0.062), and bootstrap (0,0.048). Similar trends were observed for other sample sizes.
Conclusions
When study samples demonstrate 100% sensitivity, available methods may yield inappropriately wide negative LR CIs. An alternative bootstrapping approach and accompanying free open-source R package were developed to yield realistic estimates easily. This methodology and implementation are applicable to other binomial proportions with homogeneous responses.
References
| 1. | Gallagher, EJ . Clinical utility of likelihood ratios. Ann Emerg Med 1998; 31: 391–397. Google Scholar | Crossref | Medline | ISI |
| 2. | Hayden, S, Brown, M. Likelihood ratio: a powerful tool for incorporating the results of a diagnostic test into clinical decisionmaking. Ann Emerg Med 1999; 33: 575–580. Google Scholar | Crossref | Medline | ISI |
| 3. | Deeks, JJ, Altman, DG. Diagnostic tests 4: likelihood ratios. BMJ 2004; 329: 168–169. Google Scholar | Crossref | Medline |
| 4. | Perry, JJ, Stiell, IG, Sivilotti, ML, et al. Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study. BMJ 2011; 343: d4277. Google Scholar | Crossref | Medline |
| 5. | Weber, T, Hogler, S, Auer, J, et al. D-dimer in acute aortic dissection. Chest 2003; 123: 1375–1378. Google Scholar | Crossref | Medline | ISI |
| 6. | Eggebrecht, H, Naber, CK, Bruch, C, et al. Value of plasma fibrin D-dimers for detection of acute aortic dissection. J Am Coll Cardiol 2004; 44: 804–809. Google Scholar | Crossref | Medline | ISI |
| 7. | Szucs-Farkas, Z, Christe, A, Megyeri, B, et al. Diagnostic accuracy of computed tomography pulmonary angiography with reduced radiation and contrast material dose: a prospective randomized clinical trial. Invest Radiol 2014; 49: 201–208. Google Scholar | Crossref | Medline | ISI |
| 8. | Bhat, PK, Pantham, G, Laskey, S, et al. Recognizing cardiac syncope in patients presenting to the emergency department with trauma. J Emerg Med 2014; 46: 1–8. Google Scholar | Crossref | Medline | ISI |
| 9. | Haran, JP, Beaudoin, FL, Suner, S, et al. C-reactive protein as predictor of bacterial infection among patients with an influenza-like illness. Am J Emerg Med 2013; 31: 137–144. Google Scholar | Crossref | Medline | ISI |
| 10. | Swenson, DW, Lourenco, AP, Beaudoin, FL, et al. Ovarian torsion: case-control study comparing the sensitivity and specificity of ultrasonography and computed tomography for diagnosis in the emergency department. Eur J Radiol 2014; 83: 733–738. Google Scholar | Crossref | Medline | ISI |
| 11. | Esmailian, M, Khajouei, AS, Eghtedari, N, et al. Utilization of coronary computed tomography angiography for rapid risk stratification in emergency chest pain units. J Res Med Sci 2014; 19: 134–138. Google Scholar | Medline |
| 12. | Binomial proportion confidence interval, http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval (accessed 11 May 2014). Google Scholar |
| 13. | Scherer R. “Clopper-Pearson exact CI” in “Documentation for package ‘PropCIs’ version 0.2-4, 2013-08-04,” p. 9, http://cran.r-project.org/web/packages/PropCIs/PropCIs.pdf (accessed 15 October 2013). Google Scholar |
| 14. | Hanley, JA, Lippman-Hand, A. If nothing goes wrong, is everything all right?. JAMA 1983; 249: 1743–1745. Google Scholar | Crossref | Medline | ISI |
| 15. | Gart, JJ, Nam, J. Approximate interval estimation of the ratio of binomial parameters: a review and corrections for skewness. Biometrics 1988; 44: 323–338. Google Scholar | Crossref | Medline | ISI |
| 16. | Simel, DL, Samsa, GP, Matchar, DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 1991; 44: 763–770. Google Scholar | Crossref | Medline | ISI |
| 17. | Dann, RS, Koch, GG. Review and evaluation of methods for computing confidence intervals for the ratio of two proportions and considerations for non-inferiority clinical trials. J Biopharm Stat 2005; 15: 85–107. Google Scholar | Crossref | Medline | ISI |
| 18. | Price, RM, Bonett, DG. Confidence intervals for a ratio of two independent binomial proportions. Stat Med 2008; 27: 5497–5508. Google Scholar | Crossref | Medline | ISI |
| 19. | Fagerland, MW, Lydersen, S, Laake, P. Recommended confidence intervals for two independent binomial proportions. Stat Methods Med Res 2015; 24: 224–254. Google Scholar | SAGE Journals | ISI |
| 20. | Efron, B, Tibshirani, RJ. An introduction to the bootstrap, Boca Raton, FL: Chapman & Hall/CRC, 1993. Google Scholar | Crossref |
| 21. | Haukoos, JS, Lewis, RJ. Advanced statistics: bootstrapping confidence intervals for statistics with “difficult” distributions. Acad Emerg Med 2005; 12: 360–365. Google Scholar | Crossref | Medline | ISI |
| 22. | DiCiccio, T, Tibshirani R. Bootstrap confidence intervals and bootstrap approximations. J Am Stat Assoc 1987; 82: 163–170. . Google Scholar | Crossref | ISI |
| 23. | Santner, TJ, Snell, MK. Small-sample confidence intervals for p1-p2 and p1/p2 in contingency tables. J Am Stat Assoc 1980; 75: 386–394. Google Scholar | ISI |
| 24. | Farrington, CP, Manning, G. Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk. Stat Med 1990; 9: 1447–1454. Google Scholar | Crossref | Medline | ISI |
| 25. | Chan, ISF, Zhang, Z. Test-based exact confidence intervals for the difference of two binomial proportions. Biometrics 1999; 55: 1202–1209. Google Scholar | Crossref | Medline | ISI |
| 26. | Agresti, A, Min, Y. On small-sample confidence intervals for parameters in discrete distributions. Biometrics 2001; 57: 963–971. Google Scholar | Crossref | Medline | ISI |
| 27. | Cytel Software Corp . StatXact 10 User Manual, Cambridge, MA: Cytel Software Corp, 2012, pp. 489–490. , 527–533. Google Scholar |
| 28. | Scherer R. “Score confidence interval for the relative risk in a 2x2 table” in “Documentation for package ‘PropCIs’ version 0.2-4, 2013-08-04,” p. 12, http://cran.r-project.org/web/packages/PropCIs/PropCIs.pdf (accessed 15 October 2013). Google Scholar |
| 29. | Agresti, A . Categorical data analysis, 2nd ed. Hoboken, NJ: John Wiley & Sons, 2002, pp. 73–78. Google Scholar | Crossref |
| 30. | Efron B and Tibshirani RJ. Chapter 12: confidence intervals based on bootstrap ‘tables’. In: Efron B and Tibshirani RJ (eds) An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall/CRC, 1993, pp. 153–167. Google Scholar |
| 31. | Czuczman, AD, Thomas, LE, Boulanger, AB, et al. Interpreting red blood cells in lumbar puncture: distinguishing true subarachnoid hemorrhage from traumatic tap. Acad Emerg Med 2013; 20: 247–256. Google Scholar | Crossref | Medline | ISI |
| 32. | Briggs, AH, Wonderling, DE, Mooney, CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ 1997; 6: 327–340. Google Scholar | Crossref | Medline | ISI |
| 33. | Keller, T, Zeller, T, Ojeda, F, et al. Serial changes in highly sensitive troponin I assay and early diagnosis of myocardial infarction. JAMA 2011; 306: 2684–2693. Google Scholar | Crossref | Medline | ISI |
| 34. | Efron B and Tibshirani RJ. Chapter 14: better bootstrap confidence intervals. In: Efron B and Tibshirani RJ (eds) An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall/CRC, 1993, pp. 178–201. Google Scholar |
