Review: Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature
Abstract
Previous reviews of research on electronic health record (EHR) data quality have not focused on the needs of quality measurement. The authors reviewed empirical studies of EHR data quality, published from January 2004, with an emphasis on data attributes relevant to quality measurement. Many of the 35 studies reviewed examined multiple aspects of data quality. Sixty-six percent evaluated data accuracy, 57% data completeness, and 23% data comparability. The diversity in data element, study setting, population, health condition, and EHR system studied within this body of literature made drawing specific conclusions regarding EHR data quality challenging. Future research should focus on the quality of data from specific EHR components and important data attributes for quality measurement such as granularity, timeliness, and comparability. Finally, factors associated with poor or variability in data quality need to be better understood and effective interventions developed.
Get full access to this article
View all access and purchase options for this article.
References
Agnew-Blais, J.C., Coblyn, J.S., Katz, J.N., Anderson, R.J., Mehta, J., & Solomon, D.H. ( 2009). Measuring quality of care for rheumatic diseases using an electronic medical record. Annals of the Rheumatic Diseases, 68, 680-684.
Baldwin, K.B. ( 2008). Evaluating healthcare quality using natural language processing. Journal for Healthcare Quality, 30, 24-29.
Baker, D.W., Persell, S.D., Thompson, J.A., Soman, N.S., Burgner, K.M., Liss, D., et al. (2007). Automated review of electronic health records to assess quality of care for outpatients with heart failure. Annals of Internal Medicine, 146, 270-277.
Baron, R.J. ( 2007). Quality improvement with an electronic health record: Achievable, but not automatic. Annals of Internal Medicine, 147, 549-552.
Benin, A.L., Vitkauskas, G., Thornquist, E., Shapiro, E.D., Concato, J., Aslan, M., et al. (2005). Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Medical Care, 43, 691-698.
Conroy, M.B., Majchrzak, N.E., Silverman, C.B., Chang, Y., Regan, S., Schneider, L.I., et al. (2005). Measuring provider adherence to tobacco treatment guidelines: A comparison of electronic medical record review, patient survey, and provider survey. Nicotine and Tobacco Research, 7(Suppl. 1), S35-S43.
Dean, B.B., Lam, J., Natoli, J.L., Butler, Q., Aguilar, D., & Nordyke, R.J. ( 2009). Use of electronic medical records for health outcomes research: A literature review. Medical Care Research and Review, 66, 611-638.
de Lusignan, S., Chan, T., Wood, O., Hague, N., Valentin, T., & Van Vlymen, J. (2005). Quality and variability of osteoporosis data in general practice computer records: Implications for disease registers. Public Health, 119, 771-780.
de Lusignan, S., Hague, N., Brown, A., & Majeed, A. ( 2004). An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. Journal of Public Health, 26, 34-37.
desRoches, C.M., Campbell, E.G., Rao, S.R., Donelan, K., Ferris, T.G., Jha, A., et al. (2008). Electronic health records in ambulatory care: A national survey of physicians. New England Journal of Medicine, 359, 50-60.
Faulconer, E.R., & de Lusignan, S. (2004). An eight-step method for assessing diagnostic data quality in practice: Chronic obstructive pulmonary disease as an exemplar. Informatics in Primary Care, 12, 243-253.
Fishman, P.A., Hornbrook, M.C., Meenan, R.T., & Goodman, M.J. ( 2004). Opportunities and challenges for measuring cost, quality and clinical effectiveness in health care. Medical Care Research and Review, 61, 124S-143S.
Forster, M., Bailey, C., Brinkhof, M.W., Graber, C., Boulle, A., Spohr, M., et al. (2008). Electronic medical record systems, data quality and loss to follow-up: survey of antiretroviral therapy programmes in resource-limited settings. Bulletin of the World Health Organization, 86, 939-947.
Goodyear-Smith, F., Grant, C., York, D., Kenealy, T., Copp, J., Petousis-Harris, H., et al. (2008). Determining immunisation coverage rates in primary health care practices: a simple goal but a complex task. International Journal of Medical Informatics, 77, 477-485.
Goulet, J.L., Erdos, J., Kancir, S., Levin, F.L., Wright, S.M., Daniels, S.M., et al. (2007). Measuring performance directly using the Veterans health administration electronic medical record: A comparison with external peer review. Medical Care, 45, 73-79.
Hazlehurst, B., Sittig, D.F., Stevens, V.J., Smith, K.S., Hollis, J.F., Vogt, T.M., et al. (2005). Natural language processing in the electronic medical record: Assessing clinician adherence to tobacco treatment guidelines . American Journal of Preventive Medicine, 29, 434-439.
Hicken, V.N., Thornton, S.N., & Rocha, R.A. ( 2004). Integration challenges of clinical information systems developed without a shared data dictionary. Studies in Health Technology and Informatics, 107, 1053-1057.
Institute of Medicine. (2006). Rewarding provider performance: Aligning incentives in Medicare. Washington, DC: National Academies Press.
Kaboli, P.J., McClimon, B.J., Hoth, A.B., & Barnett, M.J. ( 2004). Assessing the accuracy of computerized medication histories . American Journal of Managed Care, 10, 872-877.
Linder, J.A., Kaleba, E.O., & Kmetik, K.S. ( 2009). Using electronic health records to measure physician performance for acute conditions in primary care: Empirical evaluation of the community-acquired pneumonia clinical quality measure set. Medical Care, 47, 208-216.
McGinnis, K.A., Skanderson, M., Levin, F.L., Brandt, C., Erdos, J., & Justice, A.C. ( 2009). Comparison of two VA laboratory data repositories indicates that missing data vary despite originating from the same source. Medical Care, 47, 121-124.
Owen, R.R., Thrush, C.R., Cannon, D., Sloan, K.L., Curran, G., Hudson, T., et al. (2004). Use of electronic medical record data for quality improvement in schizophrenia treatment. Journal of the American Medical Informatics Association, 11, 351-357.
Pakhomov, S., Bjornsen, S., Hanson, P., & Smith, S. ( 2008). Quality performance measurement using the text of electronic medical records. Medical Decision Making, 28, 462-470.
Pakhomov, S.V., Jacobsen, S.J., Chute, C.G., & Roger, V.L. ( 2008). Agreement between patient-reported symptoms and their documentation in the medical record. American Journal of Managed Care, 14, 530-539.
Persell, S.D., Dunne, A.P., Lloyd-Jones, D.M., & Baker, D.W. ( 2009). Electronic health record-based cardiac risk assessment and identification of unmet preventive needs. Medical Care, 47, 418-424.
Persell, S.D., Kho, A.N., Thompson, J.A., & Baker, D.W. ( 2009). Improving hypertension quality measurement using electronic health records. Medical Care, 47, 388-394.
Persell, S.D., Wright, J.M., Thompson, J.A., Kmetik, K.S., & Baker, D.W. ( 2006). Assessing the validity of national quality measures for coronary artery disease using an electronic health record. Archives of Internal Medicine, 166, 2272-2277.
Powell, J., Fitton, R., & Fitton, C. ( 2006). Sharing electronic health records: The patient view. Informatics in Primary Care, 14, 55-57.
Pringle, M., Ward, P., & Chilvers, C. ( 1995). Assessment of the completeness and accuracy of computer medical records in four practices committed to recording data on computer . British Journal of General Practice, 45, 537-541.
Ricciardi, T.N., Lieberman, M.I., Kahn, M.G., & Masarie, F.E., Jr. (2005). Clinical terminology support for a national ambulatory practice outcomes research network. AMIA Annual Symposium Proceedings, 2005, 629-633.
Rosenman, M.B., Tao, G., Szucs, K.A., Wang, J., Ambuehl, R., & Mahon, B.E. ( 2008). Prenatal syphilis screening rates measured using Medicaid claims and electronic medical records. Sexually Transmitted Diseases, 35, 387-392.
Roukema, J., Los, R.K., Bleeker, S.E., van Ginneken, A.M., van der Lei, J., & Moll, H.A. ( 2006). Paper versus computer: Feasibility of an electronic medical record in general pediatrics. Pediatrics, 117, 15-21.
Saigh, O., Triola, M.M., & Link, R.N. ( 2006). Brief report: Failure of an electronic medical record tool to improve pain assessment documentation. Journal of General Internal Medicine, 21, 185-188.
Schneider, E.C., Riehl, V., Courte-Wienecke, S., Eddy, D.M., & Sennett, C. ( 1999). Enhancing performance measurement: NCQA’s road map for a health information framework. Journal of the American Medical Association, 282, 1184-1190.
Staes, C.J., Bennett, S.T., Evans, R.S., Narus, S.P., Huff, S.M., & Sorensen, J.B. (2006). A case for manual entry of structured, coded laboratory data from multiple sources into an ambulatory electronic health record. Journal of the American Medical Informatics Association, 13, 12-15.
Staroselsky, M., Volk, L.A., Tsurikova, R., Newmark, L.P., Lippincott, M., Litvak, I., et al. (2008). An effort to improve electronic health record medication list accuracy between visits: Patients’ and physicians’ response. International Journal of Medical Informatics, 77, 153-160.
Staroselsky, M., Volk, L.A., Tsurikova, R., Pizziferri, L., Lippincott, M., Wald, J., et al. (2006). Improving electronic health record (EHR) accuracy and increasing compliance with health maintenance clinical guidelines through patient access and input. International Journal of Medical Informatics, 75, 693-700.
Szpunar, S., Williams, P., Dagroso, D., Enberg, R., & Chesney, J. ( 2008). Improving the documentation of vital signs: A business reengineering efficiency study. Joint Commission Journal on Quality and Patient Safety, 34, 171-178.
Tang, P.C., Ralston, M., Arrigotti, M.F., Qureshi, L., & Graham, J. ( 2007). Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: Implications for performance measures. Journal of the American Medical Informatics Association, 14, 10-15.
Thielke, S., Hammond, K., & Helbig, S. ( 2007). Copying and pasting of examinations within the electronic medical record. International Journal of Medical Informatics, 76(Suppl. 1), S122-S128.
Thiru, K., Hassey, A., & Sullivan, F. ( 2003). Systematic review of scope and quality of electronic patient record data in primary care. British Medical Journal, 326, 1070-1074.
Varkey, P., Cunningham, J., & Bisping, D.S. ( 2007). Improving medication reconciliation in the outpatient setting. Joint Commission Journal on Quality and Patient Safety, 33, 286-292.
Weingart, S.N., Cleary, A., Seger, A., Eng, T.K., Saadeh, M., Gross, A., et al. (2007). Medication reconciliation in ambulatory oncology. Joint Commission Journal on Quality and Patient Safety, 33, 750-757.
Williams, J.G. ( 2003). Measuring the completeness and currency of codified clinical information. Methods of Information in Medicine, 42, 482-488.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: February 11, 2010
Issue published: October 2010
Keywords
PubMed: 20150441
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in Medical Care Research and Review.
View All Journal MetricsArticle usage*
Total views and downloads: 2492
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 0
Crossref: 247
- Qualitative Analysis of Therapist Documentation of Assessments of Orally Feeding Infants Who Require Noninvasive Respiratory Support
- Lower electronic health record adoption and interoperability in rural versus urban physician participants: a cross-sectional analysis from the CMS quality payment program
- The effect of introducing an electronic medical record system on data quality and factors associated with data quality across 187 HIV clinics in Kenya: An interrupted time series analysis from 2011-2018
- Natural Language Processing-Based Deep Learning to Predict the Loss of Consciousness Event Using Emergency Department Text Records
- The effectiveness of a multi-domain electronic feedback report on the performance of quality indicators for chronic conditions: Protocol for a randomized controlled trial in general practice
- Impact of Implementing Electronic Nursing Records on Quality and Safety Indicators in Care
- Electronic Health Record Data Quality and Performance Assessments: Scoping Review
- Appropriateness of Hysterectomy as Treatment for Benign Gynecological Conditions
- Navigating electronic health record accuracy by examination of sex incongruent conditions
- Lessons on the use of real-world data in medical device research: findings from the National Evaluation System for Health Technology Test-Cases
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
