Response Inconsistency of Patient-Reported Symptoms as a Predictor of Discrepancy Between Patient and Clinician-Reported Depression Severity
Abstract
The aim of this study was to assess the extent to which discrepancy between self-reported and clinician-rated severity of depression are due to inconsistent self-reports. Response inconsistency threatens the validity of the test score. We used data from a large sample of outpatients (N = 5,959) who completed the self-report Beck Depression Inventory–II (BDI-II) and the clinician-rated Montgomery–Åsberg Depression Rating Scale (MADRS). We used item response theory based person-fit analysis to quantify the inconsistency of the self-report item scores. Inconsistency was weakly positively related to patient–clinician discrepancy (i.e., higher BDI-II scores relative to MADRS scores). The mediating effect of response inconsistency in the relationship between discrepancy and demographic (e.g., ethnic origin) and clinical variables (e.g., cognitive problems) was negligible. The small direct and mediating effects of response inconsistency suggest that inaccurate patient self-reports are not a major cause of patient–clinician discrepancy in outpatient samples. Future research should investigate the role of clinician biases in explaining clinician–patient discrepancy.
Get full access to this article
View all access and purchase options for this article.
References
Beck A. T., Steer R. A., Ball R., Ranieri W. (1996). Comparison of Beck Depression Inventories-IA and-II in psychiatric outpatients. Journal of Personality Assessment, 67, 588-597.
Bernstein I. H., Rush A. J., Carmody T. J., Woo A., Trivedi M. H. (2007). Clinical vs. self-report versions of the Quick Inventory of Depressive Symptomatology in a public sector sample. Journal of Psychiatric Research, 41, 239-246.
Brouwer D., Meijer R., Zevalkink J. (2013). On the factor structure of the BDI-II: G is the key. Psychological Assessment, 25, 136-145.
Buckley T. C., Parker J. D., Heggie J. (2001). A psychometric evaluation of the BDI-II in treatment-seeking substance abusers. Journal of Substance Abuse Treatment, 20, 197-204.
Carroll B. J., Feinberg M., Smouse P. E., Rawson S. G., Greden J. F. (1981). The Carroll rating scale for depression. I: Development, reliability and validation. British Journal of Psychiatry, 138, 194-200.
Carter J. D., Frampton C. M., Mulder R. T., Luty S. E., Joyce P. R. (2010). The relationship of demographic, clinical, cognitive and personality variables to the discrepancy between self and clinician rated depression. Journal of Affective Disorders, 124, 202-206.
Conijn J. M., Emons W. H. M., De Jong K., Sijtsma K. (2015). Detecting and explaining aberrant responding to the Outcome Questionnaire-45. Assessment, 22, 513-524.
Conijn J. M., Emons W. H. M., Sijtsma K. (2014). Statistic lz based person-fit methods for noncognitive multiscale measures. Applied Psychological Measurement, 38, 122-136.
Conijn J. M., Sijtsma K., Emons W. H. M. (2016). Identifying person-fit latent classes and explanation of categorical and continuous person misfit. Applied Psychological Measurement, 40, 128-141.
Corruble E., Legrand J. M., Zvenigorowski H., Duret C., Guelfi J. D. (1999). Concordance between self-report and clinician’s assessment of depression. Journal of Psychiatric Research, 33, 457-465.
Cuijpers P., Li J., Hofmann S. G., Andersson G. (2010). Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: A meta-analysis. Clinical Psychology Review, 30, 768-778.
De Beurs E. (2004). De Brief Symptom Inventory: Handleiding [The Brief Symptom Inventory: Manual]. Leiden, Netherlands: Pits.
De Beurs E., Den Hollander-Gijsman M. E., Van Rood Y. R., Van der Wee N. J., Giltay E. J., Van Noorden M. S., . . . Zitman F. G. (2011). Routine outcome monitoring in the Netherlands: Practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clinical Psychology & Psychotherapy, 18, 1-12.
De La Torre J., Deng W. (2008). Improving person-fit assessment by correcting the ability estimate and its reference distribution. Journal of Educational Measurement, 45, 159-177.
De Los Reyes A., Kazdin A. E. (2004). Measuring informant discrepancies in clinical child research. Psychological Assessment, 16, 330-334.
DeFife J. A., Drill R., Nakash O., Westen D. (2010). Agreement between clinician and patient ratings of adaptive functioning and developmental history. American Journal of Psychiatry, 167, 1472-1478.
Derogatis L. R. (1977). SCL-90-R: Administration, scoring, and procedures manual 1. Baltimore, MD: Clinical Psychometric Research.
Derogatis L. R., Melisaratos N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13, 595-605.
Domken M., Scott J., Kelly P. (1994). What factors predict discrepancies between self and observer ratings of depression? Journal of Affective Disorders, 31, 253-259.
Dorz S., Borgherini D., Conforti D., Scarso C., Magni G. (2004). Comparison of self-rated and clinician-rated measures of depressive symptoms: A naturalistic study. Psychology and Psychotherapy: Theory, Research and Practice, 77, 353-361.
Drasgow F., Levine M. V., McLaughlin M. E. (1991). Appropriateness measurement for some multidimensional test batteries. Applied Psychological Measurement, 15, 171-191.
Drasgow F., Levine M. V., Williams E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67-86.
Duberstein P. R., Heisel M. J. (2007). Personality traits and the reporting of affective disorder symptoms in depressed patients. Journal of Affective Disorders, 103, 165-171.
Dunlop B. W., Li T., Kornstein S. G., Friedman E. S., Rothschild A. J., Pedersen R., . . . Keller M. (2010). Correlation between patient and clinician assessments of depression severity in the PREVENT study. Psychiatry Research, 177, 177-183.
Dunlop B. W., Li T., Kornstein S. G., Friedman E. S., Rothschild A. J., Pedersen R., . . . Trivedi M. H. (2011). Concordance between clinician and patient ratings as predictors of response, remission, and recurrence in major depressive disorder. Journal of Psychiatric Research, 45, 96-103.
Emons W. H. M. (2008). Nonparametric person-fit analysis of polytomous item scores. Applied Psychological Measurement, 32, 224-247.
Enns M. W., Larsen D. K., Cox B. J. (2000). Discrepancies between self and observer ratings of depression: The relationship to demographic, clinical and personality variables. Journal of Affective Disorders, 60, 33-41.
Ferrando P. J. (2014). A general approach for assessing person fit and person reliability in typical-response measurement. Applied Psychological Measurement, 38, 166-183.
Hamilton M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery & Psychiatry, 23, 56-62.
Hamilton M. (1976). Comparative value of rating scales. British Journal of Clinical Pharmacology, 3, 58-60.
Hasemann K. (1971). Verhaltensbeobachtung. In Heiss R. (Ed.), Handbuch der psychologie [Handbook of Psychology] (3rd ed., pp. 807-836). Göttingen, Germany: Hogrefe.
Hayes A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Press.
Keith T. Z. (2006). Multiple regression and beyond. Boston, MA: Allyn & Bacon.
Kenny D. A., Judd C. M. (2014). Power anomalies in testing mediation. Psychological Science, 25, 334-339.
Kenny D. A., Kaniskan B., McCoach D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods and Research, 44, 486-507.
Kenny D. A., Kashy D. A., Bolger N. (1998). Data analysis in social psychology. In Gilbert D., Fiske S., Lindzey G. (Eds.), The handbook of social psychology (Vol. 1, 4th ed., pp. 233-265). Boston, MA: McGraw-Hill.
Kim E. Y., Hwang S. S., Lee N. Y., Kim S. H., Lee H. J., Kim Y. S., Ahn Y. M. (2013). Intelligence, temperament, and personality are related to over- or under-reporting of affective symptoms by patients with euthymic mood disorder. Journal of Affective Disorders, 148, 235-242.
Lin C. H., Lu M. J., Wong J., Chen C. C. (2014). Comparison of physician-rating and self-rating scales for patients with major depressive disorder. Journal of Clinical Psychopharmacology, 34, 716-721.
Logan D. E., Claar R. L., Scharff L. (2008). Social desirability response bias and self-report of psychological distress in pediatric chronic pain patients. Pain, 136, 366-372.
Lyness J. M., Cox C., Curry J., Conwell Y., King D. A., Caine E. D. (1995). Older age and the underreporting of depressive symptoms. Journal of the American Geriatrics Society, 43, 216-221.
McHugh M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22, 276-282.
Meijer R. R., Niessen A. S. M., Tendeiro J. N. (2016). A practical guide to check the consistency of item response patterns in clinical research through person-fit statistics: Examples and a computer program. Assessment, 23, 52-62.
Meijer R. R., Sijtsma K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.
Meyer G., Finn S., Eyde L., Kay G., Moreland K., Dies R., . . . Reed G. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128-165.
Montgomery S. A., Åsberg M. (1979). A new depression scale designed to be sensitive to change. British Journal of Psychiatry, 134, 382-389.
Möller H. J. (1991). Outcome criteria in antidepressant drug trials: Self-rating versus observer-rating scales. Pharmacopsychiatry, 24, 71-75.
Möller H. J. (2000). Rating depressed patients: Observer- vs self-assessment. European Psychiatry, 15, 160-172.
Möller H. J. (2009). Standardised rating scales in psychiatry: Methodological basis, their possibilities and limitations and descriptions of important rating scales. World Journal of Biological Psychiatry, 10, 6-26.
Muthén B. O., Muthén L. K. (2007). Mplus: Statistical analysis with latent variables (Version 5.0). Los Angeles, CA: Author.
Oliver S. J., Toner B. B. (1990). The influence of gender role typing on the expression of depressive symptoms. Sex Roles, 22, 775-790.
Paykel E. S., Norton K. R. W. (1986). Self-report and clinical interview in the assessment of depression. In Sartorius N., Ban T. A. (Eds.), Assessment of depression (pp. 356-366). New York, NY: Springer.
Podsakoff P., MacKenzie S., Lee J., Podsakoff N. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879-903.
Reise S. P., Due A. M. (1991). The influence of test characteristics on the detection of aberrant response patterns. Applied Psychological Measurement, 15, 217-226.
Rush A. J., Carmody T. J., Ibrahim H. M., Trivedi M. H., Biggs M. M., Shores-Wilson K.,. . .Kashner T. M. (2006). Comparison of self-report and clinician ratings on two inventories of depressive symptomatology. Psychiatric Services, 57, 829-837.
Samejima F. (1997). Graded response model. In van der Linden W. J., Hambleton R. K. (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
Shain B. N., Naylor M., Alessi N. (1990). Comparison of self-rated and clinician-rated measures of depression in adolescents. American Journal of Psychiatry, 147, 793-795.
Sheehan D. V., Lecrubier Y., Sheehan K. H., Amorim P., Janavs J., Weiller E., . . . Dunbar G. C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(Suppl. 20), 22-33.
Stukenberg K., Brady C., Klinetob N. (2000). Use of the MMPI-2’s VRIN scale with severely disturbed populations: Consistent responding may be more problematic than inconsistent responding. Psychological Reports, 86, 3-14.
Tada M., Uchida H., Suzuki T., Abe T., Pollock B. G., Mimura M. (2014). Baseline difference between patients’ and clinicians’ rated illness severity scores and subsequent outcomes in major depressive disorder: Analysis of the sequenced treatment alternatives to relieve depression data. Journal of Clinical Psychopharmacology, 34, 297-302.
Tsujii N., Akashi H., Mikawa W., Tsujimoto E., Niwa A., Adachi T., Shirakawa O. (2014). Discrepancy between self- and observer-rated depression severities as a predictor of vulnerability to suicide in patients with mild depression. Journal of Affective Disorders, 161, 144-149.
Van der Ark L. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48, 1-27.
Van der Does A. J. W. (2002). BDI-II-NL. In Handleiding: De Nederlandse versie van de Beck Depression Inventory [Manual: The Dutch version of the Beck Depression Inventory] (2nd ed.). Lisse, Netherlands: Harcourt Test.
Van Noorden M. S., Van Fenema E. M., Van der Wee N. J., Van Rood Y. R., Carlier I. V., Zitman F. G., Giltay E. J. (2012). Predicting outcomes of mood, anxiety and somatoform disorders: The Leiden routine outcome monitoring study. Journal of Affective Disorders, 142, 122-131.
Wanders R. B. K., Wardenaar K. J., Penninx B. W. J. H., Meijer R. R., Jonge P. D. (2015). Data-driven atypical profiles of depressive symptoms: Identification and validation in a large cohort. Journal of Affective Disorders, 180, 36-43.
Wardell J. D., Rogers M. L., Simms L. J., Jackson K. M., Read J. P. (2014). Point and click, carefully: Investigating inconsistent response styles in middle school and college students involved in web-based longitudinal substance use research. Assessment, 21, 427-442.
Wardenaar K. J., Wanders R. B. K., Roest A. M., Meijer R. R., De Jonge P. (2015). What does the beck depression inventory measure in myocardial infarction patients? A psychometric approach using item response theory and person-fit. International Journal of Methods in Psychiatric Research, 24, 130-142.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: September 14, 2016
Issue published: October 2018
Keywords
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 600
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 8 view articles Opens in new tab
Crossref: 7
- Validity Indices for Interpreting Informant Discrepancies in ADHD Asse...
- Comparison on the clinical features in patients with or without treatm...
- Patients-centered SurvivorShIp care plan after Cancer treatments based...
- The Patient Health Questionnaire-9 vs. the Hamilton Rating Scale for D...
- Discrepancies between self-rated depression and observed depression se...
- Satisficing in Mental Health Care Patients: The Effect of Cognitive Sy...
- A Comprehensive IRT Approach for Modeling Binary, Graded, and Continuo...
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
