Strengths and Weaknesses of Two Empathy Measures: A Comparison of the Measurement Precision, Construct Validity, and Incremental Validity of Two Multidimensional Indices
Abstract
The quality of empathy research, and clinical assessment, hinges on the validity and proper interpretation of the measures used to assess the construct. This study investigates, in an online sample of 401 adult community participants, the construct validity of the Affective and Cognitive Measure of Empathy (ACME) relative to that of the Interpersonal Reactivity Index (IRI), the most widely used multidimensional empathy research measure. We investigated the factor structures of both measures, as well as their measurement precision across varying trait levels. We also examined them both in relation to convergent and discriminant criteria, including broadband personality dimensions, general emotionality, personality disorder features, and interpersonal malignancy. Our findings suggest that the ACME possesses incremental validity beyond the IRI for most constructs related to interpersonal malignancy. Our results further indicate that the IRI Personal Distress scale is severely deficient in construct validity, raising serious concerns regarding past findings that have included it when computing total empathy scores. Finally, our results indicate that both questionnaires display poor measurement precision at high trait levels, emphasizing the need for future researchers to develop indices that can reliably measure high levels of empathy.
Get full access to this article
View all access and purchase options for this article.
References
Ackerman R. A., Witt E. A., Donnellan M. B., Trzesniewski K. H., Robins R. W., Kashy D. A. (2011). What does the Narcissistic Personality Inventory really measure? Assessment, 18, 67-87.
Alterman A. I., McDermott P. A., Cacciola J. S., Rutherford M. J. (2003). Latent structure of the Davis Interpersonal Reactivity Index in methadone maintenance patients. Journal of Psychopathology and Behavioral Assessment, 25, 257-265.
American Psychiatric Association. (2013). Online assessment measures: The Personality Inventory for DSM-5–Brief Form (PID-5-BF)–Adult. Retrieved from https://www.psychiatry.org/psychiatrists/practice/dsm/educational-resources/assessment-measures
Ashton M. C., Lee K. (2009). The HEXACO-60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91, 340-345.
Asparouhov T., Muthén B. O. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438.
Baron-Cohen S. (2016, December 30). Empathy is good, right? A new book says we’re better off without it. New York Times. Retrieved from https://www.nytimes.com/2016/12/30/books/review/against-empathy-paul-bloom.html
Baron-Cohen S., Wheelwright S. (2004). The empathy quotient: An investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. Journal of Autism and Developmental Disorders, 34, 163-175.
Bird G., Viding E. (2014). The self to other model of empathy: Providing a new framework for understanding empathy impairments in psychopathy, autism, and alexithymia. Neuroscience & Biobehavioral Reviews, 47, 520-532.
Bloom P. (2016). Against empathy: The case for rational compassion. New York, NY: Ecco.
Bloom P. (2017). Empathy and its discontents. Trends in Cognitive Sciences, 21, 24-31.
Bock E. M., Hosser D. (2014). Empathy as a predictor of recidivism among young adult offenders. Psychology, Crime & Law, 20, 101-115.
Brown T. A. (2003). Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects? Behaviour Research and Therapy, 41, 1411-1426.
Campbell W. K., Bonacci A. M., Shelton J., Exline J. J., Bushman B. J. (2004). Psychological entitlement: Interpersonal consequences and validation of a self-report measure. Journal of Personality Assessment, 83, 29-45.
Chrysikou E. G., Thompson W. J. (2016). Assessing cognitive and affective empathy through the Interpersonal Reactivity Index: An argument against a two-factor model. Assessment, 23, 769-777.
Clark L. A., Watson D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.
Cohen J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
Davis M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113-126.
De Ayala R. J. (2013). The IRT tradition and its applications. Oxford Handbook of Quantitative Methods: Foundations, 1, 144-169.
Decety J., Lewis K. L., Cowell J. M. (2015). Specific electrophysiological components disentangle affective sharing and empathic concern in psychopathy. Journal of Neurophysiology, 114, 493-504.
Decety J., Michalska K. J. (2010). Neurodevelopmental changes in the circuits underlying empathy and sympathy from childhood to adulthood. Developmental Science, 13, 886-899.
Dimitrov D. M. (2012). Statistical methods for validation of assessment scale data in counseling and related fields. Alexandria, VA: American Counseling Association.
DiStefano C., Motl R. W. (2009). Personality correlates of method effects due to negatively worded items on the Rosenberg Self-Esteem scale. Personality and Individual Differences, 46, 309-313.
Domes G., Hollerbach P., Vohs K., Mokros A., Habermeyer E. (2013). Emotional empathy and psychopathy in offenders: An experimental study. Journal of Personality Disorders, 27, 67-84.
Dziobek I., Preißler S., Grozdanovic Z., Heuser I., Heekeren H. R., Roepke S. (2011). Neuronal correlates of altered empathy and social cognition in borderline personality disorder. Neuroimage, 57, 539-548.
Embretson S. E., Reise S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
Enders C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8, 128-141.
Essau C. A., Sasagawa S., Frick P. J. (2006). Callous-unemotional traits in a community sample of adolescents. Assessment, 13, 454-469.
Flight J. I., Forth A. E. (2007). Instrumentally violent youths: The roles of psychopathic traits, empathy, and attachment. Criminal Justice and Behavior, 34, 739-751.
Fraley R. C., Waller N. G., Brennan K. A. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365.
Frick P. J. (2004). The Inventory of Callous–Unemotional Traits (Unpublished rating scale). University of New Orleans, New Orleans, LA.
Gabay Y., Shamay-Tsoory S. G., Goldfarb L. (2016). Cognitive and emotional empathy in typical and impaired readers and its relationship to reading competence. Journal of Clinical and Experimental Neuropsychology, 38, 1131-1143.
Gill A. D., Stickle T. R. (2016). Affective differences between psychopathy variants and genders in adjudicated youth. Journal of Abnormal Child Psychology, 44, 295-307.
Glass L., Moody L., Grafman J., Krueger F. (2016). Neural signatures of third-party punishment: Evidence from penetrating traumatic brain injury. Social Cognitive and Affective Neuroscience, 11, 253-262.
Harkness A. R., Tellegen A., Waller N. (1995). Differential convergence of self-report and informant data for Multidimensional Personality Questionnaire traits: Implications for the construct of negative emotionality. Journal of Personality Assessment, 64, 185-204.
Hatcher S. L., Nadeau M. S., Walsh L. K., Reynolds M., Galea J., Marz K. (1994). The teaching of empathy for high school and college students: Testing Rogerian methods with the Interpersonal Reactivity Index. Adolescence, 29, 961-975.
Hawk S. T., Keijsers L., Branje S. J., Graaff J. V. D., Wied M. D., Meeus W. (2013). Examining the Interpersonal Reactivity Index (IRI) among early and late adolescents and their mothers. Journal of Personality Assessment, 95, 96-106.
Hengartner M. P., De Fruyt F., Rodgers S., Mueller M., Roessler W., Ajdacic-Gross V. (2014). An integrative examination of general personality dysfunction in a large community sample. Personality and Mental Health, 8, 276-289.
Horn J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185.
Ickes W., Stinson L., Bissonnette V., Garcia S. (1990). Naturalistic social cognition: Empathic accuracy in mixed-sex dyads. Journal of Personality and Social Psychology, 59(4), 730-742.
Jolliffe D., Farrington D. P. (2006). Development and validation of the Basic Empathy Scale. Journal of Adolescence, 29, 589-611.
Jordan M. R., Amir D., Bloom P. (2016). Are empathy and concern psychologically distinct? Emotion, 16, 1107-1116.
Keltner D., Haidt J. (1999). Social functions of emotions at four levels of analysis. Cognition & Emotion, 13, 505-521.
Keltner D., Kring A. M. (1998). Emotion, social function, and psychopathology. Review of General Psychology, 2, 320-342.
Kozee H. B., Tylka T. L., Augustus-Horvath C. L., Denchik A. (2007). Development and psychometric evaluation of the interpersonal sexual objectification scale. Psychology of Women Quarterly, 31, 176-189.
Krizan Z., Herlache A. D. (2017). The narcissism spectrum model: A synthetic view of narcissistic personality. Personality and Social Psychology Review, 22(1), 3-31.
Krueger R. F., Derringer J., Markon K. E., Watson D., Skodol A. E. (2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42(9), 1879-1890.
Lawrence E. J., Shaw P., Baker D., Baron-Cohen S., David A. S. (2004). Measuring empathy: Reliability and validity of the Empathy Quotient. Psychological Medicine, 34, 911-920.
Lilienfeld S. O., Andrews B. P. (1996). Development and preliminary validation of a self-report measure of psychopathic personality traits in noncriminal population. Journal of Personality Assessment, 66(3), 488-524.
Lilienfeld S. O., Widows M. (2005). Professional manual for the Psychopathic Personality Inventory–Revised (PPI-R). Lutz, FL: Psychological Assessment Resources.
Little R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198-1202.
Loevinger J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635-694.
Marsh H. W., Morin A. J. S., Parker P., Kaur G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110.
Michie A. M., Lindsay W. R. (2012). A treatment component designed to enhance empathy in sex offenders with an intellectual disability. British Journal of Forensic Practice, 14(1), 40-48.
Miller J. D., Crowe M., Weiss B., Maples-Keller J. L., Lynam D. R. (2017). Using online, crowdsourcing platforms for data collection in personality disorder research: The example of Amazon’s Mechanical Turk. Personality Disorders: Theory, Research, and Treatment, 8, 26-34.
Muthén L. K., Muthén B. O. (1998-2012). MPlus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
Olino T. M., Yu L., McMakin D. L., Forbes E. E., Seeley J. R., Lewinsohn P. M., Pilkonis P. A. (2013). Comparisons across depression assessment instruments in adolescence and young adulthood: An item response theory study using two linking methods. Journal of Abnormal Child Psychology, 41, 1267-1277.
Palgi S., Palgi Y., Ben-Ezra M., Shrira A. (2014). “I will fear no evil, for I am with me”: Mentalization-oriented intervention with PTSD patients. A case study. Journal of Contemporary Psychotherapy, 44, 173-182.
Patrick C. J. (2010). Triarchic Psychopathy Measure (TriPM). Retrieved from https://www.phenxtoolkit.org/index.php?pageLink=browse.protocoldetails&id=121601
Paulhus D. L., Jones D. N. (2014). Measures of dark personalities. In Boyle G. J., Saklofske D. H., Matthews G. (Eds.), Measures of personality and social psychological constructs (pp. 562-594). San Diego, CA: Academic Press.
Pulos S., Elison J., Lennon R. (2004). The hierarchical structure of the Interpersonal Reactivity Index. Social Behavior and Personality: An International Journal, 32, 355-359.
Raskin R., Terry H. (1988). A principal-components analysis of the Narcissistic Personality Inventory and further evidence of its construct validity. Journal of Personality and Social Psychology, 54, 890-902.
Realo A., Allik J., Nõlvak A., Valk R., Ruus T., Schmidt M., Eilola T. (2003). Mind-reading ability: Beliefs and performance. Journal of Research in Personality, 37, 420-445.
Reniers R. L., Corcoran R., Drake R., Shryane N. M., Völlm B. A. (2011). The QCAE: A questionnaire of cognitive and affective empathy. Journal of Personality Assessment, 93, 84-95.
Revelle W. (2016). Psych: Procedures for personality and psychological research (R package Version 1.6.6). Evanston, IL: Northwestern University.
Rizopoulos D. (2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17(5), 1-25.
Rogers C. R. (1958). The characteristics of a helping relationship. Journal of Counseling & Development, 37, 6-16.
Rosseel Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36.
Roszkowski M. J., Soven M. (2010). Shifting gears: Consequences of including two negatively worded items in the middle of a positively worded questionnaire. Assessment & Evaluation in Higher Education, 35, 113-130.
Samejima F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 17(4, Pt. 2).
Shamay-Tsoory S. G., Aharon-Peretz J., Perry D. (2009). Two systems for empathy: A double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. Brain, 132, 617-627.
Siu A. M., Shek D. T. (2005). Validation of the Interpersonal Reactivity Index in a Chinese context. Research on Social Work Practice, 15, 118-126.
Spreng R. N., McKinnon M. C., Mar R. A., Levine B. (2009). The Toronto Empathy Questionnaire: Scale development and initial validation of a factor-analytic solution to multiple empathy measures. Journal of Personality Assessment, 91, 62-71.
Steiger J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.
Strathearn L., Fonagy P., Amico J., Montague P. R. (2009). Adult attachment predicts maternal brain and oxytocin response to infant cues. Neuropsychopharmacology, 34, 2655-2666.
Tellegen A. (1982). Brief manual for the multidimensional personality questionnaire. Unpublished manuscript, University of Minnesota, Minneapolis, 1031-1010.
Tellegen A., Atkinson G. (1974). Openness to absorbing and self-altering experiences (“absorption”), a trait related to hypnotic susceptibility. Journal of Abnormal Psychology, 83, 268-277.
Thimm J. C., Jordan S., Bach B. (2016). The Personality Inventory for DSM-5 Short Form (PID-5-SF): Psychometric properties and association with big five traits and pathological beliefs in a Norwegian population. BMC Psychology, 4, 61.
Todd A. R., Forstmann M., Burgmer P., Brooks A. W., Galinsky A. D. (2015). Anxious and egocentric: How specific emotions influence perspective taking. Journal of Experimental Psychology: General, 144, 374-391.
Vachon D. D., Lynam D. R. (2016). Fixing the problem with empathy: Development and validation of the affective and cognitive measure of empathy. Assessment, 23, 135-149.
Vachon D. D., Lynam D. R., Johnson J. A. (2014). The (non)relation between empathy and aggression: Surprising results from a meta-analysis. Psychological Bulletin, 140, 751-773.
Watson D., Clark L. A. (1984). Negative affectivity: The disposition to experience aversive emotional states. Psychological Bulletin, 96, 465-490.
Wickramasekera I. E., Szlyk J. P. (2003). Could empathy be a predictor of hypnotic ability? International Journal of Clinical and Experimental Hypnosis, 51, 390-399.
Wood J. L., James M., Ciardha C. Ó. (2014). “I know how they must feel”: Empathy and judging defendants. European Journal of Psychology Applied to Legal Context, 6, 37-43.
Woods C. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28, 189-194.
Young S., Sedgwick O., Perkins D., Lister H., Southgate K., Das M., . . . Gudjonsson G. H. (2015). Measuring victim empathy among mentally disordered offenders: Validating VERA-2. Journal of Psychiatric Research, 60, 156-162.
Supplementary Material
Supplemental material files:
Summary
Supplementary material for this article is available online.
Resources
File (supplementary_figures_1-7_and_table_1.pdf)
- Download
- 575.33 KB
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: May 31, 2018
Issue published: March 2020
Keywords
PubMed: 29847994
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 3457
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 45 view articles Opens in new tab
Crossref: 42
- Cross‐cultural adaptation and clinical application of the Perth Empath...
- Measuring Empathy Across the Adult Lifespan: A Comparison of Three Ass...
- Two-Wave Stability of Grandiose and Vulnerable Narcissism During Adole...
- Reliability of the empathy selection task, a novel behavioral measure ...
- The impact of ageing simulation education on qualified acute care nurs...
- Dissociating cognitive and affective empathy across psychopathy dimens...
- The Psychometric Assessment of Empathy: Development and Validation of ...
- The multifaceted role of empathy in the transmission of postpartum dep...
- Análisis factorial de la Escala Situacional de Empatía Emocional en un...
- Structural, Construct, and Concurrent Score Validity Evidence for the ...
- Empatía en adolescentes, cogniciones y afectos durante la pandemia en ...
- Criminality, cognitive distortions, empathy, and legitimacy of laws: B...
- Intergenerational Transmission of Depression during Adolescence: The M...
- Social cognition and problematic alcohol use: An organizing theoretica...
- Emotion regulation and delusion-proneness relate to empathetic tendenc...
- Modeling Variation in Empathic Sensitivity Using Go/No-Go Social Reinf...
- Hormonal Profile in Response to an Empathic Induction Task in Perpetra...
- Refining the link between psychopathy, antisocial behavior, and empath...
- Environmental sensitivity predicts interpersonal sensitivity above and...
- Appeal of word of mouth: Influences of public opinions and sentiment o...
- Understanding Oneself to Understand Others: The Role of Alexithymia an...
- Effectiveness of mindfulness‐based interventions on psychotherapy proc...
- Confirmatory factor analysis and exploratory structural equation model...
- Why do people self‐sacrifice for their country? The roles of identity ...
- What is your empathy scale not measuring? The convergent, discriminant...
- Group cohesion in group-based personal practice
- The Initial Evaluation of a Scale for Measuring Interprofessional Pers...
- Higher Social Anxiety Severity Predicts Better Cognitive Empathy Perfo...
- Empathy: Assessment Instruments and Psychometric Quality – A Systemati...
- Empathic Psychology: A Code of Risk Prevention and Control for Behavio...
- The Relationship Between Empathy and Civic-Mindedness in First-Year En...
- Malicious mind readers? A meta-analysis on Machiavellianism and cognit...
- How do laypeople define empathy?
- Short-term impact of alcohol detoxification on facial emotions recogni...
- Investigation on the Rationality of the Extant Ways of Scoring the Int...
- Is Empathy the Default Response to Suffering? A Meta-Analytic Evaluati...
- Increased Salivary Oxytocin and Empathy in Students of Clinical and He...
- The pitfall of empathic concern with chronic fatigue after a disaster ...
- Trait Autism is a Better Predictor of Empathy than Alexithymia
- Empathy Mediates the Relationship Between Nature Connectedness and Bot...
- Measuring Empathy in Health Care
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
