Abstract
Psychometricians strive to eliminate random error from their psychological inventories. When random error affecting tests is diminished, tests more accurately characterize people on the psychological dimension of interest. We document an unusual property of the scoring algorithm for a measure used to assess a wide range of psychological states. The “D-score” algorithm for coding the Implicit Association Test (IAT) requires the presence of random noise in order to obtain variability. Without consequential degrees of random noise, all individuals receive extreme scores. We present results from an algebraic proof, a computer simulation, and an online survey of implicit racial attitudes to show how trial error can bias IAT assessments. We argue as a result that the D-score algorithm should not be used for formal assessment purposes, and we offer an alternative to this approach based on multiple regression. Our critique focuses primarily on the IAT designed to measure unconscious racial attitudes, but it applies to any IAT developed to provide psychological assessments within clinical, organizational, and developmental branches of psychology—and in any other field where the IAT might be used.
References
Section:
| Banaji M. R., Greenwald A. G. (2013). Blindspot: Hidden biases of good people. New York, NY: Random House. Google Scholar | |
| Blanton H., Jaccard J. (2006a). Arbitrary metrics in psychology. American Psychologist, 61, 27-41. Google Scholar CrossRef, Medline | |
| Blanton H., Jaccard J. (2006b). Arbitrary metrics redux. American Psychologist, 16, 62-71. Google Scholar | |
| Blanton H., Jaccard J. (2008). Unconscious racism: A concept in pursuit of a measure. Annual Review of Sociology, 34, 277–297. Google Scholar CrossRef | |
| Blanton H., Jaccard J. (2014). Not so fast: Ten challenges to importing implicit attitude measures to media psychology. Manuscript submitted for publication. Google Scholar | |
| Blanton H., Jaccard J., Gonzales P. M., Christie C. (2006). Decoding the implicit association test: Perspectives on criterion prediction. Journal of Experimental Social Psychology, 42, 192-212. Google Scholar | |
| Buhrmester M., Kwang T., Gosling S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5. Google Scholar Link | |
| Cohen J. (1988). Statistical power for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
| Gladwell M. (2005). Blink: The power of thinking without thinking. New York, NY: Little, Brown. Google Scholar | |
| Greenwald A. G. (2006, September 3). Expert report of Anthony G. Greenwald, Satchell v. FedEx Express, No. C 03-2659 (N.D. Cal.). Google Scholar | |
| Greenwald A. G., McGhee D. E., Schwartz J. K. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464-1480. Google Scholar CrossRef, Medline | |
| Greenwald A. G., Nosek B. A., Banaji M. R. (2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197-216. Google Scholar CrossRef, Medline | |
| Greenwald A. G., Poehlman T. A., Uhlmann E., Banaji M. R. (2009). Understanding and using the implicit association test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97, 17-41. Google Scholar CrossRef, Medline | |
| Kazdin A. E. (2006). Arbitrary metrics: Implications for identifying evidence-based treatments. American Psychologist, 61, 42-49. Google Scholar Medline | |
| McConahay J. B. (1986). Modern racism, ambivalence, and the Modern Racism Scale. In Dovidio J. F., Gaertner S. L. (Eds.), Prejudice, discrimination, and racism (pp. 91-125). Orlando, FL: Academic Press. Google Scholar | |
| McFarland S. G., Crouch Z. (2002). A cognitive skill confound on the implicit association test. Social Cognition, 20, 483-510. Google Scholar | |
| Mierke J., Klauer K. C. (2001). Implicit association measurement with the IAT: Evidence for effects of executive control processes. Zeitschrift für Experimentelle Psychologie, 48, 107-122. Google Scholar Medline | |
| Nosek B. A., Banaji M. R., Greenwald A. G. (2002). Harvesting implicit group attitudes and beliefs from a demonstration website. Group Dynamics, 6, 101-115. Google Scholar CrossRef | |
| Nosek B. A., Smyth F. L., Hansen J. J., Devos T., Lindner N. M., Ranganath K. A., . . .Smith C. T. (2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psychology, 18, 36-88. Google Scholar CrossRef | |
| Oswald F., Mitchell G., Blanton H., Jaccard J., Tetlock P. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT research. Journal of Personality and Social Psychology, 105, 171-192. Google Scholar Medline | |
| Pendry L. F., Driscoll D. M., Field S. T. (2007). Diversity training: Putting theory into practice. Journal of Occupational and Organizational Psychology, 80, 27-50. Google Scholar CrossRef | |
| Plous S. (2003). Understanding prejudice and discrimination. New York, NY: McGraw-Hill. Google Scholar | |
| Roefs A., Huijding J., Smulders F. Y., MacLeod C. M., de Jong P. J., Wiers R. W., Jansen A. M. (2011). Implicit measures of association in psychopathology research. Psychological Bulletin, 137, 149-193. Google Scholar Medline | |
| Raudenbush S.W., Bryk A.S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods (2nd Ed.). Thousand Oaks: Sage Publications. Google Scholar | |
| Rosen P. J., Milich R., Harris M. J. (2007). Victims of their own cognitions: Implicit social cognitions, emotional distress, and peer victimization. Journal of Applied Developmental Psychology, 28, 211-226. Google Scholar | |
| Rothermund K., Wentura D. (2004). Underlying processes in the Implicit Association Test (IAT): Dissociating salience from associations. Journal of Experimental Psychology: General, 133, 139-165. Google Scholar Medline | |
| Rudman L. A. (2008). The validity of the Implicit Association Test is a scientific certainty. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1, 426-429. Google Scholar | |
| Rüsch N., Schulz D., Valerius G., Steil R., Bohus M., Schmahl C. (2011). Disgust and implicit self-concept in women with borderline personality disorder and posttraumatic stress disorder. European Archives of Psychiatry and Clinical Neuroscience, 261, 369-376. Google Scholar Medline | |
| Rushton J. P., Brainerd C. J., Pressley M. (1983). Behavioral development and construct validity: The principle of aggregation. Psychological Bulletin, 94, 18-38. Google Scholar CrossRef | |
| Scheck J. (2004, October 28). Expert witness: Bill Bielby helped launch an industry—suing employers for unconscious bias. Retrieved from http://www.law.com/jsp/PubArticle.jsp?id=900005417471 Google Scholar | |
| Schmidt K., Nosek B. A. (2010). Implicit (and explicit) racial attitudes barely changed during Barack Obama’s presidential campaign and early presidency. Journal of Experimental Social Psychology, 46, 308-314. Google Scholar CrossRef | |
| Schreiber F., Bohn C., Aderka I. M., Stangier U., Steil R. (2012). Discrepancies between implicit and explicit self-esteem among adolescents with social anxiety disorder. Journal of Behavior Therapy and Experimental Psychiatry, 43, 1074-1081. Google Scholar Medline | |
| Sechrest L., McKnight P., McKnight K. (1996). Calibration of measures for psychotherapy outcome studies. American Psychologist, 51, 1065-1071. Google Scholar CrossRef, Medline | |
| Steffens M. (2004). Is the implicit association test immune to faking? Experimental Psychology, 51, 165-179. Google Scholar Medline | |
| Vendatam S. (2005, January 23). See no bias. The Washington Post. Retrieved from http://www.washingtonpost.com/wp-dyn/articles/A27067-2005Jan21.html Google Scholar | |
| Whitley B. E.Jr., Kite M. E. (2009). The psychology of prejudice and discrimination (2nd ed.). Belmont, CA: Thomson/Wadsworth. Google Scholar |
