The Reliability of the Wisconsin Card Sorting Test in Clinical Practice

The Wisconsin Card Sorting Test (WCST) represents the gold standard for the neuropsychological assessment of executive function. However, very little is known about its reliability. In the current study, 146 neurological inpatients received the Modified WCST (M-WCST). Four basic measures (number of correct sorts, categories, perseverative errors, set-loss errors) and their composites were evaluated for split-half reliability. The reliability estimates of the number of correct sorts, categories, and perseverative errors fell into the desirable range (rel ≥ .90). The study therefore disclosed sufficiently reliable M-WCST measures, fostering the application of this eminent psychological test to neuropsychological assessment. Our data also revealed that the M-WCST possesses substantially better psychometric properties than would be expected from previous studies of WCST test-retest reliabilities obtained from non-patient samples. Our study of split-half reliabilities from discretionary construed and from randomly built M-WCST splits exemplifies a novel approach to the psychometric foundation of neuropsychology.


Consistency reliability and clinical decision making
McManus (2012) provides a practical overview of interpretative problems that are related to the standard error of measurement (SEM). There are three different SEMs (Dudek, 1979), which McManus (2012) refers to as SEmeas, SEest, and SEpred, in order to avoid confusion. Assume that we measured two identical standard scores (z with M = 0; SD = 1) from an examinee (z = -1.50), score zA from an assessment instrument with relatively low consistency reliability (relA = .6), and score zB from a more reliable assessment instrument (relB = .9).
The standard error of measurement, SEmeas, provides an estimate of the variability of the actual scores given (unknown) true scores, with confidence intervals (CIs) that are symmetric around measured scores. SEmeas is useful as a general measure for comparing assessment instruments. According to Thus, both CIs exclude zero, leading the examiner to conclude that both actual scores fall below zero.
SEest estimates the variability of the true score, given the measured score. It is this quantity that the diagnostician should mainly be interested in. Importantly, one has to take regression to the mean into account, with the consequence that CIs are asymmetric around measured scores (and may not even include the measured score the examiner would conclude that the true score on A (with low reliability) cannot be distinguished from zero, whereas the identical observed score on B (with higher reliability) leads the examiner to conclude that the true score on B falls below zero.

Documentation of task instructions
"You see four stimulus cards in front of you. Before we start, I want you to inspect each single stimulus card and to think about what these cards depict.
In addition to the four stimulus cars, I have a deck of response cards that you will receive in a minute.
Your task, then, is to match each of the response cards, one after the other, to one of the four stimulus cards. BUT there is a matching rule, which renders only one of your choices correct, whereas the other potential choices will be incorrect.
The problem here is that I cannot inform you about the correct matching rule. It is your task to find out this correct matching rule. How can you achieve this? You achieve it by listening to the feedback that I will provide after each sort. That is, I will tell you after every sort whether the response card has its 'correct' position (the cards match according to the rule) or 'incorrect' position (the card do not match according to the rule).
Here I give you an example: If you place your response card (the one that depicts four red crosses) here (stimulus card = two green stars), my answer would probably be 'incorrect'. In case of an 'incorrect'feedback, we will not correct the position of the incorrectly placed response card. You will just pick up the next response card, and you will try to match that response card correctly with one of the stimulus cards.
Before we start, here is one additional important piece of information for you. The correct matching rule will change from time to time. I will not inform you when these rule changes will happen, but you will recognize their occurrence through changes in my feedback behavior. This means, previously correct sorts would then be incorrect sorts.