![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Although word reading has traditionally been viewed as a foundational skill for development of reading fluency and comprehension, some children demonstrate “specific” reading comprehension problems, in the context of intact word reading. The purpose of this study was to identify specific patterns of errors associated with reading profiles—basic reading difficulties (BRD), reading fluency difficulties (RFD), reading comprehension difficulties (RCD), and typical readers (total n = 821). Results indicated significant differences between the groups on most error factors. Post hoc analyses indicated there were no significant differences between the RFD and RCD groups, but these groups demonstrated different patterns of significant weakness relative to typical readers. The RFD group was weaker in spelling and oral expression whereas the RCD group demonstrated difficulties in writing mechanics and listening comprehension. These findings indicate that comprehension deficits cannot be attributed only to fluency difficulties and that specific reading difficulties may translate to other aspects of achievement.
Dyslexia or reading disability is a language-based neurodevelopmental disorder characterized by unexpected and severe reading difficulty in children with otherwise intact intelligence (Lyon, Shaywitz, & Shaywitz, 2003). Dyslexia is often marked by difficulties in phonological awareness, rapid automatized naming, decoding, and word recognition. Children manifesting these primary weaknesses commonly also have secondary problems with reading fluency and comprehension. Difficulties with reading fluency can also occur (albeit less frequently) among children with intact basic word recognition skills (Wolf, Bowers, & Biddle, 2000), especially in the context of attention-deficit hyperactivity disorder (ADHD) or other conditions that give rise to slow processing speed (Jacobson, Ryan, Denckla, Mostofsky, & Mahoneet, 2013).
Similarly, “specific” reading comprehension problems have been identified among children with intact basic word reading. Specific comprehension difficulties have been associated with weaker oral language and vocabulary skills. For example, Catts, Hogan, and Adlof (2005) followed students longitudinally and categorized different types of reading difficulties to examine developmental changes in word reading and comprehension abilities in second, fourth, and eighth grades. They classified students into four different types—non-specified (intact word recognition and language comprehension), dyslexic (poor word recognition and intact language comprehension), mixed (poor word recognition and language comprehension), and specific comprehension deficit (intact word recognition and poor language comprehension). They found that the percentage of students identified as having a specific comprehension difficulty increased at each time point, ending with nearly one third of eighth graders identified as having language comprehension difficulties. They have also identified this subtype as “late-emerging reading problems,” meaning those students who are sufficient decoders but poor comprehenders and therefore are sometimes not identified at earlier ages due to a focus on phonics and word reading in early grades (Catts et al., 2005).
This view of comprehension difficulties is aligned with the Simple View of Reading, which posits that reading comprehension consists of word recognition ability and language comprehension, or R (reading comprehension) = D (decoding) × C (language comprehension; Gough & Tunmer, 1986; Hoover & Gough, 1990). According to the Simple View, those with specific comprehension difficulties have intact word recognition ability but have less developed oral language skills to act as a foundation for reading comprehension. Specific comprehension difficulties also occur in the context of co-existing executive dysfunction—especially involving deficits in working memory and planning (Cutting, Materek, Cole, Levine, & Mahone, 2009; Jacobson et al., 2016; Locascio, Mahone, Eason, & Cutting, 2010). It has also been noted that processing assessment of reading comprehension should include assessment of oral language and executive functioning abilities along with assessment of other foundational reading skills (Dehn, 2014).
The purpose of this study was to identify distinct error patterns associated with “specific” reading difficulties in children: (a) specific reading fluency difficulties (RFD), that is, those children with intact basic word recognition, but poor fluency; and, (b) specific reading comprehension difficulties (RCD), those children with intact basic word recognition and reading fluency, but who manifest poor reading comprehension. The proposed study is novel because prior studies of specific reading comprehension have not controlled for reading fluency in the definition of RCD. We sought to identify similarities and differences between the RCD and RFD groups. In addition to differences between those groups, we were interested in the difference between these groups and students with average reading ability, and children with poor basic reading skills. We hypothesized that for most factors, the RFD and RCD groups would perform better than the basic reading difficulties (BRD) group but worse than the typical readers (TR) group. We predicted that those with BRD would demonstrate a distinct pattern of errors on phonological processes subtests that would not be found in the RCD and RFD groups. In addition, we hypothesized that the RCD group would show a specific pattern of weakness in errors made on subtests related to language comprehension and expression that would not be found in participants with other types of reading difficulties.
Method
Participants
Participants in this study were a subsample of the standardization and validation sample for the Kaufman Test of Educational Achievement, Third Edition (KTEA-3; Kaufman & Kaufman, 2014). The total standardization sample (N = 3,843) included 1,987 females and 1,856 males in Grades Pre-K to 12 (median grade = 5) who ranged in age from 4 to 19 years (M = 10.4; SD = 3.9). These data were collected between August 2012 and 2013; approximately half of the sample was tested the KTEA-3 Form A, and the other half was tested using Form B. More information about the standardization sample can be found in the KTEA-3 Technical and Interpretive Manual (Kaufman, Kaufman, & Breaux, 2014).
The sample that includes Letter & Word Recognition (LWR), Nonsense Word Decoding (NWD), and Spelling (SP), the three KTEA-3 subtests that use within-item error analysis, is a subset of the larger KTEA-3 standardization sample. The 1,781 participants included in the LWR, NWD, and SP sample included the stratified KTEA-3 error analysis normative sample (n = 1,400), 108 participants from the larger KTEA-3 standardization sample who were not part of the stratified error analysis sample, and 273 participants included in KTEA-3 cognitive validity studies. According to the KTEA-3 Technical and Interpretive Manual, trained scorers classified the errors made by students on the LWR, NWD, and SP subtests and then the total number of errors by category was calculated for each student (Kaufman et al., 2014).
Participants were selected into the present sample first if they had specific fluency or comprehension difficulties. The specific RFD (n = 59) group scored in the average range for basic reading (standard score [SS] of 100 or above on the Decoding Composite) but demonstrated poor (≤25th percentile) reading fluency (SS 90 or below on the Reading Fluency Composites). The specific RCD (n = 75) group was defined as having intact basic reading and fluency (SS 100 or above Decoding and Fluency composites) but impaired comprehension (SS of 90 or below on the Reading Comprehension Subtest).
Background data indicated that the RFD and RCD groups were similar on the demographic variables of age, grade in school, gender, region, ethnicity, and parent’s education. In that sense, they represented matched samples on important background variables. To select a matched control group of intact readers and those with BRD, a random sample—stratified by grade level—was selected from the KTEA-3 age-based and grade-based standardization samples. First, students in the sample Grades 3 through 12 were identified who had minimal missing data on the error analysis factor scores (dependent variables for the analyses) and also met the score classifications for the BRD and TR groups. The BRD (n = 431) group consisted of participants with an SS of 90 or below on the Decoding Composite. The final group was made up of TR with Decoding Composite, Fluency Composites, and Reading Comprehension scores all ≥ 100 (n = 256). Then, the two groups were created by selecting participants to match the age and grade distributions of both the RFD and RCD samples and to approximate the gender, ethnicity, and parent’s education distributions of the RFD and RCD groups. The present sample consisted of 821 participants (see Table 1, for demographic information for each of the four groups); all children in the present sample were between the ages of 8 and 18 and were in Grades 3 to 12.
|
Table 1. Sample Demographics.
Measures
KTEA-3
The KTEA-3 (Kaufman & Kaufman, 2014) is a measure of academic achievement for children ages 4 through 25. The KTEA-3 also includes an error analysis component that not only allows the examiner to know if an answer is wrong but also the type of error made and if there is a consistent pattern of errors made within and across subtests. The present study utilized error analysis from the following subtests: LWR, NWD, SP, Written Expression, Oral Expression, Reading Comprehension, Language Comprehension, and Phonological Processing. Table 2 provides the mean scores and standard deviations for subtests and composite scores of interest, by group.
|
Table 2. Group Means and Standard Deviations on Subtest and Composite Scores of Interest.
Data Analysis Plan
A multi-step process was used to investigate the relationship between students’ cognitive profiles on KTEA-3 cognitive tasks and their corresponding KTEA-3 error scores in reading, writing, spelling, and phonological processing. The first analytic step in this process was the derivation of factor scores.
The KTEA-3 utilizes a unique error analysis methodology based on the specific subskills measured by a given subtest. For 10 of the KTEA-3 subtests, curriculum experts identified the different categories of errors students are likely to make on each subtest. For each category of error on a given subtest, students received a grade-level, normative performance label of weakness, average, or strength based on a comparison of a student’s total errors to the average number of errors made by individuals in the KTEA-3 normative sample who were in the same grade and were working at approximately the same level as defined by the ceiling item on the subtest or by the item set taken (Kaufman et al., 2014). This performance label is called the skill status. Based on this error analysis system, students received multiple skill status error scores within each subtest. To facilitate the use of these skill status error scores in further analyses, exploratory factor analysis and principal components analysis were employed to create a reduced error score variable set. The derivation of factor scores for LWR, NWD, and SP was based on exploratory factor analysis (O’Brien et al., 2017); principal components analysis was used to derive factor scores for tests of reading comprehension, oral language, written expression, and phonological processing (Choi et al., 2017; Hatcher et al., 2017).
To create the factor scores, polychoric correlation matrices were generated for each subtest. The exceptions were Reading and Listening Comprehension and Oral and Written Expression subtests. Because each of these subtests has a smaller number of error scores that, in general, are the same across the subtest type (comprehension or expression), one polychoric correlation matrix was generated for each subtest type (comprehension or expression). An exploratory factor analysis using unweighted least squares extraction was conducted for each of the subtests excluding comprehension, expression, and phonological processing. Because the comprehension, expression, and phonological processing subtests include a small number of error scores, principal components analysis was used to extract the factors for these subtests.
Regardless of the factor extraction technique, a combination of parallel analysis (PA; Horn, 1965), a visual inspection of the scree plot (Cattell, 1966), and content review of the factor structure were used to determine the number of factors to extract. For the subtests related to the current study, four factors were extracted for the comprehension and expression subtests, three factors for LWR, and two factors for NWD, SP, and Phonological Processing subtests. R version 3.2.3 was used to generate Bartlett factor scores for each of the extracted factors.
Table 3 provides the mean error scores for error factors of interest by group. Factor scores were analyzed as z scores; however, they were converted to scaled scores with a mean of 10 and a standard deviation of 3 for ease of interpretation.
|
Table 3. Mean Error Scores by Group.
The final analytic step was to investigate whether students in each of the four reading groups had different mean error factor scores. One-way MANOVAs were conducted with subtest factor scores as dependent variables and reading group as the independent variable. Separate MANOVAs were conducted for (a) the four Written Expression and Oral Expression error factors, (b) the two SP error factors, (c) the two Listening Comprehension error factors, and (d) the two Phonological Processing factors. These MANOVAs were followed by univariate ANOVAs for each separate error factor. Then, pairwise comparisons were conducted to identify significant differences between groups using a Bonferroni correction. We did not use the NWD and LWR error factors as dependent variables in the MANOVAs or ANOVAs because scores on those subtests were used to categorize and create score cutoffs for our four groups.
Results
Results for the MANOVA analyses yielded significant Wilks’s lambdas (p < .0001) for all four analyses, permitting the further analysis of each error factor in subsequent univariate ANOVAs. The results of these ANOVAs are presented in Table 4, with all F values statistically significant (p < .001). Table 5 provides mean z score differences and significance levels for post hoc analyses with Bonferroni correction. The mean z score differences presented in Table 5 can be interpreted as estimated effect sizes. In general, the group with BRD scored significantly lower than the other three reading groups on most error factors (Table 4). Analogously, post hoc analyses revealed that there were no significant differences between the RFD and RCD groups for any of the error factors, in part because of the relatively small sample sizes for these two groups. Of particular interest are the comparisons between the low-fluency and low-comprehension samples and the TR sample. These differences are highlighted in Table 5 and are featured here.
|
Table 4. MANOVA Results Summary by Error Factors.
|
Table 5. Mean z Score Differences and Associated Significance Between Groups on Error Factors.
Spelling, Writing, and Oral Language Factors
For spelling, post hoc analyses revealed notable differences between the RFD and RCD groups and the TR group. The group with fluency difficulties had significantly lower scores than the TR group on both SP factors—sound-letter mapping and awareness to phonological structure of words—but the group with comprehension difficulties did not (Table 5).
Post hoc analyses for the two Written Expression factors (general written expression and writing mechanics) indicated that the RFD and TR groups did not differ significantly. In contrast, the RCD group made significantly more writing mechanics errors than TR.
The Oral Expression subtest yielded two error factors: one that is “task” oriented—the mechanics of speaking—and the other more of a general oral expression. The RFD sample made significantly more errors than the TR on both Oral Expression error factors, whereas the RCD sample did not differ significantly from the TR sample on either factor.
The Listening Comprehension subtest produced two error factors: inferential questions and narrative passages versus literal content and expository passages. The RCD sample made significantly more errors than the TR on both error factors, but the RFD sample did not differ significantly from the TR group on either Listening Comprehension factor.
Taken together, the oral language subtests produced opposite results for the RFD and RCD samples. The low-fluency students had difficulties in oral expression relative to TR, whereas the low-comprehension students had problems with listening comprehension. Similarly, the two KTEA-3 measures of Written Language yielded opposite findings for the RFD and RCD samples. The low-fluency students made significantly more spelling errors than TR whereas the low-comprehension students made more errors than the TR sample on items that involve writing mechanics.
Phonological Processing Factors
The Phonological Processing subtest yielded two factors, one that involves simple phonological awareness and the other that involves more complicated phonological processing with an emphasis on working memory. For both factors, the BRD sample made significantly more errors than the RFD, RCD, or TR groups. However, there were no other significant differences between groups. In that respect, Phonological Processing was the only subtest studied that did not produce notable findings for the RCD and RFD samples when each was compared with the sample of TR. However, both the RCD and RFD groups excluded students who had poor fluency or comprehension in addition to poor decoding, so that could be one reason that there were no significant differences between the groups.
Discussion
This study sought to investigate potential differences in error patterns between children with specific comprehension and specific fluency difficulties. We were also interested in whether the errors made by children in these groups were significantly different from those children with basic word reading difficulties or those with typical reading ability. There were no significant differences between the RFD and RCD groups on any of the error factors. Therefore, we reframed our interpretation of the results to determine how each group’s performance compared with the TR group or students with average reading ability.
The RCD group demonstrated specific difficulties with listening comprehension when compared with the RFD group in terms of how their performance aligned with the TR group. For both listening comprehension factors, the RCD group’s performance was not significantly differentiated from the BRD group in terms of error scores but made significantly more errors than the TR group. On the contrary, the RFD group was not significantly differentiated from any of the other groups on either listening comprehension factor. These findings may indicate that the RCD group was more affected by language-related difficulties than by other factors. This is consistent with a body of literature that supports a significant relationship between language and reading comprehension skills (Catts, Adlof, & Weismer, 2006; Catts et al., 2005; Nation, Clarke, Marshall, & Durand, 2004; Nation & Snowling, 1997). These findings also supports the recommendation to include a measure of language comprehension along with reading measures in evaluation for dyslexia or specific learning disability in reading.
However, a different pattern was found for oral expression error factors. Interestingly, the RFD group was more aligned with the BRD group in that there were no significant differences between the two groups, but the RFD sample made significantly more errors than the TR group. In contrast, the RCD group showed the opposite pattern. They displayed significantly higher performance than the BRD group, but they did not differ significantly from the sample of TR. Results indicate specific language-related weaknesses for the RCD group in listening comprehension but not oral expression, with the opposite pattern emerging for the RFD group. It may be that the students with fluency difficulties demonstrate difficulty with efficiency of output across domains for both reading and oral expression.
Results indicate that there are unique areas of weakness for those with specific comprehension and fluency difficulties. Although we did not find any significant differences between the RFD and RCD groups, we were still able to analyze group differences in terms of how each group compared with TR. These results provide evidence that the RCD and RFD groups differ not only on reading achievement ability and language skills but also on other academic skills. The RFD group demonstrated a specific pattern of weakness in spelling problems compared with the TR group, a difference that was not found for the RCD group. One cause of poor fluency is weak (or at least not automatized) phonetic skills, which may contribute to poor spelling.
In addition, the RCD group exhibited difficulty with writing mechanics when compared with the TR group, but this pattern was not found in the RFD group. Errors in writing mechanics may reflect fundamental misunderstandings of sentence structure and meaning, and these findings may represent specific profiles or patterns of difficulty for different types of reading problems that extend past explicit reading-related tasks.
Overall, our results indicate that even after ruling out fluency and speed problems, the RCD group was still impaired compared with TR on the core foundational skill of language comprehension whereas these differences were not found for the RFD group. These results support the Simple View of Reading and the strong contribution of language comprehension to reading comprehension skills (Gough & Tunmer, 1986; Hoover & Gough, 1990). The present results provide evidence that comprehension issues cannot be attributed only to fluency problems and there are other important foundational skills at play. Often, it is assumed that readers will acquire phonological skills, then phonics and decoding skills, and followed by fluency and automaticity of word recognition, and finally comprehension of text (Ehri, 2005). Although this may be the case for those students with typical reading development, struggling readers may fall into different subtypes when development lags in one foundational skill and they can compensate in other areas. Or as Catts and colleagues (2005) noted, “Such results clearly show that late-emerging reading problems are not the ‘downstream effects’ of poor word recognition” (p. 38). This study has implication for educational diagnosticians’ use of error analysis and also assessing language-related skills along with core reading skills.
These results must be interpreted in light of the limitations of the present study. First, the sample sizes of the RFD and RCD groups were relatively small—some group differences may have been significant with larger samples. Also, even though we placed parameters on comprehension in the RCD group, we did not place specific restriction on comprehension abilities for those in the RFD group. Although comprehension ability was still in the average range for the RFD group (SS = 99), it was lower than the TR group. It may be that both groups demonstrated poor comprehension in comparison with controls but the RCD group was worse than the RFD group. In addition, we were not able to analyze error score differences on the LWR, NWD, and Reading Comprehension factors because scores on these scores were used as cutoff criteria for includsion to one of our four groups.
The current findings indicated that further research on specific comprehension and fluency difficulties is warranted. It may be beneficial for future studies to use different methods/measures for classification of different types of readers than are used as dependent variables in the analyses. Another direction for future research would be to investigate the relationship between these reading profiles and measures of cognitive ability such as working memory and processing speed, which have been shown to have important connections to both fluency and comprehension skills (Jacobson et al., 2016; Sesma, Mahone, Levine, Eason, & Cutting, 2009). In addition, future research may benefit from utilizing curriculum-based measures or classroom permanent products in addition to standardized measures to analyze errors; this may provide a greater breadth and depth of information about errors made by different types of readers in reading, writing, and spelling.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Alan and Nadeen Kaufman received royalties on the sale of the KTEA-3.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
|
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276. Google Scholar | Crossref | Medline | ISI | |
|
Catts, H. W., Adlof, S. M., Weismer, S. E. (2006). Language deficits in poor comprehenders: A case for the simple view of reading. Journal of Speech, Language, and Hearing Research, 49, 278-293. Google Scholar | Crossref | Medline | ISI | |
|
Catts, H. W., Hogan, T. P., Adlof, S. M. (2005). Developmental changes in reading and reading disabilities. In Catts, H. W., Kamhi, A. G. (Eds.), The connections between language and reading disabilities (pp. 25-40). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Choi, D., Hatcher, R. C., Langley, S. D., Liu, X., Bray, M. A., Courville, T., . . . DeBiase, E. (2017). What do phonological processing errors tell about students’ skills in reading, writing, and oral language?. Journal of Psychoeducational Assessment, 35(1-2), 25-47. Google Scholar | SAGE Journals | |
|
Cutting, L. E., Materek, A., Cole, C. A., Levine, T. M., Mahone, E. M. (2009). Effects of fluency, oral language, and executive function on reading comprehension performance. Annals of Dyslexia, 59, 34-54. Google Scholar | Crossref | Medline | ISI | |
|
Dehn, M. J. (2014). Psychological processes and learning. In Kaufman, A., Kaufman, N. (Eds.), Essentials of processing assessment (2nd ed., pp. 25-60). Hoboken, NJ: Wiley. Google Scholar | |
|
Ehri, L. (2005). Development of sight word reading: Phases and findings. In Snowling, M., Hulme, C. (Eds.), The science of reading: A handbook (pp. 135-154). Malden, MA: Blackwell. Google Scholar | Crossref | |
|
Gough, P. B., Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7, 6-10. Google Scholar | SAGE Journals | |
|
Hatcher, R. C., Breaux, K. C., Liu, X., Bray, M. A., Ottone-Cross, K. L., Courville, T., . . . Dulong Langley, S. (2017). Analysis of children’s errors in comprehension and expression. Journal of Psychoeducational Assessment, 35(1-2), 58-74. Google Scholar | SAGE Journals | |
|
Hoover, W. A., Gough, P. B. (1990). The simple view of reading. Reading and Writing, 2, 127-160. Google Scholar | Crossref | ISI | |
|
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. doi:10.1007/BF02289447 Google Scholar | Crossref | Medline | ISI | |
|
Jacobson, L. A., Koriakin, T. A., Lipkin, P., Boada, R., Frijters, J., Lovett, M., Mahone, E. M. (2016). Executive functions contribute uniquely to reading competence in minority youth. Journal of Learning Disabilities. Advance online publication. doi:10.1177/0022219415618501. Google Scholar | SAGE Journals | |
|
Jacobson, L. A., Ryan, M., Denckla, M. B., Mostofsky, S. H., Mahone, E. M. (2013). Performance lapses in children with attention-deficit/hyperactivity disorder contribute to poor reading fluency. Archives of Clinical Neuropsychology, 28, 672-683. Google Scholar | Crossref | Medline | |
|
Kaufman, A. S., Kaufman, N. (2014). Kaufman Test of Educational Achievement (3rd ed.). Minneapolis, MN: NCS Pearson. Google Scholar | |
|
Kaufman, A. S., Kaufman, N. L., Breaux, K. C. (2014). Kaufman Test of Educational Achievement–Third Edition (KTEA-3) technical & interpretive manual. Bloomington, MN: Pearson. Google Scholar | |
|
Locascio, G., Mahone, E. M., Eason, S. H., Cutting, L. E. (2010). Executive dysfunction among children with reading comprehension deficits. Journal of Learning Disabilities, 43, 441-454. Google Scholar | SAGE Journals | ISI | |
|
Lyon, G. R., Shaywitz, S. E., Shaywitz, B. A. (2003). A definition of dyslexia. Annals of Dyslexia, 53, 1-14. Google Scholar | Crossref | ISI | |
|
Nation, K., Clarke, P., Marshall, C. M., Durand, M. (2004). Hidden language impairments in children: Parallels between poor reading comprehension and specific language impairment? Journal of Speech, Language, and Hearing Research, 47, 199-211. Google Scholar | Crossref | Medline | ISI | |
|
Nation, K., Snowling, M. J. (1997). Individual differences in contextual facilitation: Evidence from dyslexia and poor reading comprehension. Child Development, 69, 996-1011. Google Scholar | Crossref | |
|
O’Brien, R., Pan, X., Courville, T., Bray, M. A., Breaux, K. C., Avitia, M., Choi, D. (2017). Exploratory factor analysis of reading, spelling, and math errors. Journal of Psychoeducational Assessment, 35(1-2) 8-24. Google Scholar | |
|
Sesma, H. W., Mahone, E. M., Levine, T., Eason, S. H., Cutting, L. E. (2009). The contribution of executive skills to reading comprehension. Child Neuropsychology, 15, 232-246. doi:10.1080/09297040802220029 Google Scholar | Crossref | Medline | ISI | |
|
Wolf, M., Bowers, P. G., Biddle, K. (2000). Naming-speed processes, timing, and reading: A conceptual review. Journal of Learning Disabilities, 33, 387-407. Google Scholar | SAGE Journals | ISI |