Computer Versus Paper-Based Testing: Are They Equivalent When it Comes to Working Memory?
Abstract
School systems across the country are transitioning from paper-based testing (PBT) to computer-based testing (CBT). As this technological shift occurs, more research is necessary to understand the practical and performance implications of administering CBTs. Currently, there is a paucity of research using CBTs to examine working memory (WM) performance, even though CBTs may negatively influence performance. The present study compared a WM CBT and PBT and found enhanced WM performance on the PBT across several verbal and visuospatial WM tests. This pattern was evident even after age was controlled, indicating that test mode effects were persistent across ages (4-11 years). CBTs on WM performance may yield lower scores due to developmental WM differences, increased cognitive workload, test mode effects stemming from individual access to technology, and participant characteristics, such as developmental, biological, or gender differences. The presence of divergent WM in CBT and PBT indicates the need for additional options for children at risk of academic failure because of testing modality.
Get full access to this article
View all access and purchase options for this article.
References
Alloway T. P. (2007). Automated working memory assessment. London, England: Pearson Assessment.
Alloway T. P., Alloway R. (2010). Investigating the predictive roles of working memory and IQ in academic attainment. Journal of Experimental Child Psychology, 106, 20-29.
Alloway T. P., Copello E. (2013). Working memory: The what, the why, and the how. The Australian Educational and Developmental Psychologist, 30, 105-118.
Alloway T. P., Gathercole S. E., Kirkwood H., Elliott J. (2009). The cognitive and behavioral characteristics of children with low working memory. Child Development, 80(2) 606-621.
Alloway T. P., Gathercole S. E., Pickering S. J. (2006). Verbal and visuospatial short-term and working memory in children: Are they separable? Child Development, 77, 1698-1716.
Archibald L., Gathercole S. E. (2006). Short-term working memory in children with specific language impairment. International Journal of Language and Communication Disorders, 41(6), 675-693.
Baddeley A. D. (1996). Exploring the central executive. Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology, 49, 5-28.
Baddeley A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4, 417-423.
Barrouillet P., Bernardin S., Camos V. (2004). Time constraints and resource sharing in adults’ working memory spans. Journal of Experimental Psychology: General, 133, 83-100.
Bigelow J., Poremba A. (2014). Acheilles’ ear? Inferior human short-term and recognition memory in the auditory modality. PLoS ONE, 9(2), e89914.
Boeve A., Meijer R., Albers C., Beetsma Y., Bosker R. (2015). Introducing computer-based testing in high-stakes exams in higher education: Results of a field experiment. PLoS ONE, 10(12), e0143616.
Cherney I. D. (2008). Mom, let me play more computer games: They improve my mental rotation ability. Sex Roles, 59, 776-786.
Choi S., Tinker T. (2002). Evaluating comparability of paper-and-pencil computer-based assessment in a K-12 setting. Paper presented at the National Council on Measurement in Education, New Orleans, LA.
Colbert A., Bo J. (2016). Evaluating working memory: Comparing change-detection tasks and Wechsler working memory subsets in school age-children. Journal of Clinical and Experimental Neuropsychology, 39, 636-645.
Collerton J., Collerton D., Arai Y., Barrass K., Eccles M., Jagger C., Kirkwood T. (2007). A comparison of computerized and pencil-and-paper tasks in assessing cognitive function in community-dwelling older people in the Newcastle 85+ pilot study. Journal of the American Geriatrics Study, 55, 1630-1635.
Cowan N. (2005). Working memory capacity. New York, NY: Psychology Press.
Daniel M. H., Wahlstrom D., Zhang O. (2014). Equivalence of Q-interactive and paper administration of cognitive tasks: WISC-V (Q-Interactive Technical Report 7). Bloomington, MN: Pearson.
Dolan J. E. (2016). Splicing the divide: A review of research on the evolving digital divide among K-12 students. Journal of Research on Technology in Education, 48, 16-37.
Endres M. J., Houpt J. W., Donkin C., Finn P. R. (2015). Working memory capacity and redundant information processing efficiency. Frontiers in Psychology, 6, 594.
Engle R. W., Kane M. J., Tuholski S. W. (1999). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In Miyake A., Shah P. (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 102-134). New York, NY: Cambridge University Press.
Gathercole S. E., Alloway T. P. (2008). Working memory and learning: A practical guide. Los Angeles, CA: Sage.
Gathercole S. E., Baddeley A. D. (1993). Phonological working memory: A critical building block for reading development and vocabulary acquisition? European Journal of Psychology and Education, 8, 259.
Geiser C., Lehmann W., Eid M. (2006). Separating “rotators” from “non-rotators” in the mental rotations test: A multigroup latent class analysis. Multivariate Behavioral Research, 41, 261-293.
Grey L., Thomas N., Lewis L. (2010). Teachers’ use of educational technology in U.S. public schools: 2009. Washington, DC: National Center for Educational Statistics, Institute of Educational Sciences, U.S. Department of Education.
Hanho J. (2014). A comparative study of scores on computer-based tests and paper-based tests. Behaviour & Information Technology, 33, 410-422.
Hitch G. J. (1990). Developmental fractionation of working memory. In Vallar G., Shallice T. (Eds.), Neuropsychological impairments of short-term memory (pp. 221-246). Cambridge, UK: Cambridge University Press.
Hohlfeld T. N., Rizhaupt A. D., Dawson K., Wilson M. L. (2017). An examination of seven years of technology integration in Florida schools: Through the lens of the levels of digital divide in schools. Computers and Education, 113, 135-161.
Hosseini M., Abidin M. J. Z., Baghdarnia M. (2014). Comparability of test results of computer based tests (CBT) and paper and pencil tests (PPT) among English language learners in Iran. Procedia: Social and Behavioral Sciences, 98, 659-667.
Jacob B., Berger D., Hart K. C., Loeb S. (2016). Can technology help promote equality of educational opportunities? In Alexander K., Morgan S. (Eds.), The Coleman report and educational inequality fifty years later (pp. 242-271). New York, NY: Russell Sage Foundation.
Jaeggi S. M., Buschkuehl M., Jonides J., Shah P. (2011). Short- and long-term benefits of cognitive training. Proceedings of the National Academy of Sciences in the United States of America, 108, 10081-10086.
Jeong H. (2012). A comparative study of scores on computer-based tests and paper-based tests. Behaviour & Information Technology, 33, 410-422.
Katz I. R., Elliot N. (2016). Information literacy in digital environments: Construct mediation, construct modeling, and validation processes. In Information literacy: Research and collaboration across disciplines (pp. 97-116).
Khoshsima H., Toroujeni S. M. H. (2017). Transitioning to an alternative assessment: Computer-based testing and key factors related to testing mode. European Journal of English Language Teaching, 2(1), 54-73.
Koscik T. O., Leary D., Moser D. J., Andreasen N. C., Nopoulos P. (2009). Sex differences in parietal lobe morphology: Relationship to mental rotation performance. Brain and Cognition, 69, 451-459.
Krummenacher J. (1996). Zur Auswirkung der Bildwiederholfrequenz von Computermonitoren auf sakkadische Augenbewegungen bei leseähnlichen Aufgaben [On the impact of computer monitor image repetition on saccadic eye movements in reading-like tasks]. Aachen, Germany: Verlag Mainz.
Leeson H. (2009). The mode effect: A literature review of human and technological issues in computerized testing. International Journal of Technology, 6, 1-24.
Logie R., Pearson D. (1997). The inner eye and the inner scribe of visuo-spatial working memory: Evidence from developmental fractionation. European Journal of Cognitive Psychology, 9, 241-257.
Maeda Y., Yoon S. (2013). A meta-analysis on gender differences in mental rotation ability measured by the Purdue Spatial Visualization Tests: Visualization of rotations (PSVT:R). Educational Psychology Review, 25, 69-94.
Mayer R. E., Moreno R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43-53.
Mayes D. K., Sims V. K., Koonce J. M. (2001). Comprehension and workload differences for VDT and paper-based reading. International Journal of Industrial Ergonomics, 28, 367-378.
Moè A. (2009). Are males always better than females in mental rotation? Exploring a gender belief explanation. Learning and Individual Differences, 19, 21-27.
Noyes J., Garland K. (2003). VDT versus paper-based test: Reply to Mayes, Sims and Koonce. International Journal of Industrial Ergonomics, 31, 411-423.
Noyes J., Garland K. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51, 1352-1375.
Pickering S. J., Gathercole S. E., Hall M., Lloyd S. (2001). Development of memory for pattern and path: Further evidence for the fractionation of visuo-spatial memory. The Quarterly Journal of Experimental Psychology, 54, 397-420.
Prisacari A. A., Danielson J. (2017). Computer-based versus paper-based testing: Investigating testing mode with cognitive load and scratch paper use. Computers in Human Behavior, 77, 1-10.
Raiford S. E., Zhang O., Drozdick L. W., Getz K., Wahlstrom D., Gabel A., Daniel M. (2016). WISC-V coding and symbol search in digital format: Reliability, validity, special group studies, and interpretation (Q-Interactive Technical Report 12). Bloomington, MN: Pearson.
Randall D., West R. (2014). Technology integration in schools. In Handbook of research on educational communications and technology (pp. 841-853). New York, NY: Spinger
Register-Mihalik J. K., Kontos D., Guskiewicz D., Mihalik J., Conder R., Shields E. (2012). Age-related differences and reliability on computerized and paper-and-pencil neurocognitive assessment batteries. Journal of Athletic Training, 47, 297-305.
Sedlacik M., Cechova I. (2016). Computer assisted versus traditional testing: Statistical assessment of students’ performance in different types of tests. Proceedings of the European Conference on e-Learning, 650-656.
Soveri A., Antfolk J., Karlsson L., Salo B., Laine M. (2017). Working memory training revisited: A multi-level meta-analysis of n-back training studies. Psychonomic Bulletin & Review, 24, 1077-1096.
Wallace P., Clariana R. (2000). Achievement predictors for a computer-applications module delivered via the world-wide web. Journal of Information Systems Education, 11, 13-18.
Wallace P., Clariana R. (2005). Gender differences in computer-administered versus paper-based tests. International Journal of Instructional Media, 32, 171-179.
Wang S., Jiao H., Young M. J., Brooks T., Olson J. (2007). A meta-analysis of testing mode effects in grade K-12 mathematics tests. Educational and Psychological Measurement, 67, 219-238.
Wechsler D. (2008). Wechsler Adult Intelligence Scale (4th ed.). Bloomington, MN: Pearson.
Ziefle M. (1998). Effects of display resolution on visual performance. Human Factors, 40, 555-568.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: March 14, 2018
Issue published: June 2019
Keywords
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in Journal of Psychoeducational Assessment.
View All Journal MetricsArticle usage*
Total views and downloads: 1735
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 17 view articles Opens in new tab
Crossref: 19
- A testing load: a review of cognitive load in computer and paper-based learning and assessment
- Features of neurovegetative and humoral regulation of cognitive activity in adolescents when using electronic devices
- Effect of Computer Based Test on Motivation: A Meta-Analysis
- A Testing Load: Investigating Test Mode Effects on Test Score, Cognitive Load and Scratch Paper Use with Secondary School Students
- The Development of an Automatic Test Assembly System for a Formative Assessment in Mastery Learning Instruction: Case of the SQL Mastery Course
- Working memory assessment using cambridge neuropsychological test automated battery can help in the diagnosis of mild cognitive impairment: a systematic review and meta-analysis
- Surveying Disadvantaged Children’s Traffic Safety Education in a Comparison between Paper and Electronic Methods: A Case Example for the Expanded Use of Educational Technology
- 2022 IEEE Global Engineering Education Conference (EDUCON)
- Investigating the Split‐Attention Effect in Computer‐Based Assessment: Spatial Integration and Interactive Signaling Approaches
- The development of early visual-spatial abilities – considering effects of test mode
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
