The creation and validation of a listening vocabulary levels test
Abstract
An important gap in the field of second language vocabulary assessment concerns the lack of validated tests measuring aural vocabulary knowledge. The primary purpose of this study is to introduce and provide preliminary validity evidence for the Listening Vocabulary Levels Test (LVLT), which has been designed as a diagnostic tool to measure knowledge of the first five 1000-word frequency levels and the Academic Word List (AWL). Quantitative analyses based on the Rasch model utilized several aspects of Messick’s validation framework. The findings indicated that (1) the items showed sufficient spread of difficulty, (2) the majority of the items displayed good fit to the Rasch model, (3) items and persons generally performed as predicted by a priori hypotheses, (4) the LVLT correlated with Parts 1 and 2 of the TOEIC listening test at .54, (5) the items displayed a high degree of unidimensionality, (6) the items showed a strong degree of measurement invariance with disattenuated Pearson correlations of .97 and .98 for person measures estimated with different sets of items, and (7) carelessness and guessing exerted only minor influences on test scores. Follow-up interviews and qualitative analyses indicated that the LVLT measures the intended construct of aural vocabulary knowledge, the format is easily understood, and the test has high face validity. This test fills an important gap in the field of second language vocabulary assessment by providing teachers and researchers with a way to assess aural vocabulary knowledge.
Get full access to this article
View all access and purchase options for this article.
References
Beglar D. (2010). A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27, 101–118.
Bertram R., Baayen R., Schreuder R. (2000). Effects of family size for complex words. Journal of Memory and Language, 42, 390–405.
Bertram R., Laine M., Virkkala M. (2000). The role of derivational morphology in vocabulary acquisition: Get by with a little help from my morpheme friends. Scandinavian Journal of Psychology, 41, 287–296.
Borsboom D., Mellenbaugh G.J., van Heerden J. (2004). The concept of a validity. Psychological Review, 111, 1061–1071.
Buck G. (2001). Assessing listening. Cambridge: Cambridge University Press.
Coxhead A. (2000). A new academic word list. TESOL Quarterly, 34, 213–238.
Dang T.N.Y., Webb S. (2014). The lexical profile of academic English. English for Specific Purposes, 33, 66–76.
Davies A., Brown A., Elder C., Hill K., Lumley T., McNamara T. (1999). Dictionary of language testing. Cambridge: Cambridge University Press.
Fountain R.L., Nation I.S.P. (2000). A vocabulary-based graded dictation test. RELC Journal, 31, 29–44.
Fulcher G., Davidson F. (2007). Language testing and assessment. New York: Routledge.
Henning G. (1991). A study of the effects of contextualization and familiarization on responses to the TOEFL vocabulary test items. Princeton, NJ: Educational Testing Service.
Linacre J.M. (2013). A user’s guide to WINSTEPS. Chicago, IL: winsteps.com.
McLean S., Kramer B. (2014). Comparing aural and written receptive lexical knowledge. Manuscript in progress.
McLean S., Hogg N., Kramer B. (2014). Estimations of Japanese university learners’ English vocabulary sizes using the vocabulary size test. Vocabulary Learning and Instruction, 3(2), 47–55.
McNamara T. (1996). Measuring second language performance. Harlow: Addison Wesley Longman.
Meara P.M. (2005). X_Lex: The Swansea Vocabulary Levels Test (Version 2.05) [computer software and instruction manual]. Swansea: Lognostics. Retrieved from http://www.lognostics.co.uk/tools (December 2014).
Medical Outcomes Trust Scientific Advisory Committee. (1995). Instrument review criteria. Medical Outcomes Trust Bulletin, 3, 1–4.
Messick S. (1989). Validity. In: Linn R.L. (Ed.), Educational measurement. 3rd edition (pp. 13–103). New York: Macmillan.
Messick S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
Milton J. (2009). Measuring second language vocabulary acquisition. Bristol: Multilingual Matters.
Milton J., Hopkins N. (2006). Comparing phonological and orthographic vocabulary size: Do vocabulary tests underestimate the knowledge of some learners? Canadian Modern Language Review, 63, 127–147.
Milton J., Wade J., Hopkins N. (2010). Aural word recognition and oral competence in a foreign language. In: Chacón-Beltrán R., Abello-Contesse C., Torreblanca-López M., López-Jiménez M.D. (Eds.), Further insights into nonnative vocabulary teaching and learning (pp. 83–97). Bristol: Multilingual Matters.
Nagy W.E., Anderson R., Schommer M., Scott J.A., Stallman A. (1989). Morphological families in the internal lexicon. Reading Research Quarterly, 24, 263–282.
Nation I.S.P. (2006). How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review, 63, 59–82.
Nation I.S.P. (2012). The BNC/COCA word family lists (17 September 2012). Unpublished paper. Available at: http://www.victoria.ac.nz/lals/about/staff/paul-nation (December 2014).
Nation I.S.P., Beglar D. (2007). A Vocabulary Size Test. The Language Teacher, 31, 9–13.
Rasch G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedogogiske Institute.
Read J. (2000). Assessing vocabulary. Cambridge: Cambridge University Press.
Rost M. (2011). Teaching and researching listening. Harlow: Pearson Education.
Schmitt N. (1999). The relationship between TOEFL vocabulary items and meaning, association, collocation, and word-class knowledge. Language Testing, 16, 189–216.
Schmitt N. (2010). Researching vocabulary: A vocabulary research manual. Basingstoke: Palgrave Press.
Schmitt N., Schmitt D., Clapham C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18, 55–88.
Stæhr L. (2009). Vocabulary knowledge and advanced listening comprehension in English as a foreign language. Studies in Second Language Acquisition, 31, 577–607.
Stevens J. (2002). Applied multivariate statistics for the social sciences. 4th edition. Mahwah, NJ: Erlbaum.
Thorndike E.L., Lorge I. (1944). The teacher’s word book of 30,000 words. New York: Bureau of Publications, Teachers College, Columbia University.
van Zeeland H, Schmitt N. (2012). Lexical coverage and L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34, 457–479.
Webb S., Sasao Y. (2013). New directions in vocabulary testing. RELC Journal, 44, 263–277.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: January 30, 2015
Issue published: November 2015
Keywords
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 2589
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 84 view articles Opens in new tab
Crossref: 84
- Exploring Links Between Aural Lexical Knowledge and L2 ...
- The effects of a metacognitive intervention on lower-proficiency EFL l...
- The Development and Initial Validation of O-WSVLT, a Meaning-Recall On...
- Evaluating the roles of breadth and depth of aural vocabulary knowledg...
- The development of a Chinese vocabulary proficiency test (CVPT) for le...
- Assessing English language learners’ collocation knowledge: a systemat...
- Collecting language assessment data in the age of pandemic: a prelimin...
- Variations in Vocabulary Depth Among Second Language Learners with Com...
- Lexical coverage in dialogue listening
- Individual differences in L2 listening proficiency revisited: Roles of...
- Lexical Demands of Academic Written English: From Students’ Assignment...
- Can Japanese learners of English comprehend inflectional and derivatio...
- Using structural equation modeling to examine the relationship between...
- 3K-LEx-MC
- Receptive vocabulary size estimates for general and academic vocabular...
- An exploratory study of young EFL learners’ aural and written receptiv...
- The role of spoken vocabulary knowledge in second language speaking pr...
- Exploring the relationship between metacognitive awareness and Chinese...
- Increasing use of multi-word expressions in conversation through a flu...
- Listening strategies in the English Medium Instruction (EMI) classroom...
- Aural single‐word and aural phrasal verb knowledge and their relations...
- Exploring aural vocabulary knowledge for TOEIC as a language exit requ...
- The auditory picture vocabulary test for English L2: A spoken receptiv...
- Vocabulary Demands of Academic Spoken English Revisited: A Case of Uni...
- Investigating the Effects of Extensive Listening on Non-English Majors...
- How does repetition affect vocabulary learning through listening to th...
- Longitudinal measurement of growth in vocabulary size using Rasch-base...
- Contribution of English aural vocabulary size levels to L2 listening c...
- Research Methods in Vocabulary Studies
- Effects Of Field Independence, Metacognitive Knowledge, And Vocabulary...
- Incidental vocabulary acquisition from listening to English teacher ed...
- Issues in matching learners to their reading levels using receptive vo...
- Bridging local needs and national standards: Use of standards-based in...
- The relationship between vocabulary knowledge and L2 reading/listening...
- Person factors and strategic processing in L2 listening comprehension:...
- Aural vocabulary, orthographic vocabulary, and listening comprehension
- What Predicts Academic Achievement in EMI Courses? Focus on Vocabulary...
- Individual Differences in Second Language Listening: Examining the Rol...
- The relationship between two measures of L2 phonological vocabulary kn...
- Acquisition of Breadth and Depth of Vocabulary Knowledge by Secondary ...
- Lexical Profile of Newspapers Revisited: A Corpus-Based Analysis
- Vocabulary Demands of Informal Spoken English Revisited: What Does It ...
- Exploring the relationship between aural decoding and listening compre...
- Test Format and Local Dependence of Items Revisited: A Case of Two Voc...
- Vocabulary Demands of the IELTS Listening Test: An In-Depth Analysis
- The Technology of the Learning Outcomes Test Development
- Exploring the relationships between various dimensions of receptive vo...
- A Rasch-based validation of the Vietnamese version of the Listening Vo...
- THE CONTRIBUTIONS OF LISTENING AND READING VOCABULARIES TO LISTENING C...
- Vocabulary Learning Autonomy and Vocabulary Size of Turkish ELT Studen...
- Effect‐Driven Sample Sizes in Second Language Instructed Vocabulary Ac...
- Memory Strategies for Vocabulary Learning Employed by Saudi Undergradu...
- High school to university transitional challenges in English medium in...
- Learning Vocabulary Through Listening: The Role of Vocabulary Knowledg...
- Vocabulary learning through listening: Comparing L2 explanations, teac...
- Loanword proportion in vocabulary size tests
- Vocabulary in Language Teaching
- Investigating the role of vocabulary size in second language speaking ...
- Examining Second Language Listening, Vocabulary, and Executive Functio...
- THE RELATIVE SIGNIFICANCE OF SYNTACTIC KNOWLEDGE AND VOCABULARY KNOWLE...
- The role of breadth and depth of vocabulary knowledge in second langua...
- Analyzing Discourse in EMI Courses from an ELF Perspective
- Moving the field of vocabulary assessment forward: The need for more r...
- EFFECTS OF NARROW READING AND LISTENING ON L2 VOCABULARY LEARNING
- Using a Listening Vocabulary Levels Test to Explore the Effect of Voca...
- Developing a new academic vocabulary test
- Vocabulary Size Tests of Different Modality and Their Relationships wi...
- WOR-TE: Ein Ja / Nein-Wortschatztest für Kinder verschiedene...
- Effects of Removing Responses With Likely Random Guessing Under Rasch ...
- Roles of Vocabulary Knowledge for Success in English‐Medium Instructio...
- Loanword proportion in vocabulary size tests
- Receptive vocabulary size and its relationship to EFL language skills
- Vocabulary for listening: Emerging evidence for high and mid-frequency...
- The updated Vocabulary Levels Test
- Help options for L2 listening in CALL: A research agenda
- Explaining listening comprehension among L2 learners of English: The c...
- A Response to Holster and Lake Regarding Guessing and the Rasch Model
- Christopher Brumfit Thesis Award Winner 2014 – Hilde van Zeeland: Four...
- Word Families and Frequency Bands in Vocabulary Tests: Challenging Con...
- Interpreting Vocabulary Test Scores: What Do Various Item Formats Tell...
- Making and Using Word Lists for Language Learning and Testing
- Loanwords and Vocabulary Size Test Scores: A Case of Different Estimat...
- Guessing and the Rasch Model
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
