Determining cloze item difficulty from item and passage characteristics across different learner backgrounds

Jonathan Trace, James Dean Brown, Gerriet Janssen, Liudmila Kozhevnikova

Research output: Contribution to journalArticlepeer-review

24 Citations (Scopus)


Cloze tests have been the subject of numerous studies regarding their function and use in both first language and second language contexts (e.g., Jonz & Oller, 1994; Watanabe & Koyama, 2008). From a validity standpoint, one area of investigation has been the extent to which cloze tests measure reading ability beyond the sentence level. Using test data from 50 30-item cloze passages administered to 2,298 Japanese and 5,170 Russian EFL students, this study examined the degree to which linguistic features for cloze passages and items influenced item difficulty. Using a common set of 10 anchor items, all 50 tests were modeled in terms of person ability and item difficulty onto a single scale using many-faceted Rasch measurement (k = 1314). Principle components analysis was then used to categorize 25 linguistic item- and passage-level variables for the 50 cloze tests and their respective items, from which three components for each passage- and item-level variables were identified. These six factors along with item difficulty were then entered into both a hierarchical structural equation model and a linear multiple regression to determine the degree to which difficulty in cloze tests could be explained separately by passage and item features. Comparisons were further made by looking at differences in models by nationality and by proficiency level (e.g., high and low). The analyses revealed noteworthy differences in mean item difficulties and in the variance structures between passage- and item-level features, as well as between different examinee proficiency groups.

Original languageEnglish
Pages (from-to)151-174
Number of pages24
JournalLanguage Testing
Issue number2
Publication statusPublished - 2017 Apr 1
Externally publishedYes


  • Cloze
  • item difficulty
  • many-facet Rasch measurement
  • reading
  • structural equation modeling

ASJC Scopus subject areas

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language


Dive into the research topics of 'Determining cloze item difficulty from item and passage characteristics across different learner backgrounds'. Together they form a unique fingerprint.

Cite this