Mask and Cloze: Automatic Open Cloze Question Generation Using a Masked Language Model

Shoya Matsumori, Kohei Okuoka, Ryoichi Shibata, Minami Inoue, Yosuke Fukuchi, Michita Imai

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


This paper conducts the first trial to apply a masked language AI model and the 'Gini coefficient' to the field of English study. We propose an algorithm named CLOZER that generates open cloze questions that inquiry knowledge of English learners. Open cloze questions (OCQ) have been attracting attention for both measuring the ability and facilitating the learning of English learners. However, since OCQ is in free form, teachers have to ensure that only a ground truth answer and no additional words will be accepted in the blank. A remarkable benefit of CLOZER is to relieve teachers of the burden of producing OCQ. Moreover, CLOZER provides a self-study environment for English learners by automatically generating OCQ. We evaluated CLOZER through quantitative experiments on 1,600 answers and show its effectiveness statistically. Compared with human-generated questions, we also revealed that CLOZER can generate OCQs better than the average non-native English teacher. Additionally, we conducted a field study at a high school to clarify the benefits and hurdles when introducing CLOZER. Then, on the basis of our findings, we proposed several design improvements.

Original languageEnglish
Pages (from-to)9835-9850
Number of pages16
JournalIEEE Access
Publication statusPublished - 2023


  • Open cloze test
  • automatic question generation
  • field study
  • masked language model

ASJC Scopus subject areas

  • General Engineering
  • General Computer Science
  • General Materials Science


Dive into the research topics of 'Mask and Cloze: Automatic Open Cloze Question Generation Using a Masked Language Model'. Together they form a unique fingerprint.

Cite this