Prediction of performance of cross-language information retrieval using automatic evaluation of translation

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)


This study develops regression models for predicting the performance of cross-language information retrieval (CLIR). The model assumes that CLIR performance can be explained by two factors: (1) the ease of search inherent in each query and (2) the translation quality in the process of CLIR systems. As operational variables, monolingual information retrieval (IR) performance is used for measuring the ease of search, and the well-known evaluation metric BLEU is used to measure the translation quality. This study also proposes an alternative metric, weighted average for matched unigrams (WAMU), which is tailored to gauging translation quality for special IR purposes. The data for regression analysis are obtained from a retrieval experiment of English-to-Italian bilingual searches using the CLEF 2003 test collection. The CLIR and monolingual IR performances are measured by average precision score. The result shows that the proposed regression model can explain about 60% of the variation in CLIR performance, and WAMU has more predictive power than BLEU. A back translation method for applying the regression model to operational CLIR systems in real situations is discussed.

Original languageEnglish
Pages (from-to)138-144
Number of pages7
JournalLibrary and Information Science Research
Issue number2
Publication statusPublished - 2008 Jun

ASJC Scopus subject areas

  • Information Systems
  • Library and Information Sciences


Dive into the research topics of 'Prediction of performance of cross-language information retrieval using automatic evaluation of translation'. Together they form a unique fingerprint.

Cite this