Multilingual information retrieval R&D in Europe has been the focus of a digital libraries initiative called Cross-Language Evaluation Forum (CLEF) for some years now. Anyone wanting to know about the state of the art can consult Carol Peters report of the latest CLEF workshop, which summarizes how the systems tested for text, image and actual information retrieval benchmarks.

“CLEF 2004 … marked a breaking point with respect to previous campaigns. The focus was no longer concentrated on multilingual document retrieval but was diversified to include different kinds of text retrieval across languages (exact answers in the question-answering track) and retrieval on different kinds of media (not just plain text but collections containing image and speech as well). In addition, increased attention was given to issues that regard system usability and user satisfaction with tasks to measure the effectiveness of interactive systems or system components being included in both the QA and the ImageCLEF tracks, with the collaboration of the coordinators of iCLEF (the interactive track).”