A Microsoft-developed language representation model has been ranked among the best in the world on Google’s Cross-Lingual Transfer Evaluation of Multilingual Encoders (XTREME) public leaderboard. XTREME consists of a set of benchmarks used for developing language representation models, with an emphasis on languages that are relatively under-represented in natural language processing (NLP) and machine learning.

“We are excited to announce that with our latest Turing universal language representation model (T-ULRv5), a Microsoft-created model is once again the state of the art and at the top of the Google XTREME public leaderboard,” wrote Saurabh Tiwary, vice president and distinguished engineer at Microsoft, in a blog post co-written with his college Lidong Zhou, a distinguished scientist at the company.

Microsoft’s T-ULRv5 model was developed by two entities within Microsoft: the Microsoft Turing team and the Microsoft Research team. Microsoft had topped the XTREME leaderboard before with earlier models, including T-ULRv5’s predecessor, T-ULRv2. T-ULRv5 consists of 2.2 billion parameters and has a multilingual vocabulary consisting of 500,000 tokens. Currently, it outperforms Alibaba’s VECO, the second highest ranking model on the XTREME leaderboards, by 1.7 points.

The XTREME benchmarks were developed in response to major advancements in the field of NLP. At the time the benchmarks were established, the researchers behind XTREME felt that the contemporary benchmarks were limited, as they mostly focused on English. In developing XTREME, Google sought to create a better means of assessing the “cross-lingual generalization capabilities of multilingual representations across 40 languages and nine tasks.”

“We are motivated by the opportunity to further advance the state of the art and develop new multilingual capabilities to build more inclusive AI,” Tiwary and Zhou’s blog post reads. “For example, we are excited about exploring neural machine translation (NMT) and language generation with a cross-lingual encoder. We hope that our work will contribute to the community’s progress towards making AI more inclusive and accessible to all.”