Polyglot Technology, an independent machine translation consultancy, is launching the MT Decider Benchmark. The MT Decider Benchmark is a vendor-independent, transparent, quarterly evaluation of major online machine translation services. It is the broadest benchmark of major online machine translation services using publicly available high-quality evaluation data and industry-standard evaluation metrics.
Cloud services offering machine translation promise affordable, high-quality translations. Machine translation users want to use the best quality machine translations for their language pair(s) for post-editing and direct publishing of perishable, low-impact content. Determining which service is currently best for a language pair is challenging. Quality, as measured in BLEU, can vary as much as 54% or more than 9 points for some language pairs. About one fifth of services ranked best changes every quarter! The MT Decider Benchmark, for the first time, provides reliable, detailed and up-to-date data on machine translation quality by language pair and online service.
Achim Ruopp, creator of the benchmark, notes: “During the COVID pandemic we gathered training data and optimized machine translation systems for health information content. While evaluating whether these customized systems produce better quality than generic online machine translation services we realized that there is an information gap: we didn’t know which of the generic engines provided the best baseline quality. With the MT Decider Benchmark we are closing this information gap. Not just for us, but everyone using online machine translation services. Closing this gap delivers 90% of the quality gain with a fraction of the work building and evaluating domain-specific or even customized MT systems.”
Polyglot Technology will present the MT Decider Benchmark at the upcoming TAUS Massively Multilingual Conference, October 11-13 in San Jose, CA.
For more information go to https://www.polyglot.technology/
About Polyglot Technology
Polyglot Technology is an independent consultancy helping customers succeed with machine translation by evaluating machine translation quality, leveraging customer data and advising how to best integrate technology, people and processes.
Its principal Achim Ruopp has been involved in enabling computers to process different languages and the translation business for more than two decades. He contributed widely to machine translation research, market research and meaningful integration of machine translation in the human translation process. He is an adjunct professor teaching machine translation at Georgetown University.
