Debates will be transcribed and translated by a new state-of-the-art MT system that keeps humans in the loop

Translated has been selected by the European Parliament to automatically transcribe and translate parliamentary multilingual debates in real-time, covering the 24 official languages used by the institution. The service will be provided by new software available both through fully-localized web and mobile applications, and live streaming APIs for third-party developers. It it purported to be the first human-in-the-loop speech machine translation (MT) system, and should leverage context and user feedback to adapt the output in less than one second.

The product will be developed in collaboration with two companies that have already worked with Translated in building products for professional translators: Fondazione Bruno Kessler (FBK), a world-leading research center in MT and automatic speech recognition (ASR); and PerVoice, an ASR world-leading provider. Within the next 12 months, the consortium will release a prototype to be tested by the European Parliament. This solution will be considered alongside solutions provided by two other groups, following rules put forth in “Live Speech to Text and Machine Translation Tool for 24 Languages.” The best-performing tool will be confirmed as the official one for the following two years.

The new product is not a simple concatenation of ASR and MT, but a new, fully-integrated system in which the MT algorithms are tolerant of ASR errors. This approach will not only help deliver more contextualized translations, but it will also open up the opportunity to improve the quality of the output while the plenary session is happening. This is possible thanks to the human correction feedback that the tool allows by both the end-users and a team of professional translators.

“For this project, we are bringing together ten years of research in machine translation and speech recognition,” says Simone Perone, Translated’s vice president of product management. Some of the new AI models that will be used have already been put to work successfully in products such as ModernMT (an MT that improves from corrections and adapts to the context), Matecat (a computer-assisted translation tool that makes post-editing easy), and Matesub (the first subtitling tool offering suggestions during the transcription, now in beta and due to be released in September 2020).