Amazon’s Alexa AI team was recently ranked in first place on a shared task at the International Conference on Spoken Language Translation (IWSLT), which focused on developing a machine translation (MT) system that could produce output with different levels of formality in the target language.
Amazon’s English-to-Japanese MT model outperformed the second-place entry by nearly 10%. As with other context-dependent forms of language, such as slang, formality can be difficult for MT systems to get just right, especially since languages express formality quite differently.
“Machine translation (MT) models typically return a single translation for each input, without regard to the intended use case or target audience,” the company wrote in an Aug. 15 blog post. “This kind of unconditional translation is useful in many cases but fails to account for differences in language use in different parts of the world.”
In the field of sociolinguistics, sentences and other utterances can be ranked according to their level of formality — for instance, the English sentence, “I am going to the store, would you like me to buy you something?” is much more formal than: “I’m heading to the store, want anything?” The former is more likely to be uttered between acquaintances or professional colleagues in a formal situation, while the latter is more likely to be uttered among friends and family members in a casual or informal setting.
In the shared task’s overview, the IWSLT’s organizers wrote that formality can cause difficulties in translation, since some languages may express formality in ways that others don’t — for example, the English “Are you tired?” can be translated into German using the more formal “Sind Sie müde?” or “Bist du müde,” the latter being a less formal and more familiar translation.
“Leaving the model to choose between different valid options can lead to translations that use an inappropriate degree of formality, which can be perceived as rude or jarring for speakers from certain cultures and in certain use cases, such as customer support chat,” Amazon’s blog post continues.
By annotating formal and informal language in the training data, as well as leveraging post-editing techniques, Amazon was able to create an MT system with more accuracy in producing output that matches the formality of the input. When translating from English to Japanese, the model’s formal accuracy was 95.5% and informal accuracy was 100%.
