Experts at the Rome-based language service provider (LSP) say it’s likely that the quality of MT output will be roughly equivalent to that produced by high-performing human translators within the next six years or so. By observing the edits made to MT output by the 136,000 highest performing freelance translators on Matecat — the company’s computer-assisted translation tool — Translated found that the amount of time translators take to edit MT output has been declining along a “surprisingly linear trend.”
Translated has been keeping track of a measurement the company uses to evaluate MT quality: time to edit (TTE) per word. The company calculates this as “the total time a translator spends post-editing a segment of text divided by the number of words making up that segment.” Compared to other methods of assessing the quality of MT output, like BLEU or COMET, TTE allows researchers to better evaluate the cognitive effort that a translator must exert while post-editing content.
“By switching from automated estimates to measurements of human cognitive effort, we reassign the quality evaluation to those traditionally in charge of the task: professional translators,” reads the report from Translated.
A “perfect” translation (that is, one that doesn’t require any editing at all) should have a TTE of one second, accounting for the amount of time it takes the translator to read and process the content of the text. In 2017, the average TTE for MT output was around three seconds, however that number is steadily declining, along a linear pattern. Now, it’s just over 2 seconds, with TTE declining a little more than one-tenth of a second per year. If this trend continues, Translated projects that the average TTE for MT output will reach 1 second sometime around 2027 or 2028.
Contrary to what translators might fear, the folks at Translated believe that this development will allow translators to translate more content and produce work of an even higher quality. As MT produces work with fewer and fewer errors, translators will have to make fewer corrections, allowing them to catch issues they might not have noticed in a text with more errors.
“Machines won’t ever replace humans,” the report reads. “AI is already proving to be a valuable tool for translation professionals, helping them translate more content at a higher level.”