I
n the spring of 2021, this magazine published my interview with Mikhail Gorbunov — head of localization at mobile gaming company Social Quantum — in which we spoke about his experience implementing machine translation (MT) for game content. Now, four years later, a lot has changed in the MT landscape. So, Mikhail agreed to revisit this subject and discuss the new technologies and strategies that Social Quantum uses for game localization.
First and foremost, do you think the quality of MT has improved since our last conversation?
Yes, absolutely. Translators say that MT has finally started to save their time!
Every year, I survey team members about their satisfaction with the quality of MT. In 2020, our linguists were rewriting 80 to 90 percent of MT segments, so their answers were essentially that MT interferes, rather than helps. They were deleting MT output and translating from scratch.
Now, however, about 50 percent of segments remain unchanged, so translators look at MT more favorably — although they do not agree with the idea that it can be put into production without human editing.
Of course, 50 percent is a rough estimate; everything depends on the language pair. In our case, a more accurate picture can be seen in Table 1. As you see, the Taiwanese variant of Chinese is a record-holder: None of the engines we use can handle it yet.
Advertisement
How is MT versus LLMs for translation?
Before launching the translation into Dutch, Polish, Arabic, Indonesian, and Thai, we conducted blind testing of Claude, Llama, DeepL, and ChatGPT 4o in Polish and Dutch. DeepL and Claude won convincingly, ChatGPT was in the middle, and Llama completely failed.
According to current data and feedback from native speakers, MT for Polish and Dutch is only suitable for provisional translation. The low quality is due to undertraining of MT engines (since the languages are not the most common) and lack of context (artificial intelligence (AI) does not yet understand the context of a phrase itself without manual input). To achieve a radical improvement in text quality, you need to write a prompt for each segment, and then adjust it depending on the result. This kills any economic feasibility of AI translation in game localization without the participation of native speakers.
For now, the “MT plus human editing” workflow seems optimal, as it allows us to reduce the cost and time compared to purely human translation by 30 to 50 percent. Previously, we used only DeepL, and now we use DeepL and ChatGPT 4o. We are also looking towards Claude, but it has problems with integration: It is not available in either Phrase or Gridly at the moment.
So, you started working in Gridly. Can you tell us about the pros and cons of this tool?
In short, Gridly is an enhanced Google Sheets for localization. Initially, the creators of Gridly positioned their tool as a content management system (CMS). Then they incorporated computer-assisted translation (CAT) and translation management system (TMS) features into it. And it worked, but turned out a bit strange for now; in fact, these are three different products on one website united by an umbrella brand (the development team, of course, tried to integrate them with one another). And the TMS/CAT interface resembles the quintessence of Crowdin and Trados of the late 2000s.
Still, CMS Gridly is like a “spherical cow in a vacuum” that we wanted to have 10 years ago, when I was managing five large projects with hundreds of thousands of words each. The most convenient tool for a localization project manager for content management, which easily and effortlessly “digests” tens of thousands of lines and millions of words on one page, works in batches with any number of segments, allows for setting dependencies between locales, is capable of versioning, and works via application programming interface (API) with cloud tools. How cool is that?
Naturally, all this did not appear overnight. When Gridly was just making their first steps on the market, we were with them almost from the very beginning, giving regular feedback. And the Gridly development team is great: They always answer to the point and strive to improve the product. The experience of interacting with them is almost the best in my practice.
Do you continue to use Phrase?
Yes, we do, as we don’t see any alternatives in terms of the functionality. The developers try to keep up with modern trends and regularly add something new. Still, I’d like to talk about three existing flies in the Phrase ointment:
Advertisement
What issues remain unresolved and difficult to automate in modern localization processes?
So much has changed over the years, but context remains the biggest headache. There is no way to transfer the thoughts of the author of the text to a separate column with the heading “Here, I meant the following.” Of course, you can write all this manually — but where is the automation, then? We would be explaining the context to a machine instead of explaining the context to a person; for us, as an organizer of processes, nothing would change in terms of saving time.
Unstable quality of the source text and multi-format both deserve a special mention. With source quality, everything is simple: It is necessary to introduce quality control at the level of game designers or content managers, checking the source texts through AI before sending them to the game and for translation.
With multi-format, though, things are more complicated. I am waiting for an Uber-like tool that will automatically convert file formats, perform syntactic analysis, fix formatting, and all that. Roughly speaking, you feed the system a file in any format and then it “digests” it itself; loads it into the TMS/CAT environment; outputs everything in a user-friendly editor for post-editing; monitors the correct placement of tags, formatting, and placeholders; and then compiles the file in the needed format.
Another thing I would like to change is that MT does not properly take into account translation memory (TM) yet. I am not talking about training MT engines, but specifically about working with TM “on the fly,” with AI analyzing the variants from the TM and taking TM into account when producing the output. This will require serious computing power, so for now, my wish can be considered a utopia.
In addition, there is a dramatic difference in the quality of translation by language pairs; the less popular the language, the worse the MT.
It’s worth ending the interview with a popular question: Will AI replace translators?
In the short term, it won’t; statistics now show that the number of vacancies for translators has increased compared to previous years. In the middle term, AI will most likely take on a significant part of the workload, but it still won’t replace people completely. And I won’t make long-term forecasts.
AI is globally changing the landscape of today’s information technology (IT), but these changes are not instantaneous, as many people have imagined. And I can’t say that I like them, but I don’t have a strong dislike for AI, either. I rather see it as a tool that we all need to master.
Yulia Akhulkova graduated from the University of Electronic Engineering as a software engineer. Since 2010, she has worked in localization, combining strategic and control functions. Since 2018, she has worked at Nimdzi Insights as a Language Technology Researcher.
Advertisement
Related Articles
The report finds that while AI has dramatically accelerated content creation, many organizations still struggle to deliver consistent, relevant customer experiences required to support global…
→ Continue ReadingIt’s a big week for language technology company Phrase, with the company announcing the most massive product release in its history, and company CEO Georg…
→ Continue ReadingPhrase, a global leader in cloud-based localization technology, and Lionbridge, a recognized pioneer in language technology innovation and one of the world’s largest language service providers, have…
→ Continue Reading
 at Social Quantum.){kind=link}