 at Social Quantum.){kind=link}
Weekly Shorts | February 5, 2021
Alleged US Capitol rioter needs Russian interpreter One of the people accused of attacking the US Capitol on January 6th requires a Russian interpreter, reports…
→ Continue Readinggames
Machine Translation for Game Localization
An updated conversation with Social Quantum’s
Mikhail Gorbunov
Interview by Yulia Akhulkova
I
n the spring of 2021, this magazine published my interview with Mikhail Gorbunov — head of localization at mobile gaming company Social Quantum — in which we spoke about his experience implementing machine translation (MT) for game content. Now, four years later, a lot has changed in the MT landscape. So, Mikhail agreed to revisit this subject and discuss the new technologies and strategies that Social Quantum uses for game localization.
First and foremost, do you think the quality of MT has improved since our last conversation?
Yes, absolutely. Translators say that MT has finally started to save their time!
Every year, I survey team members about their satisfaction with the quality of MT. In 2020, our linguists were rewriting 80 to 90 percent of MT segments, so their answers were essentially that MT interferes, rather than helps. They were deleting MT output and translating from scratch.
Now, however, about 50 percent of segments remain unchanged, so translators look at MT more favorably — although they do not agree with the idea that it can be put into production without human editing.
Of course, 50 percent is a rough estimate; everything depends on the language pair. In our case, a more accurate picture can be seen in Table 1. As you see, the Taiwanese variant of Chinese is a record-holder: None of the engines we use can handle it yet.
Advertisement
How is MT versus LLMs for translation?
Before launching the translation into Dutch, Polish, Arabic, Indonesian, and Thai, we conducted blind testing of Claude, Llama, DeepL, and ChatGPT 4o in Polish and Dutch. DeepL and Claude won convincingly, ChatGPT was in the middle, and Llama completely failed.
According to current data and feedback from native speakers, MT for Polish and Dutch is only suitable for provisional translation. The low quality is due to undertraining of MT engines (since the languages are not the most common) and lack of context (artificial intelligence (AI) does not yet understand the context of a phrase itself without manual input). To achieve a radical improvement in text quality, you need to write a prompt for each segment, and then adjust it depending on the result. This kills any economic feasibility of AI translation in game localization without the participation of native speakers.
For now, the “MT plus human editing” workflow seems optimal, as it allows us to reduce the cost and time compared to purely human translation by 30 to 50 percent. Previously, we used only DeepL, and now we use DeepL and ChatGPT 4o. We are also looking towards Claude, but it has problems with integration: It is not available in either Phrase or Gridly at the moment.
So, you started working in Gridly. Can you tell us about the pros and cons of this tool?
In short, Gridly is an enhanced Google Sheets for localization. Initially, the creators of Gridly positioned their tool as a content management system (CMS). Then they incorporated computer-assisted translation (CAT) and translation management system (TMS) features into it. And it worked, but turned out a bit strange for now; in fact, these are three different products on one website united by an umbrella brand (the development team, of course, tried to integrate them with one another). And the TMS/CAT interface resembles the quintessence of Crowdin and Trados of the late 2000s.
Still, CMS Gridly is like a “spherical cow in a vacuum” that we wanted to have 10 years ago, when I was managing five large projects with hundreds of thousands of words each. The most convenient tool for a localization project manager for content management, which easily and effortlessly “digests” tens of thousands of lines and millions of words on one page, works in batches with any number of segments, allows for setting dependencies between locales, is capable of versioning, and works via application programming interface (API) with cloud tools. How cool is that?
Naturally, all this did not appear overnight. When Gridly was just making their first steps on the market, we were with them almost from the very beginning, giving regular feedback. And the Gridly development team is great: They always answer to the point and strive to improve the product. The experience of interacting with them is almost the best in my practice.
Do you continue to use Phrase?
Yes, we do, as we don’t see any alternatives in terms of the functionality. The developers try to keep up with modern trends and regularly add something new. Still, I’d like to talk about three existing flies in the Phrase ointment:
- MT subscription. Phrase has its own combination subscription to several MT services at once. It is convenient, since it allows you to access multiple engines at once for a fixed price. However, Phrase cuts the functionality of the engines that the user connects to independently via API. For example, if you connected DeepL yourself using the API key, MT will not take into account the glossary connected to the project. We contacted support for both Phrase and DeepL, but they pointed the finger at each other, saying, “The problem is not on our side.”
- New pricing policy. In 2024, Phrase significantly changed the terms and cost of the subscription. For example, there were no restrictions on the volume of translated text on the subscription before. We could upload hundreds of thousands of source words and experiment with various MT engines. Now, there is no such possibility, because they introduced the Processed Words parameter, which takes into account all the source words, multiplying them by the number of languages. For example, imagine you decided to translate (either by machine or human linguist) two projects of 60,000 words into ten languages and received 1.2 million words in total. That’s it — your annual subscription limit has expired (the cost of which has increased even without these innovations). The price of an additional words package is about half the price of the subscription itself.
- Favorite statistics removed. Until May 2023, Phrase had a great tool that allowed for batch collecting statistics on any important area: languages, tasks, projects, translators, and stages of the workflow. All of this was given out in a beautifully designed Microsoft Excel sheet. We used this to calculate payments to translators. It even multiplied the translator’s rate by the volume of tasks and gave us ready-made amounts that were enough to copy-paste into an invoice. And then Phrase disabled this tool with the justification that “nobody uses it.” A couple of days later, they turned it back on “by popular demand” until the end of May. As I understand it, “nobody” was quite unhappy.
Advertisement
What issues remain unresolved and difficult to automate in modern localization processes?
So much has changed over the years, but context remains the biggest headache. There is no way to transfer the thoughts of the author of the text to a separate column with the heading “Here, I meant the following.” Of course, you can write all this manually — but where is the automation, then? We would be explaining the context to a machine instead of explaining the context to a person; for us, as an organizer of processes, nothing would change in terms of saving time.
Unstable quality of the source text and multi-format both deserve a special mention. With source quality, everything is simple: It is necessary to introduce quality control at the level of game designers or content managers, checking the source texts through AI before sending them to the game and for translation.
With multi-format, though, things are more complicated. I am waiting for an Uber-like tool that will automatically convert file formats, perform syntactic analysis, fix formatting, and all that. Roughly speaking, you feed the system a file in any format and then it “digests” it itself; loads it into the TMS/CAT environment; outputs everything in a user-friendly editor for post-editing; monitors the correct placement of tags, formatting, and placeholders; and then compiles the file in the needed format.
Another thing I would like to change is that MT does not properly take into account translation memory (TM) yet. I am not talking about training MT engines, but specifically about working with TM “on the fly,” with AI analyzing the variants from the TM and taking TM into account when producing the output. This will require serious computing power, so for now, my wish can be considered a utopia.
In addition, there is a dramatic difference in the quality of translation by language pairs; the less popular the language, the worse the MT.
It’s worth ending the interview with a popular question: Will AI replace translators?
In the short term, it won’t; statistics now show that the number of vacancies for translators has increased compared to previous years. In the middle term, AI will most likely take on a significant part of the workload, but it still won’t replace people completely. And I won’t make long-term forecasts.
AI is globally changing the landscape of today’s information technology (IT), but these changes are not instantaneous, as many people have imagined. And I can’t say that I like them, but I don’t have a strong dislike for AI, either. I rather see it as a tool that we all need to master.
Yulia Akhulkova graduated from the University of Electronic Engineering as a software engineer. Since 2010, she has worked in localization, combining strategic and control functions. Since 2018, she has worked at Nimdzi Insights as a Language Technology Researcher.
Advertisement
Related Articles
Ten LSPs form translate5 Consortium
Ten European language services providers (LSPs) have recently come together to form the translate5 Consortium, a group that’s investing in the open-source translation management system…
→ Continue ReadingLionbridge Games Wins Best Overall Use of AI in Gaming Award in 2026 AI Breakthrough Awards Program
The achievement marks the third consecutive year Lionbridge has been honored by the AI Breakthrough Awards program for its industry-defining AI leadership.
→ Continue Reading