Focus

AI in the translation business: Beyond MT

Dmitry Ulanov has been involved in localization since 1998 and risen through the ranks from a project manager to CTO. Now he is focused on automation of operations and also supervising the R&D department. He holds an MS degree and is an accredited internal auditor for ISO 9001 QMS.

Dmitry Ulanov headshot

Dmitry Ulanov

AI in the translation business: Beyond MT
Dmitry Ulanov headshot

Dmitry Ulanov

Dmitry Ulanov has been involved in localization since 1998 and risen through the ranks from a project manager to CTO. Now he is focused on automation of operations and also supervising the R&D department. He holds an MS degree and is an accredited internal auditor for ISO 9001 QMS.

T

here are many areas in the translation business that may benefit from implementation of AI and particularly from machine learning (ML). Since AI is not completely predictable, implementation of any kind of AI also involves new risks. The introduction of AI may produce an unexpected impact or a positive breakthrough.

In linguistics, we have been seeing a continuous evolution of machine translation (MT), the most evident example of ML. We already know what to expect from it, what the benefits are, and how to address the risks. How we manage it also presents a great potential for applying ML to make improvements elsewhere. This includes automated decision-making based on acquired data to either provide human workers with pointers, or to eliminate certain human management in order to resolve bottlenecks in continuous development or continuous translation. We could also use ML for making predictions and bringing to light potential risks (of missing a deadline, losing a client) and maybe to propose corrective measures.

There are different areas to which an ML technique could be applied, and different learning approaches — supervised or unsupervised learning, reinforcement learning, transfer learning and so on. Searching for the best combinations is an interesting journey.

First steps: ML-powered vendor suggestion
A couple of years ago, we conducted a survey among our project managers to find out in which areas the use of ML could simplify and speed up their work. It turned out that the project manager spends quite a lot of time searching for the right vendor to do a particular job, and this is far from the project manager’s favorite activity (Figure 1). This was why we began experimenting with ML to select the best vendor for a project.
Activities that project managers enjoy doing.
Figure 1: Activities that project managers enjoy doing.
Activities that project managers do not like.
Activities that project managers do not like.
Activities that project managers would eliminate.
Activities that project managers would eliminate.
Figure 1, left: Activities that project managers enjoy doing. Middle: Activities that project managers do not like. Right: Activities that project managers would eliminate.
To begin with, we wanted to find out how realistic it was to train a neural network to make the same choices as human project managers. We took the initial data and assigned vendors from several tens of thousands of already completed jobs and used them to train a model. Although a variety of sophisticated neural network architectures are available, we chose a simple regression classification model, and it performed fairly well. Validation showed that the model made the right choice in more than 99% of cases. However, when trying to use this model’s output in production, we found that often, selected vendors were in fact not suitable. This was due inter alia to the fact that in the past, work was often given to a vendor that would be far from the best one for the job from today’s perspective. It was the principle of “garbage in/garbage out” in action.
Reevaluation of project management
Recently, we conducted another survey to understand what activities project managers spend the majority of their time on. In addition, it was interesting to learn in which other areas it makes sense to experiment with machine learning. This survey showed that the search for the best vendor for a project is still the number one task. It also became clear what other areas needed work.Figure 2 lists activities of project managers we are going to improve and ideas about how to partially automate this type of activity, or completely pass it to the machine.
Charting out improvements based on machine learning.
Figure 2: Charting out improvements based on machine learning.
Vendor suggestion 2.0
As mentioned above, the ML-based vendor suggestion worked, but often gave results which were wrong (or at least seemed to be wrong to some of the project managers). To resolve this, we decided to:

  1. Consider who is managing this project (and therefore who receives a suggestion). This allows customization of ML output on a personal/departmental basis, and thus, to continue reusing collected data across different project managers and eliminate “wrong” suggestions at the same time).
  2. Reduce the weighting of records in the training/validation data set as they become obsolete. Thus, more recent data take precedence over data that is five years old, and the data older than five years is not considered at all.

The results of using this improved architecture have not yet been announced, but we already have a list of adjustments for the next iteration. Instead of a complete regular retraining of the model on the data set over the past five years, we will try to switch to reinforcement learning on the data that will come after the initial training of the network.

Apart from that, we can just compare a new project with completed past projects considering such characteristics as client/account, language pair(s), domain, service level agreement and a linguistic snapshot of text to be translated (hello, natural language processing). And we simply reuse resources from matching past projects where appropriate.

Correct data in the system
While processing data contained in the records of completed jobs, we found a number of inconsistencies. Analyzing these, we realized that with a little extra effort it is possible to identify incorrect data with a high degree of certainty — the existence of which also poisons the lives of project managers. The appearance of incorrect data could be due to a wrong user action, an error in the program code or just a trivial typo. No matter the cause, at least we could easily identify it.

So, the way to detect erroneous data could be either an extra analysis during a scheduled data processing task or a separate scheduled data analysis task. Both are supposed to identify cases that do not match a common pattern (via regression model). Automatic correction of data that seem incorrect would be too risky. Therefore, it is enough to report a potential error to the responsible employee who can verify and fix it if needed.

Weighing the pros and cons of different ML approaches.
Figure 3: Weighing the pros and cons of different ML approaches.
Project setup and planning
Often at the start phase of a project, such as quotation or project acknowledgement, it is necessary to determine a realistic project deadline. To get an answer to this question, we need to evaluate the deadline for each stage in the project workflow, calculate the critical path and add some time buffer for contingency. The estimated deadline for each specific task can be found by a neural network that was previously trained on data from completed tasks.

There is another option that would work for a quick preliminary estimation of the deadline: training the network on the data of previously completed projects, taking into account the volume, language pairs, domain and other characteristics of the project. The estimate given by such a network will be approximate, but such an algorithm is much simpler to implement.

Deadline monitoring
For project deadline monitoring, we can use the same approach as for project planning. However, when the project has already been launched and work is in full swing, we have much more data about it. For example, it is known which tasks have already been completed and which specific vendors are assigned to the remaining tasks. Accordingly, in this case, it is enough for us to evaluate the realistic completion time for the remaining tasks, calculate the critical path, add it to the current time to estimate the most probable project completion time and compare it with the agreed deadline. If the estimated completion time is earlier than the agreed deadline, we are happy — and there is every chance we will meet the deadline. And if it’s later, then by applying the difference to a suitable normal distribution histogram, you can roughly estimate the probability of still meeting the deadline (which in this case will definitely be lower than 100%).
Manual preparation of vendor instructions
Although preparation of clear, consistent and up-to-date instruction for vendors is an important part of the project management routine, it often ends up in the copy-and-paste of client instructions, which is crude and unrewarding. While maintaining a set of up-to-date instructions for all types of vendors (like engineer, translator, editor, proofreader and DTP specialist) involved in the workflow for a certain account is a good practice, this could be excessive or just not realistic in some cases.

Here to help us are natural language processing techniques such as information extraction and named entity recognition. Running them on a project description received from a client in nonstructured format, we could get all those project properties (such as volume, language pair, account, project name, deadline and many more) extracted as separate values. Then we just fill in a structured instruction template with them to get a draft of the vendor instruction. And if you do not have a corresponding project created in your project management system yet, it may be a good idea to create a project based on the extracted project properties.

Build or buy
As technology moves forward, and the world becomes more and more digitalized, software development becomes more and more important to linguistic service providers. However, complex, relatively new and quickly evolving areas like ML could still be too difficult to adopt.

To deal with this, a company may consider either building its own ML microservices with Python/R or use excellent cognitive services provided by such IT giants like Google, Amazon and Microsoft, or integrators like Intento (Figure 3).

Although the results of adopting ML could be amazing, they are accompanied with risks. So it makes sense to implement ML solutions in an operational environment, but to keep a human eye on what they do until you can trust the automation and properly evaluate the possible risks.

Back to Issue