n the megabucks world of high tech, software and hardware vendors vie to capture the attention and spending of business buyers and consumers. Over the last few years their battleground has been artificial intelligence, with frequent announcements from mega-tech platform suppliers such as Amazon, Google, and Microsoft about the smartest algorithm or the fastest AI computer. It seems likely that the battle for the smartest-fastest-biggest AI solution will segue into the practical arena of “easiest,” with the democratization of artificial intelligence and incorporation into every electronic device you use. Language technology companies are applying AI on a smaller scale to improve their operations and efficiency, and language service providers (LSPs) across the spectrum are following suit.

It takes money and data to build AI

Much of today’s AI innovation is centered on making computing and communications faster, cheaper, and more accessible — for example, by automating processes that a machine could do more efficiently. In the language technology and services sector, that often requires processing and evaluating words in their original form, transforming them for use in other languages or channels, and analyzing what they mean.

These requirements led to deep analytics and widespread development of machine translation. In fact, the role of natural language processing (NLP) was at the core of the earliest research into machine translation (MT) as far back as the 1930s, continuing through the decades with the Turing Test, Noam Chomsky’s syntactic structures, William A. Woods’ augmented transition networks, and dozens of other influential experiments and innovations.

Success for the big AI of the mega-tech companies relies on an entire ecosystem of developers, practitioners, users, and sometimes victims as they create value by consuming and exploiting resources. AI is no different — success comes from leveraging two assets, free cash flow and enormous amounts of data.

Free cash flow

The first asset, free cash flow (FCF), is the money that a company generates after paying to support operations and maintain assets. Big AI and LSPs aren’t even in the same game here. FCF provides capital for investment in areas of near-term or future opportunity (or for some, to disburse in executive compensation or to shareholders as dividends or stock buybacks). Of course, this is cash that in larger enterprises has to be spread across multiple projects, initiatives, spending packages, and other business needs.

The amount of free cash flow can be staggering — in 2019, the last full year for which data is available, FCF at Microsoft pushed past $38 billion while Google parent Alphabet yielded $30 billion. By way of contrast with publicly traded language technology (langtech) developers and LSPs, SDL returned an estimated £32 million in FCF last year, while RWS (not a langtech company in 2019, though it became one with its acquisition of SDL) had access to £58 million. It can be misleading to compare these smaller companies with the mega-tech firms, but the disparity of the amounts — three orders of magnitude difference — says a lot about how and what the larger companies can do with all that money.

Rich data flow

The second asset is more democratic in a way — enterprises that process enormous amounts of data benefit from what CSA Research labels a “rich and reliable data flow.” This rich data flow is the legacy of big-data initiatives that began in the 1990s, along with the growing and relatively cheap availability of CPUs-on-demand starting in 2006 with AWS and Azure. Developers and LSPs can learn from content they’ve analyzed.

While the mega-tech companies have leveraged this data into MT and speech platforms, AI bots, and other innovations, the langtech vendors and tech-savvy language service providers have focused on using the data that passes through their systems to create many optimizations on the source and target content they process. They want to use this data in aggregate along with machine learning to lessen the cognitive load on linguists and project managers, letting them instead concentrate on tasks that have higher economic value and hopefully provide more meaningful returns. LSPs and langtech vendors have an advantage over the big-data platforms in that they have more rights to legally exploit the data passing through their systems, while the mega-tech companies are prohibited from many possible applications.

What AI means to the language sector

Mega-tech companies such as Amazon and Google possess these two asset classes in abundance, and their piles of money and data position them well for major investments in artificial intelligence. Each of these giant vendors offers a touch of natural language processing (NLP), the part of artificial intelligence that deals with language. Their NLP work improves, simplifies, or enhances the interaction with human-like conversation, suggestions, and quick paths based on analysis of many previous communications.

Smaller langtech companies and LSPs don’t have a lot of free cash flow, but they do have that rich data flow, and they can build on what the tech giants provide in their platforms. While the mega-platforms have billions of dollars in free cash flow to the millions that langtech vendors do, smaller independent software vendors (ISVs) and tech-savvy LSPs that have paid attention to data collection, structure, curation, and analysis have massive and leverageable amounts of data that they can use to inform, enrich, optimize, and otherwise improve interactions.

The most perspicacious among them have assiduously collected and curated data even when there was no immediate or apparent need, sure that someday this data might have value. They were right. And even for ISVs and LSPs that didn’t systematically collect and curate their assets, they may have enough data on hand to review their translation memories and logs, and harvest data they can use to train their systems.

Any enhancements that improve product function or make them easier to use will contribute to their perceived usability, and thus be noticed by buyers. They begin to expect that every tool they use will offer similar capabilities. They will look for them wherever they go, both in the same and different apps (for example, AI that they find in Microsoft Office will set their expectations for the same functions in Google Docs, and vice versa).

They will likely look for those same advances in NLP in other software as well, such as computer-aided translation tools, translation management systems, terminology databases, and language quality checkers. For example, predictive type-ahead search in a browser was the model for look-ahead adaptive MT translation tools from Lilt and SDL, then Unbabel and Lengoo. Similarly, software from suppliers such as Acrolinx and Vistatec automate the process of making text more intelligent by adding context-setting and semantic detail to content as it passes through workflows.

Small langtech and LSP AI in practice

As part of our research into this smaller form of AI, we contacted leaders at several language software vendors to learn what they’re doing with the massive amounts of data that pass through their systems.

Hideo Yanagi, founder and CEO, Cistate

“Cistate shows how companies that use even off-the-shelf components benefit from crafting utilities to address persistent problems with MT. The company has built a suite of small applications to address everything from expanding abbreviations to correcting punctuation to bridge the last gap between Google Translation and what customers need.”

Manuel Herranz, CEO, Pangeanic

“Pangeanic focuses on the processes that make things human-like or that expand human capabilities. Our ecosystem combines NLP processes and technologies that humans can adapt as they need, including deep adaptive MT, key data extraction, data classification, anonymization, and summarization. Each process is independent and adaptable to specific user needs, but can also be linked to the other processes in an intelligent way. These capabilities are transforming us into a company that extracts value from both structured and unstructured data and adds information. AI components add immediacy to these processes, crossing language barriers and adapting quickly to user scenarios.”

Ivan Smolnikov, founder and CEO, Smartcat

“Smartcat started with the goal of reducing waste in project management and uses AI to preemptively address delivery and production problems, match content to the best linguists, and eliminate non-productive overhead for translation managers and translators, all in a free-to-use system.”

Jack Welde, founder and CEO, Smartling

“Smartling sees the role of AI as eliminating extraneous human work, for example by reducing or eliminating clicks. Even something as basic as using machine learning to automatically identify file types improves the client experience and reduces errors. Similarly, we have a tool that identifies the grammatical gender of strings for translation, with higher accuracy rates than humans, and automatically tags the strings accordingly: This speeds up translation, reduces rework rates, and – most critically – saves a human from having to do ‘scut work.’ The savings from each individual service may be small, but their cumulative impact is enormous.”

Vincent Nguyen, CEO and founder, Ubiqus

“NMT has reached a very high level of accuracy and fluency, and is widely used in LSP workflow and by translators. The challenge will be to measure the quality between NMT providers. The test sets used in recent competitions were composed largely of post-edits of existing engines, which led to a major bias towards those engines. In addition, BLEU and similar metrics are really not useful at all for measuring terminology accuracy when it depends on context or a termbase. At Ubiqus Labs we will never be able to compete against Amazon, Apple, Facebook, Google, or Microsoft on fundamental research, but we do have an advantage in areas that matter to production such as domain tuning, context-awareness, taking into account terminology, and on very low latency.”

José Vega, chairman of the board and co-founder, Wordbee

“At Wordbee I question the applicability of the term ‘AI’ to most of what is going on in the industry. Our work focuses on automating and accelerating as much of the management process as we can, with an emphasis on bridging the gap between content creation and translation. AI helps maintain never-ending segment-based and not-job-based content streams, which every software development company desperately needs today.”

Andrzej Zydroń, CTO, XTM

“XTM has been investing in small AI. These applications focus on relatively discrete tasks such as terminology extraction, improved corpus alignment, better tag handling, and adaptive MT and post-editing. Taken individually, none of these are revolutionary, but they combine to make translation far more efficient.”

Leverage the data you have

The bottom line for smaller langtech developers and LSPs is that their money is short, but their data is large. Creative analysis of the voluminous reams of multilingual content flowing through their software provides the foundation for less flashy but nonetheless important innovations.

Significantly, these enhancements typically come with the standard package. They don’t require customers to rebuild their operations and workflows, but they work to enhance, amplify, and simplify operations with data custom to each application. These langtech developers are taking their user interfaces up a notch, making their products easier to use while they make them more intuitive and decrease the learning curve and cognitive load for common but nettlesome problems.

The mega-tech companies provide platforms, APIs, education, cloud servers, funding for innovation, and a community of buyers and users. Leveraging that base and their own data repositories and analytics, the smaller langtech companies are standing on the shoulders of these giants and adding additional value. This network effect produces a virtuous cycle for artificial intelligence – as more AI technology gets developed, deployed, evolved, rethought, renovated, optimized, and more widespread, its value increases. Who wins? The users — and the ISVs and LSPs that leverage that rich data flowing through their systems.