Experiential Localization Quality Assessment (xLQA)

Experiential Localization Quality Assessment (xLQA)

and User-Centricity Illustrated

and User-Centricity Illustrated

Tucker Johnson

Tucker Johnson is the founder of Nimdzi Insights, a market research and consulting agency specializing in globalized operations. He is the author of The General Theory of the Translation Company, and professor of Translation and Localization Management (TLM) at the Middlebury Institute of International Studies. Tucker lives in Seattle with his wife and three kids.

Tucker Johnson

There is more competition for consumers’ attention than ever before. As technology improves, it influences how brands engage with their global customers. Today, brands need to be aware of not only what language they are using, but also how they are using it. It is harder now than ever to comprehensively define “language quality” or “translation quality,” if not downright impossible.

This article will not be defining quality. We would need much more (infinite, perhaps) time for such an endeavor. What we will be doing is taking a look at five trends affecting how global brands manage and measure language effectiveness across markets. Specifically, we’ll be looking at the increase in user-centered design principles and their effects on how we understand language and localization.

What we will refer to broadly as “user-centricity” is known by many other names: design thinking, user experience (UX) design, interactive design, user-centered design, etc. Some of these may be unique enough to comprise their own field of study, separate from UX design. However, it is more useful to focus on the commonalities that bind them all together.

The first thing all user-centric frameworks have in common is an almost fanatical focus on satisfying the needs of the user — up to and including needs that those users themselves didn’t even know they had.

User-centricity requires a deep understanding of your customer, leading to a better understanding of how to engage with them. Global user-centricity requires this as well, multiplied by however many markets your product or service is offered in. But whereas companies are investing more and more into user-centric design principles, most have not yet gotten the memo that they can’t assume their international customers are driven by the same buying behaviors as their domestic counterparts.

Some markets are more sensitive than others. As seen in figure 1, consumers in some markets place a lot of value on design/style/user experience, whereas consumers in other markets don’t seem to care.

In this article, we are going to look at five market trends borne out of user-centricity, and how these are forcing us as an industry to reevaluate how we define and measure language quality. Spoiler alert: we are past due for a disruption in this space.

Language quality is no longer “just a check box”. In today’s customer-centric organizations, quality, masquerading as design principles, permeates all departments. While a positive trend, it presents new challenges for globalization departments.

Five user-centered trends affecting language quality management

Trend #1: Quality as a step -> Quality as a design principle

Rejoice! After years of complaining about how language quality is too often treated as just another checkbox to tick at the end of a project, I’m happy to report that more and more companies are starting to look at quality not as a step or a task, but as a mindset or a value system. There have been many contributing factors to this shift. One factor has been the proliferation of user experience (UX) design principles within global organizations.

Beyond a laser focus on the end-user, another commonality between all user-centric systems is that they all demand a holistic, cross-functional approach to achieve this goal. As user-centricity concepts grow in popularity, localization managers are finding that they have allies “upstream” who also care about the quality of the localized end-product experience. However, there is still much room to improve. Not even the best-trained UX designers and global research teams are experts in global markets or language translation, and most of them do not currently collaborate with the globalization teams within their organization.

Beyond the global UX and research departments, other teams seem to be growing their awareness of localization as well. Marketing and search engine optimization (SEO) teams work on localized keywords and product teams are learning to liaise early and often with their localization colleagues to avoid costly and embarrassing mistakes down the line. This is great news for language quality experts. The more people that share a goal of producing high-quality localized products, the less likely quality will come to be an afterthought. The price we pay for this is simply the increased amount of work that comes from managing internal stakeholders.

In the past, companies hired project managers (PMs) to manage their external vendors. Today, more and more enterprise globalization teams are hiring and training “internal evangelists” responsible for building and developing relation-ships with internal stakeholders (i.e. their clients). In today’s user-centered reality, focused on user-centered goals, cross-functional collaboration is needed more than ever not just between localization teams and design teams, but between all internal teams that play a role in the product life cycle (in other words, everybody).

Of course, this consideration does not end with internal stakeholders.

Figure 1: While user-centric design is on the rise around the world, not all markets are as sensitive to product design when making their buying decisions. Source: Nimdzi Insights.

Trend #2: Quality is reported -> quality is experienced

The ultimate focus of the modern quality program needs to be directed at global end-users.

In the past, global brands lacked the tools to confidently collect and analyze end-user feedback. They were also unable to push product updates post-launch and improve on the fly. Some of us are old enough to remember thinking that paying monthly for an Adobe or Office365 subscription was new and strange compared to the boxed-product software applications we would stand in line to purchase at Best Buy or any similar large electronics store.

In the old reality, we relied heavily on language quality assurance (LQA) and language quality evaluation (LQE) processes, which would add a layer of review to a translation and provide a score on the quality. These quality scores could then be reviewed by project stakeholders who did not understand the target language but needed to be aware of the quality of the translated content. Because these scores were the only data we had, we have developed an unhealthy relationship with them over the years.

In the past, when user satisfaction was harder to come by, LQA scores were intended as a rough leading indicator of end-user satisfaction. They were (and are) a measurement of progress toward a goal. However, for many of us, LQA scores have at times become not the measurement, but the goal itself.

Today we are in a completely new reality. Brands are inundated with user feedback from all over the world and in all languages. This means we need to listen to them. We also have the means at our disposal to push updates at any time. This means our users expect nothing less than continuous improvement.

Forward-thinking companies are realizing that their bud-get for LQA reviewers can be easily reallocated to global user experience research. As with the first trend mentioned above, this disruptive change can also be the most difficult for organizations with the most mature (read: complicated, interdependent, expensive) globalization programs.

You’ll notice that we’ve been talking about process management so far, but haven’t yet touched on shifts to quality expectations themselves. Let’s turn our attention to the ongoing trend away from metaphrase (literal translations and formal language) towards paraphrase (localized translations and modern/informal language).

Conﬁdence in traditional LQA/LQE is waning. Organizations are paying closer attention to the experience of their global customers. New user-centric quality frameworks are being deﬁned to conﬁdently measure and improve the localized user experience (both in general and for each market) over time.

Trend #3: Metaphrase -> Paraphrase

Any recent localization graduate can happily recite the differences between translation, localization, internationalization, globalization, etc. However, many of us with a few years of experience in the industry have noticed that those definitions are in reality quite pliable.

In years past, brands competed for consumer trust. They still do. Today, though, brands have unprecedented levels of access directly to their customers allowing for more intimate relationships, and the language that brands use needs to reflect this intimacy.

This has brought about a general shift away from formality and towards a more modern, accessible language. The nuances of how this shift plays out across languages and cultures are complex as not all markets are comfortable with the same levels of informality. Still, on balance, language services are moving away from literal translation (metaphrase) toward true localization (paraphrase) via transcreation, local copy-writing, cultural review, and the like.

So while recent grads may have been taught the so-called wrong definitions, I doubt this can be held against them. In ten years, the services we provide in our industry will have evolved even further. The technology we use will grow in complexity. Out of necessity, the words we use to describe these things will need to adapt as well.

Trend #4: Academically driven -> Influencer driven

Language is always evolving, though the mechanisms driving that evolution may change over time.

As long as words have existed, certain people have felt the need to criticize how other people use those words. This will never change. However, the same trends that have connected brands with their consumers are also giving those consumers a megaphone to the world.

In the past, language and grammar were discussed and debated among scholars behind closed doors. Papers were written and then published to niche audiences. Conferences were held — invitation only. Not just anybody could discuss language, at least not in a meaningful way.

Today, anybody can, and will, discuss language. Notwithstanding some frustratingly regressive governments that are still clinging to restrictive or compulsive speech policies, people are freer to exchange ideas than ever before in human history. Furthermore, people have the power to educate themselves so that they can develop informed opinions. Not entirely unreasonably, some of these people are starting to demand more say as to how language rules are written, or who they are written for. Language activism, by no means a new phenomenon, enjoys access to new tools and global networks to amplify its messages.

As much as we have tried to create flexible models to objectively measure language quality, we may as well be chasing the wind. Language is like a particularly frustrating virus, always evolving one step ahead of the host. As we define objective standards and create sophisticated models to measure language quality, language itself is growing (or at least mutating) in complexity. It’s as if we are always trying to capture all the complexities of all languages in a box, but we are too slow. The complexity of language is increasing faster than we are currently capable of measuring it.

The trend towards friendlier, more inclusive brand voice calls for prioritizing the message/intent of a translation (style/paraphrase) over strict adherence to literal translations (accuracy/metaphrase).

Trend #5: Project Deliverable KPI’s -> “experienced” customer Service

If we can agree that language is, and will continue, increasing in complexity, it sheds some light onto how and why many agree that traditional quality management models are bordering on the obsolete, though this only paints a partial picture — a picture that is still focused on the process rather than on the user.

Today, we as consumers are spoiled. For us millennials, this holds doubly true. The same trends driving the shift toward customer intimacy we discussed above are also driving a shift towards customer entitlement. When buyers feel entitled, a marvelous thing happens: brands start to listen to their customers in an effort to build customer intimacy. What is intimacy, after all, if not an ongoing attempt between two parties to better understand, empathize with, and support each other.

The most striking way this shift manifests itself is in the increasing expectations surrounding customer service. When market expectations shift, opportunities for brand differentiation arise. This creates more competition. When competition increases, the consumer always wins — in this case, consumers win in the form of being able to expect friendlier, more competent, more localized relationships with the brands they support.

It is no longer enough to localize into your customers’ language. People have always preferred to interact in their own language and still do. But today, expectations are so localized that it becomes impossible to please everybody. Despite the impossibility of the task, that is nonetheless the goal. Every day we go to work — global marketing teams, customer support departments, and account managers — all working toward the impossible task of forging strong relationships with consumers.

To put these market data into perspective, it is useful to compare to a standard benchmark that most are familiar with. Figure 2, originally published in Creating Loveable International Customer Experiences by Pactera Edge, compares the importance of UX and design against the benchmark of the United States.

While increasing expectations and maturation of user-centric design processes are influencing the way everybody works in an organization, localization managers are especially affected because not only do they have to be aware of shifting perceptions of the English language, they have to be aware of these shifts for dozens of different languages beyond English. Going further, they have to develop the skills needed to effectively communicate — and advocate for — these changes to other stakeholders who are not as well-versed in localization.

Language has always evolved and changed over time. In the past, this change was codiﬁed (and, arguably, inﬂuenced) by authoritative bodies and language scholars. Today, anybody can study and debate language and so traditional academics, adhering to rigid standards, struggle to keep up with the pace of change.

Getting a headstart on the impending disruption

Ironically, the companies with the most “mature” quality management processes are the very same companies that are struggling to redefine the way things are done. Globalization teams sometimes approach process design as an iterative process. Processes are defined, and then more processes are built on top of the old ones. Bugs are triaged and fixed. Custom automation and integration systems are developed and deployed. Knowledge base articles are published, and the end result is often a hideous yet efficient machine to churn out high-quality translations on time and on budget.

“Globalization teams sometimes approach process design as an iterative process. Processes are defined, and then more processes are built on top of the old ones. Bugs are triaged and fixed.”

This is no longer good enough

The brands that I have spoken to, or at least read research briefs on, are having the same conversations about quality. Most organizations I have worked with are focused on iterating their current language quality programs to adapt to the changes in language. An iterative continuous improvement approach has served traditional LQA programs well for decades. Today, we have flexible, modular LQA models that do what they can to take into account that not all content is created equal and adjust quality expectations accordingly.

But an iterative approach is not going to be enough to respond to the impending disruption to industry-standard quality management programs. There is nothing to salvage here. To be clear: please let me reaffirm my respect for the LQA models that came before and all they have done for the industry. That said, they have to go.

These old models, carried out without any input whatsoever from the end-users, are already being replaced. Brands are prototyping new processes and models, which measure and report quality from the user perspective, with less attention paid to what a linguist deems “correct” or “accurate.”

Figure 2: Design principles are increasing in importance at different rates across markets. Not all languages require the same level of sensitivity to customer experience/design. Globalization managers need to be aware of these differences to better lead their respective brands as they engage with new markets and cultures.

01. Figure out what matters to your customers

Your customers are not identical to other companies. Your customers are probably not even identical to the customers of your closest competitors. So why would you use the same quality model as other companies? And before you protest that your LQA program is unique, I assure you it’s not. Not for today’s consumers and certainly not for the consumers of the future.

The first step, before we even think about quality, is to figure out all we can about our customers. Only once we have this understanding can we start to define language quality.

02. Define Language Quality

We no longer assume that we know an objective standard by which quality can be measured. There are objective ways to measure the accuracy and fluency of a translation, and that is not in question. What is in question is whether or not those objective measurements still matter.

Having taken the time to understand our customers in the first step, we can now begin to understand what high quality means to them. This isn’t to say we take the feedback, survey results, notes, etc. from that step and then lock ourselves in a room with quality managers to redefine quality, though. We still need to include our customers. Get used to it. In today’s customer-centric world, we always need to be including our customers.

03. Track, measure, report, improve what is Important.

In the past, we knew — or thought we knew — what was important to track, measure, and report. Life was simpler. We focused on mistakes because LQA scores are calculated based on the number of mistakes made in a translation, as judged by a linguist, not an end-user.

Today, mistakes are not important. Customers are important. Globalization teams at major brands are trading in their LQA spreadsheets for user surveys. Localizing surveys into customer languages and asking them about their experience is a great place to start. The survey design does not need to be complex, and there are research partners out there that can help with this. Focus groups, either virtual or in-person, are even more useful at uncovering hidden truths about your customers if the budget allows.

What is the complexity? What about the cost? Obsessively tracking user experience is not simple and it is not cheap. However, neither is a traditional third-party LQA program. In my not-so-humble-yet-well-earned-opinion, third-party LQA programs that employ the use of a completely separate vendor to evaluate the translations of a primary translation vendor are one of the single most wasteful practices in our industry. When we pay a vendor to provide translation, we should expect them to deliver acceptable quality. Period. If you don’t trust the quality of your translation vendor, then what you need is not an additional LQA vendor; what you need is a new translation vendor. Perhaps one that actually charges higher rates so that they, in turn, can afford to hire the more experienced translators — but I digress.

Closing Thoughts

User-centricity requires a deep understanding of your customer, leading to a better understanding of how to engage with them. When our customers are global, speaking in different languages, and influenced by different cultural norms, this process can quickly seem overwhelming.

Saying goodbye to LQA spreadsheets may not be easy. There will be challenges just as with any process migration. However, once we can collectively mourn the loss of our old friend, LQA, we are free to work with our customers to define new measurements. Our customers are shifting their expectations around quality. It’s time that we catch up.