Requesting test translations

So you’re looking to purchase translations and onboard a new language service provider (LSP). Why not simply request a test translation from several suppliers and select the supplier whose translations you like the most? With the latest estimates indicating that there are more than 27,000 language service providers operating globally, language buyers are attracted to test translations because they seem like an easy way of narrowing the field to a more manageable number of potential suppliers.

The prevailing logic is that a test translation will give you access to both subjective and objective information, which will help you to make a decision. You will get to know potential suppliers, and you will be able to measure the quality of the output of different suppliers.

But do test translations really offer you the best measure of an LSP?

Building long-term quality to specification

Think back to when you were in the throes of exam season at high school. It is quite possible that you would “cram” for exams the night before. While this was a tried and tested method of getting the grades you needed to progress, it almost certainly didn’t result in long-term learning. Would you have been able to retake the exam a month later without extra revision time and achieve the same grades?

The same applies with test translations requested of prospective LSPs. They might pass your test, but does this really show that they have the infrastructure and processes to support your localization needs going forwards? What guarantees are there that the same translator they used for your test will translate your work later? What most businesses really need in order to choose the right supplier for their localization program is to ensure that a supplier has the ability to create a high quality, repeatable solution which meets their localization needs.

When launching a new product, businesses go through a series of process stages before the actual launch. Market assessments, competitive evaluation, prototype development, product testing, regulatory approvals where necessary, manufacturing all happen before finally commercializing a new product. These processes are necessary to ensure that a product is a success, and involve many parts of the business.

Building a high-quality, repeatable localization program requires a similar approach. Successful localization programs are built to a plan; think of it as manufacturing to specification. This means that your business needs to discuss the needs of internal stakeholders before launching a selection process. It also means that if you are to select the best supplier for your needs that you discuss the challenges you have identified with the suppliers that you invite to participate in your selection process.

This allows suppliers to bring their know-how to the table. It allows them to build a prototype process for you which meets your business specifications.

You may still decide that you want to include a test translation as part of your selection process. One of the early decisions you will need to make is whether to pay for test translations. Should you expect to pay for the tests or should potential suppliers expect to perform free test translations as part of a selection process?

When buying a new car, you can take it for a test drive. When buying a new house, you can normally view a pilot home, but for services the approach is often different. Barbers and hairstylists will often have a portfolio of photos which show different hair styles. If you want a designer to create a new ecommerce portal for your website, the best you can normally hope for is to see samples of similar sites the designer has created.

What do the experts say?

The American Translators Association’s Code of Conduct explicitly specifies that its members must not require their translators to work for free in return for the prospect of paid work.

There are two quality standards to which language service providers are most often certified:

ISO 9001:2008 is an organizational framework, which means that suppliers need to have documented policies and procedures in place. Those procedures will be defined by each certified organization, and while management of suppliers is an important part of ISO9001:2008 it does not specify any requirement to test suppliers, and nor does it make any recommendation about whether tests should be remunerated

ISO 17100:2015 (formerly EN15038:2006) is a translation services standard which specifies that there does need to be a documented procedure for the selection of appropriate project collaborators, but again, no mention is made of either test translations, or whether payment should be made for work carried out by the supplier as part of the selection process.

What do providers think?

Most LSPs are commercially realistic, and will be financially robust enough to have budgeted for some unbilled work as part of marketing spend or similar.

That said, any savvy supplier will need to perform some sort of bid decision calculation, and if the numbers don’t add up, then they will need to decline as elegantly as it possible. It may be that the supplier will want to strike some sort of a deal. For example, that should paid work be eventually awarded, the samples will be paid for as part of that initial project.

What makes a suitable test?

So, you decided to go ahead and test your potential suppliers. Does content type matter?

While it is true that a test translation will only show whether a supplier is suitable if it is representative of your eventual translation program, a general, straightforward technical test is unlikely to help you to differentiate one supplier from another.

A challenging test that requires some specialist knowledge is more likely to find a suitable partner because when it comes to evaluating the translation, your in-country specialists will recognize and appreciate what they consider a correct and appropriate translation. If an LSP has delivered a solid translation of a specialized text, it is likely that they have performed some level of linguistic evaluation of the test document and have then gone on to choose translators with an interest in the tests’ field rather than sending the text to general technical linguists.

A word of caution regarding marketing materials: it’s extremely difficult for an LSP to find an appropriate “voice” for your local marketing strategy in a one-off test scenario. What’s more, the evaluation of marketing translations is more prone to subjective criteria and preferences. It is possible to provide a high quality marketing translation service, but this is easier to achieve once you have selected the supplier you want to partner with.

What are testing best practices?

The aim of a test translation should be to help you to select a suitable LSP. While it might be tempting to hold the test under harsh conditions, it is really in everyone’s interest if you make it possible to get the results you want. That means that as a client you may well need to do some preparation work to make the process a success.

 Supply useful reference materials. For example, if your technical authors use a style guide, let those taking the test have a copy. Better still, if your organization has invested in multilingual terminology glossaries or localization style guides, get those to the language service providers so that they have the best possible tools at their disposal to perform well.

 Make sure that you build a transparent process. Allow and expect your providers to ask some questions. Explain the measurement criteria for the test, and build a feedback mechanism into your process. From an LSP’s perspective, there are few things more frustrating than being told that you have “failed” a test, but not being given proper feedback. Feedback allows a provider to evaluate their performance and make changes the next time they are in the same situation.

 Ensure there is a formal methodology for evaluating the quality of the sample rather than simply a subjective evaluation. This will enable you to provide intelligent, structured feedback to providers who took part in the test. There are a number of standard quality measurement frameworks available, each categorizing errors empirically according to a standard scale. Examples of these language QA frameworks are the Lisa QA quality model, SAE J2450 or the TAUS Dynamic Quality Framework.

 Size matters! With very few exceptions, it should be possible to evaluate a supplier properly using somewhere between 200 and 350 words.

 Finally, double-check that the section of text chosen for the test is not already readily available in translated form on your website! While most providers are of unimpeachable character, if they do happen upon the text in their research it can become difficult not to be influenced by the existing translation.

Are there any alternatives to test translations?

So, if translation tests are not for you, what are the alternatives? Luckily there are other ways you can evaluate LSPs.

A thorough initial selection process will uncover whether a supplier has quality certifications. While carrying a certificate won’t guarantee perfect quality each time, it is likely to guarantee that a supplier has been audited and has been found to have robust, documented and repeatable processes. These are the pillars upon which high quality work can be built.

Responses to a request for proposal (RFP) can also help to demonstrate how a supplier is likely to behave if contracted. By following up references, you can gain insight into how a supplier operates.

Clients often request examples of linguists CVs as part of this process, or reference clients who are part of the same industry vertical as part of an RFP process. This information can help to reassure you that your chosen language provider has the necessary subject matter expertise and competencies to work with your business.

As discussed earlier, what a business really needs is to have a reliable, high quality and repeatable process. In order to achieve this, you need to engage with potential suppliers by having a two-way discussion about the challenges you currently face and the midterm developments that might influence how a supplier collaborates with you.

Once you have chosen a language partner, there are ongoing maintenance steps you can carry out, such as on-site supplier audits, agreeing on key performance indicators and reviewing them regularly with your supplier, or carrying out random spot-checks of their work and validating quality by using suitable in-country personnel.

At best, test translations offer a small piece in the full jigsaw when evaluating and selecting a language service provider. Communicate, engage, question and listen carefully in order to successfully select the right provider for your process.

Joanna Weitz, operations director at Argos Multilingual, has 12 years’ experience in the localization industry, combining a strategic and operational view on the localization process.

Joanna Weitz

About Joanna Weitz

Joanna Weitz, operations director at Argos Multilingual, has 12 years’ experience in the localization industry, combining a strategic and operational view on the localization process.

Leave a Reply

Secured By miniOrange