How to Measure Translation Quality.
Every Language Service Provider will tell you that they provide the highest quality service. They can’t all be right, so how can you compare performance to identify who to entrust with your mission-critical translation projects?
First of all, it’s important to remember that like writing, translation is a very subjective task. For instance:
- People may have different views on the correct terminology to use on a specific subject
- One reviewer may prefer to deviate from the meaning of the original (source) text in order to achieve fluency in the translation
- Another reviewer might see this as the cardinal sin of mistranslation, preferring to stick closely to the exact phrasing of the original content
This makes it all the more important to have a system to follow which gives you an objective measure of translation quality; a score which will give you a clear result which can be used as a basis of comparison between translation companies, and individual translators.
Quality assurance in translation
There are several well-regarded Quality Assurance (QA) models within the language services industry. The downside of many is that, from the point of an industry outsider, they are often complex to the point of being impenetrable.
The person in any particular organisation responsible for choosing a Language Services Provider is often a procurement professional, or a line manager needing a vendor for projects in his or her business function, or perhaps an administrative assistant. People, in short, who are not translation specialists.
This can make it difficult to implement the most accurate translation QA models. But it doesn’t mean an effective solution can’t be achieved by studying how the various high end systems work.
Translation QA models you can use
The LISA QA model was one of the first widely adopted systems. Designed specifically for use in the translation industry, it’s still widely popular.
These days though, there are many other translation QA models available:
- MQM – is a well-regarded language QA model which is designed to adapt to the type of project at hand, the target audience for the translation, and even the quality of the source text. It’s also backwards compatible with older standards, like LISA or SAE J2450 (see below).
- TAUS – a relative newcomer, TAUS is becoming more widely accepted as its flexibility becomes clear. Because the system can be set to be as rigorous (or otherwise) as you need it to be, it’s a great choice for all sorts of content and translation standards. TAUS has different models for human translation and for post-edited machine translation.
- SAE J2450– this system was originally designed for use in the automotive industry, but also works well for translations in other industry sectors. It does tend to be used by companies with a heavy focus on technical language however.
- Various models developed by Translation Management System (TMS) developers
Choosing the right language QA model for you
Selecting the right QA model for judging Language Service Providers is easier when you start by considering your own requirements for a QA model. For example:
Will your reviewer be able to implement the system?
It’s important that the model you select doesn’t create a heavy additional workload for your reviewer, or prove too complex for their level of expertise. Neither of these will encourage a clear assessment, no matter how accurate the QA model itself actually is. This is especially pertinent when the reviewer is not a translation professional but a subject matter expert within your organisation who is helping out to review translations in addition to his or her day job.
What are your actual requirements?
What qualifies a Language Services Provider as being the “best” one for you? You need to accurately assess your requirements in terms of quality:
Do you need an internal document to be just fit for purpose? Here you’ll want translations which are simply highly accurate. Perhaps the document needs to be functional, and not allow for any confusion in the choice of language used, while terminology is acceptable so long as it’s unambiguous.
Or are you producing marketing copy for a local market? Here you’ll need a translator capable of producing a stylish translation which takes into account cultural differences and other factors to convey the essence of what your source document is saying, rather than simply a direct translation of the actual words used.
Will the quality of your source document be considered?
Many QA models simply assume the source document that the translation will be performed on is perfect. Does your source stand up to scrutiny?
In the event that you need a translation of your shiny new brand-compliant marketing literature, most likely it’s a polished piece of content. Are all of your internal communications written to the same standard? Or are you translating back into your home language so you can identify possible errors in your foreign language versions?
Whatever the nature of your project, the quality of the source document needs to be taken into account when considering the accuracy of the translation. Correcting errors in the source document may be a value-add from the Language Service Provider for one project, and a disaster on another.
In fact – consideration of source material quality included – it’s worth talking a little about how language QA models work…
How do language QA models work?
What most translation QA systems have in common is an attempt to categorise each error, and then to produce metrics on the number and severity of errors in each category. This is then converted into a quality score according to the weighting attached to each category and the severity of the error.
Categories of errors are typically based on considerations such as:
1. Is the translation conveying the right message?
2. Are there any mistranslations?
3. Are tone and style suitable for the context and the target audience?
4. Are there any typos or grammar mistakes?
5. Does it flow?
6. Is the right terminology being used?
Most language QA models have been developed with large-scale, technical translation in mind, and may therefore not be very suitable for translation of content such as marketing communications or e-learning content in a corporate environment.
Once you understand the principles on which the systems are based though, it’s relatively simple to identify one you like and then adapt it to suit your specific requirements.
The simple way – applying the principles of translation quality assurance
If you don’t have the time, or an employee with the correct expertise, to implement one of the metrics for measuring translation quality mentioned here, there’s still a lot that the systems themselves can teach you about how to judge the accuracy of a translation.
Here are the basics of a translation QA process that we can take away from how these systems are generally designed to work:
1. Have someone review the translation and make any necessary changes
2. Have an independent linguist or translation company categorise the changes and rate their severity
3. Use a tool or simply manually count to produce a report indicating the number of errors (making sure to exclude any changes which are entirely subjective) of each category and severity, and the total word count of the sample used for assessment
4. Use whatever weighting system makes sense to you to produce a translation quality score
This way you’re achieving essentially the same kind of result that any language QA system is designed to give you, but without the need to precisely toe an unnecessary line as you follow a system which might not be exactly right for your needs.
One of the key parts of the process described above is to use a completely independent linguist or translation company in step 2. It’s important that this impartial judge is ware there is no prospect of winning any sort of translation contract out of their work. This ensures there is no conflict of interest, and that their judgement of the quality of the translation is one on which you can rely.
Asian Absolute – translation quality assurance experts
If you have any doubts about how to assess the quality of a translation project, why not talk to the experts?
With over sixteen years experience delivering professional translation services to individuals, start-up businesses, and Fortune 500 clients on five different continents, Asian Absolute knows the science of translation.
Use us as your independent translation quality assurance specialists. Or get the same high quality 24-hour language services used by business leaders in all industry sectors, in locations across the globe.
Give your local Asian Absolute team a call now – 24 hours a day – and get a free, no-obligation quote on your project.