Quill, a content creation agency, has just published a report highlighting the gaps between human translation and computer aided or machine translation within the online retail sector.

‘Man vs. Machine: Is machine translation ready for retail?’ is the result of a survey of 400 male and female consumers (aged 18+) to try and understand how they felt about machine  and human translated online product descriptions.

The survey covered only France, Germany, China and Japan.

In each country they surveyed, the participants were asked to review online product descriptions which originated from the same English source text, but had been either machine-translated (using Google Translate) or localised into the target language by professional human translators.

Shoppers were asked to review the product descriptions side-by-side, without knowledge of whether they were human or machine translations and answer the following questions:

:: Level of understanding
Which product description is hardest to understand? (A/B choice)

:: Purchase intent
Which product description makes you feel more likely to buy the product? (A/B choice)

:: Brand reputation
Which product description leaves you with the best  impression of the retailer? (A/B choice)

The descriptions spanned fashion, home wear, technology & beauty.

So who won? Humans or Machines?

In short, human translators although in some cases it appears that machines are well and truly on their way to creating some redundancies in the translation & localization industry.

online shoppers prefer human translations


When it came to a basic level of understanding, where participants were asked to choose which description they found the most difficult to understand, machine translations and humans pretty much drew level – a 50/50 split on average across the board.

Where human translators really show their value is in the other metrics of ‘purchase intent’
and ‘brand reputation’. Respondents were asked which description made them feel
more likely to buy the product, and which description left them with a better impression
of the retailer, respectively.

For both  the human translations dominated with 79% of the 400 respondents stating that the human-translated copy made them more likely to buy – while 80% of respondents agreed that the human-translated descriptions gave them a more positive impression of the retailer

As the findings illustrate, whilst online shoppers can understand machine-
translated content to a certain degree for basic information, it still isn’t anywhere near as persuasive, adaptive, convincing, in-tune or brand-focused as human-translated content.

While machine translation has come a long way since the days of garbled, unintelligible output – and the shift to using deep neural networks to contextualise phrases has made a vast difference to translation quality – our study has shown that machine-generated copy is nowhere near as effective as human-translated content at driving purchase intent and brand affinity.

To sum it up, in online retail anyway, machine translation still has some learning to do!

Download your copy of the report from Quill here.