{"id":9068,"date":"2020-11-12T14:57:50","date_gmt":"2020-11-12T22:57:50","guid":{"rendered":"https://www.microsoft.com\/en-us\/translator/blog\/?p=9068"},"modified":"2020-11-24T06:13:56","modified_gmt":"2020-11-24T14:13:56","slug":"microsoft-custom-translator-pushes-the-translation-quality-bar-closer-to-human-parity","status":"publish","type":"post","link":"https://www.microsoft.com\/en-us\/translator/blog\/2020\/11\/12\/microsoft-custom-translator-pushes-the-translation-quality-bar-closer-to-human-parity\/","title":{"rendered":"Microsoft Custom Translator pushes the translation quality bar closer to human parity"},"content":{"rendered":"

\"\"<\/p>\n

The Custom Translator journey to be on the leading edge of machine translation technology continues.<\/p>\n

In early August 2020, we started our Custom Translator upgrade from Long Short-Term Memory (aka LSTM) based neural machine translation architecture (or V1) to our Microsoft Translator\u2019s state-of-the-art Transformer based architecture (or V2). V2 is the same translation architecture which powers the standard uncustomized Translator API, as well as translation in Microsoft Office 365, Speech, Bing.com\/translator, Edge, and more.<\/p>\n

The August release enabled customers to use the dictionary<\/strong> (phrase or sentence) document type to build custom models on top of the V2 platform for a quick translation quality improvement over the V1 platform.<\/p>\n

Today, Custom Translator completed the full V2 platform upgrade to deliver an even bigger translation quality gain than before. Customers can now build custom models with all document types (Training, Testing, Tuning, Phrase Dictionary and Sentence Dictionary) using full text documents, like Office documents, PDFs, HTML and plain text.<\/p>\n

With this release, enterprises, small and medium sized businesses, app developers, and language service providers can build advanced custom neural translation systems that respect their defined business terminology and seamlessly integrate those systems into existing or new applications, workflows, and websites to attract customers and grow the business.<\/p>\n

We put every new baseline language model through a rigorous human evaluation process to ensure the translation quality continues to meet high standards on generic input across all domains. However, with custom trained specialized translation systems, customers can achieve much higher adherence to the domain-specific terminology and style by training a custom translation system on previously translated, in-domain documents. These previously translated documents allow Custom Translator to learn the preferred translations in context, so Translator can apply these terms and phrases when the context calls for it, and produce a fluent translation in the target language, respecting the context-dependent grammar of the target language.<\/p>\n

Benefits of the upgrade<\/h2>\n

We use BLEU score<\/a> (a standard way in the research community) to measure the translation quality of a newly trained baseline model. A one or two BLEU point gain is a worthy achievement. The Custom Translator V2 platform upgrade will deliver significant improvements when compared to the previous V1 platform. The bar chart below depicts the translation quality BLEU score improvement for some domains and the impact of training dataset size.<\/p>\n

Sample domains translation quality BLEU score when using standard Translator, Custom Translator V1, and Custom Translator V2.<\/strong>
\nTraining dataset size in thousands (\u2018auto-28k\u2019 means 28,000 parallel sentences for the automotive domain)<\/p>\n

\"\"<\/p>\n

It is important to note that actual quality improvement is dependent upon customer data quality, training dataset size, and domain coverage.<\/p>\n

\u201cWe\u2019re hoping that translation through a neural network will not only boost quality and speed, but also offer advances in the evaluation of big data,\u201d <\/em>said Nikolas Meyer-Aun, Head of Quality and Supplier Management for Languages at Volkswagen AG<\/p><\/blockquote>\n