This tutorial will provide an in-depth look at the experiments, jointly carried out by KantanMT and eBay during 2018, to determine which Neural Model delivers the best translation performance for eBay Customer Service content. It will lay out the timeline, process and mechanisms used to customise Neural MT models and how these were used in conjunction with Human Based evaluations to determine which approach to Neural MT provided the best translation outcomes.The tutorial will cover the following topics and methods:- Structural differences in Neural Networks and how they assist the language decoding process – RNN, CNN and TNN will be covered in detailed.- Customisation of Neural MT using the KantanMT Platform- Using MQM Framework for the evaluation and comparison of Translation Outputs and comparison to Human Translation- Collation and analysis of experimental findings in reaching our decision to standardise on Transformer type networks.Participants of the tutorial will get a clear understanding of Neural Model types and the differences, it will also cover how to customise these models and then how to set up a controlled experiment to determine translation performance.

Most of the test sets used for the evaluation of MT systems reflect the frequency distribution of different phenomena found in naturally occurring data (”standard” or ”natural” test sets). However, to better understand particular strengths and weaknesses of MT systems, especially those based on neural networks, it is necessary to apply more focused evaluation procedures. Therefore, another type of test sets (”challenge” test sets, also called ”test suites”) is being increasingly employed in order to highlight points of difficulty which are relevant to model development, training, or using of the given system. This tutorial will be useful for anyone (researchers, developers, users, translators) interested in detailed evaluation and getting a better understanding of machine translation (MT) systems and models. The attendees will learn about the motivation and linguistic background of challenge test sets and a range of testing possibilities applied to the state-of-the-art MT systems, as well as a number of practical aspects and challenges.

In the last couple of years, machine translation technology has seen major changes with the breakthrough of neural machine translation (NMT), a growing number of providers and translation platforms. Machine Translation generally is experiencing a peak in demand from translation buyers, thanks to Machine Learning and AI being omnipresent in the media and at industry events. At the same time, new models for defining translation quality are becoming more widely adopted. These changes have profound implications for translators, LSPs and translation buyers: translators have to adjust their post-editing approaches, while LSPs and translation buyers are faced with decisions on selecting providers, best approaches for updating MT systems, financial investments, integrating tools, and getting the timing for implementation right for an optimum ROI.In this tutorial on MT and post-editing we would like to continue sharing the latest trends in the field of MT technologies, and discuss their impact on post-editing practices as well as integrating MT on large, multi-language translation programs. We will look at tool compatibility, different use cases of MT and dynamic quality models, and share our experience of measuring performance.

The tutorial will introduce a set of very useful statistical tests for conducting analyses in the research areas of Machine Translation (MT) and Translation Studies (TS). For each statistical test, the presenter will: 1) introduce it in the context of a common research example that pertains to the area of MT and/or TS 2) explain the technique behind the test and its assumptions 3) cover common pitfalls when the test is applied in research studies, and 4) conduct a hands-on activity so that attendees can put the knowledge acquired in practice straight-away. All examples and exercises will be in R. The following statistical tests will be covered: t-tests (both parametric and non-parametric), bootstrap resampling, Pearson and Spearman correlation coefficients, linear mixed-effects models.