Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing

Quality estimation is often a desired component for developing and deploying automatic language technologies, and has been extensively researched in machine translation Barrault et al. (2019). Its purpose is to provide some metrics measuring the overall quality. The current state-of-the-art models mostly originated from the predictor-estimator framework Kim et al. (2017), where a sequence-to-sequence model is pre-trained to extract sophisticated sequence features to be fed into a sequence level regression or classification network.

Tan et al. (2017) proposed the neural post-editing based quality estimation by streamlining together the traditional QE and APE models. Since our proposed QE module will eventually serve the APE module as well, we consider two modifications accordingly. First, we re-define the QE as a fine-grained multi-class problem, whose output indicates the number of tokens in four categories, missing / redundant / erroneous or kept tokens. A similar idea was initially proposed in Gu et al. (2017) to predict the number of copy occurrences in non-autoregressive neural machine translation. In this paper, we make significant extensions to include more categories. Secondly, we maximize our QE model performance with a novel conditional BERT architecture. Inspired by the masked language model objective in the encoder BERT Devlin et al. (2019), we introduce the training objective to the encoder-decoder framework by adapting the decoder to become a memory encoder, allowing us to pre-train the target language model similar to BERT but conditioned on the source language text.

Automatic Post Editing aims to improve the quality of an existing MT system by learning from human edited samples, converting “translationese” output into natural text. The traditional APE is based on a round-trip translation loop to mimic errors similar to the ones produced by NMT and can achieve acceptable performance with large scale monolingual data only Freitag et al. (2019). However, the prevalent trend in this area prefers the dual-source encoder-decoder architecture with parallel data Chatterjee et al. (2017b); Junczys-Dowmunt and Grundkiewicz (2018); Pal et al. (2018); Lopes et al. (2019), which obtained the best results in WMT competitions Chatterjee et al. (2019). The dual-source encoder encodes the source text and the machine translation output separately, and the decoder decodes the post-edited results. All these approaches encode each source independently and apply an auto-regressive decoder. They differ in their parameter sharing mechanisms.

While our approach still employs the multi-source APE framework, but there are two fundamental differences. First, our APE module, as aforementioned above, is built on our re-designed QE model, with which the source and the machine translation are entangled by the encoder and memory-encoder QE module. Second, our decoder consists in a versatile architecture that can choose between the left to right auto-regressive generative model and the atomic-operation based paralleled model. It dynamically determines which model to engage at runtime. The parallelizable model was broadly explored in insertion- or deletion- based transformer Chan et al. (2019); Stern et al. (2019); Gu et al. (2019), while our decoder supports more functional operations.

As described in the related work, compared to traditional translation QE task in WMT²²2http://www.statmt.org/wmt19/qe-task.html, our QE module is more fine-grained and is recast as a multi-class $\{-1,0,1,...,K\}$ sequence labeling problem. The definition of the integer labels is shown in Table 2. If $k<=1$, the label denotes one single token operation; otherwise, it means to insert $k-1$ extra tokens after the current one. The QE tag $\mathbf{q}$ for training pair $(\mathbf{m},\mathbf{e})$ can be deterministically calculated by dynamic programming Algorithm 4 in Appendix, which is basically a string matching algorithm. We define a conditionally independent sequence tagging model for the error prediction.

A transformer based neural network is employed. We present a novel encoder-memory encoder framework with memory attention as shown in the decomposition of the following equation.

where $\text{Enc}(\cdot)$ is the standard transformer encoder Vaswani et al. (2017), and $\text{Enc}^{M}(\cdot)$ is the memory encoder adapted from standard transformer decoder. It removed the future masking in the transformer decoder and use the last state as the output which contains contexts from both SRC and MT.

During inference, neither the ground truth of post-editing nor the golden translation reference is available. The fine-grained QE model can predict the human translation edit rate (HTER) $h$ through the inferred QE tags $\hat{\mathbf{q}}$.

On the one hand, the overall metric $h$ can quantitate the quality of machine translation and determine which APE algorithm will be used. On the other hand, the detailed QE tags can theoretically guide the APE which atomic operation should be applied. Thus, the QE tagging and the atomic operation APE are simultaneously and iteratively trained, which will be elaborated in 3.2 and 3.5.

The key idea of atomic operation APE is to reduce all predefined operations (insertion, deletion, substitution) into a special substitution operation by introducing an artificial token placeholder [PLH].

First, we align the machine translation $\mathbf{m}$ and the post-edits $\mathbf{e}$ by inserting [PLH]s, resulting in a new $\tilde{\mathbf{m}}$ of the same length as $\mathbf{e}$. Technically, we insert $q_{i}-1$ [PLH]s after $m_{i}$ if $q_{i}>1$; we delete the current token $m_{i}$ if $q_{i}=0$; we replace $m_{i}$ with [PLH] if $q_{i}=-1$. For convenience, this process is denoted as $\tilde{\mathbf{m}}=\text{PLH\_INS}(\mathbf{m},\mathbf{q})$.

Second, the original APE task is transformed into another sequence tagging problem, since $|\tilde{\mathbf{m}}|=|\mathbf{e}|$.

Notice that i) the encoder and memory encoder share the parameters with the QE in Equation (2); ii) the softmax layer is different, because the number of outputs in APE has a different size equal to the vocabulary size. An intuitive visualization can see the Figure 3 and the holistic pipeline sees the Figure 1.

The larger HTER $h$ is, the lower quality of $\mathbf{m}$ is, and the more atomic operations are required. In this case, the previous APE model may be not powerful enough to learn complicated editing behaviors. We propose a backup APE model via auto-regressive approach for the deteriorated translations. Concretely, we write the dual-source language model into its probabilistic formulation.

Notice that i) the encoder and memory encoder are still reused here, ii) the $\text{Dec}(\cdot;\cdot;\cdot)$ is a transformer decoder with hierarchical attention, since two memory blocks $\text{Enc}^{M}(\mathbf{m},\text{Enc}(\mathbf{s}))$ and $\text{Enc}(\mathbf{s})$ are both conditional variables for the auto-regressive language model; iii) unlike sequence tagging, the inference of the generative APE is intrinsically non-parallelizable.

Because of the lack of human post-editing data, training from scratch is typically difficult. We thus employ two workaround methods to improve the model performance.

Pre-training It is worth noting that the reduced atomic operation APE is actually equivalent to the mask language modeling problem, a.k.a. the famous BERT Devlin et al. (2019). Therefore, we pre-train the encoder-memory encoder model as a conditional BERT with the data pairs $(\mathbf{s},\mathbf{t})$ and $(\mathbf{m},\hat{\mathbf{e}})$, aiming at learning the syntactic and alignment information of the ground truth. To make the pre-training valid on downstream tasks, we consistently use [PLH] token to randomly mask the reference / post-editing sentence.

Imitation Learning As mentioned in 3.1, during inference, the predicted QE tags will causally tie to the successive APE algorithm, because $\tilde{\mathbf{m}}$ is derived from $(\mathbf{m},\hat{\mathbf{q}})$. Although we would want the model to learn to predict all three atomic operations together, the small size of real post-editing data severely limits the performance of joint QE tagging. Therefore, we propose a model specialization strategy where the model learns three separate tasks: deletion, insertion, and substitution. A reasonable amount of training data can be generated for each of the tasks and the model learns to specialize in each operation. The details are summarized in Algorithm 1.

In this section, we assemble all modules together into the final system. Because our model involves a nontrivial pipeline, we describe the details of training and inference separately and summarize them in Algorithm 2 and 3.

Training usually requires to minimize the loss function (negative data log-likelihood of probabilistic models) by stochastic gradient descent (SGD) with respect to the trainable parameters. Our QE and atomic operation APE are both sequence tagging task, while the generative APE is a sequence generation task. The three loss functions are uniformly defined as sequential cross entropy between the predicted and the true sequence. Note that the QE and atomic operation APE share the encoder-memory encoder, so these two losses can be summed together for optimization. However, the generative APE model has an isolated hierarchical transformer decoder, so we need a second update by optimizing the corresponding loss alone.

Inference of our APE system is not quite the same as the training. First, the overall inference is a continuously alternating procedure between QE and APE, where the predicted APE is assigned as a new machine translation for iterative updating. However, the inner loop in training algorithm regards to the augmented data points. Second, we introduce an early stop after the first QE tagging prediction. If the predicted quality is very low (i.e. the HTER is larger than a cross-validated threshold), the generative APE will be called and the inference will immediately exit without further iterations. Lastly, the APE results are utilized by professional translators for further editing. In the next section, we validate the gain of APE over machine translation with regards to the efficiency.

Dataset. The open public WMT17 Automatic Post-Editing Shared Task Bojar et al. (2017) data on English-German (En-De) is widely used for APE experiments. It consists of 23K real triples (source, machine translation & post-editing) for training and another 2K triples for testing from the Internet Technology (IT) domain. Besides, the shared task also provides a large-scale artificial synthetic corpus containing around 500K high quality and 4 million low quality synthetic triples. We over sample the APE real data by 20 times and merge it with the synthetic data, results in roughly 5 million of triples for both pre-training and APE training. The details of the training set are shown in Appendix Table 6. We adopt test set of the same task in WMT16 as the development set. Furthermore, we apply truecaser Koehn et al. (2007) to all files and encode every sentence into subword units Kudo (2018) with a 32K shared vocabulary.

Evaluation Metrics. We mainly evaluate our systems with metrics bilingual evaluation understudy (BLEU) Papineni et al. (2002) and translation edit rate (TER) Snover et al. (2006), since they are standard and widely employed in the APE shared task. The metric BLEU indicates how similar the candidate texts are to the reference texts, with values closer to 100 representing higher similarity. TER measures how many edits required from the predicted sentence to the ground truth sentence, and is calculated by Equation (3) as well and multiplied by 100.

Training Details. All experiments are trained on 8 NVIDIA P100 GPUs for maximum 100,000 steps for about two days until convergence, with a total batch-size of around 17,000 tokens per step and the Adam optimizer Kingma and Ba (2014). Only the source and post-edited sentence pairs are used for pre-training. During pre-training, 20% tokens in post-editing sentence are masked as [PLH]. Parameters are being tuned with 12,000 steps of learning rates warm-up Vaswani et al. (2017) for both of the GM and AOM model. However, 5 automatic post editing iterations (i.e. $S=5$ in Algorithm alg:infer) are applied during the inference for the AOM model due to its characteristic of fine-grained editing behaviors. Except these modifications, we follow the default transformer-based configuration Vaswani et al. (2017) for other hyper-parameters in our models.

The main results of automatic post-editing systems are presented in Table 3 and competitively compared with results of recent years’ winners of WMT APE shared task and several other top results. It is observed that our hierarchical single model achieves the state-of-the-art performance on both BLEU and TER metrics, outperforming not only all other single models but also the ensemble models of top ranked systems in WMT APE tasks.

Note that our hierarchical system is not a two-model ensemble. The standard ensemble method requires inference and combination of results from more than one models. In contrast, our hierarchical model contains multiple parameter-sharing modules to accomplish multi-tasks, and only need to infer once on the selected model.

We conduct real post-editing experiments with professional translators involved. There are 6 independent participating translators, randomly divided into 2 groups. They are all native speakers of German and have 10+ years of experience in translation of En-De in IT related domains. We follow two different flows in our experiments. For fair comparison, both of the two groups see the same 100 source sentences picked from the WMT17 test dataset. The MTs are provided for the first group for post-editing, while our model generated APEs for the second group. However, the information on the category of the translation is not revealed to translators. The translators are asked to record the elapsed time of their labor in total.

The statistics of averaged post-editing time for different translators are summarized in Figure 6. Besides the total time, we also analyze the duration for low and high quality translations separately (determined by QE model). In either case, post-editing from the APE costs less time. We also did case study about high-quality vs low-quality APE in Appendix A.3. From different perspectives of experimental validation, we can conclude that the APE generated by our model can ease the burden of translators and substantially improve the post-editing efficiency.