A Comparison of Sentence-Weighting Techniques for NMT

Simon Riess, Matthias Huck, Alexander Fraser

MT Summit 2021

Sentence weighting is a simple and powerful domain adaptation
technique. We carry out domain classification for computing sentence
weights with 1) language model cross entropy difference 2) a
convolutional neural network 3) a Recursive Neural Tensor Network. We
compare these approaches with regard to domain classification
accuracy, and study the posterior probability distributions. Then we
carry out NMT experiments in the scenario where we have no in-domain
parallel corpora, and only very limited in-domain monolingual
corpora. Here, we use the domain classifier to reweight the sentences
of our out-of-domain training corpus. This leads to improvements of up
to 2.1 BLEU for German to English translation.