Anchor-based Bilingual Word Embeddings for Low-Resource Languages

Tobias Eder, Viktor Hangya, Alexander Fraser

ACL 2021

Good quality monolingual word embeddings (MWEs) can be built for
languages which have large amounts of unlabeled text. MWEs can be
aligned to bilingual spaces using only a few thousand word translation
pairs. For low resource languages training MWEs monolingually
results in MWEs of poor quality, and thus poor bilingual word
embeddings (BWEs) as well. This paper proposes a new approach for
building BWEs in which the vector space of the high resource source
language is used as a starting point for training an embedding space
for the low resource target language. By using the source vectors as
anchors the vector spaces are automatically aligned during
training. We experiment on English-German, English-Hiligaynon and
English-Macedonian. We show that our approach results not only in
improved BWEs and bilingual lexicon induction performance, but also
in improved target language MWE quality as measured using
monolingual word similarity.