Joint Lemmatization and Morphological Tagging with LEMMING

Thomas Muller
Ryan Cotterell
Alexander Fraser
Hinrich Schutze

EMNLP 2015

We present LEMMING, a modular log-linear model that jointly models
lemmatization and tagging and supports the integration of arbitrary
global features. It is trainable on corpora annotated with gold
standard tags and lemmata and does not rely on morphological
dictionaries or analyzers. LEMMING sets the new state of the art in
token-based statistical lemmatization on six languages; e.g., for
Czech lemmatization, we reduce the error by 60%, from 4.05 to 1.58. We
also give empirical evidence that jointly modeling morphological tags
and lemmata is mutually beneficial.