Elamur Research⌕
☰
home / papers / 2015 / Joint Lemmatization and Morphological Tagging with Lemming

Joint Lemmatization and Morphological Tagging with Lemming

Paper · 2015 · Thomas Müller, Ryan Cotterell, Alexander Fraser et al.

Summary

Lemmatizing a token and predicting its POS and morphological attributes depend on each other. Tags often resolve which lemma is correct. In the other direction, knowing the lemma helps resolve syncretic forms, such as gender on German plural nouns. Most systems still run these steps as a pipeline, and earlier lemmatizers could not use global features of the candidate lemma.

The authors present LEMMING, a log-linear lemmatizer. It builds candidate lemmata by applying edit trees extracted from training pairs. It then scores each candidate with several feature groups:

  • edit-tree features
  • character-alignment features
  • lemma affix features
  • dictionary features (Wikipedia frequency, ASPELL)

All features are conjoined with POS and morphological attributes. In the joint version, LEMMING is combined with the MarMoT higher-order CRF tagger in a tree-structured CRF. The model is trained with SGD, uses belief propagation for inference, and is initialized from a pretrained tagger. It needs only corpora annotated with gold tags and lemmata, with no morphological analyzer.

Across six languages, LEMMING beats the MORFETTE and JCK baselines, with the biggest gains on unknown words. The results show that joint modeling helps both tagging and lemmatization.

Key points

  • Evaluated on English, German, Czech, Hungarian, Latin and Spanish. For Czech, lemmatization error falls by about 60% (4.05 → 1.58).
  • Edit-tree candidate generation yields between 7 (Hungarian) and 91 (Czech) candidates per form on average. The correct lemma is among the candidates over 99.4% of the time on dev, except Latin (98.4%).
  • The joint model (LEMMING-J) significantly improves tag+lemma accuracy for four of the six languages. For Czech, it cuts lemma errors by more than 37% and tag+lemma errors by more than 6%.
  • On unknown forms, the best models beat the best baseline by more than 12 points for Czech and Latin, and by more than 5 points for German, Spanish and Hungarian.
  • The software is released as open source at cistern.cis.lmu.de/lemming.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Johns Hopkins UniversityBaltimore · United States187 works · 92,194 citations🎓Ludwig-Maximilians-Universität MünchenMunich · Germany44 works · 55,193 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 277 works

Cite

Cite as Thomas Müller, Ryan Cotterell, Alexander Fraser and Hinrich Schütze. 2015. Joint Lemmatization and Morphological Tagging with Lemming. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d15-1272

@inproceedings{muller2015joint,
  title = {{Joint Lemmatization and Morphological Tagging with Lemming}},
  author = {Thomas Müller, Ryan Cotterell, Alexander Fraser and Hinrich Schütze},
  booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1272},
  url = {https://doi.org/10.18653/v1/d15-1272}
}

References

31 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anders Björkelund, Bernd Bohnet, Love Hafdell, and Pierre Nugues. 2010. A high-performance syntactic and semantic dependency parser. In Proceedings of COLING: Demonstrations. find
  2. Grzegorz Chrupała, Georgiana Dinu, and Josef Van Genabith. 2008. Learning morphology with Morfette. In Proceedings of LREC.
  3. Grzegorz Chrupała. 2006. Simple data-driven con-textsensitive lemmatization. Procesamiento del Lenguaje Natural. find
  4. Grzegorz Chrupała. 2008. Towards a machine-learning architecture for lexical functional grammar parsing. Ph.D. thesis, Dublin City University. find
  5. Michael Collins. 2002. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of EMNLP.
  6. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2014. Stochastic contextual edit distance and probabilistic FSTs. In Proceedings of ACL. find
  7. Markus Dreyer, Jason R Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of EMNLP. find
  8. Ondřej Dušek and Filip Jurčı́ček. 2013. Robust multilingual statistical morphological generation models. In Proceedings of ACL: Student Research Workshop. find
  9. Alexander Fraser, Marion Weller, Aoife Cahill, and Fabienne Cap. 2012. Modeling inflection and word-formation in SMT. In Proceedings of EACL. find
  10. Andrea Gesmundo and Tanja Samardzic. 2012. Lemmatisation as a tagging task. In Proceedings of ACL.
  11. Dan Gusfield. 1997. Algorithms on Strings, Trees, and Sequences - Computer Science and Computational Biology. Cambridge University Press. find
  12. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, et al. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of CoNLL.
  13. Dag TT Haug and Marius Jøhndal. 2008. Creating a parallel treebank of the old Indo-European bible translations. In Proceedings of LaTeCH. find
  14. Sittichai Jiampojamarn, Colin Cherry, and Grzegorz Kondrak. 2008. Joint processing and discriminative training for letter-to-phoneme conversion. In Proceedings of ACL. find
  15. Dong C Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming. find
  16. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of ACL: Demonstrations.
  17. Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn treebank. Computational linguistics.
  18. Andrew K McCallum. 2002. MALLET: A machine learning for language toolkit. find
  19. Thomas Müller and Hinrich Schütze. 2015. Robust morphological tagging with word representations. In Proceedings of NAACL. find
  20. Thomas Müller, Helmut Schmid, and Hinrich Schütze. 2013. Efficient higher-order CRFs for morphological tagging. In Proceedings of EMNLP. find
  21. Lluı́s Padró and Evgeny Stanilovsky. 2012. FreeLing 3.0: Towards wider multilinguality. In Proceedings of LREC.
  22. Judea Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann. find
  23. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 shared task on parsing the web. In Proceedings of SANCL. find
  24. Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Proceedings of EMNLP.
  25. Djamé Seddah, Grzegorz Chrupała, Özlem Çetinoğlu, Josef Van Genabith, and Marie Candito. 2010. Lemmatization and lexicalized statistical parsing of morphologically rich languages: the case of French. In Proceedings of SPMRL. find
  26. Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho D. Choi, Richárd Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola Gal-letebeitia, Yoav Goldberg, Spence Green, Nizar Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiórkowski, Ryan Roth, Wolfgang Seeker, Yanni… find
  27. Noah A. Smith, David A. Smith, and Roy W. Tromble. 2005. Context-based morphological disambiguation with random fields. In Proceedings of HLT-EMNLP. find
  28. Kristina Toutanova and Colin Cherry. 2009. A global model for joint lemmatization and part-of-speech prediction. In Proceedings of ACL-IJCNLP.
  29. Yoshimasa Tsuruoka, Jun’ichi Tsujii, and Sophia Ananiadou. 2009. Stochastic gradient descent training for l1-regularized log-linear models with cumulative penalty. In Proceedings of ACL-IJCNLP. find
  30. Richard Wicentowski. 2002. Modeling and learning multilingual inflectional morphology in a minimally supervised framework. Ph.D. thesis, Johns Hopkins University. find
  31. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of COLING. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/muller-2015-joint-lemmatization-morphological-tagging-lemming/