Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks

What it is about

The team's shared-task system uses one LSTM network, trained jointly to output UPOS tags, morphological features and lemmas. Lemmas are produced at character level, which extends a sequence-tagging network. Sentence segmentation, tokenization and dependency parsing come from the UDPipe 1.2 baseline. The model is validated on nine UD v2.2 treebanks, including English EWT and GUM. The authors report that the results are generally not better than the UDPipe baseline, and that the network was not tuned heavily. The work is a first step towards a network that outputs all UD labels.

Ideas

• I1. en/src/morph.rs: a fallback lemmatizer for forms that are missing from the dictionary. It uses suffix edit rules counted from data/lemmas.tsv and data/forms.tsv. Measure lemma accuracy on EWT test out-of-dictionary forms only, against the dictionary-only route.
• I2. en morph-eval: print POS and lemma errors together for each token, to see whether tag errors cause lemma errors. The paper trains the two jointly, so the question is open for us.
• I3. en/src/ptag.rs: add suffix-derived number, tense and person features to the tagger, and measure EWT test tagging accuracy with en tag-eval, twice with identical results before a claim.

For Mova

The code is en/src/morph.rs and en/src/ptag.rs; the chapters are languages/en/06-spelling-rules.md, 11-nouns.md and 13-verbs.md. The paper's lemma accuracy is on UD v2.2, not our UD 2.18, and our dictionary lemmatizer is already at 99.3 to 99.5%, so any gain is in the unknown forms.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/k18-2018 · arXiv 1809.03211

@inproceedings{arakelyan2018jointud,
  title = {{Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks}},
  author = {Arakelyan, Gor and Hambardzumyan, Karen and Khachatrian, Hrant},
  booktitle = {Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies},
  year = {2018},
  doi = {10.18653/v1/k18-2018},
  eprint = {1809.03211},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/k18-2018}
}

References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics 5:135–146. find
2. Dat Quoc Nguyen, Mark Dras, and Mark Johnson. 2017. A novel neural network model for joint pos tagging and graph-based dependency parsing. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics, Vancouver,… find
3. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pages 740–750. find our card
4. Jason P.C. Chiu and Eric Nichols. 2016. Named Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin-entity recognition with bidirectional lstm-ter, Yoav Goldberg, Jan Hajič, Christopher Man-cnns. Transactions of the Association ning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, for Computational Lingu… find our card
5. European Language Re-Schwenk, and Yoshua Bengio. 2014. Learning sources Association, Portoro, Slovenia, pages 1659– phrase representations using rnn encoder–decoder 1666. for statistical machine translation. In Proceedings of Joakim Nivre et al. 2018. Universal Dependencies 2.2. LINDAT/CLARIN digita… find
6. Slav Petrov. 2018b. CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics, Brussels, Belgium, pages 1–20. find our card
7. Martin Potthast, Tim Gollub, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, and Benno Stein. 2014. Improving the reproducibility of PAN’s shared tasks: Plagiarism detection, author identification, and author profiling. In Evangelos Kanoulas, Mihai Lupu, Paul Clough, Mark Sander-son, Mark Hall… — [doi:10.1007/978-3-319-11382-1](https://doi.org/10.1007/978-3-319-11382-1) find
8. Nils Reimers and Iryna Gurevych. 2017. Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). Copenhagen, Denmark, pages 338–348. http://aclweb.org/a… ↗ our card
9. Milan Straka, Jan Hajič, and Jana Straková. 2016. UD-Pipe: trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016). European Langua… find
10. Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 2670–2680. find
11. Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In Advances in neural information processing systems. pages 2440–2448. find
12. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. pages 5998–6008. find
13. Dan Zeman et al. 2018a. Universal Dependencies 2.2 CoNLL 2018 shared task development and test data. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics, Charles University, Prague, http://hdl.handle.net/ 11234/1-2184. http://hdl.handle.net/11234/1-2184. ↗
14. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and find