Cross-lingual and Supervised Models for Morphosyntactic Annotation: a Comparison on Romanian
Summary
Romanian is a Romance language, but few annotated corpora exist for it. As a result, standard supervised PoS taggers and dependency parsers perform considerably worse on it than on French or English. The authors test whether cross-lingual transfer can help in this setting. The work is valuable because the test is carried out on a genuinely low-resource language, whereas such methods are usually evaluated on well-resourced ones. For tagging, the approach of Täckström et al. (2013) is reproduced. Tags are projected onto Romanian Europarl sentences via alignment with English, French, Italian and Spanish, and are then filtered with dictionary constraints from Wiktionary. For parsing, the delexicalized multi-source transfer of McDonald et al. (2011) is used. Both approaches are compared with supervised models trained on subsets of the RSAC treebank of various sizes. The finding is that even a small amount of manual annotation in the target language is more useful than large parallel corpora. For parsing, about 15 annotated sentences are enough. The authors therefore recommend hybrid approaches that combine transfer with a small amount of annotation. This applies to any low-resource language.
Key points
- Data. RSAC (Perez, 2012) contains 67,686 tokens and 3,587 sentences and is split into train and test sets. The Romanian part of UD 1.2 contained only 633 sentences. For transfer, 300,000 Romanian Europarl sentences aligned with en, fr, it and es were used.
- Dictionary. 405,125 word forms were collected from the English and Romanian Wiktionary and mapped to universal PoS tags. The dictionary covers 69% of RSAC types.
- Tagging accuracy. The transfer tagger reaches 79.1–79.6. Adding 15 manual rules for clitics and function words raises accuracy to 81.8–82.7; the best source languages are fr and es (82.7). The supervised tagger reaches 88.8. English as a source language is almost as good as the Romance languages.
- Comparison with manual annotation. The best transfer tagger performs on par with a supervised tagger trained on only 363 sentences. For parsing, transfer is outperformed by a model trained on about 15 annotated Romanian sentences.
- Typical errors. Errors are most frequent on adverbs, the class with the largest share of Slavic loanwords (about 20%). The ambiguous "a", which can be DET, VERB or PRT, is also difficult. A large share of the errors comes from differences in tokenization and annotation schemes across corpora.
Cite
@inproceedings{aufrant2016cross,
title = {{Cross-lingual and Supervised Models for Morphosyntactic Annotation: a Comparison on Romanian}},
author = {Aufrant, Lauriane and Wisniewski, Guillaume and Yvon, François},
booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
year = {2016},
doi = {10.63317/5bd4x3h3dqjx},
url = {https://doi.org/10.63317/5bd4x3h3dqjx}
}References
- Calacean, M. and Nivre, J. (2009). A data-driven dependency parser for romanian. In Proceedings of the 7th International Workshop on Treebanks and Linguistic Theories (TLT 7), pages 65–76. find
- Chrupala, G., Dinu, G., and van Genabith, J. (2008). Learning morphology with morfette. In LREC. find our card
- Colhon, M. and Simionescu, R. (2012). Deriving a statistical syntactic parsing from a treebank. In Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics, page 34. find
- Goldberg, Y., Zhao, K., and Huang, L. (2013). Efficient implementation of beam-search incremental parsers. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 628–633. find
- Haspelmath, M. and Tadmor, U., (2009). Loanwords in the World’s Languages: A Comparative Handbook, chapter 8. find
- Walter de Gruyter. Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In MT Summit. find
- Lacroix, O., Aufrant, L., Wisniewski, G., and Yvon, F. (2016). Frustratingly easy cross-lingual transfer for transition-based dependency parsing. In NAACL. find
- McDonald, R., Petrov, S., and Hall, K. (2011). Multi-source transfer of delexicalized dependency parsers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 62–72. McDonald, R., Nivre, J., Quirmbach-Brundage, Y., Goldberg, Y., Das, D., Ganchev, K., Hall, K.,… find our card
- Pan, S. J. and Yang, Q. (2010). A survey on transfer learning. Knowledge and Data Engineering, IEEE Transactions on, 22(10):1345–1359. find
- Perez, C.-A. (2012). Linguistic resources for the processing of the romanian language. In University Al. I. Cuza of Iasi. find
- Petrov, S., Das, D., and McDonald, R. (2012). A universal part-of-speech tagset. In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Eight International Conference on Language Resources and Evaluation, LREC’12, Istanbul, Turkey, may. European Language Resources Association… find
- Roegiest, E. (2006). Vers les sources des langues romanes: un itinéraire linguistique à travers la Romania. Acco. Straka, M., Hajič, J., Straková, J., and Hajič Jr, J. (2015). Parsing universal dependency treebanks using neural networks and search-based oracle. In International Workshop on Tree… find
- Zeman, D. and Resnik, P. (2008). Cross-language parser adaptation between related languages. In IJCNLP, pages 35–42. find
- Zhang, Y. and Nivre, J. (2011). Transition-based dependency parsing with rich non-local features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 188–193. find
This page in Markdown → https://md.research.elamur.ai/free/en/papers/aufrant-2016-cross-lingual-supervised-models-morphosyntactic/