Elamur Research⌕
☰
home / papers / 2020 / Quantitative Linguistic Investigations across Universal Dependencies Treebanks

Quantitative Linguistic Investigations across Universal Dependencies Treebanks

Summary

The authors propose a method for quantitatively comparing how syntactic dependencies are distributed in the UD treebanks of typologically different languages. The aim is to separate genuine language-specific properties from differences in how different teams apply the shared annotation scheme. Four languages are studied: Bulgarian, English, Italian and Spanish. For each, a large corpus of news and Wikipedia texts is automatically annotated (with UDPipe). A statistical model of the language is built from it. The delexicalized algorithm LISCA then assesses how typical the context of each dependency in the manually annotated UD v2.2 treebank is. The sorted scores are divided into 20 bins, and the patterns are examined in the CLaRK system. The languages share common tendencies: relations of function words and leaf nodes tend toward the top of the ranking, while "free" and clausal relations end up at the bottom. The differences are explained either by typology (pro-drop, the postpositive article in Bulgarian) or by inconsistent annotation, as in the case of iobj or fixed. This kind of analysis is useful for typologically informed multilingual NLP and for improving treebank consistency.

Key points

  • The UD v2.2 treebanks used are the English Web Treebank (≈255 thousand tokens), the Italian Stanford Dependency Treebank (≈278 thousand), Spanish UD (≈548 thousand) and UD_Bulgarian-BTB (≈156 thousand). For each language there is also a reference corpus of about 40 million tokens, parsed with UDPipe.
  • The 29 dependency types shared by all languages were compared by their median bin. The position in the ranking hardly depends on frequency: punct makes up about 11% of all dependencies, yet it ends up in the middle in all languages. In all languages, xcomp ranks above ccomp, obj above obl, and nsubj above csubj.
  • Among the dependencies in the first 10 bins, the overwhelming share consists of leaf nodes: 91.52% (BUL), 95.56% (ENG), 98.27% (ITA) and 91.76% (SPA). In the last 10 bins, by contrast, about 64–69% are dependencies without leaves.
  • The annotation of iobj differs greatly across languages. The share of pronouns in the iobj role is 100% in Italian, 84% in English, 46.7% in Spanish and only 19% in Bulgarian, where this relation is also used to label nouns in prepositional phrases.
  • Typological differences are also visible in the ranking. In English, subjects (nsubj) rank considerably higher than in the three pro-drop languages. In Bulgarian, definiteness is usually expressed by the postpositive article, so a separate det occurs less often.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Universal Dependencies · Treebanks

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alzetta2020quantitative,
  title = {{Quantitative Linguistic Investigations across Universal Dependencies Treebanks}},
  author = {Alzetta, Chiara and Dell'Orletta, Felice and Montemagni, Simonetta and Osenova, Petya and Simov, Kiril and Venturi, Giulia},
  booktitle = {Proceedings of the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020)},
  year = {2020},
  doi = {10.4000/books.aaccademia.8210},
  url = {https://doi.org/10.4000/books.aaccademia.8210}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. F. Dell’Orletta, G. Venturi, and S. Montemagni. 2013. Linguistically-driven selection of correct arcs for dependency parsing. Computaciòn y Sistemas, 2:125– 136. find
  2. H. M. Alonso and D. Zeman. 2016. Universal dependencies for the ancora treebanks. Procesamiento del Lenguaje Natural, 57:91–98. find
  3. M. Dickinson. 2015. Detection of Annotation Errors in Corpora. Language and Linguistics Compass, 9(3):119–138. find
  4. C. Alzetta, F. Dell’Orletta, S. Montemagni, and G. Venturi. 2018a. Dangerous relations in dependency treebanks. In Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories (TLT16), pages 201–210, Prague, Czech Republic, January. find
  5. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  6. C. Alzetta, F. Dell’Orletta, S. Montemagni, and G. Venturi. 2018b. Universal dependencies and quantitative typological trends. a case study on word order. In Proceedings of the 11th Edition of International Conference on Language Resources and Evaluation (LREC 2018), pages 4540–4549. Association for… find our card
  7. C. Alzetta, F. Dell’Orletta, S. Montemagni, and G. Venturi. 2020. Uncovering typological context-sensitive features. In Proceedings of the Second Workshop on Typology for Polyglot Natural Language Processing. find
  8. Johannes Bjerva and Isabelle Augenstein. 2018. From phonology to syntax: Unsupervised linguistic typology at different levels with language embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V… find
  9. Johannes Bjerva, Yova Kementchedjhieva, Ryan Cotterell, and Isabelle Augenstein. 2019. A probabilistic generative model of linguistic typology. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1… find
  10. C. Bosco, S. Montemagni, and M. Simi. 2013. Converting italian treebanks: Towards an italian stanford dependency treebank. In Proceedings of the ACL Linguistic Annotation Workshop & Interoperability with Discourse, Sofia, Bulgaria, August. find
  11. Ryan Cotterell and Jason Eisner. 2017. Probabilistic typology: Deep generative models of vowel inventories. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1182–1192. find
  12. M.C. de Marneffe, M. Grioni, J. Kanerva, and F. Ginter. 2017. Assessing the Annotation Consistency of the Universal Dependencies Corpora. In Proceedings of the 4th International Conference on Dependency Linguistics (Depling 2007), pages 108–115, Pisa, Italy, September. find our card
  13. Roman Jakobson. 1973. Essais de linguistique générale t. 2: rapports internes et externes du langage. Les éditions de Minuit. find
  14. J. Jiang and H. Liu. 2018. Quantitative Analysis of Dependency Structures. De Gruyter Mouton, Berlin, Boston. find
  15. Yugo Murawaki. 2017. Diachrony-aware induction of binary latent representations from typological features. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 451–461. find
  16. J. Nivre, A. Željko, A. Lars, and et alii. 2017. Universal dependencies 2.0. In LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics, Charles University. find
  17. J. Nivre. 2015. Towards a universal grammar for natural language processing. In Computational Linguistics and Intelligent Text Processing - Proceedings of the 16th International Conference, CICLing 2015, Part I, pages 3–16, Cairo, Egypt, April. find
  18. E.M. Ponti, H. O’Horan, Y. Berzak, I. Vulić, R. Reichart, T. Poibeau, E. Shutova, and A. Korhonen. 2018. Modeling language variation and universals: A survey on typological linguistics for natural language processing. arXiv preprint arXiv:1807.00914. find
  19. N. Silveira, T. Dozat, M.C. de Marneffe, S. Bowman, M. Connor, J. Bauer, and C.D. Manning. 2014. A gold standard dependency corpus for english. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC). find
  20. K. Simov, A. Simov, H. Ganev, K. Ivanova, and I. Grig-orov. 2004. The CLaRK System: XML-based Corpora Development System for Rapid Prototyping. Proceedings of LREC 2004, pages 235–238. find
  21. K. Simov, P. Osenova, A. Simov, and M. Kouylekov. 2005. Design and Implementation of the Bulgarian HPSG-based Treebank. Journal of Research on Language and Computation. Special Issue, pages 495–522. find
  22. M. Straka, J. Hajic, and J. Strakova. 2016. UD-Pipe: Trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, pos tagging and parsing. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC). find
  23. E. Tusa, F. Dell’Orletta, S. Montemagni, and G. Venturi. 2016. Dieci sfumature di marcatezza sintattica: verso una nozione computazionale di comp-lessitá. In Proceedings of the Third Italian Conference on Computational Linguistics (CLiC-it), pages 3–16, Napoli, Italy, December. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/alzetta-2020-quantitative-linguistic-investigations-across-universal/