Elamur Research⌕
☰
home / papers / 2020 / A Universal Dependencies Conversion Pipeline for a Penn-format Constituency Treebank

A Universal Dependencies Conversion Pipeline for a Penn-format Constituency Treebank

Summary

The authors aimed to automatically derive a UD-format corpus from an existing constituency treebank. This treebank is annotated according to the PPCHE scheme, which descends from the Penn Treebank. Off-the-shelf converters such as PyStanfordDependencies or the LTH tool are not suitable here, because the PPCHE scheme differs too much from the classic Penn format. The team therefore built its own rule-based pipeline on top of NLTK. It works in three stages. First, the input files are cleaned; among other things, separated suffixed articles are reattached to their nouns. Next, the trees are converted to CoNLL-U: heads and dependency types are identified. Finally, post-processing marks clitics with ID ranges and joins clauses into full sentences. Part-of-speech tags and morphological features are mapped by hand-written rules. The tool was applied to the Icelandic historical corpus IcePaHC and, after minor adjustments, to the Faroese FarPaHC as well. This shows that the approach transfers to other languages annotated with the same scheme. The outcome is an open-source converter and two new UD corpora. There is no formal quality evaluation yet; sentences were only checked manually. The value of the work is that rich constituency resources become available in a standard cross-lingual format.

Key points

  • IcePaHC contains about 1 million words of text from the 12th to the 21st century. A 40,000-word Faroese UD corpus was derived from FarPaHC, but without lemmas, since the original has none.
  • The output is in CoNLL-U format with the set of 17 UPOS tags. For the Icelandic corpus, the MISC column contains the IFD tag assigned by ABLTagger.
  • The converter is distributed under the Apache 2.0 license on GitHub (thorunna/UDConverter), and the corpora under CC BY-SA 4.0. Both corpora were to be included in the UD 2.7 release.
  • The authors plan to convert another 75,000 words of modern texts: parliamentary speeches and sports news from the Icelandic Gigaword Corpus. The Icelandic corpus would then grow to about 1.1 million words.
  • Possible applications include other corpora with a scheme similar to PPCHE (Ancient Greek, historical Portuguese, Middle Low German), as well as the output of the IceNeuralParsingPipeline parser.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Universal Dependencies · Treebanks

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{arnardottir2020universal,
  title = {{A Universal Dependencies Conversion Pipeline for a Penn-format Constituency Treebank}},
  author = {Arnardóttir, THórunn and Hafsteinsson, Hinrik and Sigurdhsson, Einar Freyr and Bjarnadóttir, Kristín and Ingason, Anton Karl and Jónsdóttir, Hildur and Steingrímsson, Steinthór},
  booktitle = {Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020)},
  year = {2020},
  doi = {10.48448/aj10-3376},
  url = {https://doi.org/10.48448/aj10-3376}
}

References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Þórunn Arnardóttir and Anton Karl Ingason. 2020. A neural parsing pipeline for Icelandic using the Berkeley neural parser. In Costanza Navarretta and Maria Eskevich, editors, Proceedings of CLARIN 2020, pages 48–51. find
  2. Jana E. Beck. 2011. Penn Parsed Corpora of Historical Greek (PPCHiG). http://ling.upenn.edu/~janabeck/greek-corpora.html. ↗
  3. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O’Reilly Media, Sebastopol, CA. find
  4. Hannah Booth, Anne Breitbarth, Aaron Ecay, and Melissa Farasyn. 2020. A Penn-style treebank of Middle Low German. In Proceedings of The 12th Language Resources and Evaluation Conference (LREC 2020), pages 766–775, Marseille, France. European Language Resources Association. find
  5. Charlotte Galves. 2018. The Tycho Brahe Corpus of Historical Portuguese: Methodology and results. Linguistic Variation, 18(1):49–73. find
  6. Anton Karl Ingason, Sigrún Helgadóttir, Hrafn Loftsson, and Eiríkur Rögnvaldsson. 2008. A Mixed Method Lemmatization Algorithm Using a Hierarchy of Linguistic Identities (HOLI). In Proceedings of Sixth International Conference on Natural Language Processing, GoTAL 2008, pages 205–216, Gothenburg, Sw… find
  7. Anton Karl Ingason, Eiríkur Rögnvaldsson, Einar Freyr Sigurðsson, and Joel C. Wallenberg. 2012. Faroese Parsed Historical Corpus (FarPaHC). Version 0.1. find
  8. Anton Karl Ingason, Hrafn Loftsson, Eiríkur Rögnvaldsson, Einar Freyr Sigurðsson, and Joel Wallenberg. 2014. Rapid deployment of phrase structure parsing for related languages: A case study of Insular Scandinavian. In Proceedings of the Ninth International Conference on Language Resources and Evalua… find
  9. Richard Johansson and Pierre Nugues. 2007. Extended constituent-to-dependency conversion for English. In Proceedings of the 16th Nordic Conference of Computational Linguistics (NODALIDA 2007), pages 105–112, Tartu, Estonia. University of Tartu, Estonia. find our card
  10. Hildur Jónsdóttir and Anton Karl Ingason. 2020. Creating a parallel Icelandic dependency treebank from raw text to Universal Dependencies. In Proceedings of The 12th Language Resources and Evaluation Conference (LREC 2020), pages 2924–2931, Marseille, France. European Language Resources Association. find
  11. Jenna Kanerva, Filip Ginter, Niko Miekka, Akseli Leino, and Tapio Salakoski. 2018. Turku neural parser pipeline: An end-to-end system for the CoNLL 2018 shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 133–142, Brussels, B… find
  12. Anthony S. Kroch and Ann Taylor. 2000. Penn-Helsinki Parsed Corpus of Middle English. CD-ROM. Second edition. Size: 1.3 million words. find
  13. Anthony S. Kroch, Beatrice Santorini, and Lauren Delfs. 2004. Penn-Helsinki Parsed Corpus of Early Modern English. CD-ROM. First Edition. Size: 1.8 million words. find
  14. Hrafn Loftsson and Eiríkur Rögnvaldsson. 2007. IceNLP: A Natural Language Processing Toolkit for Icelandic. In Proceedings of Interspeech – Speech and language technology for less-resourced languages, Interspeech 2007, Antwerp, Belgium. find
  15. David Mitchell Magerman. 1994. Natural Language Parsing As Statistical Pattern Recognition. Ph.D. thesis, Stanford University. find
  16. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth In… find our card
  17. Jörgen Pind, Friðrik Magnússon, and Stefán Briem. 1991. Íslensk orðtíðnibók [The Icelandic Frequency Dictionary]. The Institute of Lexicography, University of Iceland, Reykjavík. find
  18. Eiríkur Rögnvaldsson and Sigrún Helgadóttir. 2011. Morphological tagging of Old Icelandic texts and its use in studying syntactic variation and change. In Caroline Sporleder, Antal van den Bosch, and Kalliopi Zervanou, editors, Language Technology for Cultural Heritage: Selected Papers from the LaTe… find
  19. Eiríkur Rögnvaldsson, Anton Karl Ingason, Einar Freyr Sigurðsson, and Joel Wallenberg. 2011. Creating a dual-purpose treebank. JLCL, 26(2):139–150. find
  20. Eiríkur Rögnvaldsson, Anton Karl Ingason, Einar Freyr Sigurðsson, and Joel Wallenberg. 2012. The Icelandic Parsed Historical Corpus (IcePaHC). In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), pages 1977–1984, Istanbul, Turkey. European Language R… find our card
  21. Eiríkur Rögnvaldsson. 2005. Setningafræðilegar breytingar í íslensku. In Höskuldur Thráinsson, editor, Setningar. Handbók um setningafræði. Íslensk tunga III, pages 602–635. Almenna bókafélagið, Reykjavík. find
  22. Rudolf Rosa and David Mareček. 2018. CUNI x-ling: Parsing under-resourced languages in CoNLL 2018 UD shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 187–196, Brussels, Belgium. Association for Computational Linguistics. find
  23. Kristján Rúnarsson and Einar Freyr Sigurðsson. 2020. Parsing Icelandic Alþingi transcripts: Parliamentary speeches as a genre. In Proceedings of the Second ParlaCLARIN Workshop, pages 44–50, Marseille, France. European Language Resources Association. find
  24. Steinþór Steingrímsson, Sigrún Helgadóttir, Eiríkur Rögnvaldsson, Starkaður Barkarson, and Jón Guðnason. 2018. Risamálheild: A Very Large Icelandic Text Corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Languag… find
  25. Steinþór Steingrímsson, Örvar Kárason, and Hrafn Loftsson. 2019. Augmenting a BiLSTM tagger with a morphological lexicon and a lexical category identification step. In Proceedings of RANLP 2019, Varna, Bulgaria. find
  26. Francis M. Tyers, Mariya Sheyanova, Alexandra Martynova, Pavel Stepachev, and Konstantin Vinogradovsky. 2018. Multi-source synthetic treebank creation for improved cross-lingual dependency parsing. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 144–150. find
  27. Joel C. Wallenberg, Anton Karl Ingason, Einar Freyr Sigurðsson, and Eiríkur Rögnvaldsson. 2011. Icelandic Parsed Historical Corpus (IcePaHC). Version 0.9. http://www.linguist.is/icelandic_treebank. ↗
  28. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical Dependency Analysis with Support Vector Machines. In The 8th International Workshop of Parsing Technologies (IWPT2003), pages 195–206, Nancy, France. find our card
  29. Eray Yildiz and A. Cüneyd Tantuğ. 2019. Morpheus: A neural network for jointly learning contextual lemmatization and morphological tagging. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 25–34, Florence, Italy, August. Association for Co… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/arnardottir-2020-universal-dependencies-conversion-pipeline-penn/