Elamur Research⌕
☰
home / papers / 2013 / Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages

Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages

Summary

The paper presents the inaugural shared task devoted to statistically parsing morphologically rich languages (MRLs), organized alongside the 4th SPMRL workshop at EMNLP 2013. Earlier cross-lingual parsing campaigns, such as the CoNLL dependency tasks, used mismatched data sizes and assumed morphology was already known. That made it hard to see why parsers struggle on languages where grammatical relations are encoded inside words.

The organizers built treebanks for nine typologically diverse languages, each in both constituency and dependency form, in two sizes: full training data and a 5,000-sentence subset. They defined three input scenarios:

  • gold morphology,
  • automatically predicted morphology,
  • raw input supplied as a lattice of possible analyses.

They also proposed evaluation that works across frameworks and scenarios.

Seven teams entered more than 14 systems. The strongest were mostly ensembles or pipelines built on a strong morphological analyzer. Ambiguous morphology remained hard for every system. The work matters because it created a stable benchmark and baselines for realistic MRL parsing.

Key points

  • Data: nine languages, all in both constituency and dependency annotation. Resources included the Arabic Penn Treebank (still distributed via LDC), the Hebrew treebank with its morphological analyzer, the French treebank and the Kaist Korean Treebank.
  • Training regimes: a full setup and a small 5,000-sentence setup. Some cross-framework comparisons use unlabeled TedEval on generalized trees, with training on 5k sentences and testing on 5k terminals.
  • Input scenarios: gold, predicted and raw (lattice). This is the first campaign to report parser results on realistic, morphologically ambiguous input.
  • Overall findings: more data helps, and gold morphology gives clearly higher scores than predicted input. The gap between dependency and constituency parsing is smaller than often assumed; the learning model and label granularity matter more.
  • Hardest languages: Semitic and Asian languages remain the hardest to parse whatever the method. Whether this comes from the language, the models or treebank consistency stays open.

Interesting ideas

  • I1. Lattice input for joint disambiguation. The parser receives a lattice of candidate morphological analyses and chooses segmentation together with syntax, as ALPAGE:DYALOG did. This could help any language where tokens hide several syntactic units, such as clitics or compounds, and avoids passing early tagging errors down the pipeline.
  • I2. Gold vs. predicted vs. raw scenarios. Reporting all three separates a lab-condition upper bound from real pipeline performance. This makes clear how much of the error comes from upstream morphology.
  • I3. Fixed small-data track. A training subset of the same size across languages makes treebanks of very different sizes comparable. It also shows how parsers behave under resource scarcity, which is relevant for low-resource treebanks.
  • I4. Cross-framework evaluation on generalized trees. Scoring with a metric like TedEval, which works on tree structures regardless of formalism, lets constituency and dependency outputs be compared fairly. This is useful whenever competing systems produce different output representations.
  • I5. Combining parsers as a robust baseline. Ensembles of different parsers or training runs gave stable results across languages. They are a sensible default before investing in language-specific engineering.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Topics: Treebanks

Cited by

116 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 100 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{seddah2013overview,
  title = {{Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages}},
  author = {Seddah, Djamé and Tsarfaty, Reut and Kübler, Sandra and Candito, Marie and Choi, Jinho D. and Farkas, Richárd and Foster, Jennifer and Goenaga, Iakes and Galletebeitia, Koldo Gojenola and Goldberg, Yoav and Green, Spence and Habash, Nizar and Kuhlmann, Marco and Maier, Wolfgang and Nivre, Joakim and Przepiórkowski, Adam and Roth, Ryan and Seeker, Wolfgang and Versley, Yannick and Vincze, Veronika and Woliński, Marcin and Wróblewska, Alina and Clergerie, Eric Villemonte de la},
  booktitle = {Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages},
  year = {2013},
  doi = {10.18653/v1/w13-4917},
  url = {https://doi.org/10.18653/v1/w13-4917}
}

References

141 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne Abeillé, Lionel Clément, and François Toussenel. 2003. Building a treebank for French. In Anne Abeillé, editor, Treebanks. Kluwer, Dordrecht. find
  2. Szymon Acedański. 2010. A Morphosyntactic Brill Tagger for Inflectional Languages. In Advances in Natural Language Processing, volume 6233 of Lecture Notes in Computer Science, pages 3–14. Springer-Verlag. find
  3. Meni Adler and Michael Elhadad. 2006. An unsupervised morpheme-based HMM for Hebrew morphological disambiguation. In Proceedings COLING-ACL, pages 665–672, Sydney, Australia. find our card
  4. Meni Adler, Yoav Goldberg, David Gabay, and Michael Elhadad. 2008. Unsupervised lexicon-based resolution of unknown words for full morphological analysis. In Proceedings of ACL-08: HLT, pages 728–736, Columbus, OH. find
  5. Meni Adler. 2007. Hebrew Morphological Disambiguation: An Unsupervised Stochastic Word-based Approach. Ph.D. thesis, Ben-Gurion University of the Negev. find
  6. Itziar Aduriz, José María Arriola, Xabier Artola, A Díaz de Ilarraza, et al. 1997. Morphosyntactic disambiguation for Basque based on the constraint grammar formalism. In Proceedings of RANLP, Tzigov Chark, Bulgaria. find
  7. Itziar Aduriz, Eneko Agirre, Izaskun Aldezabal, Iñaki Alegria, Xabier Arregi, Jose Maria Arriola, Xabier Artola, Koldo Gojenola, Aitor Maritxalar, Kepa Sarasola, et al. 2000. A word-grammar based morphological analyzer for agglutinative languages. In Proceedings of COLING, pages 1–7, Saarbrücken, Ge… find
  8. Itziar Aduriz, Maria Jesus Aranzabe, Jose Maria Arriola, Aitziber Atutxa, A Diaz de Ilarraza, Aitzpea Garmendia, and Maite Oronoz. 2003. Construction of a Basque dependency treebank. In Proceedings of the 2nd Workshop on Treebanks and Linguistic Theories (TLT), pages 201–204, Växjö, Sweden. find
  9. Zeljko Agic, Danijela Merkler, and Dasa Berovic. 2013. Parsing Croatian and Serbian by using Croatian dependency treebanks. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL), Seattle, WA. find
  10. I. Aldezabal, M.J. Aranzabe, A. Diaz de Ilarraza, and K. Fernández. 2008. From dependencies to constituents in the reference corpus for the processing of Basque. In Procesamiento del Lenguaje Natural, n o 41 (2008), pages 147–154. XXIV edición del Congreso Anual de la Sociedad Española para el Proce… find
  11. Bharat Ram Ambati, Samar Husain, Joakim Nivre, and Rajeev Sangal. 2010. On the role of morphosyntactic features in Hindi dependency parsing. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  12. Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi, and Josef van Genabith. 2010. Handling unknown words in statistical latent-variable parsing models for Arabic, English and French. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Lan… find
  13. Miguel Ballesteros and Joakim Nivre. 2012. MaltOptimizer: An optimization tool for MaltParser. In Proceedings of EACL, pages 58–62, Avignon, France. find
  14. Miguel Ballesteros. 2013. Effective morphological feature selection with MaltOptimizer at the SPMRL 2013 shared task. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 53–60, Seattle, WA. find
  15. Kepa Bengoetxea and Koldo Gojenola. 2010. Application of different techniques to dependency parsing of Basque. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  16. Philip Bille. 2005. A survey on tree edit distance and related problems. Theoretical Computer Science, 337(1– 3):217–239, 6. find
  17. Anders Björkelund, Ozlem Cetinoglu, Richárd Farkas, Thomas Mueller, and Wolfgang Seeker. 2013. (Re)ranking meets morphosyntax: State-of-the-art results from the SPMRL 2013 shared task. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 134–144, Seat… find
  18. Ezra Black, Steven Abney, Dan Flickinger, Claudia Gdaniec, Ralph Grishman, Philip Harrison, Donald Hindle, Robert Ingria, Frederick Jelinek, Judith Klavans, Mark Liberman, Mitchell Marcus, Salim Roukos, Beatrice Santorini, and Tomek Strzalkowski. 1991. A procedure for quantitatively comparing the sy… find
  19. Bernd Bohnet and Joakim Nivre. 2012. A transition-based system for joint part-of-speech tagging and labeled non-projective dependency parsing. In Proceedings of the EMNLP-CoNLL, pages 1455–1465, Jeju, Korea. find
  20. Bernd Bohnet. 2010. Top accuracy and fast dependency parsing is not a contradiction. In Proceedings of COL-ING, pages 89–97, Beijing, China. find our card
  21. Adriane Boyd. 2007. Discontinuity revisited: An improved conversion to context-free representations. In Proceedings of the Linguistic Annotation Workshop, Prague, Czech Republic. find
  22. Sabine Brants, Stefanie Dipper, Silvia Hansen, Wolfgang Lezius, and George Smith. 2002. The TIGER treebank. In Proceedings of the First Workshop on Treebanks and Linguistic Theories (TLT), pages 24–41, Sozopol, Bulgaria. find
  23. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In Proceedings of CoNLL, pages 149–164, New York, NY. find our card
  24. Tim Buckwalter. 2002. Arabic morphological analyzer version 1.0. Linguistic Data Consortium. find
  25. Tim Buckwalter. 2004. Arabic morphological analyzer version 2.0. Linguistic Data Consortium. find
  26. Marie Candito and Djamé Seddah. 2010. Parsing word clusters. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  27. Marie Candito, Benoit Crabbé, and Pascal Denis. 2010. Statistical French dependency parsing: Treebank conversion and first results. In Proceedings of LREC, Valletta, Malta. find
  28. Xavier Carreras, Michael Collins, and Terry Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In Proceedings of CoNLL, pages 9–16, Manchester, UK. find
  29. Eugene Charniak and Mark Johnson. 2005. Course-to-fine n-best-parsing and maxent discriminative reranking. In Proceedings of ACL, pages 173–180, Barcelona, Spain. find
  30. Eugene Charniak. 1997. Statistical parsing with a context-free grammar and word statistics. In AAAI/IAAI, pages 598–603. find
  31. Eugene Charniak. 2000. A maximum entropy inspired parser. In Proceedings of NAACL, pages 132–139, Seattle, WA. find our card
  32. Jinho D. Choi and Martha Palmer. 2011. Statistical dependency parsing in Korean: From corpus generation to automatic parsing. In Proceedings of Second Workshop on Statistical Parsing of Morphologically Rich Languages, pages 1–11, Dublin, Ireland. find
  33. Jinho D. Choi and Martha Palmer. 2012. Guidelines for the Clear Style Constituent to Dependency Conversion. Technical Report 01-12, University of Colorado at Boulder. find
  34. Key-sun Choi, Young S. Han, Young G. Han, and Oh W. Kwon. 1994. KAIST Tree Bank Project for Korean: Present and Future Development. In In Proceedings of the International Workshop on Sharable Natural Language Resources, pages 7–14, Nara, Japan. find
  35. Jinho D. Choi. 2013. Preparing Korean data for the shared task on parsing morphologically rich languages. arXiv:1309.1649. — [arXiv:1309.1649](https://arxiv.org/abs/1309.1649) find
  36. Grzegorz Chrupała, Georgiana Dinu, and Josef van Genabith. 2008. Learning morphology with Morfette. In Proceedings of LREC, Marrakech, Morocco. find our card
  37. Tagyoung Chung, Matt Post, and Daniel Gildea. 2010. Factors affecting the accuracy of Korean parsing. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  38. Volkan Cirik and Hüsnü Şensoy. 2013. The AI-KU system at the SPMRL 2013 shared task: Unsupervised features for dependency parsing. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 68–75, Seattle, WA. find
  39. Michael Collins. 2003. Head-driven statistical models for natural language parsing. Computational Linguistics, 29(4):589–637. find our card
  40. Matthieu Constant, Marie Candito, and Djamé Seddah. 2013. The LIGM-Alpage architecture for the SPMRL 2013 shared task: Multiword expression analysis and dependency parsing. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 46–52, Seattle, WA. find
  41. Anna Corazza, Alberto Lavelli, Giogio Satta, and Roberto Zanoli. 2004. Analyzing an Italian treebank with state-of-the-art statistical parsers. In Proceedings of the Third Workshop on Treebanks and Linguistic Theories (TLT), Tübingen, Germany. find
  42. Benoit Crabbé and Marie Candito. 2008. Expériences d’analyse syntaxique statistique du français. In Actes de la 15ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN’08), pages 45–54, Avignon, France. find
  43. Dóra Csendes, János Csirik, Tibor Gyimóthy, and András Kocsor. 2005. The Szeged treebank. In Proceedings of the 8th International Conference on Text, Speech and Dialogue (TSD), Lecture Notes in Computer Science, pages 123–132, Berlin / Heidelberg. Springer. find
  44. Eric De La Clergerie. 2013. Exploring beam-based shift-reduce dependency parsing with DyALog: Results from the SPMRL 2013 shared task. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 81–89, Seattle, WA. find
  45. Marie-Catherine de Marneffe and Christopher D. Manning. 2008. The stanford typed dependencies representation. In Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation. find our card
  46. Mona Diab, Nizar Habash, Owen Rambow, and Ryan Roth. 2013. LDC Arabic treebanks and associated corpora: Data divisions manual. Technical Report CCLS-13-02, Center for Computational Learning Systems, Columbia University. find
  47. Eva Ejerhed and Gunnel Källgren. 1997. Stockholm Umeå Corpus. Version 1.0. Department of Linguistics, Umeå University and Department of Linguistics, Stockholm University. find
  48. Eva Ejerhed, Gunnel Källgren, Ola Wennstedt, and Magnus Åström. 1992. The linguistic annotation system of the Stockholm–Umeå Corpus project. Technical Report 33, University of Umeå: Department of Linguistics. find
  49. Nerea Ezeiza, Iñaki Alegria, José María Arriola, Rubén Urizar, and Itziar Aduriz. 1998. Combining stochastic and rule-based methods for disambiguation in agglutinative languages. In Proceedings of COLING, pages 380–384, Montréal, Canada. find
  50. Jenny Rose Finkel, Alex Kleeman, and Christopher D. Manning. 2008. Efficient, feature-based, conditional random field parsing. In Proceedings of ACL, pages 959–967, Columbus, OH. find
  51. Alexander Fraser, Helmut Schmid, Richárd Farkas, Renjing Wang, and Hinrich Schütze. 2013. Knowledge sources for constituent parsing of German, a morphologically rich and less-configurational language. Computational Linguistics, 39(1):57–85. find
  52. Iakes Goenaga, Koldo Gojenola, and Nerea Ezeiza. 2013. Exploiting the contribution of morphological information to parsing: the BASQUE TEAM system in the SPRML’2013 shared task. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 61–67, Seattle, WA. find
  53. Yoav Goldberg and Michael Elhadad. 2010a. Easy-first dependency parsing of Modern Hebrew. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find our card
  54. Yoav Goldberg and Michael Elhadad. 2010b. An efficient algorithm for easy-first non-directional dependency parsing. In Proceedings of HLT: NAACL, pages 742–750, Los Angeles, CA. find
  55. Yoav Goldberg and Reut Tsarfaty. 2008. A single framework for joint morphological segmentation and syntactic parsing. In Proceedings of ACL, Columbus, OH. find
  56. Yoav Goldberg, Meni Adler, and Michael Elhadad. 2008. EM can find pretty good HMM POS-taggers (when given a good start). In Proc. of ACL, Columbus, OH. find
  57. Yoav Goldberg, Reut Tsarfaty, Meni Adler, and Michael Elhadad. 2009. Enhancing unlexicalized parsing performance using a wide coverage lexicon, fuzzy tag-set mapping, and EM-HMM-based lexical probabilities. In Proceedings of EALC, pages 327–335, Athens, Greece. find
  58. Yoav Goldberg. 2011. Automatic syntactic processing of Modern Hebrew. Ph.D. thesis, Ben Gurion University of the Negev. find
  59. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) version 3.1. Linguistic Data Consortium LDC2009E73. find
  60. Spence Green and Christopher D. Manning. 2010. Better Arabic parsing: Baselines, evaluations, and analysis. In Proceedings of COLING, pages 394–402, Beijing, China. find our card
  61. Nathan Green, Loganathan Ramasamy, and Zdenék Žabokrtský. 2012. Using an SVM ensemble system for improved Tamil dependency parsing. In Proceedings of the ACL 2012 Joint Workshop on Statistical Parsing and Semantic Processing of Morphologically Rich Languages, pages 72–77, Jeju, Korea. find
  62. Spence Green, Marie-Catherine de Marneffe, and Christopher D. Manning. 2013. Parsing models for identifying multiword expressions. Computational Linguistics, 39(1):195–227. find
  63. Noemie Guthmann, Yuval Krymolowski, Adi Milea, and Yoad Winter. 2009. Automatic annotation of morphosyntactic dependencies in a Modern Hebrew Treebank. In Proceedings of the Eighth International Workshop on Treebanks and Linguistic Theories (TLT), Groningen, The Netherlands. find
  64. Nizar Habash and Ryan Roth. 2009. CATiB: The Columbia Arabic Treebank. In Proceedings of ACL-IJCNLP, pages 221–224, Suntec, Singapore. find our card
  65. Nizar Habash, Ryan Gabbard, Owen Rambow, Seth Kulick, and Mitch Marcus. 2007. Determining case in Arabic: Learning complex linguistic behavior requires complex linguistic features. In Proceedings of EMNLP-CoNLL, pages 1084–1092, Prague, Czech Republic. find
  66. Nizar Habash, Reem Faraj, and Ryan Roth. 2009a. Syntactic Annotation in the Columbia Arabic Treebank. In Proceedings of MEDAR International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  67. Nizar Habash, Owen Rambow, and Ryan Roth. 2009b. MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Proceedings of the Second International Conference on Arabic Language Resources and Tools. Cairo, Egypt. find
  68. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. find
  69. Jan Hajič, Alena Böhmová, Eva Hajičová, and Barbora Vidová-Hladká. 2000. The Prague Dependency Treebank: A three-level annotation scenario. In Anne Abeillé, editor, Treebanks: Building and Using Parsed Corpora. Kluwer Academic Publishers. find
  70. Péter Halácsy, András Kornai, and Csaba Oravecz. 2007. HunPos – an open source trigram tagger. In Proceedings of ACL, pages 209–212, Prague, Czech Republic. find
  71. Johan Hall, Jens Nilsson, Joakim Nivre, Gülşen Eryiǧit, Beáta Megyesi, Mattias Nilsson, and Markus Saers. 2007. Single malt or blended? A study in multilingual parser optimization. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 933–939, Prague, Czech Republic. find
  72. Chung-hye Han, Na-Rae Han, Eon-Suk Ko, Martha Palmer, and Heejong Yi. 2002. Penn Korean Treebank: Development and evaluation. In Proceedings of the 16th Pacific Asia Conference on Language, Information and Computation, Jeju, Korea. find
  73. Tilman Höhle. 1986. Der Begriff "Mittelfeld", Anmerkungen über die Theorie der topologischen Felder. In Akten des Siebten Internationalen Germanistenkongresses 1985, pages 329–340, Göttingen, Germany. find
  74. Zhongqiang Huang, Mary Harper, and Slav Petrov. 2010. Self-training with products of latent variable grammars. In Proceedings of EMNLP, pages 12–22, Cambridge, MA. find
  75. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL, pages 586–594, Columbus, OH. find
  76. Alon Itai and Shuly Wintner. 2008. Language resources for Hebrew. Language Resources and Evaluation, 42(1):75–98, March. find
  77. Mark Johnson. 1998. PCFG models of linguistic tree representations. Computational Linguistics, 24(4):613– 632. find our card
  78. Laura Kallmeyer and Wolfgang Maier. 2013. Data-driven parsing using probabilistic linear context-free rewriting systems. Computational Linguistics, 39(1). find
  79. Fred Karlsson, Atro Voutilainen, Juha Heikkilae, and Arto Anttila. 1995. Constraint Grammar: a language-independent system for parsing unrestricted text. Walter de Gruyter. find
  80. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL, pages 423–430, Sapporo, Japan. find our card
  81. Sandra Kübler, Erhard W. Hinrichs, and Wolfgang Maier. 2006. Is it really that difficult to parse German? In Proceedings of EMNLP, pages 111–119, Sydney, Australia, July. find
  82. Sandra Kübler, Wolfgang Maier, Ines Rehbein, and Yannick Versley. 2008. How to compare treebanks. In Proceedings of LREC, pages 2322–2329, Marrakech, Morocco. find
  83. The PaGe 2008 shared task on parsing German. In Proceedings of the Workshop on Parsing German, pages 55–63, Columbus, OH. find
  84. Seth Kulick, Ryan Gabbard, and Mitch Marcus. 2006. Parsing the Arabic Treebank: Analysis and Improvements. In Proceedings of the Treebanks and Linguistic Theories Conference, pages 31–42, Prague, Czech Republic. find
  85. Joseph Le Roux, Benoit Sagot, and Djamé Seddah. 2012. Statistical parsing of Spanish and data driven lemmatization. In Proceedings of the Joint Workshop on Statistical Parsing and Semantic Processing of Morphologically Rich Languages, pages 55–61, Jeju, Korea. find
  86. Kong Joo Lee, Byung-Gyu Chang, and Gil Chang Kim. 1997. Bracketing Guidelines for Korean Syntactic Tree Tagged Corpus. Technical Report CS/TR-97-112, Department of Computer Science, KAIST. find
  87. Roger Levy and Christopher D. Manning. 2003. Is it harder to parse Chinese, or the Chinese treebank? In Proceedings of ACL, Sapporo, Japan. find
  88. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Hubert Jin. 2004a. Arabic Treebank: Part 2 v 2.0. LDC catalog number LDC2004T02. find
  89. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004b. The Penn Arabic Treebank: Building a large-scale annotated Arabic corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  90. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Hubert Jin. 2005. Arabic Treebank: Part 1 v 3.0. LDC catalog number LDC2005T02. find
  91. Mohamed Maamouri, Ann Bies, Seth Kulick, Fatma Gaddeche, Wigdan Mekki, Sondos Krouna, and Basma Bouziri. 2009. The Penn Arabic Treebank part 3 version 3.1. Linguistic Data Consortium LDC2008E22. find
  92. Wolfgang Maier, Miriam Kaeshammer, and Laura Kallmeyer. 2012. Data-driven PLCFRS parsing revisited: Restricting the fan-out to two. In Proceedings of the Eleventh International Conference on Tree Adjoining Grammars and Related Formalisms (TAG+11), Paris, France. find
  93. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn TreeBank. Computational Linguistics, 19(2):313–330. find our card
  94. Andre Martins, Noah Smith, Eric Xing, Pedro Aguiar, and Mario Figueiredo. 2010. Turbo parsers: Dependency parsing by approximate variational inference. In Proceedings of EMNLP, pages 34–44, Cambridge, MA. find our card
  95. Yuval Marton, Nizar Habash, and Owen Rambow. 2013a. Dependency parsing of Modern Standard Arabic with lexical and inflectional features. Computational Linguistics, 39(1):161–194. find our card
  96. Yuval Marton, Nizar Habash, Owen Rambow, and Sarah Alkhulani. 2013b. SPMRL’13 shared task system: The CADIM Arabic dependency parser. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 76–80, Seattle, WA. find
  97. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceedings of HLT:NAACL, pages 152–159, New York, NY. find our card
  98. Ryan T. McDonald, Koby Crammer, and Fernando C. N. Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of ACL, pages 91–98, Ann Arbor, MI. find our card
  99. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Tackstrom, Claudia Bedini, Nuria Bertomeu Castello, and Jungmee Lee. 2013. Universal dependency annotation for multilingual parsing. In Proceedings of ACL, S… find
  100. Igor Mel’čuk. 2001. Communicative Organization in Natural Language: The Semantic-Communicative Structure of Sentences. J. Benjamins. find
  101. Knowledge Center for Processing Hebrew MILA. 2008. Hebrew morphological analyzer. http://mila.cs.technion.ac.il. ↗
  102. Antonio Moreno, Ralph Grishman, Susana Lopez, Fernando Sanchez, and Satoshi Sekine. 2000. A treebank of Spanish and its application to parsing. In Proceedings of LREC, Athens, Greece. find
  103. Joakim Nivre and Beáta Megyesi. 2007. Bootstrapping a Swedish treeebank using cross-corpus harmonization and annotation projection. In Proceedings of the 6th International Workshop on Treebanks and Linguistic Theories, pages 97–102, Bergen, Norway. find
  104. Joakim Nivre, Jens Nilsson, and Johan Hall. 2006. Talbanken05: A Swedish treebank with phrase structure and dependency annotation. In Proceedings of LREC, pages 1392–1395, Genoa, Italy. find
  105. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007a. find
  106. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915–932, Prague, Czech Republic. find
  107. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gülşen Eryiǧit, Sandra Kübler, Svetoslav Marinov, and Erwin Marsi. 2007b. MaltParser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95–135. find
  108. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 Shared Task on Parsing the Web. In Proceedings of the First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL), a NAACL-HLT 2012 workshop, Montreal, Canada. find
  109. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proceedings of COLING-ACL, Sydney, Australia. find our card
  110. Slav Petrov. 2009. Coarse-to-Fine Natural Language Processing. Ph.D. thesis, University of California at Bekeley, Berkeley, CA. find
  111. Slav Petrov. 2010. Products of random latent variable grammars. In Proceedings of HLT: NAACL, pages 19– 27, Los Angeles, CA. find
  112. Adam Przepiórkowski, Mirosław Bańko, Rafał L. Górski, and Barbara Lewandowska-Tomaszczyk, editors. 2012. Narodowy Korpus Jkezyka Polskiego. Wydawnictwo Naukowe PWN, Warsaw. find
  113. Ines Rehbein and Josef van Genabith. 2007a. Evaluating Evaluation Measures. In Proceedings of the 16th Nordic Conference of Computational Linguistics NODALIDA-2007, Tartu, Estonia. find
  114. Ines Rehbein and Josef van Genabith. 2007b. Treebank annotation schemes and parser evaluation for German. In Proceedings of EMNLP-CoNLL, Prague, Czech Republic. find
  115. Ines Rehbein. 2011. Data point selection for self-training. In Proceedings of the Second Workshop on Statistical Parsing of Morphologically Rich Languages, pages 62– 67, Dublin, Ireland. find
  116. Kenji Sagae and Alon Lavie. 2006. Parser combination by reparsing. In Proceedings of HLT-NAACL, pages 129–132, New York, NY. find
  117. Geoffrey Sampson and Anna Babarczy. 2003. A test of the leaf-ancestor metric for parse accuracy. Natural Language Engineering, 9(04):365–380. find
  118. Natalie Schluter and Josef van Genabith. 2007. Preparing, restructuring, and augmenting a French Treebank: Lexicalised parsers or coherent treebanks? In Proc. of PACLING 07, Melbourne, Australia. find
  119. Helmut Schmid, Arne Fitschen, and Ulrich Heid. 2004. SMOR: A German computational morphology covering derivation, composition and inflection. In Proceedings of LREC, Lisbon, Portugal. find our card
  120. Djamé Seddah, Grzegorz Chrupała, Ozlem Cetinoglu, Josef van Genabith, and Marie Candito. 2010. Lemmatization and statistical lexicalized parsing of morphologically-rich languages. In Proceedings of the First Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL), Los Angeles, CA. find
  121. Wolfgang Seeker and Jonas Kuhn. 2012. Making ellipses explicit in dependency conversion for a German treebank. In Proceedings of LREC, pages 3132–3139, Istanbul, Turkey. find
  122. Hiroyuki Shindo, Yusuke Miyao, Akinori Fujino, and Masaaki Nagata. 2012. Bayesian symbol-refined tree substitution grammars for syntactic parsing. In Proceedings of ACL, pages 440–448, Jeju, Korea. find
  123. Anthony Sigogne, Matthieu Constant, and Eric Laporte. 2011. French parsing enhanced with a word clustering method based on a syntactic lexicon. In Proceedings of the Second Workshop on Statistical Parsing of Morphologically Rich Languages, pages 22–27, Dublin, Ireland. find
  124. Khalil Sima’an, Alon Itai, Yoad Winter, Alon Altmann, and Noa Nativ. 2001. Building a tree-bank of Modern Hebrew text. Traitement Automatique des Langues, 42:347–380. find
  125. Marek Świdziński and Marcin Woliński. 2010. Towards a bank of constituent parse trees for Polish. In Proceedings of Text, Speech and Dialogue, pages 197–204, Brno, Czech Republic. find
  126. Ulf Teleman. 1974. Manual för grammatisk beskrivning av talad och skriven svenska. Studentlitteratur. find
  127. Lucien Tesnière. 1959. Éléments De Syntaxe Structurale. Klincksieck, Paris. find
  128. Reut Tsarfaty and Khalil Sima’an. 2010. Modeling morphosyntactic agreement in constituency-based parsing of Modern Hebrew. In Proceedings of the First Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL), Los Angeles, CA. find our card
  129. Reut Tsarfaty, Djame Seddah, Yoav Goldberg, Sandra Kübler, Marie Candito, Jennifer Foster, Yannick Versley, Ines Rehbein, and Lamia Tounsi. 2010. Statistical parsing for morphologically rich language (SPMRL): What, how and whither. In Proceedings of the First workshop on Statistical Parsing of Morph… find
  130. Reut Tsarfaty, Joakim Nivre, and Evelina Andersson. 2011. Evaluating dependency parsing: Robust and heuristics-free cross-framework evaluation. In Proceedings of EMNLP, Edinburgh, UK. find
  131. Reut Tsarfaty, Joakim Nivre, and Evelina Andersson. 2012a. Cross-framework evaluation for statistical parsing. In Proceeding of EACL, Avignon, France. find
  132. Reut Tsarfaty, Joakim Nivre, and Evelina Andersson. 2012b. Joint evaluation for segmentation and parsing. In Proceedings of ACL, Jeju, Korea. find
  133. Reut Tsarfaty, Djamé Seddah, Sandra Kübler, and Joakim Nivre. 2012c. Parsing morphologically rich languages: Introduction to the special issue. Computational Linguistics, 39(1):15–22. find
  134. Reut Tsarfaty. 2010. Relational-Realizational Parsing. Ph.D. thesis, University of Amsterdam. find
  135. Reut Tsarfaty. 2013. A unified morpho-syntactic scheme of Stanford dependencies. In Proceedings of ACL, Sofia, Bulgaria. find
  136. Veronika Vincze, Dóra Szauter, Attila Almási, György Móra, Zoltán Alexin, and János Csirik. 2010. Hungarian Dependency Treebank. In Proceedings of LREC, Valletta, Malta. find
  137. Joachim Wagner. 2012. Detecting Grammatical Errors with Treebank-Induced Probabilistic Parsers. Ph.D. thesis, Dublin City University. find
  138. Marcin Woliński, Katarzyna Głowińska, and Marek Świdziński. 2011. A preliminary version of Składnica—a treebank of Polish. In Proceedings of the 5th Language & Technology Conference, pages 299–303, Poznań, Poland. find
  139. Alina Wróblewska. 2012. Polish Dependency Bank. Linguistic Issues in Language Technology, 7(1):1–15. find
  140. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of ACL:HLT, pages 188–193, Portland, OR. find
  141. János Zsibrita, Veronika Vincze, and Richárd Farkas. 2013. magyarlanc: A toolkit for morphological and dependency parsing of Hungarian. In Proceedings of RANLP, pages 763–771, Hissar, Bulgaria. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seddah-2013-overview-spmrl-2013-shared-task/