Elamur Research⌕
☰
home / papers / 2010 / Statistical Parsing of Morphologically Rich Languages (SPMRL) What, How and Whither

Statistical Parsing of Morphologically Rich Languages (SPMRL) What, How and Whither

Paper · 2010 · Reut Tsarfaty, Djamé Seddah, Yoav Goldberg et al.

Summary

The paper is the opening overview of the first SPMRL workshop (2010). It asks why statistical parsers that perform well on the English Penn Treebank lose much of their accuracy on morphologically rich languages (MRLs), where word forms carry much of the grammatical information and word order tends to be freer. The authors survey earlier constituency and dependency work and synthesize 11 workshop contributions, which together address seven languages: Korean, Hindi, Hebrew, German, French, Basque and Arabic. They group the difficulties into three families:

  • Architecture: what the parser receives as input, and how segmentation errors propagate through the pipeline.
  • Representation: where morphology should sit in the model, such as PoS tags, dependency labels or non-terminals.
  • Estimation: how to get reliable lexical probabilities from small treebanks with very large vocabularies.

The main finding is that these problems recur across languages and frameworks. Whether a morphological feature helps depends on the model and on whether the tags are gold or predicted. The authors also stress that good handling of rare and unknown words is central to accuracy. The paper matters because it frames MRL parsing as a shared research agenda. It calls for raw-text parsing tasks, cross-language constituency shared tasks and evaluation metrics that work across frameworks.

Key points

  • English WSJ/PTB constituency parsing had reached about 92% F1 under PARSEVAL. In contrast, a Collins-style adaptation for Czech peaked around 80 F1.
  • In the CoNLL 2006/2007 dependency shared tasks (18 languages), English and Catalan were parsed best. Arabic, Basque and Greek were the hardest, for both MaltParser and MST.
  • Annotation scheme strongly affects scores:
    • Restructuring Spanish and French treebanks toward PTB style improved Head-Driven parsers.
    • German results differ between the flat NEGRA and the more hierarchical TüBa-D/Z.
  • The split-merge latent-variable Berkeley parser (Petrov et al., 2006) is described as MRL-friendly because its design is language-agnostic. It outperformed several lexicalized and unlexicalized models for French.
  • Mapping unknown words to morphology-based classes (Attia et al., 2010) helped English, French and Arabic. Arabic, the most morphologically rich of the three, gained the most.

Interesting ideas

  • I1. Replace word forms with lemmas or distributional word clusters at the terminal level. Clusters can be learned from a large raw corpus, and lemma-based variants were reported to help French parsing. Each symbol then occurs more often and the effective vocabulary grows, which is attractive for any small-treebank, highly inflected language such as Ukrainian.
  • I2. Back off unseen words to coarse signatures built from morphological cues (affixes, inflection) instead of a single "unknown" token. This is cheap and fits well into latent-variable parsers or other sparse-lexicon models.
  • I3. Evaluate on raw text with predicted morphology, not gold segmentation and tags. Gold-input experiments can overstate accuracy and can even favour different features than realistic pipelines do.
  • I4. Treat treebank annotation style as an experimental variable. Flat versus hierarchical structure can shift PARSEVAL scores substantially, so cross-treebank comparisons should account for it before drawing conclusions about models.
  • I5. Compare systems on framework-neutral information, such as predicate-argument relations. This makes it possible to compare constituency and dependency parsers fairly across languages and annotation schemes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany102 works · 17,447 citations🎓Uppsala UniversityUppsala · Sweden56 works · 13,046 citations🎓Dublin City UniversityDublin · Ireland50 works · 8,937 citations🔬Institut national de recherche en sciences et technologies du numériqueLe Chesnay-Rocquencourt · France62 works · 17,271 citations🎓Ben-Gurion University of the NegevBeersheba · Israel36 works · 3,066 citations🎓University of TübingenTübingen · Germany30 works · 40,834 citations🎓Indiana UniversityBloomington · United States24 works · 2,995 citations🎓Paris-Sorbonne UniversityParis · France3 works · 429 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 1,484 worksSyntax & grammar formalisms · 161 worksMorphology · 277 works

Cite

Cite as Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein and Lamia Tounsi. 2010. Statistical Parsing of Morphologically Rich Languages (SPMRL) What, How and Whither. Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tsarfaty2010statistical,
  title = {{Statistical Parsing of Morphologically Rich Languages (SPMRL) What, How and Whither}},
  author = {Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein and Lamia Tounsi},
  booktitle = {Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages},
  year = {2010},
  url = {https://aclanthology.org/W10-1401/}
}

References

64 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bharat Ram Ambati, Samar Husain, Sambhav Jain, Dipti Misra Sharma, and Rajeev Sangal. 2010a. Two methods to incorporate local morphosyntactic features in Hindi dependency parsing. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Ange… find
  2. Bharat Ram Ambati, Samar Husain, Joakim Nivre, and Rajeev Sangal. 2010b. On the role of morphosyntactic features in Hindi dependency parsing. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  3. Abhishek Arun and Frank Keller. 2005. Lexicalization in crosslinguistic probabilistic parsing: The case of French. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 306–313, Ann Arbor, MI. find
  4. Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi, and Josef van Genabith. 2010. Handling unknown words in statistical latent-variable parsing models for Arabic, English and French. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Lan… find
  5. Kepa Bengoetxea and Koldo Gojenola. 2010. Application of different techniques to dependency parsing of Basque. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  6. Daniel M. Bikel. 2002. Design of a multi-lingual, parallel-processing statistical parsing engine. In Proceedings of the Second International Conference on Human Language Technology Research, pages 178– 182. Morgan Kaufmann Publishers Inc. San Francisco, CA, USA. find
  7. E. Black, S. Abney, D. Flickinger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini, and T. Strzalkowski. 1991. A procedure for quantitatively comparing the syntactic coverage of English grammars. In Proceedings of th… find
  8. Rens Bod. 2003. An efficient implementation of a new DOP model. In Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, pages 19–26, Budapest, Hungary. find
  9. Joan Bresnan. 2001. Lexical-Functional Syntax. Blackwell, Oxford. find
  10. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In Proceedings of the Tenth Conference on Computational Language Learning (CoNLL), pages 149–164, New York, NY.
  11. Marie Candito and Djamé Seddah. 2010. Parsing word clusters. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  12. Xavier Carreras, Michael Collins, and Terry Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In Proceedings of the Twelfth Conference on Computational Natural Language Learning (CoNLL), pages 9–16, Manchester, UK. find
  13. Eugene Charniak and Mark Johnson. 2005. Course-to-fine n-best-parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the ACL, pages 173–180, Barcelona, Spain, June. find
  14. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st Annual Meeting of the North American Chapter of the ACL (NAACL), Seattle.
  15. David Chiang. 2000. Statistical parsing with an automatically-extracted Tree Adjoining Grammar. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 456–463, Hong Kong. Association for Computational Linguistics Morristown, NJ, USA.
  16. Tagyoung Chung, Matt Post, and Daniel Gildea. 2010. Factors affecting the accuracy of Korean parsing. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  17. Michael Collins, Jan Hajič, Lance Ramshaw, and Christoph Tillmann. 1999. A statistical parser for Czech. In Proceedings of the 37th Annual Meeting of the ACL, volume 37, pages 505–512, College Park, MD.
  18. Michael Collins. 1997. Three Generative, Lexicalized Models for Statistical Parsing. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pages 16–23, Madrid, Spain. find
  19. Anna Corazza, Alberto Lavelli, Giogio Satta, and Roberto Zanoli. 2004. Analyzing an Italian treebank with state-of-the-art statistical parsers. In Proceedings of the Third Third Workshop on Treebanks and Linguistic Theories (TLT 2004), Tübingen, Germany. find
  20. Brooke Cowan and Michael Collins. 2005. Morphology and reranking for the statistical parsing of Spanish. In in Proceedins of EMNLP. find
  21. Benoit Crabbé and Marie Candito. 2008. Expériences d’analyse syntaxique statistique du français. In Actes de la 15ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN’08), pages 45–54, Avignon, France. find
  22. Amit Dubey and Frank Keller. 2003. Probabilistic parsing for German using sister-head dependencies. In In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 96–103, Ann Arbor, MI. find
  23. Amit Dubey. 2005. What to do when lexicalization fails: parsing German with suffix analysis and smoothing. In 43rd Annual Meeting of the Association for Computational Linguistics. find
  24. Jenny Rose Finkel, Alex Kleeman, and Christopher D. Manning. 2008. Efficient, feature-based, conditional random field parsing. In Proceedings of ACL.
  25. Yoav Goldberg and Michael Elhadad. 2010. Easy-first dependency parsing of Modern Hebrew. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA.
  26. Yoav Goldberg and Reut Tsarfaty. 2008. A single framework for joint morphological segmentation and syntactic parsing. In Proceedings of the 46nd Annual Meeting of the Association for Computational Linguistics. find
  27. Yoav Goldberg, Reut Tsarfaty, Meni Adler, and Michael Elhadad. 2009. Enhancing unlexicalized parsing performance using a wide coverage lexicon, fuzzy tag-set mapping, and em-hmm-based lexical probabilities. In Proceedings of the 12th Conference of the European Chapter of the Association for Computat… find
  28. Kenneth L. Hale. 1983. Warlpiri and the grammar of non-configurational languages. Natural Language and Linguistic Theory, 1(1). find
  29. Erhard W. Hinrichs, Sandra Kübler, and Karin Naumann. 2005. A unified representation for morphological, syntactic, semantic, and referential annotations. In Proceedings of the ACL Workshop on Frontiers in Corpus Annotation II: Pie in the Sky, pages 13–20, Ann Arbor, MI. find
  30. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL. find
  31. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL, pages 423–430.
  32. Sandra Kübler, Erhard W. Hinrichs, and Wolfgang Maier. 2006. Is it really that difficult to parse German? In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 111– 119, Sydney, Australia, July. Association for Computational Linguistics. find
  33. The PaGe 2008 shared task on parsing German. In Proceedings of the Workshop on Parsing German, pages 55–63. Association for Computational Linguistics. find
  34. Seth Kulick, Ryan Gabbard, and Mitchell Marcus. 2006. Parsing the Arabic treebank: Analysis and improvements. In Proceedings of TLT. find
  35. Dekang Lin. 1995. A dependency-based method for evaluating broad-coverage parsers. In International Joint Conference on Artificial Intelligence, pages 1420–1425, Montreal. find
  36. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic treebank: Building a large-scale annotated Arabic corpus. In Proceedings of NEMLAR International Conference on Arabic Language Resources and Tools. find
  37. Mohamed Maamouri, Ann Bies, and Seth Kulick. 2008. Enhanced annotation and parsing of the Arabic treebank. In Proceedings of INFOS. find
  38. David M. Magerman. 1995. Statistical decision-tree models for parsing. In Proceedings of the 33rd Annual Meeting on Association for Computational Linguistics, pages 276–283, Cambridge, MA.
  39. Wolfgang Maier. 2010. Direct parsing of discontinuous constituents in german. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA. find
  40. Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330.
  41. Yuval Marton, Nizar Habash, and Owen Rambow. 2010. Improving Arabic dependency parsing with lexical and inflectional morphological features. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA.
  42. Ryan T. McDonald and Fernando C. N. Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proc. of EACL’06.
  43. Ryan T. McDonald, Koby Crammer, and Fernando C. N. Pereira. 2005. Online large-margin training of dependency parsers. In Proc. of ACL’05, Ann Arbor, USA.
  44. Igor Mel’čuk. 1988. Dependency Syntax: Theory and Practice. State University of New York Press. find
  45. Marianne Mithun. 1992. Is basic word order universal? In Doris L. Payne, editor, Pragmatics of Word Order Flexibility. John Benjamins, Amsterdam. find
  46. Joakim Nivre and Jens Nilsson. 2005. Pseudo-projective dependency parsing. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), Ann Arbor, MI.
  47. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007a. find
  48. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915–932, Prague, Czech Republic, June. Association for Computational Linguistics.
  49. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007b. find
  50. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915–932.
  51. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gülşen Eryiǧit, Sandra Kübler, Svetoslav Marinov, and Erwin Marsi. 2007c. MaltParser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95–135. find
  52. Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.
  53. Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference.
  54. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia,…
  55. Ines Rehbein and Josef van Genabith. 2007. Treebank annotation schemes and parser evaluation for German. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic. find
  56. Geoffrey Sampson and Anna Babarczy. 2003. A test of the leaf-ancestor metric for parse accuracy. Natural Language Engineering, 9(04):365–380. find
  57. Natalie Schluter and Josef van Genabith. 2007. Preparing, restructuring, and augmenting a French Treebank: Lexicalised parsers or coherent treebanks? In Proc. of PACLING 07, Melbourne, Australia. find
  58. Djamé Seddah, Marie Candito, and Benoit Crabbé. 2009. Cross parser evaluation and tagset variation: A French Treebank study. In Proceedings of the 11th Internation Conference on Parsing Technologies (IWPT’09), pages 150–161, Paris, France, October. Association for Computational Linguistics. find
  59. Djamé Seddah, Grzegorz Chrupała, Ozlem Cetinoglu, Josef van Genabith, and Marie Candito. 2010. Lemmatization and statistical lexicalized parsing of morphologically-rich languages. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Ang… find
  60. Wojciech Skut, Thorsten Brants, Brigitte Krenn, and Hans Uszkoreit. 1998. A linguistically interpreted corpus of German newspaper texts. In ESSLLI Workshop on Recent Advances in Corpus Annotation, Saarbrücken, Germany. find
  61. Reut Tsarfaty and Khalil Sima’an. 2007. Three-dimensional parametrization for parsing morphologically rich languages. In Proceedings of the 10th International Conference on Parsing Technologies (IWPT), pages 156–167. find
  62. Reut Tsarfaty and Khalil Sima’an. 2008. Relational-Realizational parsing. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 889–896. find
  63. Reut Tsarfaty and Khalil Sima’an. 2010. Modeling morphosyntactic agreement in constituency-based parsing of Modern Hebrew. In Proceedings of the NAACL/HLT Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), Los Angeles, CA.
  64. Yannick Versley and Ines Rehbein. 2009. Scalable discriminative parsing for german. In Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09), pages 134–137, Paris, France, October. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tsarfaty-2010-statistical-parsing-morphologically-rich-languages/