Elamur Research⌕
☰
home / papers / 2013 / SPMRL‘13 Shared Task System: The CADIM Arabic Dependency Parser

SPMRL‘13 Shared Task System: The CADIM Arabic Dependency Parser

Paper · 2013 · Yuval Marton, Nizar Habash, Owen Rambow et al.

Summary

This paper describes the Columbia Arabic & Dialect Modeling group (CADIM) entry to the SPMRL 2013 Shared Task. The team competed only in Arabic dependency parsing and only for the setting with predicted (non-gold) POS tags and morphological features. They reused the system from Marton et al. (2013), which uses the Easy-First Parser with a feature set chosen by searching the morphological feature space. That search judged each feature on three things: how relevant it is to attachment decisions, whether other features already cover it, and how reliably it can be predicted. The chosen set pairs a 12-tag core POS inventory with agreement-related features: determiner, person, functional number and gender, rationality, and the undiacritized lemma. For the shared task the models were retrained on the new data splits. Lemmas and functional features were regenerated with the ALMOR analyzer and an existing functional-feature converter. The parser was the strongest single-parser system with predicted tokenization and the second strongest with gold tokenization. This supports the view that agreement morphology helps parsing when it is modeled functionally.

Key points

  • The parser is the Easy-First Parser (Goldberg and Elhadad, 2010), chosen because it beat MaltParser in the earlier study. It predicts attachments first and labels afterwards, so the team tuned UAS first and the labeler second.
  • The final features are CORE12 POS, DET, PERSON, functional number (FNN), functional gender (FNG), rationality (FN*R) and the undiacritized lemma (LMM). Training data mixes gold and predicted feature values.
  • CASE and STATE help a lot with gold input. CASE is rarely written in undiacritized MSA, so predicting it is very inaccurate, and it is useless in realistic conditions.
  • Data preparation used ALMOR with the SAMA databases to derive lemmas from the word form plus POS. The MLE:W2+CATiB model (Alkuhlani and Habash, 2012) produced the functional number, gender and rationality features. Two models were trained: one on 5K sentences and one on the full training set.
  • With gold tokenization and predicted tags, the full-data model reached 83.2 LAS and 85.8 UAS on test, and 84.8 LAS on dev sentences of 70 words or fewer. With predicted tokenization it scored 87.8 labeled and 90.8 unlabeled Tedeval. Only the IMS/Szeged system combination (Run 2) beat it on every measure.

Interesting ideas

  • I1. Judge each candidate parser feature on three separate axes: relevance to attachment, redundancy with stronger features, and predictability on real input. A feature that is great in theory can still be worthless in practice.
  • I2. Model functional morphology (the number or gender a word actually behaves with in agreement) rather than its surface form, as with Arabic broken plurals that look singular but agree as plurals.
  • I3. Train on a mixture of gold and automatically predicted feature values, so the model sees clean signal and also the noise it will meet at test time.
  • I4. Run the feature search separately for gold and predicted input conditions, because the best feature set differs between them.
  • I5. Treat rationality (humanness) as an explicit grammatical feature because it takes part in agreement.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Before adding morphological or lexical features to the perceptron parser or gparse, measure each one under predicted tagging, not only gold, and drop features whose tagger accuracy is too low to pay off.
  • I2 (tokens): The en crate could store a functional number alongside the form-based tag for words like "people", "police" or "data", so agreement checks and later stages use the value that actually governs syntax.
  • I3 (learn): When building parser training sets from silver data, mix gold-tagged and tagger-predicted versions of the same sentences instead of choosing one, and use evaluation to check which mix works best.
  • I4 (learn): Evaluation reports should always pair gold-input and pipeline-input scores, so that improvements that only hold with oracle input are not mistaken for real gains.
  • I5 (world): A compact animate/human flag on entities, drawn from the WordNet ontology, could feed back to choices like who/which and he/it, and help coreference agree with its antecedents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States325 works · 142,931 citations🎓Columbia UniversityNew York · United States241 works · 54,053 citations🏢Microsoft Research AsiaBeijing · China73 works · 60,539 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 worksDependency parsing · 131 works

Cite

Cite as Yuval Marton, Nizar Habash, Owen Rambow and Sarah Alkhulani. 2013. SPMRL‘13 Shared Task System: The CADIM Arabic Dependency Parser. Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{marton2013spmrl,
  title = {{SPMRL‘13 Shared Task System: The CADIM Arabic Dependency Parser}},
  author = {Yuval Marton, Nizar Habash, Owen Rambow and Sarah Alkhulani},
  booktitle = {Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages},
  year = {2013},
  doi = {10.18653/v1/w13-4910},
  url = {https://doi.org/10.18653/v1/w13-4910}
}

References

20 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sarah Alkuhlani and Nizar Habash. 2011. A corpus for modeling morpho-syntactic agreement in Arabic: gender, number and rationality. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL), Portland, Oregon, USA.
  2. Sarah Alkuhlani and Nizar Habash. 2012. Identifying broken plurals, irregular gender, and rationality in Arabic text. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 675–685. Association for Computational Linguistics.
  3. Michael Collins, Jan Hajic, Lance Ramshaw, and Christoph Tillmann. 1999. A statistical parser for Czech. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL), pages 505–512, College Park, Maryland, USA, June.
  4. Mona Diab, Nizar Habash, Owen Rambow, and Ryan Roth. 2013. LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual. Technical Report CCLS-13-02, Center for Computational Learning Systems, Columbia University. ↗
  5. Gülsen Eryigit, Joakim Nivre, and Kemal Oflazer. 2008. Dependency parsing of Turkish. Computational Linguistics, 34(3):357–389. ↗
  6. Yoav Goldberg and Michael Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Proceedings of Human Language Technology (HLT): the North American Chapter of the Association for Computational Linguistics (NAACL), pages 742–750, Los Angeles, California. find
  7. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
  8. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 573–580, Ann Arbor, Michigan.
  9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  10. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools. The MEDAR Consortium, April. find
  11. Nizar Habash, Owen Rambow, and Ryan Roth. 2012. MADA+TOKAN Manual. Technical report, Technical Report CCLS-12-01, Columbia University. find
  12. Nizar Habash. 2007. Arabic Morphological Representations for Machine Translation. In Antal van den Bosch and Abdelhadi Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Kluwer/Springer. ↗
  13. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  14. Yuval Marton, Nizar Habash, and Owen Rambow. 2013. Dependency parsing of Modern Standard Arabic with lexical and inflectional features. Computational Linguistics, 39(1). ↗
  15. Joakim Nivre, Igor M. Boguslavsky, and Leonid K. Iomdin. 2008. Parsing the SynTagRus Treebank of Russian. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING), pages 641–648. find
  16. Joakim Nivre. 2008. Algorithms for Deterministic Incremental Dependency Parsing. Computational Linguistics, 34(4). ↗
  17. Joakim Nivre. 2009. Parsing Indian languages with MaltParser. In Proceedings of the ICON09 NLP Tools Contest: Indian Language Dependency Parsing, pages 12–18. find
  18. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proceedings of the Conference on Language Resources and Evaluation (LREC), May.
  19. Owen Rambow, Bonnie Dorr, David Farwell, Rebecca Green, Nizar Habash, Stephen Helmreich, Eduard Hovy, Lori Levin, Keith J. Miller, Teruko Mitamura, Florence Reeder, and Siddharthan Advaith. 2006. Parallel syntactic annotation of multiple languages. In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC), Genoa, Italy. find
  20. Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho Choi, Richárd Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola, Yoav Goldberg, Spence Green, Nizar Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiorkowski, Ryan Roth, Wolfgang Seeker, Yannick Versley, Veronika Vincze, Marcin Woliński, Alina Wróblewska, and Eric Villemonte de la Clérgerie. 2013. Overview of the spmrl 2013 shared task: A cross-framework evaluation of parsing morphologically rich languages. In Proceedings of the 4th Workshop on Statistical Parsing of Morphologically Rich Languages: Shared Task, Seattle, WA.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marton-2013-spmrl-13-shared-task-system/