Elamur Research⌕
☰
home / papers / 2018 / Complementary Strategies for Low Resourced Morphological Modeling

Complementary Strategies for Low Resourced Morphological Modeling

Paper · 2018 · Alexander Erdmann and Nizar Habash

Summary

Morphologically rich languages spread each lexeme across many inflected forms. Individual word types therefore occur rarely, and models get little contextual evidence for each one. The authors ask which of two common remedies is more worthwhile for a low-resourced language: hand-written, context-free morphological knowledge, or learning methods built to cope with sparsity. They use Modern Standard Arabic as a proxy for under-resourced Arabic dialects. The task is clustering words into paradigms, meaning sets of forms that share a lemma. Training uses only an unannotated corpus of about 13 million words, and SAMA analyses serve solely as the evaluation reference. They compare embeddings that exploit subword strings and noise reduction with a very small set of de-lexicalized rules for closed-class morphology. Those rules propose candidate orthographic roots. When the rules filter the paradigm mates suggested by the best embedding model, the system beats either part alone by about 20 points absolute and reaches an average maximum F-score of 71.5%. The result points to a cheap, practical recipe for building morphological resources for dialects and other low-resource languages.

Key points

  • Training data: 500,000 sentences (~13M words) sampled from the Almahairi et al. (2016) corpus, a size meant to match what many dialects have. SAMA, applied out of context, defines the gold lemma sets.
  • EVAL contains 1,036 words from 91 paradigms. Paradigms were drawn from 10 frequency buckets, and whole clans were included together. 36% of EVAL words are lemma-ambiguous, compared with 39% in the full corpus.
  • The paper defines a hierarchy of ambiguity:
    • subparadigm (words with the same lemma ambiguity);
    • paradigm;
    • family (paradigms linked by derivational ambiguity from one root);
    • clan (families linked by coincidental ambiguity, e.g. b in brd read as a clitic preposition or as a root letter).
  • Evaluation arranges words in a tree. For each word, the best F-score its branch reaches against its true paradigm mates at any depth is recorded, and these maxima are averaged. The depth at which templatic versus concatenative mates join is also reported.
  • The best system uses the DELEX rules to prune FT+ embedding neighbours, reaching 71.5%. Templatic processes such as broken plurals (about 55% of Arabic plurals) remain the main weakness.

Interesting ideas

  • I1. Rules that over-generate candidates with near-perfect recall work as a safe filter on a learned model: they remove implausible options without adding errors of their own.
  • I2. The system keeps all plausible analyses (several orthographic roots per word) instead of committing to one, so ambiguity is not resolved too early.
  • I3. The evaluation set is sampled across frequency buckets and closed under ambiguity links. This keeps test items independent of excluded ones and keeps the ambiguity distribution realistic.
  • I4. An oracle-style metric (the best F-score over all depths) measures how good a representation could be, separately from how its cutoff is chosen.
  • I5. Separating derivational ambiguity (shared root) from coincidental ambiguity (unrelated homographs) clarifies which errors are linguistically meaningful.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Cheap English suffix rules and closed irregular lists could produce every possible lemma candidate, and these could prune lemma or tagger proposals coming from statistical components.
  • I2 (tokens): The morphology crate could output sets of analyses for forms like "saw" or "leaves" and leave the choice to the parser and the frames.
  • I3 (learn): Test sets for lemmatization or word-sense evaluation could include every word linked by shared lemma ambiguity, so the scores are neither inflated nor leaky.
  • I4 (learn): Reporting ceiling scores alongside thresholded scores would show whether errors come from the representation itself or from the decision rule.
  • I5 (world): Ontology links built from WordNet could mark homonymy and derivational relatedness separately, so the world model does not merge unrelated senses of the same word.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Morphology · 405 worksLow-resource languages · 267 works

Cite

Cite as Alexander Erdmann and Nizar Habash. 2018. Complementary Strategies for Low Resourced Morphological Modeling. Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-5806

@inproceedings{erdmann2018complementary,
  title = {{Complementary Strategies for Low Resourced Morphological Modeling}},
  author = {Alexander Erdmann and Nizar Habash},
  booktitle = {Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology},
  year = {2018},
  doi = {10.18653/v1/w18-5806},
  url = {https://doi.org/10.18653/v1/w18-5806}
}

References

66 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xinxiong Chen, Zhiyuan Liu, and Maosong Sun. 2014. A unified model for word sense representation and disambiguation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1025–1035.
  2. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  3. David Chiang, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. 2006. Parsing Arabic Dialects. In Proceedings of EACL, Trento, Italy. EACL. ↗
  4. Faisal Al-Shargi, Aidan Kaplan, Ramy Eskander, Nizar Habash, and Owen Rambow. 2016. Morphologically annotated corpora and morphological analyzers for Moroccan and Sanaani Yemeni Arabic. In 10th Language Resources and Evaluation Conference (LREC 2016).
  5. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, and Mans Hulden. 2017. CoNLL-SIGMORPHON 2017 shared task: Universal morphological reinflection in 52 languages. CoRR, abs/1706.09031. ↗
  6. Sarah Alkuhlani and Nizar Habash. 2011. A Corpus for Modeling Morpho-Syntactic Agreement in Arabic: Gender, Number and Rationality. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL’11), Portland, Oregon, USA.
  7. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. find
  8. The SIGMORPHON 2016 shared task— morphological reinflection. In Proceedings of the 2016 Meeting of SIGMORPHON, Berlin, Germany. Association for Computational Linguistics. find
  9. Amjad Almahairi, Kyunghyun Cho, Nizar Habash, and Aaron C. Courville. 2016. First result on Arabic neural machine translation. CoRR, abs/1606.02680. ↗
  10. Ryan Cotterell and Hinrich Schütze. 2015. Morphological word-embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1287–1292. find
  11. Oded Avraham and Yoav Goldberg. 2017. The interplay of semantics and morphology in word embeddings. arXiv preprint arXiv:1704.01938. ↗
  12. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. 2014. A large scale Arabic sentiment lexicon for Arabic opinion mining. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 165–173.
  13. Mathias Creutz and Krista Lagus. 2005. Unsupervised morpheme segmentation and morphology induction from text corpora using Morfessor 1.0. Helsinki University of Technology. find
  14. Kareem Darwish. 2002. Building a shallow Arabic morphological analyzer in one day. In Computational Approaches to Semitic Languages, an ACL’02 Workshop, pages 47–54, Philadelphia, PA. find
  15. Sergey Bartunov, Dmitry Kondrashkin, Anton Osokin, and Dmitry Vetrov. 2016. Breaking sticks and ambiguities with adaptive skip-gram. In Artificial Intelligence and Statistics, pages 130–138. ↗
  16. Ahmed El Kholy and Nizar Habash. 2012. Orthographic and morphological processing for English– Arabic statistical machine translation. Machine Translation, 26(1-2):25–45. ↗
  17. Kenneth Beesley. 1998. Arabic morphology using only finite-state operations. In Proceedings of the Workshop on Computational Approaches to Semitic Languages, pages 50–7, Montereal.
  18. Mohammed Elrazzaz, Shady Elbassuoni, Khaled Shaban, and Chadi Helwe. 2017. Methodical evaluation of Arabic word embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 454–458, Vancouver, Canada.
  19. Toms Bergmanis and Sharon Goldwater. 2017. From segmentation to analyses: A probabilistic model for unsupervised morphology induction. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 337–346. find
  20. Alexander Erdmann, Nizar Habash, Dima Taji, and Houda Bouamor. 2017. Low resourced machine translation via morpho-syntactic modeling: The case of dialectal arabic. In Proceedings of MT Summit 2017, Nagoya, Japan. ↗
  21. William Black, Sabri Elkateb, Horacio Rodriguez, Musa Alkhalifa, Piek Vossen, Adam Pease, and Christiane Fellbaum. 2006. Introducing the Arabic wordnet project. In Proceedings of the third international WordNet conference, pages 295–300. Citeseer. find
  22. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. ↗
  23. Alexander Erdmann, Nasser Zalmout, and Nizar Habash. 2018. Addressing noise in multidialectal word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 558–565. find
  24. Kris Cao and Marek Rei. 2016. A joint model for word embedding and word morphology. arXiv preprint arXiv:1606.02601. ↗
  25. Ramy Eskander, Nizar Habash, and Owen Rambow. 2013. Automatic extraction of morphological lexicons from morphologically annotated corpora. In representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873–882. Association for Computational Linguistics. find
  26. Go Inoue, Hiroyuki Shindo, and Yuji Matsumoto. 2017. Joint prediction of morphosyntactic categories for fine-grained Arabic part-of-speech tagging exploiting tag dictionary information. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 421–431, Vancouver, Canada. Association for Computational Linguistics.
  27. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a corpus for Palestinian Arabic: a preliminary study. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 18–27.
  28. Salam Khalifa, Nizar Habash, Fadhl Eryani, Ossama Obeid, Dana Abdulrahim, and Meera Al Kaabi. 2018. A Morphologically Annotated Corpus of Emirati Arabic. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
  29. Salam Khalifa, Sara Hassan, and Nizar Habash. 2017. A morphological analyzer for Gulf Arabic verbs. WANLP 2017 (co-located with EACL 2017), page 35.
  30. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2016. Yamama: Yet another multi-dialect Arabic morphological analyzer. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 223–227.
  31. Shereen Khoja and Roger Garside. 1999. Stemming Arabic text. Lancaster, UK, Computing Department, Lancaster University. find
  32. Jiaming Luo, Karthik Narasimhan, and Regina Barzilay. 2017. Unsupervised learning of morphological forests. arXiv preprint arXiv:1702.07015. ↗
  33. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  34. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. ↗
  35. Daniel Müllner et al. 2013. fastcluster: Fast hierarchical, agglomerative clustering routines for R and Python. Journal of Statistical Software, 53(9):1–18. ↗
  36. Karthik Narasimhan, Regina Barzilay, and Tommi Jaakkola. 2015. An unsupervised method for uncovering morphological chains. arXiv preprint arXiv:1503.02335. ↗
  37. Robert Parker, David Graff, Ke Chen, Junbo Kong, and Kazuaki Maeda. 2011. Arabic Gigaword Fifth Edition. LDC catalog number No. LDC2011T11, ISBN 1-58563-595-2. ↗
  38. Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1032–1043. find
  39. Ramy Eskander, Nizar Habash, Owen Rambow, and Arfath Pasha. 2016a. Creating resources for Dialectal Arabic from a single annotation: A case study on Egyptian and Levantine. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3455– 3465, Osaka, Japan.
  40. Ramy Eskander, Owen Rambow, and Tianchun Yang. 2016b. Extending the use of adaptor grammars for unsupervised morphological segmentation of unseen languages. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 900–910. find
  41. Manaal Faruqui, Ryan McDonald, and Radu Soricut. 2016. Morpho-syntactic lexicon generation using graph-based semi-supervised learning. Transactions of the Association for Computational Linguistics, 4:1–16. ↗
  42. Sharon Gieske. 2017. Inflecting verbs with word embeddings: A systematic investigation of morphological information captured by German verb embeddings. Master’s thesis, University of Amsterdam. find
  43. Yoav Goldberg. 2016. A primer on neural network models for natural language processing. J. Artif. Intell. Res.(JAIR), 57:345–420. ↗
  44. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic morphological analyzer (SAMA) version 3.1. Linguistic Data Consortium LDC2009E73. find
  45. Amaru Cuba Gyllensten and Magnus Sahlgren. 2015. Navigating the semantic horizon using relative neighborhood graphs. arXiv preprint arXiv:1501.02670. ↗
  46. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for Egyptian Arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9. Association for Computational Linguistics.
  47. Nizar Habash and Owen Rambow. 2006. MAGEAD: a morphological analyzer and generator for the Arabic dialects. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 681–688. Association for Computational Linguistics. ↗
  48. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  49. Nizar Y Habash. 2010. Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  50. Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. Improving word Lifu Tu, Kevin Gimpel, and Karen Livescu. 2017. Learning to embed words in context for syntactic tasks. arXiv preprint arXiv:1706.02807. ↗
  51. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of LREC, Reykjavik, Iceland.
  52. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. ↗
  53. Mohammad Sadegh Rasooli, Thomas Lippincott, Nizar Habash, and Owen Rambow. 2014. Unsupervised morphology-based vocabulary expansion. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1349–1359. find
  54. Joe H Ward Jr. 1963. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 58(301):236–244. ↗
  55. Nasser Zalmout, Alexander Erdmann, and Nizar Habash. 2018. Noise-robust morphological disambiguation for dialectal Arabic. In Proceedings of the 16th Meeting of the North American Chapter of the Association for Computational Linguistics/Human Language Technologies Conference (HLT-NAACL18), New Orleans, Louisiana, USA.
  56. Nasser Zalmout and Nizar Habash. 2017. Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 715– 724.
  57. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta. ELRA. http://is.muni.cz/ publication/884893/en. ↗
  58. Joseph Reisinger and Raymond J Mooney. 2010. Multi-prototype vector-space models of word meaning. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 109–117. Association for Computational Linguistics.
  59. Tarek Sakakini, Suma Bhat, and Pramod Viswanath. 2017. Fixing the infix: Unsupervised discovery of root-and-pattern morphology. arXiv preprint arXiv:1702.02211. ↗
  60. Ibrahim M. Saleh and Nizar Habash. 2009. Automatic extraction of lemma-based bilingual dictionaries for morphologically rich languages. In Proceedings of MT Summit, Ottawa, Canada. find
  61. Wael Salloum and Nizar Habash. 2014. ADAM: Analyzer for Dialectal Arabic Morphology. Journal of King Saud University-Computer and Information Sciences, 26(4):372–378. ↗
  62. Otakar Smrž. 2007. Functional Arabic Morphology. Formal System and Implementation. Ph.D. thesis, Charles University, Prague. find
  63. Benjamin Snyder and Regina Barzilay. 2010. Climbing the tower of Babel: Unsupervised multilingual learning. In Proceedings of the International Conference on Machine Learning (ICML-10), Haifa, Israel. find
  64. Radu Soricut and Franz Och. 2015. Unsupervised morphology induction using word embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1627–1637. find
  65. Andrew Trask, David Gilmore, and Matthew Russell. 2015. Modeling order in neural word embeddings at scale. arXiv preprint arXiv:1506.02338. ↗
  66. Inès Zribi, Mariem Ellouze, Lamia Hadrich Belguith, and Philippe Blache. 2017. Morphological disambiguation of Tunisian dialect. Journal of King Saud University-Computer and Information Sciences, 29(2):147–155. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/erdmann-2018-complementary-strategies-low-resourced-morphological/