Elamur Research⌕
☰
home / papers / 2017 / A Morphological Analyzer for Gulf Arabic Verbs

A Morphological Analyzer for Gulf Arabic Verbs

Paper · 2017 · Salam Khalifa, Sara Hassan and Nizar Habash

Summary

Most Arabic NLP tools target Modern Standard Arabic (MSA) and perform poorly on dialects. Before this work, no full analyzer for Gulf Arabic (GLF) produced segmentation, POS analysis and lemmas together. The authors build CALIMA GLF, a morphological analyzer and generator for GLF verbs with a focus on Emirati Arabic. They start from phonetic entries of the Qafisheh Gulf Arabic Dictionary and map them, first automatically and then with manual checking, into CODA GLF orthography.

The system has two parts:

  • a set of root-abstracted inflectional paradigms;
  • a lexicon that pairs each root with a paradigm.

These are compiled into the database format used by MADAMIRA and CALIMA EGY. Roots are also given phonological spellings, so the analyzer can handle raw, non-CODA text.

The authors evaluate token recall on verbs from part of a Gulf Arabic novel. With CODA input, CALIMA GLF finds the correct POS tag far more often than SAMA or CALIMA EGY. This shows that a dialect-specific lexicon built largely by rule can fill a real resource gap.

Key points

  • CALIMA GLF covers over 2,600 verbal lemmas. It is built from the Qafisheh dictionary, uses the Gumar Corpus, and follows CODA GLF conventions.

  • With CODA input, POS-tag token recall is 87.7%, compared with 60.3% for SAMA and 70.8% for CALIMA EGY. That is a gain of 27.4 and 16.9 points absolute.

  • Modeling non-CODA spelling variants adds over 11 points absolute of recall on raw input in every condition (CALIMA GLF-CODA vs CALIMA GLF).

  • The best system merges CALIMA GLF with CALIMA EGY, so it covers GLF, EGY and MSA. It reaches 87.3% POS recall on raw input and 92.3% on CODA input.

  • Error analysis shows three main error types:

    • typos: about 19%;
    • non-CODA spellings that lead to wrong segmentation: about 18%;
    • out-of-vocabulary items: about 63%.

    Merging with CALIMA EGY cuts OOV errors by 13 points absolute.

Interesting ideas

  • I1. Instead of listing every inflected form, the analyzer stores abstract paradigm templates and a lexicon of root-paradigm pairs, then compiles them into one model that works in both directions (analysis and generation). This keeps the lexicon small and makes it easy to check.
  • I2. Each root is recorded twice, once in the canonical orthography and once as it is pronounced and often written. Messy input can then be analyzed while the output stays normalized.
  • I3. The evaluation crosses two model versions (with and without variant modeling) with two input types (raw and normalized). This isolates exactly how much the robustness extension contributes.
  • I4. A narrow dialect resource is merged with broader analyzers as a fallback, and the union beats each part alone because coverage gaps are filled.
  • I5. Errors are split into typos, normalization mismatches and lexicon gaps, each with a share, and the split points directly to the next step: annotating more corpus data to grow the lemma list.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): English morphology in crate en could store lemma plus inflection-class pairs and compile analysis and generation tables from them, so every form traces back to an explicit class.
  • I2 (tokens): the tokenizer could map known nonstandard or phonetic spellings to canonical lemmas and mark the analysis as a variant, so the explanation shows what was normalized.
  • I3 (learn): evaluations could routinely run each component on raw and on cleaned input, with and without a feature, to measure real gains rather than gains that only appear on clean data.
  • I4 (tokens): a domain lexicon (for example, school-course vocabulary) could be unioned with the general English lexicon, keeping the source of each analysis visible.
  • I5 (learn): self-repair could sort failures into similar buckets, so that OOV-type failures trigger lexicon growth and spelling-type failures trigger normalization rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 works

Cite

Cite as Salam Khalifa, Sara Hassan and Nizar Habash. 2017. A Morphological Analyzer for Gulf Arabic Verbs. Proceedings of the Third Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1305

@inproceedings{khalifa2017morphological,
  title = {{A Morphological Analyzer for Gulf Arabic Verbs}},
  author = {Salam Khalifa, Sara Hassan and Nizar Habash},
  booktitle = {Proceedings of the Third Arabic Natural Language Processing Workshop},
  year = {2017},
  doi = {10.18653/v1/w17-1305},
  url = {https://doi.org/10.18653/v1/w17-1305}
}

References

34 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Belal Abuata and Asma Al-Omari. 2015. A rule-based stemmer for Arabic Gulf Dialect. Journal of King Saud University - Computer and Information Sciences, 27(2):104 – 112. ↗
  2. Ramy Eskander, Nizar Habash, Owen Rambow, and Arfath Pasha. 2016. Creating resources for dialectal arabic from a single annotation: A case study on egyptian and levantine. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3455– 3465, Osaka, Japan, December.
  3. Rania Al-Sabbagh and Roxana Girju. 2012. A supervised POS tagger for written Arabic social networking corpora. In Jeremy Jancsary, editor, Proceedings of KONVENS 2012, pages 39–52. ÖGAI, September. Main track: oral presentations. find
  4. Hassan Gadalla, Hanaa Kilany, Howaida Arram, Ashraf Yacoub, Alaa El-Habashi, Amr Shalaby, Krisjanis Karins, Everett Rowson, Robert MacIntyre, Paul Kingsbury, David Graff, and Cynthia McLemore. 1997. CALLHOME Egyptian Arabic Transcripts. In Linguistic Data Consortium, Philadelphia. find
  5. Faisal Al-Shargi, Aidan Kaplan, Ramy Eskander, Nizar Habash, and Owen Rambow. 2016. A Morphologically Annotated Corpus and a Morphological Analyzer for Moroccan and Sanaani Yemeni Arabic. In Proceedings of the International Conference on Mohamed Maamouri, Ann Bies, Seth Kulick, Dalila Tabessi, and Sondos Krouna. 2012a. Egyptian Arabic Treebank DF Parts 1-8 V2.0 - LDC catalog numbers LDC2012E93, LDC2012E98, LDC2012E89, LDC2012E99, LDC2012E107, LDC2012E125, LDC2013E12, LDC2013E21. find
  6. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
  7. Nizar Habash and Owen Rambow. 2006. MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects. In Proceedings of ACL, pages 681– 688, Sydney, Australia. ↗
  8. Nizar Habash and Ryan Roth. 2009. CATiB: The Columbia Arabic Treebank. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 221–224, Suntec, Singapore. ↗
  9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  10. N. Habash, R. Eskander, and A. Hawwari. 2012a. A Morphological Analyzer for Egyptian Arabic. In NAACL-HLT 2012 Workshop on Computational Morphology and Phonology (SIGMORPHON2012), pages 1–9.
  11. Nizar Habash, Mona T Diab, and Owen Rambow. 2012b. Conventional Orthography for Dialectal Arabic. In LREC, pages 711–718.
  12. Clive Holes. 1990. Gulf Arabic. Croom Helm Descriptive Grammars. Routledge, London / New York. find
  13. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a Corpus for Palestinian Arabic: a Preliminary Study. ANLP 2014, page 18.
  14. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2016. Curras: an annotated corpus for the Palestinian Arabic dialect. Language Resources and Evaluation, pages 1–31. ↗
  15. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016. A Large Scale Corpus of Gulf Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia.
  16. H. Kilany, H. Gadalla, H. Arram, A. Yacoub, A. El-Habashi, and C. McLemore. 2002. Egyptian Colloquial Arabic Lexicon. LDC catalog number LDC99L22. find
  17. Mohamed Maamouri and Christopher Cieri. 2002. Resources for Arabic Natural Language Processing. In International Symposium on Processing Arabic, volume 1. find
  18. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  19. Mohamed Maamouri, Ann Bies, Tim Buckwalter, Mona Diab, Nizar Habash, Owen Rambow, and Dalila Tabessi. 2006. Developing and Using a Pilot Dialectal Arabic Treebank. In Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC06.
  20. Mohamed Maamouri, Sondos Krouna, Dalila Tabessi, Nadia Hamrouni, and Nizar Habash. 2012b. Egyptian Arabic Morphological Annotation Guidelines. find
  21. Mohamed Maamouri, Ann Bies, Seth Kulick, Michael Ciul, Nizar Habash, and Ramy Eskander. 2014. Developing an Egyptian Arabic Treebank: Impact of Dialectal Morphology on Annotation and Tool Development. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Association (ELRA). find
  22. Abir Masmoudi, Mariem Ellouze Khmekhem, Yannick Esteve, Lamia Hadrich Belguith, and Nizar Habash. 2014. A corpus and phonetic dictionary for tunisian arabic speech recognition. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Association (ELRA).
  23. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. Machine translation experiments on padic: A parallel arabic dialect corpus. In The 29th Pacific Asia conference on language, information and computation.
  24. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of LREC, Reykjavik, Iceland.
  25. Hamdi A Qafisheh. 1977. A Short Reference Grammar of Gulf Arabic. find
  26. H.A. Qafisheh. 1997. NTC’s Gulf Arabic-English dictionary. NTC Pub. Group. find
  27. Houda Saadane and Nizar Habash. 2015. A Conventional Orthography for Algerian Arabic. In ANLP Workshop 2015, page 69.
  28. Wael Salloum and Nizar Habash. 2011. Dialectal to Standard Arabic Paraphrasing to Improve Arabic-English Statistical Machine Translation. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties, pages 10–21, Edinburgh, Scotland. ↗
  29. Kamel Smaı̈li, Mourad Abbas, Karima Meftouh, and Salima Harrat. 2014. Building resources for Algerian Arabic dialects. In 15th Annual Conference of the International Communication Association Interspeech. find
  30. Otakar Smrž and Jan Hajič. 2006. The Other Arabic Treebank: Prague Dependencies and Functions. In Ali Farghaly, editor, Arabic Computational Linguistics: Current Implementations. CSLI Publications. find
  31. Houcemeddine Turki, Emad Adel, Tariq Daouda, and Nassim Regragui. 2016. A Conventional Orthography for Maghrebi Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia. find
  32. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large scale Arabic error annotation: Guidelines and framework. In International Conference on Language Resources and Evaluation (LREC 2014). ↗
  33. Omar F Zaidan and Chris Callison-Burch. 2011. The Arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37–41. Association for Computational Linguistics.
  34. Ines Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith, and Nizar Habash. 2014. A Conventional Orthography for Tunisian Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khalifa-2017-morphological-analyzer-gulf-arabic-verbs/