Elamur Research⌕
☰
home / papers / 2016 / DALILA: The Dialectal Arabic Linguistic Learning Assistant

DALILA: The Dialectal Arabic Linguistic Learning Assistant

Summary

Learners of Arabic who read real online text keep running into Dialectal Arabic (DA). Tools built for Modern Standard Arabic (MSA) handle it poorly, and general machine translators often fail on dialect words, sometimes just transliterating them. The authors built DALILA, a Google Chrome extension. A user double-clicks a word on any web page and gets a pop-up with its morphological analysis and an English gloss.

The backend is a JavaScript port of the ALMOR analysis algorithm, which descends from BAMA. It reads the Egyptian CALIMA-ARZ morphological database, converted into JSON files for prefixes, stems and suffixes. It ranks candidate analyses with unigram POS and lemma models estimated from the Penn Arabic Treebank. Because everything runs on the client, the tool works offline.

The authors checked 1,000 Egyptian blog words by hand. DALILA reached 88.5% POS accuracy and 72.7% gloss accuracy. The much heavier MADAMIRA-EGY tagger reached 93.3% and 79.5%. The authors present this modest gap as an acceptable price for a far simpler system that ordinary users can install.

Key points

  • Earlier work cited in the paper shows the size of the gap that motivates DALILA:
    • MSA analyzers cover only about 60% of Levantine verbs and 64% of Egyptian words.
    • Around 42% of sampled Arabic Facebook posts are dialectal.
  • The backend combines the ALMOR algorithm (BAMA-based) with the CALIMA-ARZ Egyptian database, exported as separate JSON dictionaries for prefixes, stems and suffixes.
  • Disambiguation uses two unigram models built from PATB parts 1–3, using the Diab et al. (2013) data split:
    • a POS model with 3,016 entries;
    • a lemma model with 12,823 entries.
  • On 1,000 manually checked words from an Egyptian blog:
    • DALILA scored 88.50 POS and 72.70 English gloss;
    • MADAMIRA-EGY scored 93.30 POS and 79.50 English gloss.
  • Planned future work:
    • add more dialect databases;
    • use context-aware disambiguation (the current version analyzes isolated words);
    • show ambiguity and pronunciation in the pop-up.

Interesting ideas

  • I1. The analyzer enumerates every prefix+stem+suffix split, including empty affixes, and keeps only splits whose parts all appear in the lexicons; this makes the candidate space explicit and auditable. — For Mova (tokens): An affix-split fallback for unknown English words (un-, -ness, -ize) would yield traceable lemma and POS candidates instead of a blind guess.
  • I2. Among the valid analyses, DALILA picks one using only context-free unigram priors over POS and lemma; this is a cheap baseline whose every decision can be explained by a single lexical frequency. — For Mova (tokens): Unigram lemma/POS priors could act as a transparent tie-breaker, and as a reference point that shows how much the contextual tagger actually adds.
  • I3. The algorithm stays fixed while the morphological database is a swappable, static JSON resource, so a new dialect means a new data file rather than new code. — For Mova (world): Compiled knowledge such as verb classes or regional lexical variants could ship as versioned data files behind a stable lookup engine.
  • I4. The lightweight tool is scored against a heavyweight tagger on the same hand-checked, out-of-domain sample, and the accuracy gap is framed as a cost-versus-complexity tradeoff. — For Mova (learn): Each simplified component could report its loss against the strongest available witness on identical held-out text, so simplifications are accepted knowingly.
  • I5. Gloss correctness stands in for lemma correctness, which ties evaluation to whether the reader actually gets the right meaning. — For Mova (world): Lemma and sense decisions could be scored by whether the chosen WordNet gloss fits, measuring meaning rather than string identity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semitic languages

Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States278 works · 138 810 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates45 works · 2 055 citations🎓Carnegie Mellon University QatarDoha · Qatar8 works · 1 587 citations

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{khalifa2016dalila,
  title = {{DALILA: The Dialectal Arabic Linguistic Learning Assistant}},
  author = {Khalifa, Salam and Bouamor, Houda and Habash, Nizar},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/23g8y72wvrt4},
  url = {https://doi.org/10.63317/23g8y72wvrt4}
}

References

33 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al-Badrashiny, M., Eskander, R., Habash, N., and Rambow, O. (2014). Automatic Transliteration of Romanized Dialectal Arabic. CoNLL-2014, page 30. find
  2. Al-Shargi, F. and Rambow, O. (2015). DIWAN: A Dialectal Word Annotation Tool for Arabic. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 49–58, Beijing, China. find
  3. Al-Shargi, F., Kaplan, A., Eskander, R., Habash, N., and Rambow, O. (2016). A Morphologically Annotated Corpus and a Morphological Analyzer for Moroccan and San’ani Yemeni Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia. find
  4. Bouamor, H., Habash, N., and Oflazer, K. (2014). A Multidialectal Parallel Corpus of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1240–1245, Reykjavik, Iceland. find our card
  5. Brustad, K. (2000). The Syntax of Spoken Arabic: A Comparative Study of Moroccan, Egyptian, Syrian, and Kuwaiti Dialects. Georgetown University Press. find
  6. Buckwalter, T. (2004). Buckwalter Arabic Morphological Analyzer Version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. find
  7. Chapelle, C. A. (2008). Computer Assisted Language Learning. Wiley Online Library. find
  8. Diab, M., Habash, N., Rambow, O., and Roth, R. (2013). LDC Arabic treebanks and associated corpora: Data divisions manual. arXiv preprint arXiv:1309.5652. ↗
  9. Diab, M. T., Al-Badrashiny, M., Aminian, M., Attia, M., Elfardy, H., Habash, N., Hawwari, A., Salloum, W., Dasigi, P., and Eskander, R. (2014). Tharwa: A large scale dialectal arabic-standard arabic-english lexicon. In LREC, pages 3782–3789. find
  10. Elfardy, H., Al-Badrashiny, M., and Diab, M. (2014). Aida: Identifying code switching in informal arabic text. EMNLP 2014, pages 94–101. find
  11. Habash, N. and Rambow, O. (2006). MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects. In Proceedings of ACL, pages 681–688, Sydney, Australia. find our card
  12. Habash, N., Soudi, A., and Buckwalter, T. (2007). On Arabic Transliteration. In A. van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
  13. Habash, N., Eskander, R., and Hawwari, A. (2012). A Morphological Analyzer for Egyptian Arabic. In Proceedings of the Twelfth Meeting of the Special Interest Group on Computational Morphology and Phonology, pages 1–9, Montréal, Canada, June. Association for Computational Linguistics. find our card
  14. Habash, N., Roth, R., Rambow, O., Eskander, R., and Tomeh, N. (2013). Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of NAACL-HLT, pages 426–432, Atlanta, Georgia, June. find
  15. Habash, N. (2007). Arabic Morphological Representations for Machine Translation. In A. van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
  16. Habash, N. Y. (2010). Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  17. Holes, C. (1986). Variation in the Morphophonology of Arabic Dialects. Transactions of the Philological Society, 84(1):167–190. find
  18. Holes, C. (2004). Modern Arabic: Structures, Functions, and Varieties. Georgetown University Press. Revised Edition. find
  19. Huang, F. (2015). Improved Arabic Dialect Classification with Social Media Data. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2118–2126, Lisbon, Portugal. find
  20. Khalifa, S., Habash, N., Abdulrahim, D., and Hassan, S. (2016). A Large Scale Corpus of Gulf Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia. find
  21. Levy, M. and Stockwell, G. (2013). CALL Dimensions: Options and Issues in Computer-assisted Language Learning. Routledge. find
  22. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  23. Maamouri, M., Bies, A., and Kulick, S. (2006). Diacritization: A challenge to arabic treebank annotation and parsing. In Proceedings of the Conference of the Machine Translation SIG of the British Computer Society. find
  24. Maamouri, M., Bies, A., and Kulick, S. (2009). Creating a Methodology for Large-Scale Correction of Treebank Annotation: The Case of the Arabic Treebank. In Proceedings of MEDAR International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  25. Pasha, A., Al-Badrashiny, M., Kholy, A. E., Eskander, R., Diab, M., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of LREC, Reykjavik, Iceland. find our card
  26. Sadat, F., Kazemi, F., and Farzindar, A. (2014). Automatic Identification of Arabic Dialects in Social Media. In Proceedings of the Second Workshop on Natural Language Processing for Social Media (SocialNLP), pages 22–27, Dublin, Ireland. find
  27. Sajjad, H., Darwish, K., and Belinkov, Y. (2013). Translating dialectal arabic to english. In Proceedings of ACL, Sofia, Bulgaria. find
  28. Salama, A., Bouamor, H., Mohit, B., and Oflazer, K. (2014). YouDACC: the Youtube Dialectal Arabic Comment Corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1246–1251, Reykjavik, Iceland. find
  29. Salloum, W. and Habash, N. (2011). Dialectal to Standard Arabic Paraphrasing to Improve Arabic-English Statistical Machine Translation. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties, pages 10–21, Edinburgh, Scotland. find
  30. Shoufan, A. and Alameri, S. (2015). Natural Language Processing for Dialectical Arabic: A Survey. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 36–48, Beijing, China. find
  31. Watson, J. C. (2007). The Phonology and Morphology of Arabic. Oxford University Press. find
  32. Zaidan, O. F. and Callison-Burch, C. (2011). The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In Proceedings of the Association for Computational Linguistics, Portland, Oregon, USA. find our card
  33. Zbib, R., Malchiodi, E., Devlin, J., Stallard, D., Matsoukas, S., Schwartz, R., Makhoul, J., Zaidan, O. F., and Callison-Burch, C. (2012). Machine Translation of Arabic Dialects. In Proceedings of NAACL-HLT, Montréal, Canada. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khalifa-2016-dalila-dialectal-arabic-linguistic-learning/