Elamur Research⌕
☰
home / papers / 2024 / Exploiting Dialect Identification in Automatic Dialectal Text Normalization

Exploiting Dialect Identification in Automatic Dialectal Text Normalization

Paper · 2024 · Bashar Alhafni, Sarah Al-Towaity, Ziyad Fawzy et al.

Summary

Written Dialectal Arabic has no standard spelling, so the same word can show up in many spontaneous forms. Social media noise makes this worse, and the variation creates sparsity problems for NLP systems. The authors treat CODAfication, which rewrites raw dialectal text into the CODA convention, as a standalone normalization task. They use the MADAR CODA Corpus, a parallel set of raw and CODA-normalized sentences covering five city dialects. On this data they benchmark pretrained Seq2Seq models, with AraT5 as the main baseline. Their key addition is a sentence-level dialect identification (DID) label: a CAMeL Tools classifier predicts it, and it is fed to the model as a control token. Adding DID information improves results, and with the best configuration the test-set gain over the AraT5 baseline holds for every dialect. Feeding gold dialect labels instead of predicted ones barely helps, so a reliable off-the-shelf classifier is enough. The work matters for tasks where dialect is the output, such as speech recognition, because those systems need a consistent target spelling to evaluate against. Code, data and models are publicly released.

Key points

  • Data: The MADAR CODA Corpus has 10,000 sentences from Beirut, Cairo, Doha, Rabat and Tunis, each paired with its CODA form. CODA here means the unified CODA* guidelines.
  • Dialect labels: DID comes from the CAMeL Tools implementation of Salameh et al. (2018). It reaches 92.1% accuracy on Dev, and most of its mistakes fall on sentences that lack distinctive dialect cues.
  • Dev results: The best Dev system is AraT5 with a City DID control token, which gains up to 3.11 F0.5 for Beirut. Cairo and Tunis drop slightly because precision falls, which suggests the model makes extra unnecessary rewrites.
  • Test results: The best Test system uses a DA Phrase DID control token and beats the baseline for all five dialects, with the largest gain of 3.58 F0.5 for Doha.
  • Error analysis: The authors examined 100 of the 471 erroneous Dev outputs. The breakdown is:
    • 46% non-CODA but plausible spellings
    • 19% hallucinations
    • 13% valid alternative spellings
    • 9% deletions
    • 9% related hallucinations
    • 4% punctuation errors

Interesting ideas

  • I1. Prefixing the input with a predicted variety label lets a single normalization model adjust its rewrites to the source dialect, without training a separate model per variety.
  • I2. Rerunning with gold auxiliary labels gives an oracle ceiling, which shows how much the upstream classifier's mistakes actually cost the downstream task.
  • I3. The error taxonomy separates genuinely acceptable alternative outputs from real mistakes, which shows that a single-reference metric overstates the error rate.
  • I4. Classifier "errors" on sentences without distinguishing cues are better treated as ambiguity, best handled with multi-label annotation rather than forced single labels.
  • I5. A precision drop on some dialects revealed over-eager rewriting, and the precision-weighted F0.5 metric makes that failure visible.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A detected register tag (for example British vs American spelling, or chat-style vs edited text) could be fed to the tokenizer and normalizer, so spelling normalization becomes variety-aware and the chosen tag is logged as an explanation.
  • I2 (learn): Each pipeline stage could be evaluated once with predicted and once with gold inputs from earlier stages, which shows where errors really propagate and where improving an upstream stage is not worth the effort.
  • I3 (learn): When silver data from several witness parsers disagree, legitimate alternative analyses could be stored as multiple references rather than counted as errors.
  • I4 (tree): During UD dialect conversion and treebank merging, sentences valid under several conventions could carry several labels, so the parser is not penalized for picking either one.
  • I5 (back): The text → graph → text compressor could be scored with a precision-weighted measure, so that unnecessary rewrites of the original wording are penalized more than missed edits.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Bashar Alhafni, Sarah Al-Towaity, Ziyad Fawzy, Fatema Nassar, Fadhl Eryani, Houda Bouamor and Nizar Habash. 2024. Exploiting Dialect Identification in Automatic Dialectal Text Normalization. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.arabicnlp-1.4

@inproceedings{alhafni2024exploiting,
  title = {{Exploiting Dialect Identification in Automatic Dialectal Text Normalization}},
  author = {Bashar Alhafni, Sarah Al-Towaity, Ziyad Fawzy, Fatema Nassar, Fadhl Eryani, Houda Bouamor and Nizar Habash},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.4},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.4}
}

References

85 references, 54 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sweta Agrawal and Marine Carpuat. 2019. Controlling text complexity in neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1549– 1564, Hong Kong, China. Association for Computational Linguistics. ↗
  2. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2021. QADI: Arabic dialect identification in the wild. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 1–10, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  3. Sweta Agrawal and Marine Carpuat. 2023. Controlling pre-trained language models for grade-specific text simplification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12807–12819, Singapore. Association for Computational Linguistics. ↗
  4. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. You tweet what you speak: A city-level dataset of Arabic dialects. In Proceedings of the 11th Language Resources and Evaluation Conference, Miyazaki, Japan. European Language Resource Association.
  5. Mohamed Al-Badrashiny and Mona Diab. 2016. LILI: A simple language independent approach for language identification. In Proceedings of the International Conference on Computational Linguistics (COLING), Osaka, Japan. find
  6. Mohamed Al-Badrashiny, Ramy Eskander, Nizar Habash, and Owen Rambow. 2014. Automatic transliteration of Romanized dialectal Arabic. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 30–38, Ann Arbor, Michigan. Association for Computational Linguistics.
  7. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The fourth nuanced Arabic dialect identification shared task. In Proceedings of ArabicNLP 2023, pages 600– 613, Singapore (Hybrid). Association for Computational Linguistics.
  8. Bashar Alhafni, Nizar Habash, and Houda Bouamor. 2022. User-centric gender rewriting. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 618–631, Seattle, United States. Association for Computational Linguistics. ↗
  9. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task. In Proceedings of The Second Arabic Natural Language Processing Conference (ArabicNLP 2024).
  10. Bashar Alhafni, Go Inoue, Christian Khairallah, and Nizar Habash. 2023. Advancements in Arabic grammatical error detection and correction: An empirical investigation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6430–6448, Singapore. Association for Computational Linguistics.
  11. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. NADI 2021: The second nuanced Arabic dialect identification shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  12. Ahmed Ali, Salam Khalifa, and Nizar Habash. 2019. Towards Variability Resistant Dialectal Speech Evaluation. In Proc. Interspeech 2019, pages 336–340. find
  13. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. NADI 2022: The third nuanced Arabic dialect identification shared task. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  14. Khalid Alnajjar and Mika Hämäläinen. 2024. Normalization of arabic dialects into modern standard arabic using. Journal of Data Mining & Digital Humanities, NLP4DH. find
  15. Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. 2018. DART: A large dataset of dialectal Arabic tweets. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan.
  16. Kathrein Abu Kwaik, Motaz Saad, Stergios Chatzikyriakidis, and Simon Dobnik. 2018. Shami: A corpus of Levantine Arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  17. Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2022. Hierarchical aggregation of dialectal data for Arabic dialect identification. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4586–4596, Marseille, France. European Language Resources Association.
  18. Abdel-Rahman Abu-Melhim. 1991. Code-switching and linguistic accommodation in Arabic. In Proceed-Reem Bassiouney. 2009. Arabic Sociolinguistics: Topics in Diglossia, Gender, Identity, and Politics. Georgetown University Press. ↗
  19. Fadhl Eryani, Nizar Habash, Houda Bouamor, and Salam Khalifa. 2020. A spelling correction corpus for multiple Arabic dialects. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4130–4138, Marseille, France. European Language Resources Association.
  20. Riadh Belkebir and Nizar Habash. 2021. Automatic error type annotation for Arabic. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 596–606, Online. Association for Computational Linguistics. ↗
  21. Ramy Eskander, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing spontaneous orthography. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 585–595, Atlanta, Georgia. find
  22. Houda Bouamor, Hanan Alshikhabobakr, Behrang Mohit, and Kemal Oflazer. 2014. A Human Judgement Corpus and a Metric for Arabic MT Evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 207–213, Doha, Qatar.
  23. Asma Etman and Louis Beex. 2015. Language and Dialect Identification: A Survey. In Proceedings of the Intelligent Systems Conference (IntelliSys), London, UK. find
  24. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  25. Charles F Ferguson. 1959. Diglossia. Word, 15(2):325– 340. find
  26. Nizar Habash, Mona Diab, and Owen Rambow. 2012a. Conventional Orthography for Dialectal Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 711–718, Istanbul, Turkey.
  27. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR Shared Task on Arabic Fine-Grained Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP19), Florence, Italy.
  28. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012b. A Morphological Analyzer for Egyptian Arabic. In Proceedings of the Workshop of the Special Interest Group on Computational Morphology and Phonology (SIGMORPHON), pages 1–9, Montréal, Canada.
  29. Ryan Cotterell and Chris Callison-Burch. 2014. A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 241–245, Reykjavik, Iceland.
  30. Nizar Habash, Salam Khalifa, Fadhl Eryani, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified Guidelines and Resources for Arabic Dialect Orthography. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
  31. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–572, Montréal, Canada. Association for Computational Linguistics. find
  32. Nizar Habash and David Palfreyman. 2022. ZAEBUC: An annotated Arabic-English bilingual writer corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 79–88, Marseille, France. European Language Resources Association. find
  33. Mona T Diab, Mohamed Al-Badrashiny, Maryam Aminian, Mohammed Attia, Heba Elfardy, Nizar Habash, Abdelati Hawwari, Wael Salloum, Pradeep Dasigi, and Ramy Eskander. 2014. Tharwa: A Large Scale Dialectal Arabic-Standard Arabic-English Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 3782–3789, Reykjavik, Iceland.
  34. Nizar Habash, Owen Rambow, Mona Diab, and Reem Kanjawi-Faraj. 2008. Guidelines for Annotation of Arabic Dialectness. In Proceedings of the Workshop on HLT & NLP within the Arabic World, Marrakech, Morocco. find
  35. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. Octopus: A multitask model and toolkit for Arabic natural language generation. In Proceedings of ArabicNLP 2023, pages 232–243, Singapore (Hybrid). Association for Computational Linguistics.
  36. Nizar Habash, Abdelhadi Soudi, and Timothy Buckwalter. 2007. On Arabic Transliteration, pages 15–22. Springer Netherlands, Dordrecht. find
  37. Nizar Y Habash. 2010. Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  38. Alexander Erdmann, Nizar Habash, Dima Taji, and Houda Bouamor. 2017. Low resourced machine translation via morpho-syntactic modeling: The case of dialectal Arabic. In Proceedings of Machine Translation Summit XVI: Research Track, pages 185–200, Nagoya Japan. ↗
  39. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2016. Curras: an annotated corpus for the Palestinian Arabic dialect. Language Resources and Evaluation, pages 1–31. ↗
  40. Serena Jeblee, Weston Feely, Houda Bouamor, Alon Lavie, Nizar Habash, and Kemal Oflazer. 2014. Domain and dialect adaptation for machine translation into egyptian Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 196–206, Doha, Qatar.
  41. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. 2014. The first QALB shared task on automatic text correction for Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 39–47, Doha, Qatar.
  42. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2016. Google’s multilingual neural machine translation system: Enabling zero-shot translation. arXiv preprint arXiv:1611.04558. ↗
  43. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2022. AraT5: Text-to-text transformers for Arabic language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 628–647, Dublin, Ireland. Association for Computational Linguistics.
  44. Moussa Kamal Eddine, Nadi Tomeh, Nizar Habash, Joseph Le Roux, and Michalis Vazirgiannis. 2022. AraBART: a pretrained Arabic sequence-to-sequence model for abstractive summarization. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 31–42, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  45. Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Controlling the formality of machine translation output. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2814–2819, Copenhagen, Denmark. Association for Computational Linguistics. find
  46. Xing Niu, Sudha Rao, and Marine Carpuat. 2018. Multi-task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1008–1021, Santa Fe, New Mexico, USA. Association for Computational Linguistics. ↗
  47. Amr Keleg and Walid Magdy. 2023. Arabic dialect identification under scrutiny: Limitations of single-label classification. In Proceedings of ArabicNLP 2023, pages 385–398, Singapore (Hybrid). Association for Computational Linguistics.
  48. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016. A Large Scale Corpus of Gulf Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Portorož, Slovenia.
  49. Ossama Obeid, Go Inoue, and Nizar Habash. 2022. Camelira: An Arabic multi-dialect morphological disambiguator. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 319–326, Abu Dhabi, UAE. Association for Computational Linguistics.
  50. Salam Khalifa, Nizar Habash, Fadhl Eryani, Ossama Obeid, Dana Abdulrahim, and Meera Al Kaabi. 2018. A morphologically annotated corpus of Emirati Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan.
  51. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  52. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2020. Morphological analysis and disambiguation for Gulf Arabic: The interplay between resources and methods. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3895–3904, Marseille, France. European Language Resources Association.
  53. Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR – grammatical error correction: Tag, not rewrite. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163–170, Seattle, WA, USA → Online. Association for Computational Linguistics. find
  54. Sang Kwon, Gagan Bhatia, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. Beyond English: Evaluating LLMs for Arabic grammatical error correction. In Proceedings of ArabicNLP 2023, pages 101–119, Singapore (Hybrid). Association for Computational Linguistics. find
  55. Mohamed Maamouri, Ann Bies, Seth Kulick, Michael Ciul, Nizar Habash, and Ramy Eskander. 2014. Developing an Egyptian Arabic Treebank: Impact of dialectal morphology on annotation and tool development. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Association (ELRA). find
  56. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014a. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland. European Language Resources Association (ELRA).
  57. Karen McNeil and Miled Faiza. 2011. Tunisian Arabic Corpus : Creating a Written Corpus of an "Unwritten" Language. In Proceedings of the Workshop on Arabic Corpus Linguistics (WACL). find
  58. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014b. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1094–1101, Reykjavik, Iceland.
  59. Francisco Rangel, Paolo Rosso, Anis Charfi, Wajdi Zaghouani, Bilal Ghanem, and Javier Sánchez-Junquera. 2019. On the author profiling and deception detection in Arabic shared task at FIRE. In Proceedings of the 11th Annual Meeting of the Forum for Information Retrieval Evaluation, pages 7–9. find
  60. Alla Rozovskaya, Houda Bouamor, Nizar Habash, Wajdi Zaghouani, Ossama Obeid, and Behrang Mohit. 2015. The second QALB shared task on automatic text correction for arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 26–35, Beijing, China.
  61. Houda Saadane and Nizar Habash. 2015. A Conventional Orthography for Algerian Arabic. In Proceedings of the Second Workshop on Arabic Natural Language Processing, page 69, Beijing, China.
  62. Abdulwahab Sahyoun and Shady Shehata. 2023. AraDi-aWER: An explainable metric for dialectical Arabic ASR. In Proceedings of the Second Workshop on NLP Applications to Field Linguistics, pages 64–73, Dubrovnik, Croatia. Association for Computational Linguistics. find
  63. Ahmed Salama, Houda Bouamor, Behrang Mohit, and Kemal Oflazer. 2014. YouDACC: the Youtube Dialectal Arabic Comment Corpus. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1246–1251, Reykjavik, Iceland.
  64. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332– 1344, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  65. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. 2014. Sentence level dialect identification for machine translation system selection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 772–778, Baltimore, Maryland. Association for Computational Linguistics. find
  66. Wael Salloum and Nizar Habash. 2011. Dialectal to Standard Arabic Paraphrasing to Improve Arabic-English Statistical Machine Translation. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties, pages 10–21, Edinburgh, Scotland. ↗
  67. Wael Salloum and Nizar Habash. 2012. Elissa: A Dialectal to Standard Arabic Machine Translation System. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 385– 392, Mumbai, India.
  68. Rico Sennrich and Barry Haddow. 2016. Linguistic input features improve neural machine translation. In Proceedings of the First Conference on Machine Translation: Volume 1, Research Papers, volume 1, pages 83–91.
  69. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, San Diego, California. Association for Computational Linguistics.
  70. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725. Association for Computational Linguistics.
  71. K. Shaalan, H.M.A. Bakr, and I. Ziedan. 2007. Transferring Egyptian colloquial dialect into modern standard Arabic. In Proceedings of the Conference on Recent Advances in Natural Language Processing (RANLP), pages 525–529. find
  72. Ali Shazal, Aiza Usman, and Nizar Habash. 2020. A unified model for Arabizi detection and transliteration using sequence-to-sequence models. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 167–177, Barcelona, Spain (Online). Association for Computational Linguistics.
  73. Houcemeddine Turki, Emad Adel, Tariq Daouda, and Nassim Regragui. 2016. A Conventional Orthography for Maghrebi Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Portorož, Slovenia. find
  74. Daniel Watson, Nasser Zalmout, and Nizar Habash. 2018. Utilizing character and word embeddings for text normalization with sequence-to-sequence models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 837–843, Brussels, Belgium. Association for Computational Linguistics. ↗
  75. Janet CE Watson. 2007. The Phonology and Morphology of Arabic. Oxford University Press. find
  76. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. ↗
  77. Daiki Yanamoto, Tomoki Ikawa, Tomoyuki Kajiwara, Takashi Ninomiya, Satoru Uchida, and Yuki Arase. 2022. Controllable text simplification with deep reinforcement learning. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 398–404, Online only. Association for Computational Linguistics. find
  78. Wajdi Zaghouani and Anis Charfi. 2018. ArapTweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  79. Wajdi Zaghouani, Nizar Habash, Houda Bouamor, Alla Rozovskaya, Behrang Mohit, Abeer Heider, and Kemal Oflazer. 2015. Correction annotation for non-native Arabic texts: Guidelines and corpus. In Proceedings of the Linguistic Annotation Workshop (LAW), pages 129–139.
  80. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland. ↗
  81. Omar F Zaidan and Chris Callison-Burch. 2011. The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic With High Dialectal Content. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), pages 37–41.
  82. Nasser Zalmout, Alexander Erdmann, and Nizar Habash. 2018. Noise-robust morphological disambiguation for dialectal Arabic. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 953–964, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  83. Nasser Zalmout and Nizar Habash. 2020. Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8297–8307, Online. Association for Computational Linguistics.
  84. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine Translation of Arabic Dialects. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 49–59, Montréal, Canada.
  85. Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith, and Nizar Habash. 2014. A conventional orthography for Tunisian Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alhafni-2024-exploiting-dialect-identification-automatic-dialectal/