Elamur Research⌕
☰
home / papers / 2020 / Automatic Arabic Dialect Identification Systems for Written Texts: A Survey

Automatic Arabic Dialect Identification Systems for Written Texts: A Survey

Summary

The survey covers automatic Arabic Dialect Identification (ADI) for written text: deciding which Arabic variety a piece of text is in. Dialects began appearing in writing with user-generated web content. They have no spelling standard and often mix with Modern Standard Arabic (MSA). Tools built for MSA perform clearly worse on them, so ADI is treated as a front-end step for machine translation, generation and social-media analysis.

Althobaiti organizes about a decade of work along several axes:

  • Dialect taxonomies: Glottolog and Ethnologue.
  • Granularity of labels: region, country and city.
  • Unit of analysis: token, sentence and document.
  • Features and their representations.
  • Learning methods: classical machine learning, deep networks and more complex hybrids.
  • Resources: annotated corpora, benchmarks, metrics and shared tasks.

On the fine-grained MADAR city-level benchmark, reported systems reach only about 58–66% macro F1, so the problem is far from solved. The paper closes with open issues: scarce fine-grained data, noisy online text, code switching, Arabizi, rich cliticization, and little study of how preprocessing affects results.

Key points

  • Glottolog sorts Arabic into 5 dialect groups covering 38 dialects. Ethnologue (23rd edition) lists 36, and roughly 22 of those are written in Arabic script.
  • The two Mesopotamian groups (Gilit vs. Qeltu) are distinguished by a single diagnostic: how /q/ is pronounced in the word for "I said".
  • On MADAR Corpus-26 (25 city dialects plus MSA), the systems compared score macro F1 between 58% and 65.66%. A separate MADAR Twitter corpus targets 21 country-level dialects.
  • Written Arabic may be fully diacritized, partially diacritized or not diacritized at all. Arabizi (Latin letters plus digits) adds further variation in spelling.
  • Linguistic Code Switching (LCS), where MSA and dialect mix within one utterance, has received only a handful of studies.

Interesting ideas

  • I1. Treating variety identification as a routing step placed before every downstream tool, because tools tuned to the standard variety degrade on dialects. — For Mova (tree): A cheap register/dialect detector could select the right UD conversion or parser model (e.g., informal web English vs. edited text) before gparse runs.
  • I2. Organizing labels in a coarse-to-fine hierarchy (region → country → city), so a system can stop at the level its evidence supports. — For Mova (world): Places and other ontology entities could be resolved hierarchically, committing only to the granularity the text actually justifies and recording why.
  • I3. Doing identification at the token level, not only per sentence, so that mixed-variety utterances can be labeled at all. — For Mova (tokens): Per-token origin tags (e.g., foreign word, slang, code-switched span) from the tagger would let later stages explain unusual parses instead of silently mis-analyzing them.
  • I4. Single diagnostic features, like the /q/ reflex separating Gilit and Qeltu, show that one interpretable cue can carry a whole class distinction. — For Mova (frames): Speech-act or irony detection could rely on a small set of named, inspectable cues rather than opaque scores, which keeps each decision explainable.
  • I5. The survey points out that the effect of preprocessing (normalization, stemming, spelling correction) on results was rarely isolated. — For Mova (learn): Evaluation could include ablations that switch each normalization step off, serving as controls that show which step actually earns its accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

21 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 19 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2009.12622 · arXiv 2009.12622

@misc{althobaiti2020automatic,
  title = {{Automatic Arabic Dialect Identification Systems for Written Texts: A Survey}},
  author = {Althobaiti, Maha J.},
  year = {2020},
  doi = {10.48550/arxiv.2009.12622},
  eprint = {2009.12622},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2009.12622}
}

References

142 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nizar Habash, Owen Rambow, Mona Diab, and Reem Kanjawi-Faraj. Guidelines for annotation of arabic dialectness. In Proceedings of the LREC Workshop on HLT & NLP within the Arabic world, pages 49–53, 2008. find
  2. Nizar Y Habash. Introduction to arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187, 2010. find
  3. Chiyu Zhang and Muhammad Abdul-Mageed. No army, no navy: Bert semi-supervised learning of arabic dialects. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 279–284, 2019. find
  4. Mona Diab, Nizar Habash, Owen Rambow, Mohamed Altantawy, and Yassine Benajiba. COLABA: Arabic dialect annotation and processing. In LREC workshop on Semitic language processing, pages 66–74, 2010. find
  5. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab. Code switch point detection in arabic. In International Conference on Application of Natural Language to Information Systems, pages 412–416. Springer, 2013. find
  6. Omar F Zaidan and Chris Callison-Burch. Arabic dialect identification. Computational Linguistics, 40(1):171–202, 2014. find our card
  7. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. Morphological analysis and disambiguation for dialectal arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 426–432, 2013. find
  8. Salima Harrat, Karima Meftouh, and Kamel Smaili. Machine translation for arabic dialects (survey). Information Processing & Management, 2017. find
  9. David Chiang, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. Parsing arabic dialects. In 11th Conference of the European Chapter of the Association for Computational Linguistics, pages 369–376, 2006. find our card
  10. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F Zaidan, and Chris Callison-Burch. Machine translation of arabic dialects. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 49–59. Association for Computational Linguistics, 2012. find our card
  11. Omar F Zaidan and Chris Callison-Burch. The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37–41. Association for Computational Linguistics, 2011. find our card
  12. Houda Bouamor, Nizar Habash, and Kemal Oflazer. A multidialectal parallel corpus of arabic. In LREC, pages 1240–1245, 2014. find our card
  13. Shervin Malmasi and Marcos Zampieri. Arabic dialect identification in speech transcripts. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 106–113, 2016. find
  14. Gaël de Francony, Victor Guichard, Praveen Joshi, Haithem Afli, and Abdessalam Bouchekif. Hierarchical deep learning for arabic dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 249–253, 2019. find
  15. Houda Bouamor, Sabit Hassan, and Nizar Habash. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, 2019. find our card
  16. Mona Diab and Nizar Habash. Arabic dialect processing tutorial. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Tutorial Abstracts, pages 5–6. Association for Computational Linguistics, 2007. find
  17. Harald Hammarström, Robert Forkel, and Martin Haspelmath. Glottolog 4.1. Max Planck Institute for the Science of Human History, 2018. Online at https://glottolog.org/; accessed 4 March 2018. ↗
  18. Alexander Borg. A comparative glossary of Cypriot Maronite Arabic (Arabic-English): with an introductory essay, volume 70. Brill Academic Pub, 2004. find
  19. Xenia Hadjioannou, Stavroula Tsiplakou, and Matthias Kappler. Language policy and language planning in cyprus. Current Issues in Language Planning, 12(4):503–569, 2011. find
  20. Marie Azzopardi-Alexander and Albert Borg. Maltese. Routledge, 2013. find
  21. Jonathan Owens. Arabic-based pidgins and creoles. Contact languages: A wider perspective, pages 125–172, 1997. find
  22. Inneke Hilda Werner Wellens. The Nubi language of Uganda: an Arabic creole in Africa, volume 45. Brill, 2005. find
  23. X Luffin. Kinubi. The Survey of Pidgin and Creole Languages, 3:50–53, 2013. find
  24. Terence Frederick Mitchell. Colloquial Arabic. Hodder and Stoughton, 1962. find
  25. David M. Eberhard, Gary F. Simons, and Charles D. Fennig. Ethnologue: Languages of the world. twenty-third edition. Dallas, Texas: SIL International., 2019. Online at http://www.ethnologue.com/; accessed 14 January 2019. ↗
  26. Ushari Ahmad Mahmud. Arabic in the southern Sudan: History and spread of a pidgin-creole. FAL Advertising and Printing, 1983. find
  27. Jacques Arends, Pieter Muysken, and Norval Smith. Pidgins and creoles: An introduction, volume 15. John Benjamins Publishing, 1995. find
  28. Pieter Muysken, Norval Smith, et al. The study of pidgin and creole languages. Pidgins and creoles: An introduction, pages 3–14, 1995. find
  29. Tim O’reilly. What is web 2.0. ” O’Reilly Media, Inc.”, 2009. find
  30. David Testen. Semitic Languages. Encyclopedia Britannica, 2018. Online at http://www.britannica.com/EBchecked/topic/534171/Semitic-languages; accessed 10 September 2018. ↗
  31. Nizar Habash, Abdelhadi Soudi, and Timothy Buckwalter. On arabic transliteration. In Arabic computational morphology, pages 15–22. Springer, 2007. find
  32. Mohammad Ali Yaghan. “arabizi”: A contemporary style of arabic slang. Design Issues, 24(2):39–52, 2008. find
  33. David Palfreyman and Muhamed al Khalil. A Funky Language for Teenzz to Use: Representing Gulf Arabic in Instant Messaging. Journal of computer-mediated communication, 9(1):JCMC917, 2003. find
  34. Kees Versteegh. Arabic Language. Edinburgh University Press, 2014. find
  35. Dominique Caubet. Moroccan arabic. Encyclopedia of Arabic Language and Linguistics (EALL), 3:273–287, 2008. find
  36. Joseph Aoun, Elabbas Benmamoun, and Dominique Sportiche. Agreement, word order, and conjunction in some varieties of arabic. Linguistic inquiry, pages 195–220, 1994. find
  37. Ur Shlonsky. Clause structure and word order in Hebrew and Arabic: An essay in comparative Semitic syntax. Oxford University Press, 1997. find
  38. Hassan AH Gadalla and AME Abdel-Hamid. Genitive constructions in english and arabic: A contrastive study. Bulletin of the Faculty of Arts, 6:1–64, 2000. find
  39. Mark W Cowell. Reference grammar of Syrian Arabic: Based on the dialect of Damascus (Richard Slade Harrell Arabic Series 6). Washington, DC: Georgetown University Press, 1964. find
  40. Elabbas Benmamoun. The feature structure of functional categories: A comparative study of Arabic dialects. Oxford University Press, 2000. find
  41. Alan S Kaye and Judith Rosenhouse. Arabic dialects and maltese. The Semitic languages, page 263, 2013. find
  42. Tommi Sakari Jauhiainen, Marco Lui, Marcos Zampieri, Timothy Baldwin, and Krister Lindén. Automatic language identification in texts: A survey. Journal of Artificial Intelligence Research, 65:675–782, 2019. find our card
  43. Heba Elfardy and Mona Diab. Token level identification of linguistic code switching. In Proceedings of COLING 2012: Posters, pages 287–296, 2012. find
  44. Heba Elfardy and Mona Diab. Sentence level dialect identification in arabic. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 456–461, 2013. find
  45. Clive Holes. Colloquial Arabic of the Gulf and Saudi Arabia. Routledge & Kegan Paul Books, 1984. find
  46. Clive Holes. Bahraini dialects: sectarian differences exemplified through texts. Zeitschrift für arabische Linguistik, (13):27–67, 1984. find
  47. Jeffrey Heath. Moroccan arabic phonology. Phonologies of Asia and Africa (including the Caucasus), 1:205–217, 1997. find
  48. Consortium Unicode. The character contents of the unicode standard: Technical reports and standards. Unicode, Inc, 2020. Online at http://www.unicode.org/reports/; accessed 22 February 2020. ↗
  49. Ridouane Tachicart, Karim Bouzoubaa, Si Lhoussaine Aouragh, and Hamid Jaafa. Automatic identification of moroccan colloquial arabic. In International Conference on Arabic Language Processing, pages 201–214. Springer, 2017. find
  50. Kareem Darwish. Arabizi detection and conversion to arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 217–224. Association for Computational Linguistics, 2014. find
  51. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, et al. Unified guidelines and resources for arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3628–3637, 2018. find our card
  52. Mona T Diab, Mohamed Al-Badrashiny, Maryam Aminian, Mohammed Attia, Heba Elfardy, Nizar Habash, Abdelati Hawwari, Wael Salloum, Pradeep Dasigi, and Ramy Eskander. Tharwa: A Large Scale Dialectal Arabic-Standard Arabic-English Lexicon. In LREC, pages 3782–3789, 2014. find
  53. Ryan Cotterell and Chris Callison-Burch. A multi-dialect, multi-genre corpus of informal written arabic. In LREC, pages 241–245, 2014. find
  54. Christoph Tillmann, Saab Mansour, and Yaser Al-Onaizan. Improved sentence-level arabic dialect classification. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 110–119, 2014. find
  55. Mohamed Al-Badrashiny, Heba Elfardy, and Mona Diab. Aida2: A hybrid approach for token and sentence level dialect identification in arabic. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, pages 42–51, 2015. find
  56. Fei Huang. Improved arabic dialect classification with social media data. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2118–2126, 2015. find
  57. Shervin Malmasi, Eshrag Refaee, and Mark Dras. Arabic dialect identification using a parallel multidialectal corpus. In Conference of the Pacific Association for Computational Linguistics, pages 35–53. Springer, 2015. find
  58. Leena Lulu and Ashraf Elnagar. Automatic arabic dialect classification using deep learning models. Procedia computer science, 142:262–269, 2018. find
  59. Mohamed Elaraby and Muhammad Abdul-Mageed. Deep models for arabic dialect identification on benchmarked data. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 263–274, 2018. find
  60. Karen McNeil. Tunisian arabic corpus: Creating a written corpus of an ‘unwritten’language. Arabic Corpus Linguistics, page 30, 2018. find
  61. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. Automatic identification of Arabic dialects in social media. In Proceedings of the first international workshop on Social media retrieval and analysis, pages 35–40. ACM, 2014. find
  62. Hamdy Mubarak. Dial2msa: A tweets corpus for converting dialectal arabic to modern standard arabic. In Hend Al-Khalifa, King Saud University, KSA Walid Magdy, University of Edinburgh, UK Kareem Darwish, Qatar Computing Research Institute, Qatar Tamer Elsayed, Qatar University, and Qatar, editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), OSACT2018 Workshop, pages 49–53, Paris, France, may 2018. European Language Resources Association (ELRA). find
  63. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, et al. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), pages 3387–3396, 2018. find
  64. Toshiyuki Takezawa, Genichiro Kikui, Masahide Mizushima, and Eiichiro Sumita. Multilingual spoken language corpus development for communication research. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 3, September 2007: Special Issue on Invited Papers from ISCSLP 2006, pages 303–324, 2007. find
  65. Thamar Solorio, Elizabeth Blair, Suraj Maharjan, Steven Bethard, Mona Diab, Mahmoud Ghoneim, Abdelati Hawwari, Fahad AlGhamdi, Julia Hirschberg, Alison Chang, et al. Overview for the first shared task on language identification in code-switched data. In Proceedings of the First Workshop on Computational Approaches to Code Switching, pages 62–72, 2014. find our card
  66. Giovanni Molina, Nicolas Rey-Villamizar, Thamar Solorio, Fahad AlGhamdi, Mahmoud Ghoneim, Abdelati Hawwari, and Mona Diab. Overview for the second shared task on language identification in code-switched data. In Proceedings of The EMNLP 2016 Second Workshop on Computational Approaches to Linguistic Code Switching (CALCS), pages 40–49, 2016. find
  67. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab. AIDA: Identifying code switching in informal Arabic text. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 94–101, 2014. find
  68. Mohamed Al-Badrashiny and Mona Diab. Lili: A simple language independent approach for language identification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1211–1219, 2016. find
  69. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. Discriminating between similar languages and Arabic dialect identification: A report on the third DSL shared task. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial6), pages 1–14, 2016. find our card
  70. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. Automatic dialect detection in Arabic broadcast speech. In Proceedings of Interspeech, pages 2934–2938, 2016. find
  71. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. Findings of the VarDial evaluation campaign 2017. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2017), pages 1–15, 2017. find our card
  72. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, et al. Language identification and morphosyntactic tagging: The second VarDial evaluation campaign. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 1–17, 2018. find our card
  73. Liviu P Dinu, Alina Maria Ciobanu, Marcos Zampieri, and Shervin Malmasi. Classifier ensembles for dialect and language variety identification. arXiv preprint arXiv:1808.04800, 2018. ↗
  74. Khalid Almeman and Mark Lee. Automatic building of arabic multi dialect text corpora by bootstrapping dialect words. In 1st international conference on communications, signal processing, and their applications (iccspa), pages 1–6, 2013. find
  75. Jihene Younes and Emna Souissi. A quantitative view of tunisian dialect electronic writing. In 5th International Conference on Arabic Language Processing, pages 63–72, 2014. find
  76. Ahmed Salama, Houda Bouamor, Behrang Mohit, and Kemal Oflazer. YouDACC: the Youtube Dialectal Arabic Comment Corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1246–1251, Reykjavik, Iceland, may 2014. European Language Resources Association (ELRA). find
  77. Jihen Younes, Hadhemi Achour, and Emna Souissi. Constructing linguistic resources for the tunisian dialect using textual user-generated contents on the social web. In International Conference on Web Engineering, pages 3–14. Springer, 2015. find
  78. Hamdy Mubarak and Kareem Darwish. Using twitter to collect a multi-dialectal corpus of arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 1–7, 2014. find
  79. AOO Alshutayri and Eric Atwell. Exploring twitter as a source of an arabic dialect corpus. International Journal of Computational Linguistics (IJCL), 8(2):37–44, 2017. find
  80. Kathrein Abu Kwaik, Motaz K Saad, Stergios Chatzikyriakidis, and Simon Dobnik. Shami: A corpus of levantine arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC’18, Miyazaki, Japan, May 7-12, 2018., pages 3645–3652, 2018. find
  81. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. You tweet what you speak: A city-level dataset of arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC’18), pages 3653–3659, 2018. find
  82. LDC. Corpora created for the gale program., 2006, 2007, 2008, 2009. LDC2006E32, LDC2006E77, LDC2006E90, LDC2007E04, LDC2007E44, LDC2007E102, LDC2008E41, LDC2008E54, LDC2009E14, LDC2009E93. find
  83. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. Machine translation experiments on padic: A parallel arabic dialect corpus. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pages 26–34, 2015. find
  84. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. A large scale corpus of gulf arabic. CoRR, abs/1609.02960, 2016. ↗
  85. Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. Dart: A large dataset of dialectal arabic tweets. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC’18), pages 3666–3670, 2018. find
  86. Wajdi Zaghouani and Anis Charfi. Arap-tweet: A large multi-dialect twitter corpus for gender, age and language variety identification. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC’18, Miyazaki, Japan, May 7-12, 2018., pages 694–700, 2018. find
  87. Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saied Safaei, Elizabeth D. Trippe, Juan B. Gutierrez, and Krys Kochut. A brief survey of text mining: Classification, clustering and extraction techniques, 2017. find
  88. David Martin Powers. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies, 2(1):37––63, 2011. find
  89. Wafia Adouane, Nasredine Semmar, Richard Johansson, and Victoria Bobicev. Automatic detection of arabicized berber and arabic varieties. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 63–72, 2016. find
  90. Salima Harrat, Karima Meftouh, Mourad Abbas, Salma Jamoussi, Motaz Saad, and Kamel Smaili. Cross-dialectal arabic processing. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 620–632. Springer, 2015. find
  91. Charles E Metz. Basic principles of roc analysis. In Seminars in nuclear medicine, volume 8, pages 283–298. Elsevier, 1978. find
  92. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. Verifiably effective Arabic dialect identification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1465–1468, 2014. find
  93. Yoav Goldberg. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1):1–309, 2017. find
  94. Andreas Stolcke. SRILM-an extensible language modeling toolkit. In Seventh international conference on spoken language processing, pages 901–904, 2002. find
  95. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9. Association for Computational Linguistics, 2012. find our card
  96. Paul McNamee. Language and dialect discrimination using compression-inspired language models. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 195–203, 2016. find
  97. Tom Lippincott, Pamela Shapiro, Kevin Duh, and Paul McNamee. JHU System Description for the MADAR Arabic Dialect Identification Shared Task. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 264–268, 2019. find
  98. Stuart Russel et al. Artificial intelligence: a modern approach. Prentice Hall, 2010. find
  99. Mohammad Salameh, Houda Bouamor, and Nizar Habash. Fine-grained arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344, 2018. find our card
  100. Wafia Adouane, Nasredine Semmar, and Richard Johansson. ASIREM participation at the discriminating similar languages shared task 2016. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 163–169, 2016. find
  101. Alina Maria Ciobanu, Sergiu Nisioi, and Liviu P Dinu. Vanilla classifiers for distinguishing between similar languages. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 235–242, 2016. find
  102. Çağrı Çöltekin and Taraka Rama. Discriminating similar languages with linear SVMs and neural networks. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 15–24, 2016. find
  103. Areej Alshutayri, ES Atwell, Abdulrahman Alosaimy, James Dickins, Michael Ingleby, and Janet Watson. Arabic language weka-based dialect classifier for arabic automatic speech recognition transcripts. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2016), pages 204–211. Leeds, 2016. find
  104. Abualsoud Hanani, Aziz Qaroush, and Stephen Taylor. Classifying ASR transcriptions according to Arabic dialect. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 126–134, 2016. find
  105. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. Qcri@ DSL 2016: Spoken Arabic dialect identification using textual features. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 221–226, 2016. find
  106. Huma Lodhi, Craig Saunders, John Shawe-Taylor, Nello Cristianini, and Chris Watkins. Text classification using string kernels. Journal of Machine Learning Research, 2(Feb):419–444, 2002. find
  107. Radu Tudor Ionescu and Marius Popescu. UnibucKernel: An approach for Arabic dialect identification based on multiple string kernels. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 135–144, 2016. find
  108. Radu Tudor Ionescu and Andrei Butnaru. Learning to identify Arabic and German dialects using multiple kernels. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 200–209, 2017. find
  109. Andrei M Butnaru and Radu Tudor Ionescu. UnibucKernel Reloaded: First place in Arabic dialect identification for the second year in a row. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 77–87, 2018. find
  110. Radu Tudor Ionescu and Andrei M Butnaru. Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 1084–1090, 2018. find
  111. Ahmad Ragab, Haitham Seelawi, Mostafa Samir, Abdelrahman Mattar, Hesham Al-Bataineh, Mohammad Zaghloul, Ahmad Mustafa, Bashar Talafha, Abed Alhakim Freihat, and Hussein Al-Natsheh. Mawdoo3 AI at MADAR Shared Task: Arabic Fine-Grained Dialect Identification with Ensemble Learning. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 244–248, 2019. find
  112. Abualsoud Hanani, Aziz Qaroush, and Stephen Taylor. Identifying dialects with textual and acoustic cues. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 93–101, 2017. find
  113. Shervin Malmasi and Marcos Zampieri. Arabic dialect identification using iVectors and ASR transcripts. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 178–183, 2017. find
  114. Dhaou Ghoul and Gaël Lejeune. MICHAEL: Mining Character-level Patterns for Arabic Dialect Identification (MADAR Challenge). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 229–233, 2019. find
  115. Pavel Přibáň and Stephen Taylor. ZCU-NLP at MADAR 2019: Recognizing Arabic Dialects. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 208–213, 2019. find
  116. Li Deng, Dong Yu, et al. Deep learning: methods and applications. Foundations and Trends® in Signal Processing, 7(3–4):197–387, 2014. find
  117. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR 2013), pages 1–12, 2013. find
  118. Bashar Talafha, Wael Farhan, Ahmed Altakrouri, and Hussein Al-Natsheh. Mawdoo3 AI at MADAR Shared Task: Arabic Tweet Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 239–243, 2019. find
  119. Yonatan Belinkov and James Glass. A character-level convolutional neural network for distinguishing similar languages and dialects. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 145–152, 2016. find
  120. Mohamed Ali. Character level convolutional neural network for arabic dialect identification. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 122–127, 2018. find
  121. Younes Samih, Hamdy Mubarak, Ahmed Abdelali, Mohammed Attia, Mohamed Eldesouki, and Kareem Darwish. QC-GO Submission for MADAR Shared Task: Arabic Fine-Grained Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 290–294, 2019. find
  122. Francisco Rangel, Paolo Rosso, Martin Potthast, and Benno Stein. Overview of the 5th author profiling task at pan 2017: Gender and language variety identification in twitter. Working Notes Papers of the CLEF, pages 1613–0073, 2017. find
  123. Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1746––1751, 2014. find our card
  124. Yong Du, Wei Wang, and Liang Wang. Hierarchical recurrent neural network for skeleton based action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1110–1118, 2015. find
  125. Duyu Tang, Bing Qin, and Ting Liu. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1422–1432, 2015. find our card
  126. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642, 2013. find
  127. Agenlo Basile, Gareth Dwyer, Maria Medvedeva, Josine Rawee, Hessel Haagsma, and Malvina Nissim. Is there life beyond n-grams? a simple svm-based author profiling system. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  128. Eric Sadit Tellez, Sabino Miranda-Jiménez, Mario Graff, and Daniela Moctezuma. Gender and language-variety identification with microtc. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  129. Matej Martinc, Iza Skrjanec, Katja Zupan, and Senja Pollak. Pan 2017: Author profiling-gender and language variety prediction. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  130. Don Kodiyan, Florin Hardegger, Stephan Neuhaus, and Mark Cieliebak. Author profiling with bidirectional rnns using attention with grus: Notebook for pan at clef 2017. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  131. Sebastian Sierra, Manuel Montes-y Gómez, Thamar Solorio, and Fabio A González. Convolutional neural networks for author profiling. Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  132. Nils Schaetti. Unine at clef 2017: Tf-idf and deep-learning for author profiling. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  133. Yasuhide Miura, Tomoki Taniguchi, Motoki Taniguchi, and Tomoko Ohkuma. Author profiling with word+ character neural attention network. In Working Notes of CLEF 2017-Conference and Labs of the Evaluation Forum, Ireland, 11-14 September, 2017. find
  134. Bashar Talafha, Ali Fadel, Mahmoud Al-Ayyoub, Yaser Jararweh, AL-Smadi Mohammad, and Patrick Juola. Team JUST at the MADAR Shared Task on Arabic Fine-Grained Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 285–289, 2019. find our card
  135. Pruthwik Mishra and Vandan Mujadia. Arabic dialect identification for travel and twitter text. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 234–238, 2019. find
  136. Salima Harrat, Karima Meftouh, Karima Abidi, and Kamel Smaïli. Automatic identification methods on a corpus of twenty five fine-grained arabic dialects. In International Conference on Arabic Language Processing, pages 79–92. Springer, 2019. find
  137. Ayah Zirikly, Bart Desmet, and Mona Diab. The GW/LT3 VarDial 2016 shared task system for dialects and similar languages detection. In COLING, pages 33–41. The COLING 2016 Organizing Committee, 2016. find
  138. Chinnappa Guggilla. Discrimination between similar languages, varieties and dialects using cnn-and lstm-based deep neural networks. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 185–194, 2016. find
  139. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. Character-aware neural language models. In Thirtieth AAAI Conference on Artificial Intelligence, 2016. find our card
  140. Youssef Fares, Zeyad El-Zanaty, Kareem Abdel-Salam, Muhammed Ezzeldin, Aliaa Mohamed, Karim El-Awaad, and Marwan Torki. Arabic dialect identification with deep learning and hybrid frequency based features. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 224–228, 2019. find
  141. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. Advances in pre-training distributed word representations. In Proceedings of the International Conference on LanguageResources and Evaluation (LREC), 2018. find our card
  142. Mohamed Elaraby and Ahmed Zahran. A Character Level Convolutional BiLSTM for Arabic Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 274–278, 2019. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/