Elamur Research⌕
☰
home / papers / 2021 / QADI: Arabic Dialect Identification in the Wild

QADI: Arabic Dialect Identification in the Wild

Paper · 2021 · Ahmed Abdelali, Hamdy Mubarak, Younes Samih et al.

Summary

The paper tackles fine-grained Arabic dialect identification at the country level. Earlier resources were limited by genre, covered few countries, or were skewed toward a few countries. The authors built QADI automatically from Twitter using a chain of filters. First, a gazetteer of country names, city names and nationality adjectives matches self-descriptions in user bios to assign each account a country. Second, a classifier trained with distant supervision drops accounts that write mostly in MSA. Third, a filter removes accounts with heavy vulgar language. The result is a balanced corpus of about 540k tweets from 2,525 accounts across 18 Arab countries, plus a native-speaker-checked test set. Manual checks of random samples put label accuracy at 91.5%. Classifiers trained on QADI range from SVMs on n-grams and embeddings to fine-tuned mBERT and AraBERT. The best reaches a macro-F1 of 60.6% over 18 classes, well above training on MADAR. The work matters because the pipeline is fully automatic and can be rerun to collect fresh dialect data.

Key points

  • QADI has roughly 540k training tweets from 2,525 accounts (about 140 per country) and 3,303 manually verified test tweets (about 182 per country). It covers 18 countries; Mauritania, Somalia, Djibouti and Comoros were left out for lack of Arabic-tweeting users.
  • The MSA-vs-dialect classifier was trained on heuristic labels taken from 50M tweets. Tweets containing MSA relative pronouns were labelled MSA (3.09M), and tweets containing the pan-dialectal pronoun "Ally" were labelled dialectal (3.17M).
  • Native speakers from each country judged 200 sampled tweets per country, giving 91.5% average label accuracy.
  • AraBERT was the best model, at 60.6% macro-F1 at tweet level. Models trained on MADAR (with user labels copied onto each tweet) scored significantly lower.
  • Most errors fall within a region, chiefly the Gulf, Levant (LB vs SY), Maghreb (MA vs DZ) and EG vs SD. Only about 2% of errors land outside the true region, often in tweets quoting songs, poems or MSA.

Interesting ideas

  • I1. The authors labelled millions of tweets for free by using a single function word as a signal: one relative pronoun marks dialect, others mark MSA. A cheap, high-precision lexical cue can bootstrap a classifier that generalizes beyond the cue.
  • I2. Before training, they masked the very words used as labelling cues (replacing them with a RELATIVE token), along with mentions, URLs and digits. This forces the model to learn real context rather than memorizing the seed rule.
  • I3. Labels come from authors' explicit self-declarations in their bios rather than from keyword searches for dialect words. This avoids biasing the data toward a predefined vocabulary.
  • I4. Annotators were asked whether a tweet fits the variety spoken in their own country, not to pick one exact label. This accepts that some expressions are shared across neighbours.
  • I5. The error analysis groups confusions by region and separates near misses from far-off outliers. This shows whether a model fails gracefully.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data for register or genre labels could be seeded from unambiguous grammatical markers, then widened by a trained model and checked against witness parsers.
  • I2 (learn): Any rule-seeded silver set should hide its trigger tokens during training, and a negative-control run could confirm the model doesn't simply rediscover the rule.
  • I3 (world): Self-identifying statements in a text ("I am a farmer from X") can be compiled as high-confidence entity attributes, kept separate from attributes inferred from word choice.
  • I4 (learn): Evaluation of variety-sensitive steps could use acceptability judgments rather than single gold labels, so outputs valid under several readings aren't counted as wrong.
  • I5 (tree): When converting or parsing across English varieties or UD treebank dialects, errors could be split into neighbour-style confusions and true outliers, so repair rules target the outliers first.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan and Kareem Darwish. 2021. QADI: Arabic Dialect Identification in the Wild. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abdelali2021qadi,
  title = {{QADI: Arabic Dialect Identification in the Wild}},
  author = {Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan and Kareem Darwish},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.1/}
}

References

37 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed Eldesouki, and Laura Kallmeyer. 2020. Effective multi dialectal arabic pos tagging. Natural Language Engineering, 1(1):18. find
  2. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. You tweet what you speak: A city-level dataset of Arabic dialects. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan.
  3. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014. Verifiably Effective Arabic Dialect Identification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.
  4. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  5. Thierry Declerck and Piroska Lendvai. 2015. Processing and normalizing hashtags. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 104–109. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  7. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics.
  8. Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. 2018. DART: A Large Dataset of Dialectal Arabic Tweets. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  9. Mona T Diab, Mohamed Al-Badrashiny, Maryam Aminian, Mohammed Attia, Heba Elfardy, Nizar Habash, Abdelati Hawwari, Wael Salloum, Pradeep Dasigi, and Ramy Eskander. 2014. Tharwa: A Large Scale Dialectal Arabic-Standard Arabic-English Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 3782–3789, Reykjavik, Iceland.
  10. Areej Alshutayri and Erik Atwell. 2017. Exploring twitter as a source of an arabic dialect corpus. International Journal of Computational Linguistics (IJCL), 8(2):37–44. This is an open access article under the terms of the Creative Commons Attribution License (CC-BY). find
  11. Mahmoud El-Haj, Paul Rayson, and Mariam Aboelezz. 2018. Arabic dialect identification in the context of bivalency and code-switching. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), Miyazaki, Japan. European Languages Resources Association (ELRA).
  12. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of The 4th Workshop on Open-Source Arabic Corpora and Processing Tools, pages 9–15. ↗
  13. Heba Elfardy and Mona Diab. 2013. Sentence Level Dialect Identification in Arabic. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), Sofia, Bulgaria.
  14. Piyush Bansal, Romil Bansal, and Vasudeva Varma. 2015. Towards deep semantic analysis of hashtags. In European conference on information retrieval, pages 453–464. Springer. ↗
  15. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia. Association for Computational Linguistics. find
  16. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic Dialect Corpus and Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  17. Radu Tudor Ionescu and Marius Popescu. 2016. UnibucKernel: An approach for Arabic dialect identification based on multiple string kernels. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 135–144, Osaka, Japan.
  18. The COLING 2016 Organizing Committee. find
  19. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics.
  20. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. ↗
  21. Michael Conover, Jacob Ratkiewicz, Matthew R Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on twitter. ICWSM, 133:89–96. find
  22. X. Li and J. Huang. 2009. Shc: A spectral algorithm for hierarchical clustering. In 2009 International Conference on Multimedia Information Networking and Security, volume 2, pages 197–200. Kareem Darwish, Mohammed Attia, Hamdy Mubarak, Younes Samih, Ahmed Abdelali, Lluı́s Màrquez, ciation for Computational Linguistics (ACL), pages 37–41. find
  23. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. Discriminating between similar languages and Arabic dialect identification: A report on the third DSL shared task. In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 1–14, Osaka, Japan.
  24. The COLING 2016 Organizing Committee. find
  25. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗
  26. Chiyu Zhang and Muhammad Abdul-Mageed. 2019. No army, no navy: Bert semi-supervised learning of arabic dialects. pages 279–284.
  27. Hamdy Mubarak and Kareem Darwish. 2014. Using twitter to collect a multi-dialectal corpus of arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 1–7.
  28. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
  29. Damian Radcliffe and Payton Bruni. 2019. State of Social Media Middle East: 2018. University of Oregon Libraries. ↗
  30. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2.amazonaws. com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf. ↗
  31. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. 2014. Automatic Identification of Arabic Dialects in Social Media. In Proceedings of the Workshop on Natural Language Processing for Social Media (SocialNLP), pages 22–27, Dublin, Ireland. find
  32. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media.
  33. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  34. Younes Samih, Mohamed Eldesouki, Mohammed Attia, Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Laura Kallmeyer. 2017. Learning from relatives: Unified dialectal Arabic segmentation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 432–441, Vancouver, Canada. Association for Computational Linguistics.
  35. Wajdi Zaghouani and Anis Charfi. 2018a. Arap-tweet: A large multi-dialect twitter corpus for gender, age and language variety identification. CoRR, abs/1808.07674. ↗
  36. Wajdi Zaghouani and Anis Charfi. 2018b. Guidelines and annotation framework for arabic author profiling. ↗
  37. Omar F Zaidan and Chris Callison-Burch. 2011. The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic With High Dialectal Content. In Proceedings of the Conference of the Asso-

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abdelali-2021-qadi-arabic-dialect-identification-wild/