Elamur Research⌕
☰
home / papers / 2023 / UoT at NADI 2023 shared task: Automatic Arabic Dialect Identification is Made Possible

UoT at NADI 2023 shared task: Automatic Arabic Dialect Identification is Made Possible

Paper · 2023 · Abduslam F A Nwesri, Nabila A S Shinbir and Hassan Ebrahem

Summary

The paper describes the University of Tripoli (UoT) entry to Subtask 1 of NADI 2023. The task is closed country-level Arabic dialect identification over 18 dialects in tweets, using only data supplied by the organizers. The authors first tried classic scikit-learn classifiers (SVC, MLPClassifier, MultinomialNB, BernoulliNB), which reached only about 0.6 F1. They then fine-tuned two Arabic BERT variants, MARBERTv2 and bert-base-arabertv02-twitter.

Several common cleanup steps lowered scores: character normalization, removal of non-Arabic characters, stopword removal and stemming. What helped was data augmentation by concatenation. They sorted the training tweets by dialect and joined each pair of neighbouring same-dialect tweets into a longer synthetic example, which roughly doubled the training set. MARBERTv2 fine-tuned on this expanded set scored 82.87 macro F1 on the official test set, placing seventh of 16 teams.

The work matters because it shows two things. Short, ambiguous inputs can be made more informative by giving the model more same-label context. And "standard" text normalization can quietly destroy signal in non-standard writing.

Key points

  • Data: NADI 2023 Subtask 1 had 18 dialects, split into 18k training, 1.8k development and 3.6k test tweets. Earlier NADI sets and MADAR were also allowed.
  • Classic ML: the best classical classifier (MultinomialNB) reached about 0.63 F1 on development. Stopword removal and the Snowball stemmer made every classifier worse.
  • Transformer baseline on development: MARBERTv2 scored 84.47 F1 and arabertv02-twitter 79.31 F1. Preprocessing dropped MARBERTv2 to 80.12, and removing the top-50 frequent words dropped it to 83.19.
  • Tweet expansion: the expanded set had 35,898 tweets, with a maximum length of 540 and a sequence limit of 512.
    • Development F1 was 84.70.
    • Official test F1 was 82.87 for UoT-1 (MARBERTv2), 80.70 for UoT-2 (with preprocessing) and 74.45 for UoT-3 (AraBERT).
    • The winning team scored 87.27.
  • Errors and cost:
    • Most errors fell within regional clusters (Gulf, Maghrebi, Levantine), while Moroccan was recognized best.
    • Fine-tuning took about 6 hours on a CPU and 30 minutes on a Colab T4 GPU.
    • Using all datasets at length 512 ran out of memory, so length was capped at 256.

Interesting ideas

  • I1. Same-label concatenation: gluing together two short training items that share a label produces longer, richer examples, which helps when single inputs are too short even for humans to classify.
  • I2. Cleanup can hurt: normalization, stopword removal and stemming all hurt, because the "noise" they removed carried the very signal being classified (dialect-specific function words and spellings).
  • I3. Character-range filtering destroys words: deleting characters outside a nominal script range broke words written with Farsi-style letters. The fix is to map such variants to their standard counterparts instead of dropping them.
  • I4. Error analysis by confusion clusters: most errors fell between neighbouring dialects, which shows that the label space has a geographic hierarchy.
  • I5. Longer inputs have a hardware cost: context expansion helped, but long sequences on the full data exhausted memory, forcing a length cap.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when building silver data for document-level labels such as genre or register, concatenating same-label snippets could cheaply enlarge the data and add context.
  • I2 (tokens): ablate each normalization rule in the tokenizer against a held-out set rather than assuming it helps, especially for informal or dialectal English input.
  • I3 (tokens): unknown or variant code points (curly quotes, ligatures, homoglyphs) should be mapped explicitly rather than stripped, so lemmas and tags are not silently corrupted.
  • I4 (learn): evaluation could report confusions grouped by a label hierarchy, separating near misses from real failures, for example among UD relation subtypes or VerbNet subclasses.
  • I5 (tree): when gparse is trained on concatenated or multi-sentence inputs, plan sequence-length caps or bucketing up front so memory stays bounded.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TripoliTripoli · Libya5 works · 8 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Abduslam F A Nwesri, Nabila A S Shinbir and Hassan Ebrahem. 2023. UoT at NADI 2023 shared task: Automatic Arabic Dialect Identification is Made Possible. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.64

@inproceedings{nwesri2023uot,
  title = {{UoT at NADI 2023 shared task: Automatic Arabic Dialect Identification is Made Possible}},
  author = {Abduslam F A Nwesri, Nabila A S Shinbir and Hassan Ebrahem},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.64},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.64}
}

References

15 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021a. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
  3. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  4. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021b. NADI 2021: The second nuanced Arabic dialect identification shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  5. MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  6. Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, and Inderjit Dhillon. 2020. Taming pretrained transformers for extreme multi-label text classification. find
  7. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Samhaa R. El-Beltagy, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A panoramic survey of natural language processing in the arab world. ↗
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. ↗
  9. Hani Elgabou and Dimitar Kazakov. 2017. Building dialectal Arabic corpora. In Proceedings of the Workshop Human-Informed Translation and Interpreting Technology, pages 52–57, Varna, Bulgaria. Association for Computational Linguistics, Shoumen, Bulgaria. find
  10. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016. A large scale corpus of Gulf Arabic. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4282–4289, Portorož, Slovenia. European Language Resources Association (ELRA).
  11. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  12. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. NADI 2022: The third nuanced Arabic dialect identification shared task. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  13. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. Arabert: Transformer-based model for arabic language understanding. ↗
  14. Ekaba Bisong. 2019. Google Colaboratory, pages 59– 64. Apress, Berkeley, CA. find
  15. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nwesri-2023-uot-nadi-2023-shared-task/