Elamur Research⌕
☰
home / papers / 2023 / ISL-AAST at NADI 2023 shared task: Enhancing Arabic Dialect Identification in the Era of Globalization and Technological Progress

ISL-AAST at NADI 2023 shared task: Enhancing Arabic Dialect Identification in the Era of Globalization and Technological Progress

Paper · 2023 · Shorouk Adel and Noureldin Elmadany

Summary

The paper describes the ISL-AAST entry to Subtask 1 of the NADI 2023 shared task: classifying Arabic tweets into one of 18 country-level dialects. The authors argue that dialects keep changing under globalization and online communication, which makes the task harder. They used the organizers' data (about 18,000 training tweets, 1,800 dev, 3,600 test). Preprocessing was light: diacritics, mentions, URLs, numbers, emojis and non-Arabic characters were removed, and elongated letters were collapsed. Stemming was skipped on purpose so that dialect cues would be kept. They ran 18 experiments on MARBERT and MARBERTv2. The variants included adapters, concatenation of the top layers, averaging of the middle layers (4–7), CNN or BiLSTM heads, and voting and averaging ensembles. The best single model pooled hidden layers and passed them through three convolution filters, reaching 81.67% test F1. A majority-vote ensemble of experiments 3–16 scored 83.73% test F1, which placed sixth. The work shows that cheap architectural tweaks plus ensembling can beat plain fine-tuning on fine-grained dialect labels.

Key points

  • Data: NADI 2023 Subtask 1, with 18 country labels, ~18k train / 1.8k dev / 3.6k test tweets.
  • Plain fine-tuned MARBERT reached ~80.0% test F1. Adapter variants did clearly worse (~74.6% and ~73.4% test F1).
  • Best single model (Exp.14): the Exp.7 layer pooling (average of layers 4–7 plus concatenation of the last four layers), then three Conv1D filters (kernels 10, 8, 6) with max-pooling. It scored 81.67% test F1 and 83.14% dev F1.
  • The voting ensemble (Exp.17) reached 83.73% test F1 and 85.27% dev F1, giving 6th place. The probability-averaging ensemble was slightly lower at 83.36% test F1.
  • Most errors fell between neighbouring dialect pairs: Jordan vs. Palestine and Kuwait vs. Bahrain.

Interesting ideas

  • I1. The sentence representation mixes a mean of middle encoder layers with the concatenated top layers, which captures both lower-level and task-specific signals instead of relying on the final layer alone.
  • I2. They deliberately skipped stemming and lemmatization, because collapsing word forms would erase exactly the surface cues that separate the classes.
  • I3. A simple majority vote across many slightly different fine-tuned variants beat every individual model by about two F1 points.
  • I4. The confusion-matrix analysis shows that errors concentrate on a few closely related class pairs, not spread evenly.
  • I5. Adapters were tested against full fine-tuning and lost clearly, which serves as a useful reminder to verify parameter-efficient shortcuts empirically.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): gparse could feed its arc/label scorers a learned mix of middle and top transformer layers, which may give a cheap gain for syntax, since syntax often sits in middle layers.
  • I2 (tokens): the en crate should keep raw surface forms next to lemmas, so that register or dialect markers (e.g., spelling variants) stay available to later stages.
  • I3 (learn): this supports building silver data by agreement among several diverse witness parsers, since even cheap variants of one base model add useful disagreement.
  • I4 (learn): evaluation reports could list the top confused label pairs (e.g., UD relations like obl vs. nmod) so that repair rules target the real failure points.
  • I5 (learn): before adopting adapter-style training for the GPU parser, run a head-to-head against full fine-tuning on a held-out treebank.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Arab Academy for Science, Technology, and Maritime TransportAlexandria · Egypt5 works · 50 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Shorouk Adel and Noureldin Elmadany. 2023. ISL-AAST at NADI 2023 shared task: Enhancing Arabic Dialect Identification in the Era of Globalization and Technological Progress. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.66

@inproceedings{adel2023isl,
  title = {{ISL-AAST at NADI 2023 shared task: Enhancing Arabic Dialect Identification in the Era of Globalization and Technological Progress}},
  author = {Shorouk Adel and Noureldin Elmadany},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.66},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.66}
}

References

24 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding ↗
  2. Reem Abdel-Salam. 2022. Dialect & sentiment identification in nuanced arabic tweets using an ensemble of prompt-based, fine-tuned, and multitask bert-based models. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 452–457. ↗
  3. Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. 2005. Bidirectional lstm networks for improved phoneme classification and recognition. In International conference on artificial neural networks, pages 799–804. Springer. find
  4. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020a. Arbert & marbert: deep bidirectional transformers for arabic. arXiv preprint arXiv:2101.01785. ↗
  5. Alon Jacovi, Oren Sar Shalom, and Yoav Goldberg. 2018. Understanding convolutional neural networks for text classification. arXiv preprint arXiv:1809.08037. ↗
  6. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
  7. Imran Khan Mohd Jais, Amelia Ritahani Ismail, and Syed Qamrun Nisa. 2019. Adam optimization algorithm for wide and deep neural network. Knowledge Engineering and Data Science, 2(1):41–46. find
  8. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020b. Nadi 2020: The first nuanced arabic dialect identification shared task. arXiv preprint arXiv:2010.11334. ↗
  9. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. Nadi 2021: The second nuanced arabic dialect identification shared task. arXiv preprint arXiv:2103.08466. ↗
  10. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. Nadi 2022: The third nuanced arabic dialect identification shared task. arXiv preprint arXiv:2210.09582. ↗
  11. Badr AlKhamissi, Mohamed Gabr, Muhammad El-Nokrashy, and Khaled Essam. 2021. Adapting marbert for improved arabic dialect identification: Submission to the nadi 2021 shared task. arXiv preprint arXiv:2103.01065. ↗
  12. Joseph Attieh and Fadi Hassan. 2022. Arabic dialect identification and sentiment classification using transformer-based models. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 485–490.
  13. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A multidialectal parallel corpus of arabic. In LREC, pages 1240–1245.
  14. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The madar shared task on arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199– 207.
  15. Ron Darvin. 2016. Language and identity in the digital age. The Routledge handbook of language and identity, pages 523–540. ↗
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep Abir Messaoudi, Chayma Fourati, Hatem Haddad, and Moez BenHajhmida. 2022. icompass working notes for the nuanced arabic dialect identification shared task. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 415–419.
  17. Hamada Nayel, Ahmed Hassan, Mahmoud Sobhi, and Ahmed El-Sawy. 2021. Machine learning-based approach for arabic dialect identification. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 287–290.
  18. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterhub: A framework for adapting transformers. arXiv preprint arXiv:2007.07779. ↗
  19. Matteo Re and Giorgio Valentini. 2012. Ensemble methods. Advances in machine learning and data mining for astronomy, pages 563–593. find
  20. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. 2014. Automatic identification of arabic dialects in social media. In Proceedings of the first international workshop on Social media retrieval and analysis, pages 35–40. find
  21. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained arabic dialect identification. In Proceedings of the 27th international conference on computational linguistics, pages 1332– 1344.
  22. Wael Salloum. 2018. Machine Translation of Arabic Dialects. Columbia University. ↗
  23. John Smith and Lisa Johnson. 2022. Categorical cross entropy loss for multi-class classification. Journal of Machine Learning Research, 30(1):100–120. find
  24. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/adel-2023-isl-aast-nadi-2023-shared/