Elamur Research⌕
☰
home / papers / 2023 / Mavericks at NADI 2023 Shared Task: Unravelling Regional Nuances through Dialect Identification using Transformer-based Approach

Mavericks at NADI 2023 Shared Task: Unravelling Regional Nuances through Dialect Identification using Transformer-based Approach

Paper · 2023 · Vedant Deshpande, Yash Patwardhan, Kshitij Deshpande et al.

Summary

The paper describes the Mavericks team's system for subtask 1 of the NADI 2023 shared task, which assigns Arabic tweets to one of 18 country-level dialects. Dialect recognition matters because it can improve downstream tasks such as speech recognition and machine translation, and many Arabic varieties still have little data or study. The authors used the TWT-2023 Twitter corpus and cleaned it with regular expressions that strip the placeholder tokens "USER", "NUM" and "URL". They then fine-tuned three Arabic pre-trained transformers: AraBERTv02-Twitter-base, AraBERTv02-base and CAMeLBERT-DA. The final prediction comes from a hard-voting ensemble, where the label chosen by most models wins. The Twitter-pretrained AraBERT variant was the strongest single model. The ensemble improved on it further, reaching an F1 of 76.65 on the test set and 11th place on the leaderboard. The main lesson is practical: when the pretraining corpus resembles the target text, the model gains clearly. Simple majority voting also adds a modest but consistent improvement and makes predictions more stable.

Key points

  • Data: 23,400 tweets in 18 dialects, split into 18,000 train, 1,800 development and 3,600 test. Datasets from earlier NADI rounds and MADAR were also offered for training.
  • Fine-tuning setup: 10 epochs, learning rate 1e-5, batch size 32, AdamW optimizer.
  • Development F1: AraBERTv02-Twitter-base 77.03, AraBERTv02-base 73.07, CAMeLBERT-DA 72.78, hard-voting ensemble 77.62.
  • Test F1: AraBERTv02-Twitter-base 75.17, AraBERTv02-base 72.09, CAMeLBERT-DA 71.99, ensemble 76.65 (11th rank).
  • AraBERTv02-Twitter-base was additionally pretrained on 60M multi-dialect tweets. The authors credit this match with the tweet domain for its roughly 3-point lead over the base model.

Interesting ideas

  • I1. Pretraining on text from the same genre as the task gave a large gain over an otherwise identical model, so encoder choice should follow the target domain.
  • I2. A plain majority vote over a few differently pretrained models beat the best single member, with no learned combiner needed.
  • I3. The ensemble's margin over its best member was larger on test (+1.48) than on development (+0.59), which suggests voting mainly buys robustness on unseen data.
  • I4. Anonymization placeholders such as "USER" or "URL" were treated as noise and removed before classification, because they carry no dialect signal.
  • I5. The paper frames identifying the language variety as an upstream step that helps later processing.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): when picking or continuing pretraining of the gparse encoder, prefer text that resembles our inputs (school texts, fables, word problems), and measure the gain against a generic encoder.
  • I2 (learn): witness parsers can vote per token on head and relation when building silver data, and only labels with a clear majority are kept as trusted.
  • I3 (learn): our evaluations should always report the ensemble next to its best single member on truly held-out data, so we can tell real robustness gains from tuning to the dev set.
  • I4 (tokens): the tokenizer could map such placeholders to one dedicated token class, so they stay parseable as nouns but cannot mislead the morphology or lexical features.
  • I5 (tree): a light variety or register detector could decide which UD dialect conversion or treebank-specific parser settings to apply before parsing.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Vedant Deshpande, Yash Patwardhan, Kshitij Deshpande, Sudeep Mangalvedhekar and Ravindra Murumkar. 2023. Mavericks at NADI 2023 Shared Task: Unravelling Regional Nuances through Dialect Identification using Transformer-based Approach. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.74

@inproceedings{deshpande2023mavericks,
  title = {{Mavericks at NADI 2023 Shared Task: Unravelling Regional Nuances through Dialect Identification using Transformer-based Approach}},
  author = {Vedant Deshpande, Yash Patwardhan, Kshitij Deshpande, Sudeep Mangalvedhekar and Ravindra Murumkar},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.74},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.74}
}

References

19 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
  2. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  3. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. NADI 2021: The second nuanced Arabic dialect identification shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  4. Mahmoud Al-Ayyoub, Aya Nuseir, Kholoud Alsmearat, Yaser Jararweh, and Brij Gupta. 2018. Deep learning for arabic nlp: A survey. Journal of Computational Science, 26:522–531. ↗
  5. Badr AlKhamissi, Mohamed Gabr, Muhammad El-Nokrashy, and Khaled Essam. 2021. Adapting MAR-BERT for improved Arabic dialect identification: Submission to the NADI 2021 shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 260–264, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. find
  6. Nouf AlShenaifi and Aqil Azmi. 2022. Arabic dialect identification using machine learning and transformer-based models: Submission to the NADI 2022 shared task. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 464–467, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. ↗
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  8. Giyaseddin Bayrak and Abdul Majeed Issifu. 2022. Domain-adapted BERT-based models for nuanced Arabic dialect identification and tweet sentiment analysis. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 425–430, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  9. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  11. Abdellah El Mekki, Ahmed Alami, Hamza Alami, Ahmed Khoumsi, and Ismail Berrada. 2020. Weighted combination of BERT and n-GRAM features for nuanced Arabic dialect identification. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 268–274, Barcelona, Spain (Online). Association for Computational Linguistics.
  12. Moataz Mansour, Moustafa Tohamy, Zeyad Ezzat, and Marwan Torki. 2020. Arabic dialect identification using BERT fine-tuning. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 308–312, Barcelona, Spain (Online). Association for Computational Linguistics.
  13. Ahmed Oumar and Khalil Mrini. 2022. Ahmed and khalil at NADI 2022: Transfer learning and addressing class imbalance for Arabic dialect identification and sentiment analysis. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 442–446, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. ↗
  14. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332– 1344, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  15. Abdulhadi Shoufan and Sumaya Alameri. 2015. Natural language processing for dialectical Arabic: A survey. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 36–48, Beijing, China. Association for Computational Linguistics.
  16. Rawan Tahssin, Youssef Kishk, and Marwan Torki. 2020. Identifying nuanced dialect for Arabic tweets with deep learning and reverse translation corpus extension system. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 288–294, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  17. Kamel Gaanoun and Imade Benelallam. 2020. Arabic dialect identification: An Arabic-BERT model with data augmentation and ensembling strategy. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 275–281, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  18. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  19. Abdullah Salem Khered, Ingy Yasser Hassan Abdou Abdelhalim, and Riza Batista-Navarro. 2022. Building an ensemble of transformer models for Arabic dialect classification and sentiment analysis. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 479–484, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/deshpande-2023-mavericks-nadi-2023-shared-task/