Elamur Research⌕
☰
home / papers / 2023 / DialectNLU at NADI 2023 Shared Task: Transformer Based Multitask Approach Jointly Integrating Dialect and Machine Translation Tasks in Arabic

DialectNLU at NADI 2023 Shared Task: Transformer Based Multitask Approach Jointly Integrating Dialect and Machine Translation Tasks in Arabic

Paper · 2023 · Hariram Veeramani, Surendrabikram Thapa and Usman Naseem

Summary

The paper is a system description for the NADI 2023 shared task at ArabicNLP 2023. It covers two closed-track subtasks: identifying which of 18 countries an Arabic tweet comes from, and translating four urban dialects into Modern Standard Arabic (MSA). For dialect identification, the authors combine ARBERT, which is oriented to MSA, with MARBERT, which is oriented to tweets. The strongest variant joins the two encoders' hidden vectors into one representation and adds dropout, reaching a macro-F1 of 80.56 on the test set. For translation, they compare AraT5 variants with AraBART. AraBART wins clearly. A small multitask ensemble does slightly better: AraBART is first trained to classify dialects, then fine-tuned to translate, using two learning-rate schedules, and the variant that classifies dialects better is kept. The team placed 10th on subtask 1 and 3rd on subtask 2, without external data or augmentation. The broader point is that encoders trained on different registers complement each other, and joining them at the representation level beats mixing their outputs.

Key points

  • Subtask 1 uses NADI-2023-TWT: 18,000 training tweets (1,000 per country), 1,800 dev and 3,600 test, scored by macro-F1. Subtask 2 trains on 40,000 MADAR-parallel sentences covering Egyptian, Emirati, Jordanian and Palestinian, with 400 dev and 2,000 test sentences, scored by BLEU.
  • Subtask 1 test F1 by system:
    • ARBERT alone: 73.16.
    • MARBERT alone: 73.25.
    • Logit-weighted ensemble (0.4 ARBERT / 0.6 MARBERT): 77.19.
    • Fusion representation (2×768): 79.06.
    • Fusion + dropout: 80.56, the best.
  • Prepending the label to the input (the "label-aware technique", LAT) brought no gain on test, staying at 79.06.
  • Subtask 2 test BLEU: AraT5 (base) 0.014, AraT5 (base-1024) 0.07, AraBART 13.42 and the classification-then-translation ensemble 13.43. On eval, the ensemble scored 12.9 against AraBART's 12.01.
  • The dev-set confusion matrix shows Sudanese and Moroccan as easy to recognise, while Palestinian and Kuwaiti are the most often confused.

Interesting ideas

  • I1. Concatenating the hidden states of two encoders trained on different registers (formal vs. social-media) worked better than averaging their output scores, so complementary knowledge seems best merged before the decision layer.
  • I2. An intermediate classification stage (train on dialect labels, discard that head, then fine-tune for generation) gives the encoder awareness of language variety before the main task.
  • I3. Choosing between otherwise identical runs, differing only in learning-rate schedule (linear vs. cosine), by their score on an auxiliary task is a cheap selection rule when the main metric is noisy.
  • I4. A per-class confusion matrix exposed which varieties merge into each other, not just the overall F1.
  • I5. Some "obvious" add-ons, like putting the label into the input text, gave no real benefit once checked on held-out data, a reminder to keep ablations honest.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): gparse could feed its parser head with joined embeddings from an encoder tuned on edited prose and one tuned on informal web text, improving robustness on chatty inputs without training two parsers.
  • I2 (learn): before parser training, the encoder could first learn to tell genres or registers apart (fable, textbook, news, dialogue), which may help it handle the varied texts in Mova's school-course and tale material.
  • I3 (learn): when several training runs or checkpoints of a witness parser compete, a stable auxiliary probe such as POS or morphology accuracy could act as a tie-breaker alongside LAS.
  • I4 (learn): evaluation reports could include confusion tables over dependency labels or frame roles, to show which distinctions (e.g. obj vs. obl) silver data keeps blurring and where repair rules should focus.
  • I5 (learn): every proposed feature or repair rule should be measured against a plain baseline and a negative control, so that apparent dev gains are not kept by default.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Los AngelesLos Angeles · United States87 works · 54,961 citations🎓Virginia TechBlacksburg · United States36 works · 43,036 citations🎓James Cook UniversityTownsville · Australia3 works · 23 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 works

Cite

Cite as Hariram Veeramani, Surendrabikram Thapa and Usman Naseem. 2023. DialectNLU at NADI 2023 Shared Task: Transformer Based Multitask Approach Jointly Integrating Dialect and Machine Translation Tasks in Arabic. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.63

@inproceedings{veeramani2023dialectnlu,
  title = {{DialectNLU at NADI 2023 Shared Task: Transformer Based Multitask Approach Jointly Integrating Dialect and Machine Translation Tasks in Arabic}},
  author = {Hariram Veeramani, Surendrabikram Thapa and Usman Naseem},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.63},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.63}
}

References

22 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, et al. 2021a. Arbert & marbert: Deep bidirectional transformers for arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105. ↗
  3. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. NADI 2020: The first nuanced Arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. ↗
  4. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021b. NADI 2021: The second nuanced Arabic dialect identification shared task. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. ↗
  5. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022a. Nadi 2022: The third nuanced arabic dialect identification shared task. WANLP 2022, page 85.
  6. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022b. NADI 2022: The third nuanced Arabic dialect identification shared task. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  7. Roqayah Al-Ibrahim and Rehab M Duwairi. 2020. Neural machine translation from jordanian dialect to modern standard arabic. In 2020 11th International Conference on Information and Communication Systems (ICICS), pages 173–178. IEEE. find
  8. Mohamed Seghir Hadj Ameur, Farid Meziane, and Ahmed Guessoum. 2020. Arabic machine translation: A survey of the latest trends and challenges. Computer Science Review, 38:100305. ↗
  9. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  10. Moussa Kamal Eddine, Nadi Tomeh, Nizar Habash, Joseph Le Roux, and Michalis Vazirgiannis. 2022. AraBART: a pretrained Arabic sequence-to-sequence model for abstractive summarization. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 31–42, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  11. Rajaraman Kanagasabai, Saravanan Rajamanickam, Hariram Veeramani, Adam Westerski, and Kim Jung Jae. 2023. Semantists at ImageArg-2023: Exploring Cross-modal Contrastive and Ensemble Models for Multimodal Stance and Persuasiveness Classification. In Proceedings of the 10th Workshop on Argument Mining, Online and in Singapore. Association for Computational Linguistics. find
  12. Saméh Kchaou, Rahma Boujelbane, and Lamia Hadrich. 2023. Hybrid pipeline for building arabic tunisian dialect-standard arabic neural machine translation model from scratch. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(3):1–21. ↗
  13. Shervin Malmasi, Eshrag Refaee, and Mark Dras. 2015. Arabic dialect identification using a parallel multidialectal corpus. In International conference of the pacific association for computational linguistics, pages 35–53. Springer. find
  14. Zinah J Mohammed Ameen, Abdulkareem Abdulrahman Kadhim, et al. 2023. Deep learning methods for arabic autoencoder speech recognition system for electro-larynx device. Advances in Human-Computer Interaction, 2023. ↗
  15. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained arabic dialect identification. In Proceedings of the 27th international conference on computational linguistics, pages 1332– 1344.
  16. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023a. Automated Citation Function Classification and Context Extraction in Astrophysics: Leveraging Paraphrasing and Question Answering. In Proceedings of the second Workshop on Information Extraction from Scientific Publications, Online. Association for Computational Linguistics. find
  17. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023b. Enhancing ESG Impact Type Identification through Early Fusion and Multilingual Models. In Proceedings of the Sixth Workshop on Financial Technology and Natural Language Processing (FinNLP). Association for Computational Linguistics. find
  18. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023c. KnowTellConvince at ArAIEval Shared Task: Disinformation and Persuasion Detection in Arabic using Similar and Contrastive Representation Alignment. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023), Singapore. Association for Computational Linguistics.
  19. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023d. LowResContextQA at Qur’an QA 2023 Shared Task: Temporal and Sequential Representation Augmented Question Answering Span Detection in Arabic. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023), Singapore. Association for Computational Linguistics.
  20. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023e. LowResourceNLU at BLP-2023 Task 1 2: Enhancing Sentiment Classification and Violence Incitement Detection in Bangla Through Aggregated Language Models. In Proceedings of the 1st International Workshop on Bangla Language Processing (BLP-2023), Singapore. Association for Computational Linguistics. find
  21. Hariram Veeramani, Surendrabikram Thapa, and Usman Naseem. 2023f. Temporally Dynamic Session-Keyword Aware Sequential Recommendation system. In 2023 International Conference on Data Mining Workshops (ICDMW). find
  22. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/veeramani-2023-dialectnlu-nadi-2023-shared-task/