Elamur Research⌕
☰
home / papers / 2026 / Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding

Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding

Paper · 2026 · Abdulhai Alali and Abderrahmane Issam

Summary

Large language models still handle regional Arabic varieties poorly compared with Modern Standard Arabic. For the AMIYA shared task (closed data track), the authors adapted the JAIS-2 model to Syrian, Moroccan and Saudi Arabic. They trained two kinds of LoRA adapter for each dialect. One learns from raw monolingual dialect text with a causal LM objective. The other learns English↔dialect translation framed as instructions, with the loss computed only on the output tokens. The two adapters were then combined using TIES-Merging. At inference, the system samples 20 candidates and keeps the one with the highest ADI2 dialectness score (the paper calls this MBR decoding). On Moroccan development data:

  • Monolingual training mostly boosts dialectness.
  • Translation training mostly boosts chrF++.
  • The merged model balances the two.
  • ADI2-based selection raises dialectness further, at a small cost in chrF++.

The resulting recipe is compact and cheap to train, but generating and scoring many candidates slows inference considerably.

Key points

  • Training data per dialect:

    • Syrian: Shami Corpus (25,136 monolingual) and UFAL (120,600 parallel).
    • Moroccan: DoDa (10,000 monolingual and 10,000 parallel).
    • Saudi: SDC (14,891 monolingual) and SauDial (1,000 parallel).
    • All sets were sub-sampled to save compute.

  • Training setup: LoRA on the attention layers, maximum sequence length 512, 5 epochs, learning rate 3e-5, effective batch size 32, BF16.

  • Backbone comparison (Syrian): LLaMA 3.2 3B reached ADI2 0.78 but only 0.14 chrF++. JAIS-2 8B reached ADI2 0.33 and chrF++ 0.43, and was chosen for its balance.

  • Moroccan ablation (ADI2 / chrF++):

    ConfigurationADI2chrF++
    Base0.180.31
    + monolingual FT0.440.33
    + MT FT0.260.42
    TIES-merged0.380.44

    With ADI2-driven selection, monolingual ADI2 rose to 0.51 and chrF++ fell to 0.40. Selecting by chrF++ instead pushed outputs toward neutral, standard-like Arabic (monolingual ADI2 0.24).

  • Official results:

    • ADI2: 0.679 Moroccan, 0.389 Syrian, 0.464 Saudi.
    • Saudi DA→EN collapsed to 0.03.
    • Human evaluation was best for Moroccan (fluency 3.37, adequacy 1.97).

Interesting ideas

  • I1. Train one adapter for each kind of supervision (surface fluency vs. meaning alignment), then merge them, so the two signals do not interfere during training.
  • I2. Sample several outputs and pick one with a task-specific scorer; the choice of scorer visibly steers style, as chrF++ pulled results toward standard Arabic.
  • I3. Report two axes that trade off against each other instead of a single number; a model can win one axis (LLaMA on dialectness) while failing the other (translation).
  • I4. Use a multiplicative score (degree of dialectness × probability of the requested dialect), so a candidate must satisfy both conditions at once to score well.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): gparse could get separate adapters for standard treebanks and for UD dialect-converted data, merged afterwards, so the dialect conversion does not degrade the core English parse.
  • I2 (back): generation from the meaning graph could produce several realisations and keep the one whose re-parse best reproduces the graph, while checking whether the scorer favours bland, generic wording.
  • I3 (learn): evaluations should always pair parse/frame accuracy with an answer-correctness axis, so that a gain in one stage cannot hide a regression elsewhere.
  • I4 (answer): a reader could rank candidate answers by the product of their grounding in the text world and their fit to the question type, so an answer that is strong on only one cannot win.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Maastricht UniversityMaastricht · Netherlands3 works · 426 citations

Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 1,344 worksSemitic languages · 1,485 works

Cite

Cite as Abdulhai Alali and Abderrahmane Issam. 2026. Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding. Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.vardial-1.28

@inproceedings{alali2026maastricht,
  title = {{Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding}},
  author = {Abdulhai Alali and Abderrahmane Issam},
  booktitle = {Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2026},
  doi = {10.18653/v1/2026.vardial-1.28},
  url = {https://doi.org/10.18653/v1/2026.vardial-1.28}
}

References

22 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. NADI 2024: The fifth nuanced arabic dialect identification shared task. In Proceedings of The Second Arabic Natural Language Processing Conference (Arabic-NLP 2024), pages 709–728, Bangkok, Thailand. Association for Computational Linguistics. find
  2. Kathrein Abu Kwaik, Motaz Saad, Stergios Chatzikyriakidis, and Simon Dobnik. 2018. Shami: A corpus of levantine arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA). Dataset includes Levantine Arabic tweets covering Palestinian, Jordanian, Syrian, and Lebanese dialects. find
  3. Across submissions from other teams, our system achieved the highest ADI2 for Syrian and Abdullah Alabdullah, Lifeng Han, and Chenghua Lin. Saudi, and the highest chrF++ scores on translation 2025. Advancing dialectal arabic to modern stan-from English and MSA into Syrian (ENG→DA dard arabic machine translation. arXiv preprint and MSA→DA). On Moroccan Arabic, our model arXiv:2507.20301. Dialectal Arabic–MSA MT chal-performs best on the human evaluation of fluency. lenges and resource-efficient strategies. 356 ↗
  4. Naif Alanazi, Mohammed Al-Batineh, and Hussein Abu-Rayyash. 2025. Saudial: Saudi arabic dialects game localization dataset. https://data.mendeley.com/datasets/mzdwkb2t6d/2. Parallel Saudi dialect text dataset (English, MSA, Saudi varieties) with cultural context, age ratings, and dialect notes. CC-BY 4.0 license. ↗
  5. Ankur Bapna and Orhan Firat. 2019. Simple, scalable adaptation for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2414–2425. ↗
  6. A. Bergman and Mona Diab. 2022. Towards responsible natural language annotation for the varieties of Arabic. In Findings of the Association for Computational Linguistics: ACL 2022, pages 364–371, Dublin, Ireland. Association for Computational Linguistics. find
  7. Peter Bickel and Kjell Doksum. 2007. Mathematical Statistics: Basic Ideas and Selected Topics., volume 56. find
  8. Darija Open Dataset Contributors. 2023. Doda: Darija open dataset. https://github.com/ darija-open-dataset/dataset/tree/main? tab=readme-ov-file#citation. Dataset of 48,849 Darija sentences, with English translations. CC-BY-NC license. Downloadable via link. High quality. ↗
  9. Hiroyuki Deguchi, Yusuke Sakai, Hidetaka Kamigaito, and Taro Watanabe. 2024. mbrs: A library for minimum Bayes risk decoding. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 351–362, Miami, Florida, USA. Association for Computational Linguistics. find
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783. ↗
  11. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. ALDi: Quantifying the Arabic level of dialectness of text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10597–10611, Singapore. Association for Computational Linguistics. find
  12. Mateusz Krubiński, Hashem Sellat, Shadi Saleh, Adam Pospísil, Petr Zemánek, and Pavel Pecina. 2023. Multi-parallel corpus of north levantine arabic. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023), pages 411–417. Association for Computational Linguistics. Corpus includes 120,600 multiparallel sentences in English, French, German, Greek, Spanish and MSA manually translated into North Levantine Arabic. find
  13. Shankar Kumar and William Byrne. 2004. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 169–176, Boston, Massachusetts, USA. Association for Computational Linguistics.
  14. Mohamed bin Zayed University of Artificial Intelligence (MBZUAI). 2025. Inception, cerebras and mbzuai release Jais 2 – the next generation arabic open-weight llm. https://mbzuai.ac.ae. Accessed: 2026-01-21. ↗
  15. Maja Popović. 2015. chrf: character n-gram f-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. ↗
  16. Maja Popović. 2015. chrf: Character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395.
  17. Nathaniel Robinson, Shahd Abdelmoneim, Anjali Kantharuban, Otba Alsboul, Salima Lamsiyah, Kelly Marchisio, and Kenton Murray. 2026. AMIYA shared task: Arabic Modeling In Your Accent at Var-Dial 2026. In Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects, Rabat, Morocco. Association for Computational Linguistics. find
  18. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. Preprint, arXiv:1902.00751. ↗
  19. Nathaniel R. Robinson, Shahd Abdelmoneim, Kelly Marchisio, and Sebastian Ruder. 2025. Al-qasida: Analyzing llm quality and accuracy systematically in dialectal arabic. Preprint, arXiv:2412.04193. ↗
  20. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. https://arxiv.org/abs/2106.09685. Presented at ICLR 2022; parameter-efficient adaptation method for large pre-trained models. ↗
  21. TaghreedT/SDC Contributors. 2020. Saudi dialect corpus (sdc). https://github.com/TaghreedT/SDC/ blob/main/SDC.txt. A 210,396-word corpus of Saudi Arabic social media posts collected from platforms such as Facebook and Twitter. High quality text format, accessible via link; please cite Tarmom et al. (2020) when using this corpus. ↗
  22. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. TIES-merging: Resolving interference when merging models. In Thirty-seventh Conference on Neural Information Processing Systems.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alali-2026-maastricht-university-amiya-adapting-llms/