Elamur Research⌕
☰
home / papers / 2026 / Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer

Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer

Paper · 2026 · Ahmed Haj Ahmed, Ruochen Zhang and Alvin C Grissom II

Summary

The paper asks whether fine-tuning an LLM on one Semitic language helps it on related languages because of shared linguistic structure, or for a more mundane reason. The authors fine-tuned seven models with LoRA on 4,800 Belebele reading-comprehension items in six Arabic dialects. The models range from 4B to 671B parameters and include dense Qwen3 models and the Mixture-of-Experts models GPT-OSS and DeepSeek-V3.1. They then tested zero-shot on Hebrew, Amharic and Maltese, which are related to Arabic but use different scripts. Japanese, Korean and French served as unrelated controls. A split by question ID kept parallel versions of training questions out of every test set.

The result is negative. Arabic training gave no Semitic-specific advantage. Weak-baseline models, especially the two GPT-OSS models, improved sharply and about equally on all languages, while strong models barely changed. A chain-of-thought ablation showed the same models gaining similar amounts from reasoning at inference time. The authors conclude that the gains come from learning the task format, not from transferring language knowledge. This is a caution against reading multilingual benchmark gains as true cross-lingual transfer.

Key points

  • Training data: for each of six Arabic dialects (Moroccan, Najdi, Tunisian, Mesopotamian, and the North and South Levantine varieties), 800 of the 900 Belebele items were used. Each evaluation language gets 100 items from the Belebele validation split.
  • One configuration was used for every model:
    • LoRA rank 32, three epochs, batch size 64;
    • learning rate 2×10⁻⁵ with linear decay, Adam;
    • loss computed only on the single answer digit;
    • training run through the Tinker API.
  • On Amharic, the lowest-resource target, DeepSeek-V3.1 gained +6.0 pp and GPT-OSS-120B gained +22.0 pp. Neither gain was larger than the gains on high-resource non-Semitic controls.
  • The evidence combines three analyses:
    • a regression that controls for baseline accuracy;
    • paired per-question uncertainty bands, under which several dense-model gains cannot be told apart from zero;
    • an analysis of answer-choice logprob distributions, showing both interventions sharpen poorly calibrated distributions.
  • Stated limitations:
    • only multiple-choice tasks scored by logprob ranking;
    • Arabic is the only source language;
    • CoT traces often reveal the answer explicitly;
    • MoE expert routing was not examined.

Interesting ideas

  • I1. Pair the "related" test sets with deliberately unrelated control sets, so a family-specific effect can be told apart from generic adaptation.
  • I2. Split data by a shared item identifier across all parallel versions, so no translated or paraphrased copy of a training question leaks into any test set.
  • I3. Model the gains as a function of baseline accuracy, because weak starters improve most and can fake a "transfer" effect.
  • I4. If the same systems benefit by similar amounts from two quite different interventions, the interventions probably repair the same deficit.
  • I5. Track how peaked the score distribution over answer options is, not just accuracy, to tell real understanding apart from mere commitment to an answer.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when a repair rule or retraining helps on one text genre, also score an unrelated genre as a negative control. This shows whether the fix is specific or just general tuning.
  • I2 (learn): when silver data includes paraphrases or multiple renderings of the same source item, split train and test by source ID rather than by sentence.
  • I3 (learn): evaluation reports should regress improvement on prior per-component accuracy before crediting a new module with a targeted benefit.
  • I4 (answer): on school tests, compare which items improve under a question-format fix versus a world-reasoning fix. Large overlap would suggest the bottleneck is reading the test format, not understanding the content.
  • I5 (answer): multiple-choice readers could log their per-option scores. A sharpening that appears only on items already answered correctly would reveal calibration gains rather than new comprehension.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Brown UniversityProvidence · United States76 works · 25,952 citations🎓Haverford CollegeHaverford · United States3 works · 457 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Ahmed Haj Ahmed, Ruochen Zhang and Alvin C Grissom II. 2026. Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.62

@inproceedings{ahmed2026disentangling,
  title = {{Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer}},
  author = {Ahmed Haj Ahmed, Ruochen Zhang and Alvin C Grissom II},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.62},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.62}
}

References

42 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. Arabert: Transformer-based model for arabic language understanding. Preprint, arXiv:2003.00104. ↗
  3. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. ↗
  4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. Preprint, arXiv:2204.05862. ↗
  5. Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024. The belebele benchmark: a parallel reading comprehension dataset in 122 language variants. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 749–775. Association for Computational Linguistics.
  6. Ajitesh Bankula and Praney Bankula. 2025. Cross-linguistic transfer in multilingual nlp: The role of language families and morphology. Preprint, arXiv:2505.13908. ↗
  7. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pages 610–623, New York, NY, USA. Association for Computing Machinery. Emily M Bender and Alvin Grissom. Power shift: Toward inclusive natural language processing. find
  8. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics.
  9. Tyler A. Chang, Catherine Arnett, Zhuowen Tu, and Benjamin K. Bergen. 2023. When is multilinguality a curse? language modeling for 250 high- and low-resource languages. Preprint, arXiv:2311.09205. ↗
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805. ↗
  11. Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. Loramoe: Alleviate world knowledge forgetting in large language models via moe-style plugin. Preprint, arXiv:2312.09979. ↗
  12. Juuso Eronen, Michal Ptaszynski, and Fumito Masui. 2023. Zero-shot cross-lingual transfer language selection using linguistic similarity. Information Processing & Management, 60(3):103250. ↗
  13. Alice Faber. 1997. Genetic subgrouping of the semitic languages. The Semitic Languages, 15:3–15. ↗
  14. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Preprint, arXiv:2101.03961. ↗
  15. Tingchen Fu, Deng Cai, Lemao Liu, Shuming Shi, and Rui Yan. 2024. Disperse-then-merge: Pushing the limits of instruction tuning via alignment tax reduction. Preprint, arXiv:2405.13432. ↗
  16. Andrew Gelman and Sander Greenland. 2019. Are confidence intervals better termed “uncertainty intervals”? BMJ, 366. find
  17. Rink Hoekstra, Richard D Morey, Jeffrey N Rouder, and Eric-Jan Wagenmakers. 2014. Robust misinterpretation of confidence intervals. Psychonomic bulletin & review, 21(5):1157–1164. ↗
  18. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. Preprint, arXiv:1905.10044. ↗
  19. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. ↗
  20. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. find
  21. Unsupervised Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. Preprint, arXiv:1705.03551. ↗
  22. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2021. The state and fate of linguistic diversity and inclusion in the nlp world. Preprint, arXiv:2004.09095. ↗
  23. and Samuel R. Bowman. 2020. English intermediate-task training improves zero-shot cross-lingual transfer too. Preprint, arXiv:2005.13013. ↗
  24. Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020. Unifiedqa: Crossing format boundaries with a single qa system. Preprint, arXiv:2005.00700. ↗
  25. Diederik P. Kingma and Jimmy Ba. 2017. Adam: A method for stochastic optimization. Preprint, arXiv:1412.6980. ↗
  26. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2024. Understanding the effects of rlhf on llm generalisation and diversity. Preprint, arXiv:2310.06452. ↗
  27. Thinking Machines Lab. 2025. Tinker. find
  28. Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers. Preprint, arXiv:2005.00633. ↗
  29. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, and 2 others. 2022. Few-shot learning with multilingual language models. Preprint, arXiv:2112.10668. ↗
  30. Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, and Tong Zhang. 2024. Mitigating the alignment tax of rlhf. Preprint, arXiv:2309.06256. ↗
  31. John J. McCarthy. 1981. A prosodic theory of nonconcatenative morphology. Linguistic Inquiry, 12:373. find
  32. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2021. Show your work: Scratchpads for intermediate computation with language models. Preprint, arXiv:2112.00114. ↗
  33. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. gpt-oss-120b gpt-oss-20b model card. Preprint, arXiv:2508.10925. ↗
  34. Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual bert? Preprint, arXiv:1906.01502. ↗
  35. Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai, and Le Sun. 2024. Learning or self-aligning? rethinking instruction fine-tuning. Preprint, arXiv:2402.18243. ↗
  36. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, and 373 others. 2023. Bloom: A 176b-parameter open-access multilingual language model. Preprint, arXiv:2211.05100. ↗
  37. Mittul Singh, Peter Smit, Sami Virpioja, and Mikko Kurimo. 2020. Effects of language relatedness for cross-lingual transfer learning in character-based language models. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), pages 41– 45, Marseille, France. European Language Resources association. ↗
  38. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, and 20 others. 2022. No language left behind: Scaling human-centered machine translation. Preprint, arXiv:2207.04672. ↗
  39. Zirui Wang, Zachary C. Lipton, and Yulia Tsvetkov. 2020. On negative interference in multilingual models: Findings and a meta-learning treatment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4438–4450, Online. Association for Computational Linguistics.
  40. Janet C E Watson. 2002. The Phonology And Morphology Of Arabic. Oxford University Press. ↗
  41. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 after controlling for baseline accuracy, Semitic target languages gain less from Arabic fine-tuning than non-Semitic controls. The incremental variance explained by the Semitic indicator is modest (∆R 2 = 0.065) and runs counter to the direction predicted by language-family transfer. others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. ↗
  42. Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. 2018. Record: Bridging the gap between human and machine commonsense reading comprehension. Preprint, arXiv:1810.12885. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ahmed-2026-disentangling-linguistic-relatedness-task-alignment/