Elamur Research⌕
☰
home / papers / 2026 / When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

Paper · 2026 · Fred Philippy, Siwen Guo, Jacques Klein et al.

Summary

The paper asks whether the rules for choosing a source language in fine-tuning-based cross-lingual transfer also apply to few-shot In-Context Learning (ICL). The authors test every source–target pairing across 18 typologically diverse languages. They use seven parallel benchmarks and six small LLMs (up to 4B parameters), with 4-shot prompts and at most 1,000 test items per benchmark. They then relate transfer scores to several factors: typological similarity from URIEL/lang2vec, embedding alignment measured on FLORES-200, script, and resource level. They also analyse language confusion in generative output.

The results break with fine-tuning intuitions. Linguistic similarity barely predicts ICL transfer, while alignment inside the model's representations is a better indicator. Languages that score well as targets tend to be poor sources. Low-resource, non-Latin-script languages make the strongest demonstration languages. These findings matter for low-resource NLP: picking English or the target language itself for demonstrations is often not the best choice.

Key points

  • Setup: The benchmarks are PAWS-X, XStoryCloze, Global-MMLU, XNLI, MGSM, SIB-200 and XCOPA. The models are Llama 3.2 (1B, 3B), Gemma 3 (1B, 4B), Qwen3 (1.7B) and Phi-4-Mini. Reruns with different seeds produced correlations of ≥ 0.9.
  • Common defaults often lose: Only around 24% of the time did the target language turn out to be the most effective source for itself. English was the worst source in roughly 16% of experiments.
  • Donor vs. recipient: A language's ability to help others and its ability to benefit from others are strongly inverse, with Pearson r = −0.932.
    • English, Spanish, German and Italian are strong targets but weak sources.
    • Thai, Telugu and Bengali show the reverse pattern.
  • Predictors of transfer: The authors tested four lang2vec similarity types: syntactic, genetic, phonological and featural. None correlated consistently with transfer. Cross-lingual embedding alignment explained much more of the variance.
  • Language confusion: Output language was detected line by line with fastText LID.
    • A language's tendency to cause confusion as a source correlates with its confusion as a target (r ≈ 0.63).
    • Non-Latin-script languages suffer more confusion as targets but induce less confusion as sources.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of LuxembourgLuxembourg · Luxembourg27 works · 337 citations🔬Luxembourg Institute of Science and TechnologyBelvaux · Luxembourg6 works · 9 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEnglish · 2,263 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Fred Philippy, Siwen Guo, Jacques Klein and Tegawendé F. Bissyandé. 2026. When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning. Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.mellm-1.31

@inproceedings{philippy2026when,
  title = {{When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning}},
  author = {Fred Philippy, Siwen Guo, Jacques Klein and Tegawendé F. Bissyandé},
  booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.mellm-1.31},
  url = {https://doi.org/10.18653/v1/2026.mellm-1.31}
}

References

32 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and En-Shiun Annie Lee. 2024. SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects. In Proceedings of the 18th Conference of the Eu…
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  3. Samuel Cahyawijaya, Holy Lovenia, and Pascale Fung. 2024. LLMs are few-shot in-context low-resource language learners. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 405–…
  4. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Bel… find
  5. Kajetan Dymkiewicz, Ivan Vulic, Helen Yannakoudakis, Eilam Shapira, Roi Reichart, and Anna Korhonen. 2025. Donors and recipients: On asymmetric transfer across tasks and languages with parameter-efficient fine-tuning. Preprint, arXiv:2511.13368. — [arXiv:2511.13368](https://arxiv.org/abs/2511.13368) find
  6. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. find
  7. Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-Lingual Ability of Multilingual BERT: An Empirical Study. In Proc. of the International Conference on Learning Representations. find
  8. Masahiro Kaneko, Alham Fikri Aji, and Timothy Baldwin. 2025. Balanced multi-factor in-context learning for multilingual large language models. Preprint, arXiv:2502.11495. find
  9. Sunkyoung Kim, Dayeon Ki, Yireun Kim, and Jinsik Lee. 2024. Cross-lingual qa: A key to unlocking in-context cross-lingual performance. Preprint, arXiv:2305.15233. — [arXiv:2305.15233](https://arxiv.org/abs/2305.15233) find
  10. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In International Conference on Machine Learning, pages 3519–3529. PMlR.
  11. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023. Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation. Preprint, arXiv:2305.15011.
  12. Peiqin Lin, Andre Martins, and Hinrich Schuetze. 2025. XAMPLER: Learning to retrieve cross-lingual in-context examples. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3968–3977, Albuquerque, New Mexico. Association for Computational Linguistics. find
  13. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, and 2 others. 2… — Few-shot Learning with Multilingual Language Models
  14. Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational…
  15. Dan Malkin, Tomasz Limisiewicz, and Gabriel Stanovsky. 2022. A balanced data approach for evaluating cross-lingual transfer: Mapping the linguistic blood bank. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… find
  16. Kelly Marchisio, Wei-Yin Ko, Alexandre Berard, Théo Dehaze, and Sebastian Ruder. 2024. Understanding and mitigating language confusion in LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6653– 6677, Miami, Florida, USA. Association for Computatio…
  17. Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, and 57 others. 2025. Phi-4-mini technical…
  18. Benjamin Muller, Yanai Elazar, Benoît Sagot, and Djamé Seddah. 2021. First align, then predict: Understanding the cross-lingual ability of multilingual BERT. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2214–2231,…
  19. Fred Philippy, Siwen Guo, and Shohreh Haddadan. 2023. Towards a common understanding of contributing factors for cross-lingual transfer in multilingual language models: A review. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… find
  20. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
  21. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020. XCOPA: A multilingual dataset for causal commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362–2376, Online. Association…
  22. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022. Language models are multilingual chain-of-thought reasoners. Preprint, arXiv:2210.03057.
  23. Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Chosh… — Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
  24. Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur, and Tanmoy Chakraborty. 2023. Multilingual LLMs are better cross-lingual in-context learners with alignment. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6292–6307, Toronto…
  25. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  26. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, and 20 o… find
  27. Yilei Tu, Andrew Xue, and Freda Shi. 2025. Blessing of multilinguality: A systematic analysis of multilingual in-context learning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6213–6248, Vienna, Austria. Association for Computational Linguistics. find
  28. Genta Winata, Shijie Wu, Mayank Kulkarni, Thamar Solorio, and Daniel Preotiuc-Pietro. 2022. Cross-lingual few-shot learning on unseen languages. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conferen… find
  29. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  30. Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin…
  31. Haneul Yoo, Jiho Jin, Kyunghyun Cho, and Alice Oh. 2025. Code-switching in-context learning for cross-lingual transfer of large language models. Preprint, arXiv:2510.05678. find
  32. Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba Alabi, Xiaoyu Shen, Dietrich Klakow, and Marius Mosbach. 2024. The impact of demonstrations on multilingual in-context learning: A multidimensional analysis. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7342–7371… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/philippy-2026-when-english-isn-t-best/