Elamur Research⌕
☰
home / papers / 2026 / All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG

All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG

Paper · 2026 · Dan Wang, Guozhao Mo, Yafei Shi et al.

Summary

Multilingual RAG (mRAG) systems can draw on documents written in many languages. This paper shows that the reranking step skews heavily toward English and toward the language of the query, so relevant evidence in other languages tends to be ignored. To test this, the authors built a pipeline over a 13-language Wikipedia corpus. BGE-M3 retrieves 50 candidates, a reranker keeps the top 5, and an LLM generates the answer, with MKQA as the evaluation set. They also designed an "oracle evidence" estimate: candidates are reranked separately within each language, and the language group that produces the best answer is taken as the upper bound. Standard rerankers fall well short of this bound. The oracle's best evidence is also spread across many languages, not concentrated in English or the query language. To close the gap, the authors propose LAURA, which trains the reranker on signals from generation quality rather than plain relevance. LAURA reduces the language skew and improves answer quality. The work is relevant to anyone building retrieval over mixed-language collections, including non-English settings such as Ukrainian.

Key points

  • With BGE-Reranker-V2-M3, more than 70% of the top-5 documents (averaged over 13 languages) are either English or in the query language.
  • On MKQA, the 2.7K questions that overlap with KILT NQ, vanilla rerankers score nearly 20% below the estimated oracle. The metric is character 3-gram recall.
  • Models tested:
    • Rerankers: BGE-Reranker-V2-M3 (encoder-only) and Qwen3-Reranker-0.6B (LLM-based).
    • Generators: Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct.
  • LAURA was trained on 17,360 queries. Compared with the base BGE-Reranker, it raised 3-gram recall from 48.9 to 49.9 with Llama and from 46.7 to 47.7 with Qwen. The correlation between reranker scores and answer quality also strengthened, for example Pearson 0.188 → 0.247 with Qwen.
  • Two alternative fine-tuning baselines did not match LAURA:
    • Naive self-training gave no gain.
    • Fine-tuning on mMARCO (20k queries) slightly reduced performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🏢Ant GroupHangzhou · China42 works · 1,131 citations🔬Institute of SoftwareBeijing · China21 works · 4,980 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Dan Wang, Guozhao Mo, Yafei Shi, Cheng Zhang, Bo Zheng, Boxi Cao, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han and Le Sun. 2026. All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.338

@inproceedings{wang2026all,
  title = {{All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG}},
  author = {Dan Wang, Guozhao Mo, Yafei Shi, Cheng Zhang, Bo Zheng, Boxi Cao, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han and Le Sun},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.338},
  url = {https://doi.org/10.18653/v1/2026.acl-long.338}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen Amiraz, Yaroslav Fyodorov, Elad Haramaty, Zohar Karnin, and Liane Lewin-Eytan. 2025. The cross-lingual cost: Retrieval biases in RAG over Arabic-English corpora. In Proceedings of The Third Arabic Natural Language Processing Conference, pages 69–83, Suzhou, China. Association for Computational… find
  2. Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021a. XOR QA: Cross-lingual open-retrieval question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  3. Akari Asai, Xinyan Yu, Jungo Kasai, and Hanna Hajishirzi. 2021b. One question answering model for many languages with cross-lingual dense passage retrieval. Advances in Neural Information Processing Systems, 34:7547–7560. find
  4. Akari Asai, Xinyan Yu, Jungo Kasai, and Hannaneh Hajishirzi. 2021c. One question answering model for many languages with cross-lingual dense passage retrieval. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS ’21, Red Hook, NY, USA. Curran Associates… find
  5. Luiz Bonifacio, Vitor Jeronymo, Hugo Queiroz Abonizio, Israel Campiotti, Marzieh Fadaee, Roberto Lotufo, and Rodrigo Nogueira. 2022. mmarco: A multilingual version of the ms marco passage ranking dataset. Preprint, arXiv:2108.13897. — [arXiv:2108.13897](https://arxiv.org/abs/2108.13897) find
  6. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. find
  7. Nadezhda Chirkova, David Rau, Hervé Déjean, Thibault Formal, Stéphane Clinchant, and Vassilina Nikoulina. 2024. Retrieval-augmented generation in multilingual settings. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 177–188, Bangkok, Thailand. Assoc… find
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, and others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.…
  9. Yifan Gao, Qingyu Yin, Zheng Li, Rui Meng, Tong Zhao, Bing Yin, Irwin King, and Michael Lyu. 2022. Retrieval-augmented multilingual keyphrase generation with retriever-generator iterative training. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1233–1246, Seattle, Un… find
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  11. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  12. Patrick Lewis, Ethan Perez, Aleksandara Piktus, Filippo Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. arXiv: Computation and Language,arX…
  13. Bryan Li, Samar Haider, Fiona Luo, Adwait Agashe, and Chris Callison-Burch. 2024. BordIRlines: A dataset for evaluating cross-lingual retrieval augmented generation. In Proceedings of the First Workshop on Advancing Natural Language Processing for Wikipedia, pages 1–13, Miami, Florida, USA. Associat… find
  14. Wei Liu, Sony Trenous, Leonardo F. R. Ribeiro, Bill Byrne, and Felix Hieber. 2025. XRAG: Cross-lingual retrieval-augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 15669–15690, Suzhou, China. Association for Computational Linguistics. find
  15. Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. Mkqa: A linguistically diverse benchmark for multilingual open domain question answering. Transactions of the Association for Computational Linguistics, page 1389–1406.
  16. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  17. Hoyeon Moon, Byeolhee Kim, and Nikhil Verma. 2025. Quality-aware translation tagging in multilingual RAG system. In Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025), pages 161–177, Suzhuo, China. Association for Computational Linguistics. find
  18. Jeonghyun Park and Hwanhee Lee. 2025. Investigating language preference of multilingual RAG systems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5647–5675, Vienna, Austria. Association for Computational Linguistics. find
  19. Jirui Qi, Raquel Fernández, and Arianna Bisazza. 2025. On the consistency of multilingual context utilization in retrieval-augmented generation. In Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025), pages 199–225, Suzhuo, China. Association for Computational Linguist… find
  20. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  21. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
  22. Eugene Yang, Thomas Jänich, James Mayfield, and Dawn Lawrie. 2024. Language fairness in multilingual information retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 2487–2491, New York, NY, USA. Association fo… find
  23. Xinyu Zhang, Kelechi Ogueji, Xueguang Ma, and Jimmy Lin. 2023. Toward best practices for training multilingual dense retrieval models. ACM Trans. Inf. Syst., 42(2). find
  24. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  25. Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. 2025. Swift: A scalable lightweight infrastructure for fine-tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(28):29733–297… find
  26. Metric Implementation Details We report some evaluation metrics in our experiments: character 3-gram Recall, Precision@k, NDCG@k, and PEER. Below, we describe their implementations in detail. (2) This metric is robust to tokenization differences and is particularly suitable for multilingual evaluati… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-all-languages-matter-understanding-mitigating/