Elamur Research⌕
☰
home / papers / 2025 / M-RewardBench: Evaluating Reward Models in Multilingual Settings

M-RewardBench: Evaluating Reward Models in Multilingual Settings

Paper · 2025 · Srishti Gureja, Lester James V. Miranda, Shayekh Bin Islam et al.

Summary

Reward models (RMs) steer LLM alignment, but they are built and tested almost entirely in English. This paper asks how well RMs judge responses in other languages. The authors built M-RewardBench, a benchmark of about 2.87k preference instances spanning 23 typologically varied languages, including Ukrainian.

For general skills (Chat, Chat-Hard, Safety, Reasoning), they machine-translated RewardBench with the Google Translate API. Native speakers then reviewed the translations, and items with translation errors or English-bound content were dropped. For multilingual skill, they added translation-preference pairs derived from MAPLE, which is built on WMT20/21 data. Each item compares a chosen and a rejected response, and accuracy is the share of cases where the RM prefers the chosen one.

They evaluated 25 classifier, generative, and DPO-based implicit RMs. Every model scored lower than on English RewardBench, with drops of up to 13%. Better translation quality and higher language resource availability both tracked with better RM accuracy. The released data and code give multilingual alignment work a common yardstick for comparing models.

Key points

  • Coverage: M-RewardBench spans 23 languages (excluding English) across 8 scripts, 8 language families and 12 subgroups. It is published on Hugging Face as CohereLabsCommunity/multilingual-reward-bench.
  • Translation tasks: Built from MAPLE for de↔en and zh↔en. There are TRANSLATION-EASY and TRANSLATION-HARD subsets of 100 pairs per direction, with minimum human-score gaps of 0.25 and 0.50 respectively.
  • Models tested: 25 RMs ranging from 3B to 104B parameters. GPT-4 Turbo leads with an average of 83.5 (83.1 on Ukrainian), followed by GPT-4o at 81.1 and Gemma 2 9B at 76.6.
  • Consistency with English results: Scores on English RewardBench and averages on M-RewardBench correlate strongly (Pearson r = 0.92, Spearman ρ = 0.89). Models performed best on Latin- and Cyrillic-script languages.
  • Label preservation: After refinement, native-speaker agreement with the original preference labels reached 94.2% for Hindi, 95.3% for Indonesian and 98.0% for Spanish.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations🏢CohereSan Francisco · United States15 works · 1,223 citations🏢ServiceNowSanta Clara · United States10 works · 934 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Srishti Gureja, Lester James V. Miranda, Shayekh Bin Islam, Rishabh Maheshwary, Drishti Sharma, Gusti Winata, Nathan Lambert, Sebastian Ruder, Sara Hooker and Marzieh Fadaee. 2025. M-RewardBench: Evaluating Reward Models in Multilingual Settings. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.3

@inproceedings{gureja2025m,
  title = {{M-RewardBench: Evaluating Reward Models in Multilingual Settings}},
  author = {Srishti Gureja, Lester James V. Miranda, Shayekh {Bin Islam}, Rishabh Maheshwary, Drishti Sharma, Gusti Winata, Nathan Lambert, Sebastian Ruder, Sara Hooker and Marzieh Fadaee},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.3},
  url = {https://doi.org/10.18653/v1/2025.acl-long.3}
}

References

4 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. \#\# Limitations Aakanksha, Arash Ahmadian, Beyza Ermis, Seraphina Goldfarb-Tarrant, Julia Kreutzer, Marzieh Fadaee, and Sara Hooker. 2024. The multilingual alignment prism: Aligning global and local preferences to reduce harm.
  2. Generalization to downstream DPO or policy model performance. Although we evaluated how different RMs perform on M-R EWARD B ENCH , it is unclear if high performance on M-R EWARD B ENCH correlates to high performance on downstream multilingual benchmarks. Meanwhile, Ivison et al. (2024) found that i… find
  3. Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, et al. 2024. Aya 23: Open weight releases to further multilingual progress. arXiv preprint arXiv:2405.15032.
  4. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — Qwen Technical Report

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gureja-2025-m-rewardbench-evaluating-reward-models/