Elamur Research⌕
☰
home / papers / 2026 / MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation

MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation

Paper · 2026 · Dawei Wang, Di Zhao, Xinyuan Liu et al.

Summary

The paper addresses credit assignment in cooperative multi-agent reinforcement learning (MARL) for embodied AI. In this setting, agents see only noisy egocentric views, rewards are sparse and delayed, and agents may join or drop out mid-task. Classic methods such as VDN, QMIX and COMA struggle under these conditions.

MARS-RA treats each agent's credit as a hidden contribution score. A large multimodal model (LMM) looks at the agents' visual observations and judges which of two agents helps the team more. A rank aggregation model, by default Bradley–Terry, then turns these noisy relative judgments into continuous per-agent scores. The scores become a potential function for potential-based reward shaping, which supplies dense rewards. Because comparisons are pairwise, the method does not depend on team size.

The authors also introduce MARS-Bench, a ManiSkill3-based benchmark in which agents may be added or removed during an episode. They provide theoretical arguments for convergence and robustness, with Shapley values as an interpretive reference. MARS-RA outperforms strong baselines, and its results improve with more accurate LMMs and more comparison queries.

Key points

  • MARS-Bench: 2–4 decentralized agents work on three tasks (Pass Gate, Herd Sheep, Collect Ball) with partial observability, sparse rewards and agents entering or leaving.
  • Comparison accuracy: Gemini-2.5-Pro and GPT-5.1 exceed 70% accuracy on pairwise comparisons. Qwen3-VL (2B/4B/8B) exceeds 43%. Errors cluster in cases where every agent faces a wall and no informative cues are visible.
  • Accuracy vs. query count: more accurate LMMs and more queries both raise success rates. However, extra queries help less once the LMM is already accurate.
  • Rank aggregation choice: Bradley–Terry slightly beats Rank Centrality.
    • Pass Gate: 0.52 vs. 0.50
    • Herd Sheep: 0.68 vs. 0.67
    • Collect Ball: 0.46 vs. 0.46
  • Real-world transfer: policies trained in a 3D-scanned simulated room were deployed on two XLeRobot robots for Pass Gate, reaching a 64% success rate over 25 runs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksEngineering · 133 worksControl and Systems Engineering · 45 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Dawei Wang, Di Zhao, Xinyuan Liu, Marci Chi Ma, Xiaoyang Liu, Chengming Zhou, Gary Ushaw and Richard Davison. 2026. MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1349

@inproceedings{wang2026mars,
  title = {{MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation}},
  author = {Dawei Wang, Di Zhao, Xinyuan Liu, Marci Chi Ma, Xiaoyang Liu, Chengming Zhou, Gary Ushaw and Richard Davison},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1349},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1349}
}

References

67 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alireza Saleh Abadi and Leen-Kiat Soh. 2025. Challenges in credit assignment for multi-agent reinforcement learning in open agent systems. arXiv preprint arXiv:2510.27659. — [arXiv:2510.27659](https://arxiv.org/abs/2510.27659) find
  2. Anatoly Bakushinsky and A Goncharsky. 2012. Ill-posed problems: theory and applications, volume 301. Springer Science & Business Media. find
  3. Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning, volume 4. Springer. find
  4. On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems, 32. find
  5. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. find
  6. Andy Clark. 1998. Being there: Putting brain, body, and world together again. MIT press. find
  7. Jonathan Cohen, Jilles-Steeve Dibangoye, and Abdel-Illah Mouaddib. 2017. Open decentralized pomdps. In 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI), pages 977–984. IEEE. find
  8. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  9. Douglas E Critchlow, Michael A Fligner, and Joseph S Verducci. 1991. Probability models on rankings. Journal of mathematical psychology, 35(3):294–318. find
  10. Gerard Debreu. 1960. Individual choice behavior: A theoretical analysis. find
  11. Sam Michael Devlin and Daniel Kudenko. 2012. Dynamic potential-based reward shaping. In 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), pages 433–440. IFAAMAS. find
  12. Jingfeng Zhang Di Zhao, Hongsheng Hu, Philippe Fournier-Viger, Gillian Dobbie, and Yun Sing Koh. 2025. Balancing invariant and specific knowledge for domain generalization with online knowledge distillation. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligenc… find
  13. Shuai Dong, Siyuan Wang, Xingyu Liu, Chenglin Li, Haowen Hou, and Zhongyu Wei. 2025. Interleaved latent visual reasoning with selective perceptual modeling. arXiv preprint arXiv:2512.05665.
  14. Tongtong Feng, Xin Wang, Yu-Gang Jiang, and Wenwu Zhu. 2025. Embodied ai: From llms to world models. arXiv preprint arXiv:2509.20021. — [arXiv:2509.20021](https://arxiv.org/abs/2509.20021) find
  15. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324– 345. find
  16. Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2017. Counterfactual multi-agent policy gradients (2017). arXiv preprint arXiv:1705.08926. — [arXiv:1705.08926](https://arxiv.org/abs/1705.08926) find
  17. Rodney A Brooks. 1991. New approaches to robotics. Science, 253(5025):1227–1232. find
  18. Anna Harutyunyan, Sam Devlin, Peter Vrancx, and Ann Nowé. 2015. Expressing arbitrary reward functions as potential-based advice. In Proceedings of the AAAI conference on artificial intelligence, volume 29. find
  19. Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. 2019. find
  20. Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. Trueskill™: a bayesian skill rating system. Advances in neural information processing systems, 19. find
  21. Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, and 1 others. 2023. Language is not all you need: Aligning perception with language models. Advances in Neural Information Processing Systems, 36:72096– 72109.
  22. Raivo Kolde, Sven Laur, Priit Adler, and Jaak Vilo. 2012. Robust rank aggregation for gene list integration and meta-analysis. Bioinformatics, 28(4):573–580. find
  23. Chenglin Li, Qianglong Chen, Feng Han, Yikun Wang, Xingxi Yin, Yan Gong, Ruilin Li, Yin Zhang, and Jiaqi Wang. 2026. Videothinker: Building agentic videollms with llm-guided tool reasoning. arXiv preprint arXiv:2601.15724. — [arXiv:2601.15724](https://arxiv.org/abs/2601.15724) find
  24. Chenglin Li, Feng Han, Yikun Wang, Ruilin Li, Shuai Dong, Haowen Hou, Haitao Li, Qianglong Chen, Feng Tao, Jingqi Tong, and 1 others. 2025a. Video-pro: Adaptive program reasoning for long video understanding. arXiv preprint arXiv:2509.17743. — [arXiv:2509.17743](https://arxiv.org/abs/2509.17743) find
  25. Wenhao Li, Dan Qiao, Baoxiang Wang, Xiangfeng Wang, Wei Yin, Hao Shen, Bo Jin, and Hongyuan Zha. 2025b. Multi-agent credit assignment with pre-trained language models. In International Conference on Artificial Intelligence and Statistics, pages 1945–1953. PMLR. find
  26. Yunxin Li, Baotian Hu, Xinyu Chen, Lin Ma, Yong Xu, and Min Zhang. 2024. Lmeye: An interactive perception network for large language models. IEEE Transactions on Multimedia. find
  27. Muhan Lin, Shuyang Shi, Yue Guo, Vaishnav Tadiparthi, Behdad Chalaki, Ehsan Moradi Pari, Simon Stepputtis, Woojun Kim, Joseph Campbell, and Katia Sycara. 2025. Speaking the language of teamwork: Llm-guided credit assignment in multi-agent reinforcement learning. arXiv preprint arXiv:2502.03723. — [arXiv:2502.03723](https://arxiv.org/abs/2502.03723) find
  28. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. Visual instruction tuning. Advances in neural information processing systems, 36:34892– 34916.
  29. Wenzhang Liu, Wenzhe Cai, Kun Jiang, Guangran Cheng, Yuanda Wang, Jiawei Wang, Jingyu Cao, Lele Xu, Chaoxu Mu, and Changyin Sun. 2023b. Xuance: A comprehensive and unified deep reinforcement learning library. arXiv preprint arXiv:2312.16248. find
  30. Hao Ma, Shijie Wang, Zhiqiang Pu, Siyao Zhao, and Xiaolin Ai. 2025. Vision-based generic potential function for policy alignment in multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 19287–19295. find
  31. Ke Ma, Qianqian Xu, Jinshan Zeng, Guorong Li, Xiaochun Cao, and Qingming Huang. 2022. A tale of hodgerank and spectral method: Target attack against rank aggregation is the fixed point of adversarial game. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4090–4108. find
  32. Marvin Minsky. 2007. Steps toward artificial intelligence. Proceedings of the IRE, 49(1):8–30. find
  33. Kartik Nagpal, Dayi Dong, Jean-Baptiste Bouvier, and Negar Mehr. 2025. Leveraging large language models for effective and explainable multi-agent credit assignment. arXiv preprint arXiv:2502.16863. — [arXiv:2502.16863](https://arxiv.org/abs/2502.16863) find
  34. Sahand Negahban, Sewoong Oh, and Devavrat Shah. 2012a. Iterative ranking from pair-wise comparisons. Advances in neural information processing systems, 25. find
  35. Sahand N Negahban, Pradeep Ravikumar, Martin J Wainwright, and Bin Yu. 2012b. A unified framework for high-dimensional analysis of m-estimators with decomposable regularizers. Statistical Science, 27(4):538–557. find
  36. Andrew Y Ng, Daishi Harada, and Stuart Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In Icml, volume 99, pages 278–287. Citeseer. find
  37. OpenAI. 2025. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf. ↗
  38. Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V Albrecht. 2020. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. arXiv preprint arXiv:2006.07869. — [arXiv:2006.07869](https://arxiv.org/abs/2006.07869) find
  39. Dipam Patel, Phu Pham, Kshitij Tiwari, and Aniket Bera. 2023. Dream: Decentralized reinforcement learning for exploration and efficient energy management in multi-robot systems. arXiv preprint arXiv:2309.17433. — [arXiv:2309.17433](https://arxiv.org/abs/2309.17433) find
  40. Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. 2020. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178):1–51. find
  41. Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. 2023. Vision-language models are zero-shot reward models for reinforcement learning. arXiv preprint arXiv:2310.12921. find
  42. Álvaro Serra-Gómez, Hai Zhu, Bruno Brito, Wendelin Böhmer, and Javier Alonso-Mora. 2023. Learning scalable and efficient communication policies for multi-robot collision avoidance. Autonomous Robots, 47(8):1275–1297. find
  43. Nihar B Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, and Martin J Wainwright. 2016. Estimation from pairwise comparisons: Sharp minimax bounds with topology dependence. Journal of Machine Learning Research, 17(58):1–47. find
  44. Lloyd S Shapley and 1 others. 1953. A value for n-person games. find
  45. Lin Shi, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, and Soroush Vosoughi. 2024. Judging the judges: A systematic study of position bias in llm-as-a-judge. arXiv preprint arXiv:2406.07791.
  46. Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, and 1 others. 2017. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296. — [arXiv:1706.05296](https://arxiv.org/abs/1706.05296) find
  47. Xuting Tang, Jia Xu, and Shusen Wang. 2023. Roma: Resilient multi-agent reinforcement learning with dynamic participating agents. In 2023 IEEE 12th International Conference on Cloud Networking (Cloud-Net), pages 247–255. IEEE. find
  48. Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse-kai Chan, and 1 others. 2024. Maniskill3: Gpu parallelized robotics simulation and rendering for generalizable embodied ai. arXiv preprint arXiv:2410.00425. find
  49. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  50. J Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis S Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo Perez-Vicente, and 1 others. 2021. Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34:15032… find
  51. Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. 2025. Identifying and mitigating position bias of multi-image vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10599–10609. find
  52. Alan M Turing. 2021. Computing machinery and intelligence (1950). Mind, 59(236):33–60. find
  53. Dawei Wang, Chengming Zhou, Di Zhao, Xinyuan Liu, Marci Chi Ma, Gary Ushaw, and Richard Davison. 2026. Towermind: A tower defence game learning environment and benchmark for llm as agents. arXiv preprint arXiv:2601.05899. — [arXiv:2601.05899](https://arxiv.org/abs/2601.05899) find
  54. Gaotian Wang and Zhuoyi Lu. 2025. Xlerobot: A practical low-cost household dual-arm mobile robot design for general manipulation. https://github.com/ Vector-Wangel/XLeRobot. ↗
  55. Jianhong Wang, Yuan Zhang, Yunjie Gu, and Tae-Kyun Kim. 2022. Shaq: Incorporating shapley value theory into multi-agent q-learning. Advances in Neural Information Processing Systems, 35:5941–5954. find
  56. Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, and Yunjie Gu. 2020. Shapley q-value: A local reward approach to solve global reward games. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7285–7292. find
  57. Yuan Wei, Xiaohan Shan, Ran Miao, and Jianmin Li. 2025. Lero: Llm-driven evolutionary framework with hybrid rewards and enhanced observation for multi-agent reinforcement learning. In International Conference on Intelligent Computing, pages 15–26. Springer. find
  58. Daan Wierstra, Tom Schaul, Jan Peters, and Juergen Schmidhuber. 2008. Episodic reinforcement learning by logistic reward-weighted regression. In International Conference on Artificial Neural Networks, pages 407–416. Springer. find
  59. Annie Wong, Thomas Bäck, Anna V Kononova, and Aske Plaat. 2023. Deep multiagent reinforcement learning: Challenges and directions. Artificial Intelligence Review, 56(6):5023–5056. find
  60. Sarah A Wu, Rose E Wang, James A Evans, Joshua B Tenenbaum, David C Parkes, and Max Kleiman-Weiner. 2021. Too many cooks: Bayesian inference for coordinating multi-agent collaboration. Topics in Cognitive Science, 13(2):414–432. find
  61. Ziyan Xiong, Bo Chen, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, and Yang Gao. 2024. Mqe: Unleashing the power of interaction with multi-agent quadruped environment. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5918–5924. IEEE. find
  62. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  63. Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems, 35:24611–24624. find
  64. Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Behzad Dar-iush, Kwonjoon Lee, Yilun Du, and Chuang Gan. 2024. Combo: compositional world models for embodied multi-agent cooperation. arXiv preprint arXiv:2404.10775. — [arXiv:2404.10775](https://arxiv.org/abs/2404.10775) find
  65. Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control, pages 321–384. find
  66. Di Zhao, Yun Sing Koh, Gillian Dobbie, Hongsheng Hu, and Philippe Fournier-Viger. 2024. Symmetric self-paced learning for domain generalization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16961–16969. find
  67. Di Zhao, Jingfeng Zhang, Hongsheng Hu, Philippe Fournier-Viger, Gillian Dobbie, and Yun Sing Koh. 2026. Unlearning during training: Domain-specific gradient ascent for domain generalization. In The Fourteenth International Conference on Learning Representations. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-mars-ra-rank-aggregation-credit/