Elamur Research⌕
☰
home / papers / 2026 / ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation

ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation

Paper · 2026 · Ziyi Liu, Bahareh Sarrafzadeh, Pei Zhou et al.

Summary

Most LLM assistants serve one user at a time, yet many real tasks depend on groups reaching agreement. There has been no systematic way to test whether an AI can step into a group discussion on its own initiative and help it move forward. ProMediate addresses this gap with a simulation testbed for proactive mediation in negotiations that involve several parties and several issues. It adapts six multi-party cases from Harvard's Program on Negotiation. LLM-simulated participants receive structured preference rankings and a shared conflict style, which together set Easy, Medium and Hard difficulty levels. A plug-in mediator agent decides separately whether to intervene and what to say. Evaluation follows a socio-cognitive approach: it tracks how agreement develops over the conversation and scores the mediator's perceptual, emotional, cognitive and communicative skills. In the Hard setting, a socially intelligent mediator achieved larger consensus gains than a generic one and responded faster. However, good interventions did not always produce immediate agreement. The testbed offers a reproducible way to test collaborative agents before they are used with real people.

Key points

  • Three conditions are compared across six scenarios and three difficulty levels: NoAgent, Generic Mediator and Socially Intelligent Mediator. The difficulty levels are built from the Competing, Avoiding and Accommodating conflict modes.
  • In ProMediate-Hard, consensus change was 10.65% for the social mediator versus 7.01% for the generic baseline. The reported response time was about 77% faster, with latencies of 3.71s and 15.98s.
  • Three annotators rated 200 simulated dialogues on a 5-point scale. Naturalness averaged 4.35 and conflict-mode fidelity averaged 3.87.
  • Exploratory factor analysis found two factors among the metrics: consensus and topic efficiency, and intervention tempo. Mediator Intelligence did not load clearly on either factor.
  • At the intervention level, Mediator Effectiveness and Mediator Intelligence were essentially uncorrelated (Spearman 0.01, p = 0.89). The code is at github.com/microsoft/ProMediate-Eval.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States325 works · 142,931 citations🎓University of Southern CaliforniaLos Angeles · United States162 works · 35,836 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 worksEconomics, Econometrics and Finance · 17 worksEconomics and Econometrics · 12 works

Cite

Cite as Ziyi Liu, Bahareh Sarrafzadeh, Pei Zhou, Longqi Yang, Jieyu Zhao and Ashish Sharma. 2026. ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1479

@inproceedings{liu2026promediate,
  title = {{ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation}},
  author = {Ziyi Liu, Bahareh Sarrafzadeh, Pei Zhou, Longqi Yang, Jieyu Zhao and Ashish Sharma},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1479},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1479}
}

References

51 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. 2024. Cooperation, competition, and maliciousness: Llm-stakeholders interactive negotiation. Advances in Neural Information Processing Systems, 37:83548–83599. find
  2. Mohammed Alsobay, David M. Rothschild, Jake M. Hofman, and Daniel G. Goldstein. 2025. Bringing everyone to the table: An experimental study of llm-facilitated group decision making. Preprint, arXiv:2508.08242. — [arXiv:2508.08242](https://arxiv.org/abs/2508.08242) find
  3. Wendy L Bedwell, Jessica L Wildman, Deborah Diaz-Granados, Maritza Salazar, William S Kramer, and Eduardo Salas. 2012. Collaboration at work: An integrative multilevel conceptualization. Human resource management review, 22(2):128–145. find
  4. Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, and Owain Evans. 2023. Taken out of context: On measuring situational awareness in llms. arXiv preprint arXiv:2309.00667. — [arXiv:2309.00667](https://arxiv.org/abs/2309.00667) find
  5. Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou. 2024. How well can llms negotiate? negotiationarena platform and analysis. Preprint, arXiv:2402.05863. find
  6. Alysoun Boyle. 2017. Effectiveness in mediation: A new approach. Newcastle Law Review, The, 12:148– 161. find
  7. Fabrizio Butera, Nicolas Sommet, and Céline Darnon. 2019. Sociocognitive conflict regulation: How to make sense of diverging ideas. Current Directions in Psychological Science, 28(2):145–151. find
  8. Deborah Cai and Edward Fink. 2002. Conflict style differences between individualists and collectivists. Communication Monographs, 69(1):67–87. find
  9. Junjie Chen, Haitao Li, Minghao Qin, Yujia Zhou, Yanxue Ren, Wuyue Wang, Yiqun Liu, Yueyue Wu, and Qingyao Ai. 2025. Simulating dispute mediation with llm-based agents for legal research. Preprint, arXiv:2509.06586. — [arXiv:2509.06586](https://arxiv.org/abs/2509.06586) find
  10. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, and 1 others. 2023. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2(4):6.
  11. Chun-Wei Chiang, Zhuoran Lu, Zhuoyan Li, and Ming Yin. 2024. Enhancing ai-assisted group decision making through llm-powered devil’s advocate. In Proceedings of the 29th International Conference on Intelligent User Interfaces, IUI ’24, page 103–119, New York, NY, USA. Association for Computing Machi… find
  12. Xiachong Feng, Longxu Dou, Ella Li, Qinghao Wang, Haochuan Wang, Yu Guo, Chang Ma, and Lingpeng Kong. 2025. A survey on large language model-based social agents in game-theoretic scenarios. Preprint, arXiv:2412.03920. — [arXiv:2412.03920](https://arxiv.org/abs/2412.03920) find
  13. Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, and Ji-Rong Wen. 2024. Large language model-based human-agent collaboration for complex task solving. arXiv preprint arXiv:2402.12914. find
  14. Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. Improving language model negotiation with self-play and in-context learning from ai feedback. Preprint, arXiv:2305.10142.
  15. Hitesh Goel and Hao Zhu. 2025. Lifelong sotopia: Evaluating social intelligence of language agents over lifelong social interactions. arXiv preprint arXiv:2506.12666. — [arXiv:2506.12666](https://arxiv.org/abs/2506.12666) find
  16. Amy-Jane Griffiths, James Alsip, Shelley R Hart, Rachel L Round, and John Brady. 2021. Together we can do so much: A systematic review and conceptual framework of collaboration in schools. Canadian Journal of School Psychology, 36(1):59–85. find
  17. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, and 1 others. 2024. Metagpt: Meta programming for a multi-agent collaborative framework. International Conference on Learning Representations, ICLR.
  18. Chun Wei Patrick Chiang. 2025. Enhancing Human-AI Collaboration in AI-Assisted Decision-Making for Individuals and Groups. Ph.D. thesis, Purdue University Graduate School. find
  19. Stephanie Houde, Kristina Brimijoin, Michael Muller, Steven I. Ross, Dario Andres Silva Moran, Gabriel Enrique Gonzalez, Siya Kunde, Morgan A. Foreman, and Justin D. Weisz. 2025. Controlling ai agent participation in group conversations: A human-centered approach. In Proceedings of the 30th Internat… find
  20. Jared R. Curhan, Hillary Anger Elfenbein, and Heng Xu. 2006. What do people value when they negotiate? mapping the domain of subjective value in negotiation. Conflict & Dispute Resolution. find
  21. Steve WJ Kozlowski and Daniel R Ilgen. 2006. Enhancing the effectiveness of work groups and teams. Psychological science in the public interest, 7(3):77– 124. find
  22. Tim R. Davidson, Veniamin Veselovsky, Martin Josifoski, Maxime Peyrard, Antoine Bosselut, Michal Kosinski, and Robert West. 2024. Evaluating language model agency through negotiations. Preprint, arXiv:2401.04536. — [arXiv:2401.04536](https://arxiv.org/abs/2401.04536) find
  23. Deuksin Kwon, Emily Weiss, Tara Kulshrestha, Kushal Chawla, Gale Lucas, and Jonathan Gratch. 2024. Are llms effective negotiators? systematic evaluation of the multifaceted capabilities of llms in negotiation dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages… find
  24. María José del Moral, Francisco Chiclana, Juan Miguel Tapia, and Enrique Herrera-Viedma. 2018. A comparative study on consensus measures in group decision making. International Journal of Intelligent Systems, 33(8):1624–1638. find
  25. Eva Eigner and Thorsten Händler. 2024. Determinants of llm-assisted decision-making. arXiv preprint arXiv:2402.17385. — [arXiv:2402.17385](https://arxiv.org/abs/2402.17385) find
  26. Rudolf Laine, Alexander Meinke, and Owain Evans. 2023. Towards a situational awareness benchmark for llms. In Socially responsible language modelling research. find
  27. John M Levine. 2018. Socially-shared cognition and consensus in small groups. Current opinion in psychology, 23:52–56. find
  28. Shanghao Li, Taylor Lane, Alicia Hernandez, Vinayak Kabra, Karthik Singh, Stefany Sit, and Nikita Soni. 2024. Towards understanding group collaboration patterns around mobile augmented-reality interfaces for geospatial science data visualizations. In Extended Abstracts of the CHI Conference on Human… find
  29. Roderick Swaab, Tom Postmes, Ilja Van Beest, and Russell Spears. 2007. Shared cognition as a product of, and precursor to, shared identity in negotiations. Personality and Social Psychology Bulletin, 33(2):187–199. find
  30. Xingyu Bruce Liu, Shitao Fang, Weiyan Shi, Chien-Sheng Wu, Takeo Igarashi, and Xiang Anthony Chen. 2025a. Proactive Conversational Agents with Inner Thoughts. arXiv preprint. ArXiv:2501.00383 [cs]. — [arXiv:2501.00383](https://arxiv.org/abs/2501.00383) find
  31. Ann Marie Thomson, James L Perry, and Theodore K Miller. 2009. Conceptualizing and measuring collaboration. Journal of public administration research and theory, 19(1):23–56. find
  32. Ziyi Liu, Abhishek Anand, Pei Zhou, Jen-tse Huang, and Jieyu Zhao. 2024. Interintent: Investigating social intelligence of llms via intention understanding in an interactive game context. arXiv preprint arXiv:2406.12203. find
  33. Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D Nguyen. 2025. Multi-agent collaboration mechanisms: A survey of llms. arXiv preprint arXiv:2501.06322. find
  34. Ziyi Liu, Priyanka Dey, Zhenyu Zhao, Jen tse Huang, Rahul Gupta, Yang Liu, and Jieyu Zhao. 2025b. Can llms grasp implicit cultural values? benchmarking llms’ metacognitive cultural intelligence with cq-bench. Preprint, arXiv:2504.01127. — [arXiv:2504.01127](https://arxiv.org/abs/2504.01127) find
  35. Zhenzhong Ma. 2007. Competing or accommodating? an empirical test of chinese conflict management styles. Contemporary Management Research, 3(1):3– 3. find
  36. Michelle A Marks, John E Mathieu, and Stephen J Zaccaro. 2001. A temporally based framework and taxonomy of team processes. Academy of management review, 26(3):356–376. find
  37. Donna Margaret McKenzie. 2015. The role of mediation in resolving workplace relationship conflict. International journal of law and psychiatry, 39:52–59. find
  38. Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, and 1 others. 2024. Agentsense: Benchmarking social intelligence of language agents through interactive scenarios. arXiv preprint arXiv:2410.19346. find
  39. Lourdes Munduate, Francisco J Medina, and Martin C Euwema. 2022. Mediation: Understanding a constructive conflict management tool in the workplace. Journal of Work and Organizational Psychology, 38(3):165–173. find
  40. Omar Shaikh, Valentino Emil Chai, Michele Gelfand, Diyi Yang, and Michael S. Bernstein. 2024. Rehearsal: Simulating conflict to teach conflict resolution. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY, USA. Association for Computing Machinery. find
  41. Yijia Shao, Vinay Samuel, Yucheng Jiang, John Yang, and Diyi Yang. 2024. Collaborative gym: A framework for enabling and evaluating human-agent collaboration. arXiv preprint arXiv:2412.15701. find
  42. Kenneth W Thomas. 2008. Thomas-kilmann conflict mode. TKI Profile and Interpretive Report, 1(11):1– 11. find
  43. Bruce W. Tuckman. 1965. Developmental sequence in small groups. Psychological bulletin, 63:384–99. find
  44. Richard E. Walton and Robert B. Mckersie. 1965. A behavioral theory of labor negotiations: an analysis of a social interaction system. find
  45. Zhefan Wang, Yuanqing Yu, Wendi Zheng, Weizhi Ma, and Min Zhang. 2024. Macrec: A multi-agent collaboration framework for recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2760–2764. find
  46. Marley W Watkins. 2018. Exploratory factor analysis: A guide to best practice. Journal of black psychology, 44(3):219–246. find
  47. Jenny S. Wesche and Andreas Sonderegger. 2019. When computers take the lead: The automation of leadership. Computers in Human Behavior, 101:197– 209. find
  48. Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, and Jianfeng Gao. 2025. Collabllm: From passive responders to active collaborators. Preprint, arXiv:2502.00640. find
  49. Congluo Xu, Zhaobin Liu, and Ziyang Li. 2025. Finarena: A human-agent collaboration framework for financial market analysis and forecasting. Preprint, arXiv:2503.02692. — [arXiv:2503.02692](https://arxiv.org/abs/2503.02692) find
  50. Ruoxi Xu, Hongyu Lin, Xianpei Han, Le Sun, and Yingfei Sun. 2024. Academically intelligent llms are not necessarily socially intelligent. arXiv preprint arXiv:2403.06591. — [arXiv:2403.06591](https://arxiv.org/abs/2403.06591) find
  51. Diyi Yang, Caleb Ziems, William Held, Omar Shaikh, Michael S Bernstein, and John Mitchell. 2024a. Social skill training with large language models. arXiv preprint arXiv:2404.04204. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-promediate-simulation-testbed-evaluating-proactive/