Elamur Research⌕
☰
home / papers / 2026 / GBC: Gradient-Based Connections for Optimizing Multi-Agent Systems

GBC: Gradient-Based Connections for Optimizing Multi-Agent Systems

Summary

LLM-based multi-agent systems often fail to beat strong single-agent setups. One key reason is that when the final answer is wrong, it is hard to tell which agent or which interaction step caused the error. Existing optimizers mostly use only overall task success as feedback. The authors propose Gradient-Based Connections (GBC) to address this.

GBC treats the system as a directed acyclic graph of prompt–model agents. It uses token-level gradient signals to score how strongly each upstream output shapes each downstream agent. Keeping the strongest links yields an attribution graph. A task-specific loss written in natural language is then traced backward through this graph to find the outputs responsible for errors. An LLM optimizer rewrites the prompts of those agents.

The framework is released as AgentChord, which uses prefix-based gradient computation to save memory. On MultiWOZ and τ-bench, optimization consistently helped multi-agent systems, sometimes lifting them past single-agent baselines. Better attribution also tended to produce larger gains. This makes cross-agent credit assignment a practical tool for diagnosing and improving such systems.

Key points

  • Four connection-weight variants are tested: mean or max of the gradient L1 norm, and mean or max of the gradient–input product. By default only the top-1 predecessor per output is retained (m = 1).
  • On τ-bench with Qwen-3-32B, the max-L1-norm variant raised overall reward from 13.0 to 24.3, beating the single-agent baseline of 22.6. The mean gradient–input variant also reached 24.3, with action reward rising from 13.9 to 27.0.
  • With Llama-3.3-70B-It, every optimized multi-agent variant improved on the pre-optimization baseline (overall reward 6.1). The τ-bench setup used 10 training tasks, with GPT-4o-mini as the user simulator.
  • In τ-bench error analysis, retrieval/identification failures were the most frequent of five error types. The others were tool misuse, unclear manager instructions, premature escalation and incorrect explanations.
  • The authors list several limitations:
    • repeated forward and backward LLM passes make the method expensive;
    • results depend on how well the verbal loss is designed;
    • gradients give only a first-order view of influence;
    • testing is limited to MultiWOZ, τ-bench and manager–worker architectures.
    • Code is available on GitHub (yxc-cyber/AgentChord).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.24

@inproceedings{yang2026gbc,
  title = {{GBC: Gradient-Based Connections for Optimizing Multi-Agent Systems}},
  author = {Yang, Xiaocheng and Alrabah, Abdulrahman and Hakkani-Tür, Dilek and Tur, Gokhan},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.24},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.24}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alex Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. 2026. GEPA: Reflective prompt evolution can ou… find
  2. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language… find
  3. Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, and Yi Yang. 2024. VillagerAgent: A graph-based multi-agent framework for coordinating complex task dependencies in Minecraft. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16290–16314, Bangkok, Thailand. Association fo… find
  4. Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur. 2020. MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. In Proceedings of the Twelfth Language… find
  5. Aman Gupta, Anirudh Ravichandran, Ziji Zhang, Swair Shah, Anurag Beniwal, and Narayanan Sadagopan. 2024. Dard: A multi-agent approach for task-oriented dialog systems. Preprint, arXiv:2411.00427. — [arXiv:2411.00427](https://arxiv.org/abs/2411.00427) find
  6. Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, Wolfram Barfuss, Jakob Foerster, Tomáš Gavenčiak, The Anh Han, Edward Hughes, Vojtěch Kovařík, Jan Kulveit, Joel Z. Leibo, Caspar Oesterheld, Christian Schroeder de Witt, Nisarg Shah, Mich… find
  7. Ting Han, Ximing Liu, Ryuichi Takanabu, Yixin Lian, Chongxuan Huang, Dazhen Wan, Wei Peng, and Minlie Huang. 2021. Multiwoz 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotation. In Natural Language Processing and Chinese Computing: 10th… find
  8. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan A, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. DSPy: Compiling declarative language model calls into state-of-the-art pipeline… find
  9. Jun Liu, Zhenglun Kong, Changdi Yang, Fan Yang, Tianqi Li, Peiyan Dong, Joannah Nanjekye, Hao Tang, Geng Yuan, Wei Niu, Wenbin Zhang, Pu Zhao, Xue Lin, Dong Huang, and Yanzhi Wang. 2025. Rcr-router: Efficient role-aware context routing for multi-agent llm systems with structured memory. Preprint, ar… — [arXiv:2508.04903](https://arxiv.org/abs/2508.04903) find
  10. Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K. Qiu, and Yuqing Yang. 2025. Agent lightning: Train any ai agents with reinforcement learning. Preprint, arXiv:2508.03680. — [arXiv:2508.03680](https://arxiv.org/abs/2508.03680) find
  11. Melissa Z Pan, Mert Cemri, Lakshya A Agrawal, Shuyi Yang, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Kannan Ramchandran, Dan Klein, Joseph E. Gonzalez, Matei Zaharia, and Ion Stoica. 2025. Why do multiagent systems fail? In ICLR 2025 Workshop on Building Trust in Language Mode… find
  12. Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with “gradient descent” and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7957–7968, Singapore. Association for Computationa… find our card
  13. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computat… find our card
  14. Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025. Scaling large language model-based multi-agent collaboration. In The Thirteenth International Conference on Learning Representations. find
  15. Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3145–3153. JMLR.org. find our card
  16. Xavier Sécheresse, Jacques-Yves Guilbert-Ly, and Antoine Villedieu de Torcy. 2025. Gaapo: Genetic algorithmic applied to prompt optimization. Preprint, arXiv:2504.07157. — [arXiv:2504.07157](https://arxiv.org/abs/2504.07157) find
  17. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. Preprint, arXiv:2308.08155. find our card
  18. Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, and Yuyu Luo. 2025. Self-supervised prompt optimization. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 9017–9041, Suzhou, China. Associa… find
  19. Guowei Xu, Mert Yuksekgonul, Carlos Guestrin, and James Zou. 2025. metatextgrad: Automatically optimizing language model optimizers. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  20. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large language models as optimizers. Preprint, arXiv:2309.03409. find our card
  21. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik R Narasimhan. 2025. τ -bench: A benchmark for Tool-Agent-User interaction in real-world domains. In The Thirteenth International Conference on Learning Representations. find
  22. Eric Ye and Natasha Jaques. 2024. An efficient open world benchmark for multi-agent reinforcement learning. In NeurIPS 2024 Workshop on Open-World Agents. find
  23. Fanghua Ye, Jarana Manotumruksa, and Emine Yilmaz. 2022. MultiWOZ 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 35… find our card
  24. Haofei Yu, Zirui Cheng, Zhaochen Hong, Kunlun Zhu, Jinwei Yao, Tao Feng, and Jiaxuan You. 2025. Research town: Simulator of research community. find
  25. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. Textgrad: Automatic "differentiation" via text. Preprint, arXiv:2406.07496. find
  26. Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, and Qingyun Wu. 2025a. Which agent causes task failures and when? on automated failure attribution of LLM multi-agent systems. In Forty-second International Conference on Ma… find
  27. Yaolun Zhang, Xiaogeng Liu, and Chaowei Xiao. 2025b. Metaagent: Automatically constructing multi-agent systems based on finite state machines. In Forty-second International Conference on Machine Learning. find
  28. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations. find our card
  29. Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. 2024. Gptswarm: language agents as optimizable graphs. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-gbc-gradient-based-connections-optimizing/