Elamur Research⌕
☰
home / papers / 2026 / Attribution-Based Analysis and Optimization of Modular Agentic Workflows

Attribution-Based Analysis and Optimization of Modular Agentic Workflows

Paper · 2026 · Yingxuan Yang, Bo Huang, Siyuan Qi et al.

Summary

Agent systems built from separate modules (planning, reasoning, action, reflection) are usually judged only by final task success. That hides how much each module contributes and where a stronger LLM would help most. The authors treat this model-allocation question as a cooperative game. Each component's switch from a shared baseline model to a candidate model counts as a player, and every subset of switches is run as a real end-to-end configuration. Running all 16 subsets of a four-module workflow yields exact Shapley values, which capture both individual contributions and non-additive interactions between modules. Per-model Shapley profiles then serve as a heuristic for picking a model for each component. To test this framework, ShapleyFlow, the authors build CapaBench and evaluate 9 LLMs on it. The attributions stay stable when the baseline model changes and agree with an LLM judge. The resulting mixed assignments outperform strong single-model baselines. This gives practitioners a principled way to decide where to spend model capacity in agent pipelines.

Key points

  • CapaBench: more than 1,500 tasks spread across seven areas: math problems, automated theorem proving, OS operations, coordinating robots, navigating, booking tickets and online shopping.
  • Interaction patterns:
    • Reasoning and Action show the strongest synergy in compute-heavy domains (Math, ATP).
    • Planning and Reasoning pair best in interactive ones (Robot Cooperation, Operating System).
    • Reflection contributes little or inconsistently.
  • Robustness to baseline choice: swapping the baseline from Llama3-8B-Instruct to GPT-3.5-Turbo keeps component rankings largely intact, with 85.18% pairwise consistency overall (Reasoning 91.67%, Action 86.11%, Planning 72.22%).
  • Agreement with an LLM judge: on 2,180 Algebra workflow steps, Shapley values correlate with GPT-o1-mini scores at 0.81 for Planning, 0.77 for Reasoning and 0.67 for Action.
  • Main limitation: exact attribution needs 2^|N| runs (16 versus 5 for one-factor ablation when there are four components). Larger workflows would need approximations such as KernelSHAP or SVARM, and the pool-level assignment carries no optimality guarantee.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TorontoToronto · Canada110 works · 179,061 citations🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓University of ChicagoChicago · United States52 works · 50,893 citations🏢MeituanBeijing · China26 works · 3,953 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksDecision Sciences · 249 worksInformation Systems and Management · 52 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, ZongYu Wang, Muning Wen, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu and Weinan Zhang. 2026. Attribution-Based Analysis and Optimization of Modular Agentic Workflows. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.359

@inproceedings{yang2026attribution,
  title = {{Attribution-Based Analysis and Optimization of Modular Agentic Workflows}},
  author = {Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, ZongYu Wang, Muning Wen, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu and Weinan Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.359},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.359}
}

References

49 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B. Divya. 2025. Agentic ai: Autonomous intelligence for complex goals—a comprehensive survey. IEEE Access, 13:18912–18936. find
  2. Doubao AI. 2024a. Doubao-pro-4k. Accessed from Doubao AI’s GitHub repository. find
  3. Meta AI. 2024b. Llama 3.1 70b instruct. Accessed from HuggingFace repository. find
  4. Mistral AI. 2023. Mistral 8b instruct. find
  5. Anthropic. 2024. Claude 3.5 sonnet. Accessed from Anthropic’s official website. find
  6. Tom B Brown, Benjamin Mann, Nick Ryder, and 1 others. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901.
  7. Huacan Chai, Zijie Cao, Maolin Ran, Yingxuan Yang, Jianghao Lin, Xin Peng, Hairui Wang, Renjie Ding, Ziyu Wan, Muning Wen, Weiwen Liu, Weinan Zhang, Fei Huang, and Ying Wen. 2025. Parl-mt: Learning to call functions in multi-turn conversation with progress awareness. Preprint, arXiv:2509.23206. — [arXiv:2509.23206](https://arxiv.org/abs/2509.23206) find
  8. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. 2025. MLE-bench: Evaluating machine learning agents on machine learning engineering. In The Thirteenth International Confer…
  9. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating LLMs by human preference. In Forty-first International Conference on Mac…
  10. Shay Cohen, Gideon Dror, and Eytan Ruppin. 2007. Feature selection via coalitional game theory. Neural Comput., 19(7):1939–1961. find
  11. Irinel Dragan. 1996. New mathematical properties of the banzhaf value. European Journal of Operational Research, 95(2):451–463. find
  12. Amirata Ghorbani and James Zou. 2019. Data shapley: Equitable valuation of data for machine learning. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2242–2251. PMLR. find
  13. Significant Gravitas. 2023. Autogpt: An open-source platform for autonomous ai agents. https://github.com/Significant-Gravitas/AutoGPT. Accessed: 2025-11-04. ↗
  14. Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. 2024. Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models. Preprint, arXiv:2403.07714. find
  15. Sergiu Hart. 1989. Shapley Value, pages 210–216. Palgrave Macmillan UK, London. find
  16. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS.
  17. Sirui Hong, Xiawu Zheng, Jonathan P. Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zi Hen Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, and Chenglin Wu. 2023. Metagpt: Meta programming for multi-agent collaborative framework. ArXiv, abs/2308.00352.
  18. Shengran Hu, Cong Lu, and Jeff Clune. 2025. Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations.
  19. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… find
  20. Patrick Kolpaczki, Viktor Bengs, Maximilian Muschalik, and Eyke Hüllermeier. 2024. Approximating the shapley value without marginal contributions. Preprint, arXiv:2302.00736. — [arXiv:2302.00736](https://arxiv.org/abs/2302.00736) find
  21. Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Jason Eisner. 2024. Decision-oriented dialogue for human-ai collaboration. Transactions of the Association for Computational Linguistics.
  22. Xiao Liu, Hao Zhou, Zhiheng Zhang, Dian Peng, and 1 others. 2023. Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688.
  23. Scott Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Preprint, arXiv:1705.07874.
  24. Zhao Mandi, Shreeya Jain, and Shuran Song. 2023. Roco: Dialectic multi-robot collaboration with large language models. Preprint, arXiv:2307.04738. — [arXiv:2307.04738](https://arxiv.org/abs/2307.04738) find
  25. Maximilian Muschalik, Hubert Baniecki, Fabian Fumagalli, Patrick Kolpaczki, Barbara Hammer, and Eyke Hüllermeier. 2024. shapiq: Shapley interactions for machine learning. Preprint, arXiv:2410.01649. — [arXiv:2410.01649](https://arxiv.org/abs/2410.01649) find
  26. OpenAI. 2024. Gpt-4o mini. Accessed from OpenAI’s developer resources. find
  27. Josh Achi OpenAI, Steven Adler, Sandhini Agarwal, and 1 others. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  28. Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2025. Benchmarking agentic workflow generation. In The Thirteenth International Conference on Learning Representations. find
  29. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  30. Matthew Renze and Erhan Guven. 2024. The benefits of a concise chain of thought on problem-solving in large language models. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM), page 476–483. IEEE. find
  31. Ranjan Sapkota, Konstantinos I Roumeliotis, and Manoj Karkee. 2025. Ai agents vs. agentic ai: A conceptual taxonomy, applications and challenge. arXiv preprint arXiv:2505.10468. — [arXiv:2505.10468](https://arxiv.org/abs/2505.10468) find
  32. Lloyd S. Shapley. 1952. A Value for N-Person Games. RAND Corporation, Santa Monica, CA. find
  33. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634–8652. Curran Associates, Inc.
  34. The Coq Development Team. 2024. The coq proof assistant. https://coq.inria.fr/. ↗
  35. The Isabelle Team. 2024. The isabelle theorem prover. https://isabelle.in.tum.de/. ↗
  36. The Lean Prover Team. 2024. The lean theorem prover. https://leanprover.github.io/. ↗
  37. The learnGitBranching Team. Learn git branching. https://learngitbranching.js.org//. ↗
  38. THUDM. 2024. Glm-4 air. https://github.com/ THUDM/GLM-4. Accessed: 2025-07-31. ↗
  39. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.
  40. John Yang, Carlos E Jimenez, Alex L Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R Narasimhan, Diyi Yang, Sida Wang, and Ofir Press. 2025. SWE-bench multimodal: Do AI systems generalize to visual software domains? In The Thirteenth Internationa… find
  41. Yifei Yang, Haoqiang Wu, Chen Zhao, Mingzhe Liu, and 1 others. 2024a. Agentquest: A multi-phase task planning and execution benchmark for autonomous agents. arXiv preprint arXiv:2402.01786. — [arXiv:2402.01786](https://arxiv.org/abs/2402.01786) find
  42. Yingxuan Yang, Huayi Wang, Muning Wen, Xiaoyun Mo, Qiuying Peng, Jun Wang, and Weinan Zhang. 2024b. P3: A policy-driven, pace-adaptive, and diversity-promoted framework for data pruning in llm training. Preprint, arXiv:2408.05541. — [arXiv:2408.05541](https://arxiv.org/abs/2408.05541) find
  43. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2023a. Webshop: Towards scalable real-world web interaction with grounded language agents. Preprint, arXiv:2207.01206.
  44. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023b. React: Synergizing reasoning and acting in language models. Preprint, arXiv:2210.03629.
  45. Guoli Yin, Haoping Bai, Shuang Ma, Feng Nan, Yanchao Sun, Zhaoyang Xu, Shen Ma, Jiarui Lu, Xiang Kong, Aonan Zhang, Dian Ang Yap, Yizhe zhang, Karsten Ahnert, Vik Kamath, Mathias Berglund, Dominic Walsh, Tobias Gindele, Juergen Wiest, Zhengfeng Lai, and 5 others. 2024. Mmau: A holistic benchmark of… find
  46. Wei Zhang, Junnan Wu, Tianhao Wang, Zhihao Hu, and 1 others. 2024. Omniact: A dataset and benchmark for enabling multi-modal task completion in large language models. arXiv preprint arXiv:2402.00858. — [arXiv:2402.00858](https://arxiv.org/abs/2402.00858) find
  47. Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. 2021. Minif2f: a cross-system benchmark for formal olympiad-level mathematics. arXiv preprint arXiv:2109.00110.
  48. Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulić, Anna Korhonen, and Sercan Ö. Arık. 2025. Multi-agent design: Optimizing agents with better prompts and topologies. Preprint, arXiv:2502.02533. — [arXiv:2502.02533](https://arxiv.org/abs/2502.02533) find
  49. Ruiwen Zhou, Yingxuan Yang, Muning Wen, Ying Wen, Wenhao Wang, Chunling Xi, Guoqiang Xu, Yong Yu, and Weinan Zhang. 2024. Trad: Enhancing llm agents with step-wise thought retrieval and aligned decision. Preprint, arXiv:2403.06221. — [arXiv:2403.06221](https://arxiv.org/abs/2403.06221) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-attribution-based-analysis-optimization-modular/