Elamur Research⌕
☰
home / papers / 2026 / TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs

TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs

Paper · 2026 · Ziyi Wang, Chen Zhang, Wenjun Peng et al.

Summary

Explaining the behaviour of LLM agents is especially hard when they interact with other agents under hidden information, because their choices depend on beliefs that change over time. Standard tools such as feature attribution or post-hoc rationales were built for static, single-turn tasks. The authors, from Adelaide University, propose TriEx, which records every decision point in one shared trace and examines it from three angles:

  • First-person view: a structured self-account written before the move. It combines a short narrative with a machine-readable DecisionJSON.
  • Second-person view: explicit profiles of opponents along interpretable behavioural traits, revised as play continues.
  • Third-person view: audits by oracle LLMs that check consistency against signals computed by the environment, such as pot odds and hand equity.

Texas Hold'em poker serves as a controlled diagnostic environment. The analysis shows three patterns. Stated reasoning becomes less faithful as decisions grow more complex. Opponent beliefs react to interventions and shape play. LLM judges agree on categorical and ranking judgments but not on absolute scores. The authors argue that explainability depends on interaction and requires multi-view, evidence-anchored evaluation.

Key points

  • Shared trace: each record holds the observation, the legal actions, the executed action, the self-reasoning artifact, the opponent profiles and reference features (pot odds, stack-to-pot ratios, equity estimates). First-person claims are normalised into an "explanation signature" over a fixed ontology.
  • Belief audits: opponent beliefs were audited on traits such as Aggressiveness and Risk Tolerance. Oracles almost always agreed on the direction of an error (over- vs. underestimate), but their continuous alignment scores varied considerably.
  • Grounding of abstract traits: divergence on Risk Tolerance suggests that current LLMs lack stable grounding for abstract behavioural concepts.
  • Cross-oracle agreement: measured with quadratic Cohen's κ, it was mostly moderate to high. The oracles included llama-4-maverick and qwen3-32b. Reliability ranks categorical above rank-based above absolute scores.
  • Publication details: ACL 2026 (Long Papers), pp. 6448–6479. Code is at github.com/Einsam1819/TriEx. The work was funded by ARC Discovery Project DP260102534.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of AdelaideAdelaide · Australia6 works · 1,417 citations🎓Adelaide UniversityAdelaide · Australia5 works · 3 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Ziyi Wang, Chen Zhang, Wenjun Peng, Qi Wu and Xinyu Wang. 2026. TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.292

@inproceedings{wang2026triex,
  title = {{TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs}},
  author = {Ziyi Wang, Chen Zhang, Wenjun Peng, Qi Wu and Xinyu Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.292},
  url = {https://doi.org/10.18653/v1/2026.acl-long.292}
}

References

34 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alpha Arena Team. 2025. Alpha arena: Ai trading competition in real crypto markets. https://www.alpha-arena.org/. Accessed: 2025-12-04. ↗
  2. Mark W Baldwin. 1992. Relational schemas and the processing of social information. Psychological bulletin, 112(3):461. find
  3. Oliver Bentham, Nathan Stringham, and Ana Marasović. 2024. Chain-of-thought unfaithfulness as disguised accuracy. Trans. Mach. Learn. Res. find
  4. Colin Camerer and Teck Hua Ho. 1999. Experience-weighted attraction learning in normal form games. Econometrica, 67(4):827–874. find
  5. Colin F Camerer, Teck-Hua Ho, and Juin-Kuan Chong. 2004. A cognitive hierarchy model of games. The quarterly journal of economics, 119(3):861–898. find
  6. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3):1–45.
  7. Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. 2024. Gtbench: Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. In Proc. Int. Conf. Learn. Representations, volume 37,… find
  8. Chris D Frith and Uta Frith. 2006. The neural basis of mentalizing. Neuron, 50(4):531–534. find
  9. Danny Halawi, Jean-Stanislas Denain, and Jacob Steinhardt. 2024. Overthinking the truth: Understanding how language models process false demonstrations. In Proc. Int. Conf. Learn. Representations. find
  10. Wenjun Peng, Xinyu Wang, and Qi Wu. 2026. Proxy-war: Dynamic assessment of llm code generation in game arenas. In Proc. Int. Conf. Softw. Eng. find
  11. David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and brain sciences, 1(4):515–526. find
  12. Lanxiang Hu, Mingjia Huo, Yuxuan Zhang, Haoyang Yu, Eric P Xing, Ion Stoica, Tajana Rosing, Haojian Jin, and Hao Zhang. 2025. lmgame-bench: How good are llms at playing games? arXiv preprint arXiv:2505.15146. find
  13. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should i trust you?" explaining the predictions of any classifier. In Proc. ACM SIGKDD Int. Conf. Knowledge discovery & data mining, pages 1135–1144.
  14. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proc. Annu. Meet. Assoc. Comput. Linguist., pages 4198–4205. Association for Computational Linguistics (ACL).
  15. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proc. Int. Conf. Mach. Learn., pages 3319–3328. PMLR. find
  16. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, and 1 others. 2025. From generation to judgment: Opportunities and challenges of llm-as-a-judge. In Proc. Conf. Empir. Methods in Natural Language Process.… find
  17. Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xiang Li, and 6 others. 2024. Personal llm agents: Insights and survey about the capa… find
  18. Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024. Are self-explanations from large language models faithful? In Proc. Annu. Meet. Assoc. Comput. Linguist., pages 295–337. find
  19. Aleksandar Makelov, Georg Lange, and Neel Nanda. 2024. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching. In Proc. Int. Conf. Learn. Representations. find
  20. Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2025. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In Proc. Int. Conf. Learn. Representations. find
  21. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Proc. Advances in Neural Inf. Process. Syst., 35:17359–17372. find
  22. Richard E Nisbett and Timothy D Wilson. 1977. Telling more than we can know: Verbal reports on mental processes. Psychological review, 84(3):231. find
  23. Letitia Parcalabescu and Anette Frank. 2024. On measuring faithfulness or self-consistency of natural language explanations. In Proc. Annu. Meet. Assoc. Comput. Linguist., pages 6048–6089. find
  24. Michael Tomasello, Malinda Carpenter, Josep Call, Tanya Behne, and Henrike Moll. 2005. Understanding and sharing intentions: The origins of cultural cognition. Behavioral and brain sciences, 28(5):675– 691. find
  25. Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. Proc. Advances in Neural Inf. Process. Syst., 36:74952–74965.
  26. Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. In Proc. Int. Conf. Learn. Representations. find
  27. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, and 1 others. 2024a. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345.
  28. Xinyu Wang, Bohan Zhuang, and Qi Wu. 2024b. Moda-verse: Efficiently transforming modalities with llms. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 26606–26616. find
  29. Xinyu Wang, Bohan Zhuang, and Qi Wu. 2025. Are large vision language models good game players? In Proc. Int. Conf. Learn. Representations. find
  30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Proc. Advances in Neural Inf. Process. Syst., 35:24824– 24837.
  31. Yue Wu, Xuan Tang, Tom M Mitchell, and Yuanzhi Li. 2024. Smartplay: A benchmark for llms as intelligent agents. In Proc. Int. Conf. Learn. Representations. find
  32. Wako Yoshida, Ray J Dolan, and Karl J Friston. 2008. Game theory of mind. PLoS computational biology, 4(12):e1000254. find
  33. Fred Zhang and Neel Nanda. 2024. Towards best practices of activation patching in language models: Metrics and methods. In Proc. Int. Conf. Learn. Representations. find
  34. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol., 15(2):1–38.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-triex-game-based-tri-view/