Elamur Research⌕
☰
home / papers / 2026 / Context-Value-Action Architecture for Value-Driven Large Language Model Agents

Context-Value-Action Architecture for Value-Driven Large Language Model Agents

Paper · 2026 · TianZe Zhang, Sirui Sun, Yuhang Xie et al.

Summary

LLM-based agents meant to imitate people often behave in rigid, stereotyped ways. Evaluations that use another LLM as the judge tend to hide this, because the judge shares the agent's biases. The authors compare agents against real human behavior and find a surprising effect. Stronger prompt-based psychological reasoning, such as chain-of-thought about values, makes value profiles more extreme and shrinks diversity across simulated populations.

They propose the Context-Value-Action (CVA) architecture, built on the Stimulus-Organism-Response model and Schwartz's theory of basic values. In CVA, context dynamically activates value dimensions, and these values then drive actions. The system works in two stages. First, a generator is calibrated with SFT and DPO so it produces realistic candidate actions. Second, a separate Value Verifier, trained on real human context-value-action data, scores and selects among those candidates. This replaces self-verification by the generating LLM.

Tested on the new CVABench, CVA outperforms baselines, reduces polarization, matches human distributions more closely and is easier to interpret. This matters for social simulation that needs realistic, diverse behavior.

Key points

  • CVABench contains more than 1.1 million real interaction traces from over 15,000 participants. It covers three domains, Social Media, Conversation and Mobility, and no user appears in more than one domain.
  • Values are represented as a context-dependent activation vector over Schwartz's 10 basic value dimensions. They are measured with the GPV (Generative Psychometrics for Values) tool.
  • The pipeline has two parts:
    • Value-Action Mapping Calibration: SFT on benchmark trajectories, followed by DPO with preference pairs that favor nuanced choices over caricatured ones.
    • Value-Driven Reasoning: a verifier that uses cross-attention and contrastive training to score N candidate actions.
  • Main diagnostic finding: making prompt-driven reasoning more intensive increases value polarization and reduces population variance.
  • The code is released at github.com/ValueByte-AI/psyche-driven-behavior-modeling. The stated limitations are the restricted domain coverage, possible bias in value measurement and a small number of baselines.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,268 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSafety Research · 626 works

Cite

Cite as TianZe Zhang, Sirui Sun, Yuhang Xie, Xin Zhang, Zhiqiang Wu and Guojie Song. 2026. Context-Value-Action Architecture for Value-Driven Large Language Model Agents. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.248

@inproceedings{zhang2026context,
  title = {{Context-Value-Action Architecture for Value-Driven Large Language Model Agents}},
  author = {TianZe Zhang, Sirui Sun, Yuhang Xie, Xin Zhang, Zhiqiang Wu and Guojie Song},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.248},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.248}
}

References

46 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacy Reese Anthis, Ryan Liu, Sean M. Richardson, Austin C. Kozlowski, Bernard Koch, James Evans, Erik Brynjolfsson, and Michael Bernstein. 2025. Llm social simulations are a promising research method. Preprint, arXiv:2504.02234. — [arXiv:2504.02234](https://arxiv.org/abs/2504.02234) find
  2. Nabiha Asghar. 2016. Yelp dataset challenge: Review rating prediction. Preprint, arXiv:1605.05362. — [arXiv:1605.05362](https://arxiv.org/abs/1605.05362) find
  3. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. 2023. A general theoretical paradigm to understand learning from human preferences. Preprint, arXiv:2310.12036.
  4. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 830–839. find
  5. Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. 2024. Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization. Preprint, arXiv:2406.00045. — [arXiv:2406.00045](https://arxiv.org/abs/2406.00045) find
  6. Kaiyan Chang, Songcheng Xu, Chenglong Wang, Yingfeng Luo, Xiaoqian Liu, Tong Xiao, and Jingbo Zhu. 2024. Efficient prompting methods for large language models: A survey. Preprint, arXiv:2404.01077. — [arXiv:2404.01077](https://arxiv.org/abs/2404.01077)
  7. Nuo Chen, Yan Wang, Yang Deng, and Jia Li. 2025. The oscars of ai theater: A survey on role-playing with language models. Preprint, arXiv:2407.11484. — [arXiv:2407.11484](https://arxiv.org/abs/2407.11484) find
  8. Andrew Fuchs, Andrea Passarella, and Marco Conti. 2023. Modeling, replicating, and predicting human behavior: A survey. ACM Trans. Auton. Adapt. Syst., 18(2). find
  9. Roberto Gallotta, Graham Todd, Marvin Zammit, Sam Earle, Antonios Liapis, Julian Togelius, and Georgios N. Yannakakis. 2024. Large language models and games: A survey and roadmap. IEEE Transactions on Games, page 1–18. find
  10. Adithya V Ganesan, Yash Kumar Lal, August Håkan Nilsson, and H. Andrew Schwartz. 2023. Systematic evaluation of gpt-3 for zero-shot personality estimation. Preprint, arXiv:2306.01183. find
  11. Jen-Tse Huang, Wenxuan Wang, Eric John Li, Man Ho Lam, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, Michael R. Lyu, and AI Lab. 2024. On the humanity of conversational ai: Evaluating the psychological portrayal of llms. In International Conference on Learning Representations. find
  12. Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Lukas Vierling, Donghai Hong, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Juntao Dai, Xuehai Pan, Kwan Yee Ng, Aidan O’Gara, Hua Xu, Brian Tse, and 7 others. 2025a. Ai alignment: A comprehensive survey. P…
  13. Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai. 2025b. Enhancing persona consistency for llms’ role-playing using persona-aware contrastive learning. Preprint, arXiv:2503.17662. find
  14. Ang Li, Haozhe Chen, Hongseok Namkoong, and Tianyi Peng. 2025. Llm generated persona is a promise with a catch. Preprint, arXiv:2503.16527. — [arXiv:2503.16527](https://arxiv.org/abs/2503.16527) find
  15. Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, and Marc-Alexandre Côté. 2023. Augmenting autotelic agents with large language models. Preprint, arXiv:2305.12487. find
  16. Xingxuan Li, Yutong Li, Lin Qiu, Shafiq Joty, and Lidong Bing. 2024. Evaluating psychological safety of large language models. Preprint, arXiv:2212.10529. find
  17. Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. Preprint, arXiv:2307.08691.
  18. Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. 2025. A survey on vision-language-action models for embodied ai. Preprint, arXiv:2405.14093. — [arXiv:2405.14093](https://arxiv.org/abs/2405.14093) find
  19. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Preprint, arXiv:2305.14314.
  20. Albert Mehrabian and James A Russell. 1974. An approach to environmental psychology. the MIT Press. find
  21. Qing Dong, Pengyuan Liu, Dong Yu, and Chen Kang. 2025. Simulating human behavior with the psychological-mechanism agent: Integrating feeling, thought, and action. Preprint, arXiv:2507.19495. — [arXiv:2507.19495](https://arxiv.org/abs/2507.19495) find
  22. Danny Driess, F. Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Q. Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, P. Sermanet, Daniel Duckworth, S. Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, and 3 others. 2023.… find
  23. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  24. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. Preprint, arXiv:2304.03442. find
  25. Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang, Di Zhou, Chen Gao, Fengli Xu, Fang Zhang, Ke Rong, Jun Su, and Yong Li. 2025. Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors… — [arXiv:2502.08691](https://arxiv.org/abs/2502.08691) find
  26. Aske Plaat, Max Van Duijn, Niki Van Stein, Mike Preuss, Peter Van der Putten, and Kees Joost Batenburg. 2025. Agentic large language models, a survey. Journal of Artificial Intelligence Research, 84.
  27. Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, and Guojie Song. 2024. Valuebench: Towards comprehensively evaluating value orientations and understanding of large language models. Preprint, arXiv:2406.04214.
  28. Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2025. A systematic survey of prompt engineering in large language models: Techniques and applications. Preprint, arXiv:2402.07927.
  29. Shalom H Schwartz. 1992a. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. In Advances in experimental social psychology, volume 25, pages 1–65. Elsevier. find
  30. Shalom H. Schwartz. 1992b. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25:1–65. find
  31. Shalom H Schwartz. 1994. Are there universal aspects in the structure and contents of human values? Journal of social issues, 50(4):19–45. find
  32. Shalom H. Schwartz, Gila Melech, Arielle Lehmann, Steven Burgess, Mari Harris, and Vicki Owens. 2001. Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology, 32:519 – 542. find
  33. Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2025. Personality traits in large language models. Preprint, arXiv:2307.00184.
  34. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-llm: A trainable agent for role-playing. Preprint, arXiv:2310.10158.
  35. Xiaoyang Song, Akshat Gupta, Kiyan Mohebbizadeh, Shujie Hu, and Anant Singh. 2023. Have large language models developed a personality?: Applicability of self-assessment tests in measuring personality in llms. Preprint, arXiv:2305.14693. — [arXiv:2305.14693](https://arxiv.org/abs/2305.14693) find
  36. Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister. 2025. In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents. Prepr…
  37. Yiding Wang, Yuxuan Chen, Fangwei Zhong, Long Ma, and Yizhou Wang. 2025. Simulating human-like daily activities with desire-driven autonomy. Preprint, arXiv:2412.06435. — [arXiv:2412.06435](https://arxiv.org/abs/2412.06435) find
  38. Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Stephen W. Huang, Jie Fu, and Junran Peng. 2024. Rolellm: Benchmarking, eliciting, and enhancing role-playing abili…
  39. Zhilin Wang, Yu Ying Chiu, and Yu Cheung Chiu. 2023. Humanoid agents: Platform for simulating human-like generative agents. Preprint, arXiv:2310.05418. find
  40. Qiuejie Xie, Qiming Feng, Tianqi Zhang, Qingqiu Li, Linyi Yang, Yuejie Zhang, Rui Feng, Liang He, Shang Gao, and Yue Zhang. 2025. Human simulacra: Benchmarking the personification of large language models. Preprint, arXiv:2402.18180. — [arXiv:2402.18180](https://arxiv.org/abs/2402.18180) find
  41. Dingqi Yang, Daqing Zhang, Longbiao Chen, and Bingqing Qu. 2015. Nationtelescope: Monitoring and visualizing large-scale collective behavior in lbsns. Journal of Network and Computer Applications, 55. find
  42. Dingqi Yang, Daqing Zhang, and Bingqing Qu. 2016. Participatory cultural mapping based on collective behavior data in location-based social networks. ACM Transactions on Intelligent Systems and Technology (TIST), 7:1 – 23. find
  43. Haoran Ye, Jing Jin, Yuhang Xie, Xin Zhang, and Guojie Song. 2026. Large language model psychometrics: A systematic review of evaluation, validation, and enhancement. Preprint, arXiv:2505.08245. — [arXiv:2505.08245](https://arxiv.org/abs/2505.08245) find
  44. Haoran Ye, Yuhang Xie, Yuanyi Ren, Hanjun Fang, Xin Zhang, and Guojie Song. 2025a. Measuring human and ai values based on generative psychometrics with large language models. Preprint, arXiv:2409.12106. — [arXiv:2409.12106](https://arxiv.org/abs/2409.12106) find
  45. Haoran Ye, TianZe Zhang, Yuhang Xie, Liyuan Zhang, Yuanyi Ren, Xin Zhang, and Guojie Song. 2025b. Generative psycho-lexical approach for constructing value systems in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap… find
  46. Xinnong Zhang, Jiayu Lin, Xinyi Mou, Shiyue Yang, Xiawei Liu, Libo Sun, Hanjia Lyu, Yihang Yang, Weihong Qi, Yue Chen, Guanying Li, Ling Yan, Yao Hu, Siming Chen, Yu Wang, Xuanjing Huang, Jiebo Luo, Shiping Tang, Libo Wu, and 2 others. 2025. Socioverse: A world model for social simulation powered by… — [arXiv:2504.10157](https://arxiv.org/abs/2504.10157) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-context-value-action-architecture-value/