Elamur Research⌕
☰
home / papers / 2026 / DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agents

DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agents

Summary

LLM-based agents struggle to persuade people in emergencies. Residents differ in their worries, resources and willingness to comply, so one fixed approach often fails. The authors study this in a wildfire evacuation setting, where an AI operator must convince a town resident to leave. They build on the simulation and Wizard-of-Oz operator–resident dialogues from Chaffey et al. (2020). The task is framed as a POMDP, and the proposed method is DiPS (Dialogue Policy Selection). DiPS does not use RL to produce utterances directly. Instead, an offline critic trained with Implicit Q-Learning (IQL) reads an embedding of the latest turns and picks one of several persuasion personas at each turn, such as empathetic, directive or informational. The chosen persona's description and retrieved example dialogues then prompt an LLM to write the operator's reply. Evaluation uses LLM-simulated residents, human role-players and a refined simulator that models persistent resistance. DiPS persuades more residents in fewer turns than zero-shot and RAG baselines. This suggests that offline RL over discrete strategies is a practical route to adaptive persuasion in high-stakes dialogue.

Key points

  • DiPS treats each persona as a discrete action and learns Q(h_t, z) with IQL from logged dialogues containing both successful and failed evacuations; an LLM judge decides when a dialogue ends.
  • In the refined simulation, DiPS reaches 92% evacuation success with 11.2 turns on average. This beats Zero-shot (88%, 11.9 turns), RAG (78%, 11.8 turns), random persona selection (69–78%) and a mismatched fixed policy (51%, 13.6 turns).
  • Ablations show every component matters:
    • Swapping per-policy retrieval for global retrieval lowers success to 86%.
    • Keeping only the persona, without retrieval, gives 80%.
    • Removing the explicit strategy prompt also gives 80%.
  • After revision, the LLM judge agreed with human decisions in roughly 90% of cases. The authors still note that LLM-as-judge is not consistent across all conditions.
  • Limitations include:
    • English-only data and experiments.
    • A fixed set of predefined personas.
    • Rewards that depend mainly on the final outcome, which makes turn-level credit assignment hard.
    • Reliance on simulated residents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.27

@inproceedings{zhang2026dips,
  title = {{DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agents}},
  author = {Zhang, Tianyi and Das, Mousumi and Anwar, Abrar and Thomason, Jesse and Traum, David},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.27},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.27}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  2. Patricia Chaffey, Ron Artstein, Kallirroi Georgila, Kimberly A Pollard, Setareh Nasihati Gilani, David M Krum, David Nelson, Kevin Huynh, Alesia Gainer, Seyed Hossein Alavi, Rhys Yahataa, Anton Leuski, Volodymyr Yanov, and David Traum. 2020. Human swarm interaction using plays, audibles, and a virtu… find
  3. Sam Davidson, Salvatore Romeo, Raphael Shu, James Gung, Arshit Gupta, Saab Mansour, and Yi Zhang. 2023. User simulation with large language models for evaluating task-oriented dialogue. arXiv preprint arXiv:2309.13233. — [arXiv:2309.13233](https://arxiv.org/abs/2309.13233) find
  4. Esin Durmus, Faisal Ladhak, and Claire Cardie. 2019. The role of pragmatic and discourse context in determining argument impact. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-… find our card
  5. Scott Fujimoto, David Meger, and Doina Precup. 2019. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, pages 2052–2062. PMLR. find
  6. He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333–2343. find our card
  7. Ilya Kostrikov, Ashvin Nair, and Sergey Levine. 2021. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169. — [arXiv:2110.06169](https://arxiv.org/abs/2110.06169) find
  8. Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020a. Conservative q-learning for offline reinforcement learning. In Advances in Neural Information Processing Systems, volume 33, pages 1179– 1191. Curran Associates, Inc. find
  9. Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020b. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33:1179–1191. find
  10. Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443–2453. find our card
  11. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 1192– 1202. find our card
  12. Setareh Nasihati Gilani, Kimberly Pollard, and David Traum. 2023. Multimodal prediction of user’s performance in high-stress dialogue interactions. In Companion Publication of the 25th International Conference on Multimodal Interaction, pages 71–75. find
  13. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. find
  14. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. find
  15. Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong. 2017. Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 22… find
  16. Francesco Salvi, Manoel Horta Ribeiro, Riccardo Gallotti, and Robert West. 2024. On the conversational persuasiveness of large language models: A randomized controlled trial. find
  17. Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature, 623(7987):493–498. find our card
  18. Charlie Snell, Ilya Kostrikov, Yi Su, Mengjiao Yang, and Sergey Levine. 2022. Offline rl for natural language generation with implicit language q learning. arXiv preprint arXiv:2206.11871. — [arXiv:2206.11871](https://arxiv.org/abs/2206.11871) find
  19. Claire To, Setareh Nasihati Gilani, and David Traum. 2023. Common strategy patterns of persuasion in a mission critical and time sensitive task. In Proceedings of the 27th Workshop on the Semantics and Pragmatics of Dialogue - Poster Abstracts, Maribor, Slovenia. SEMDIAL. find
  20. Skanda Vaidyanath and Kallirroi Georgila. 2020. Using reinforcement learning to manage communications between humans and artificial agents in an evacuation scenario. In FLAIRS, pages 396–400. find
  21. Siddharth Verma, Justin Fu, Sherry Yang, and Sergey Levine. 2022. Chai: A chatbot ai for task-oriented dialogue with offline reinforcement learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page… find
  22. Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 5635–5649. find our card
  23. Jason D Williams and Steve Young. 2007. Partially observable markov decision processes for spoken dialog systems. Computer Speech & Language, 21(2):393– 422. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-dips-dialogue-policy-selection-high/