Elamur Research⌕
☰
home / papers / 2026 / DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses

DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses

Paper · 2026 · Han Luo and Guy Laban

Summary

Clinicians and safety engineers are increasingly asked to audit LLM chatbots used for mental-health support and crisis counselling. Their current tools are poorly suited to the job. Developer-oriented red-teaming frameworks mostly hunt for jailbreaks and policy violations. They miss quieter harms such as manipulation, discrimination or emotional distress, and they give no explanations a clinician could cite. DialogGuard is an open-source, locally deployable web system that wraps any LLM backend. It rates responses on five psychosocial risk dimensions using a shared 0–2 rubric. Four LLM-as-a-judge pipelines are available: a single judge, a corrective two-agent setup, multi-agent debate, and majority voting. Practitioners use it through a Live Chat mode or a Manual Input mode. The interface shows colour-coded scores per dimension and a panel of natural-language reasoning. In experiments on PKU-SafeRLHF, the dual-agent correction pipeline gave the best balance of accuracy and robustness. A formative study with 12 practitioners suggested the tool fits their real auditing and supervision workflows, giving clinicians a practical second opinion.

Key points

  • Five risk dimensions: Privacy Violation, Discriminatory Behaviour, Mental Manipulation, Psychological Harm and Insulting Behaviour. They are derived from the PKU-SafeRLHF harm taxonomy and scored 0 (no concern), 1 (mild risk) or 2 (clear violation).
  • Four judge mechanisms:
    • Single-Agent Scoring
    • Dual-Agent Correction
    • Multi-Agent Debate, with proponent, opponent and judge roles
    • Majority Voting
  • Results: Dual-agent correction was stable across weight settings from 0.0 to 1.0, and the pattern held across three tested LLMs. Single-agent scoring was moderately sensitive to temperature, especially for the privacy and manipulation dimensions.
  • Implementation: A FastAPI backend with an HTML/CSS/JS frontend runs on localhost. It accepts any OpenAI-compatible endpoint, including Ollama and vLLM. Live Chat connects to the LEXI platform, and the code is released under Apache 2.0.
  • Limitations: The experiments cover only English single-turn data, and the usability sample is small. In the study, 12 practitioners used both modes, including eight scripted multi-turn conversations, four of them high-risk.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ben-Gurion University of the NegevBeersheba · Israel40 works · 3,099 citations🎓University of LeedsLeeds · United Kingdom40 works · 1,822 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Social Sciences · 1,772 worksPsychology · 1,552 worksSafety Research · 543 worksMedicine · 241 worksHealth Informatics · 184 worksApplied Psychology · 21 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 517 works

Cite

Cite as Han Luo and Guy Laban. 2026. DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-demo.19

@inproceedings{luo2026dialogguard,
  title = {{DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses}},
  author = {Han Luo and Guy Laban},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-demo.19},
  url = {https://doi.org/10.18653/v1/2026.acl-demo.19}
}

References

40 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, and 1 others. 2024. Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks. arXiv preprint arX…
  2. Rui Cao, Roy Ka-Wei Lee, and Jing Jiang. 2024. Modularized networks for few-shot hateful meme detection. In WWW ’24: Proceedings of the ACM Web Conference 2024, pages 4575–4584, Singapore, Singapore. Association for Computing Machinery. find
  3. Manav Chaudhary, Harshit Gupta, Savita Bhat, and Vasudeva Varma. 2024. Towards understanding the robustness of llm-based evaluations under perturbations. arXiv preprint arXiv:2412.09269. find
  4. Kang Chen, Xiuze Zhou, Yuanguo Lin, Shibo Feng, Li Shen, and Pengcheng Wu. 2025a. A survey on privacy risks and protection in large language models. Journal of King Saud University Computer and Information Sciences, 37(7):163. find
  5. Xiuyuan Chen, Jian Zhao, Yuchen Yuan, Tianle Zhang, Huilin Zhou, Zheng Zhu, Ping Hu, Linghe Kong, Chi Zhang, Weiran Huang, and 1 others. 2025b. Radar: A risk-aware dynamic multi-agent framework for llm safety evaluation via role-specialized collaboration. arXiv preprint arXiv:2509.25271. — [arXiv:2509.25271](https://arxiv.org/abs/2509.25271) find
  6. Andrew Clark. 2025. The ability of AI therapy bots to set limits with distressed adolescents: Simulation-based comparison study. JMIR Mental Health, 12:e78414. find
  7. Confident AI. 2025. DeepTeam: A framework to red team LLMs and LLM systems. https://github.com/confident-ai/deepteam. Accessed: 2026-02-17. ↗
  8. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, volume 11, pages 512–515.
  9. Julian De Freitas and I. Glenn Cohen. 2024. The health risks of generative AI-based wellness apps. find
  10. Zainab Iftikhar, Amy Xiao, Sean Ransom, Jeff Huang, and Harini Suresh. 2025. How llm counselors violate ethical standards in mental health practice: A practitioner-informed framework. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 8, pages 1311–1323. find
  11. Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang. 2024a. Pku-saferlhf: Towards multi-level safety alignment for llms with human preference. arXiv preprint arXiv:2406.15513.
  12. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2024b. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36.
  13. Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023. Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems, 36:20750–20762. find
  14. Guy Laban and Emily S. Cross. 2024. Sharing our Emotions with Robots: Why do we do it and how does it make us feel? IEEE Transactions on Affective Computing. find
  15. Guy Laban, Julian Hough, Minha Lee, Alva Markelius, Mary Ellen Foster, Jane Stuart-Smith, and Muneeb Imtiaz Ahmad. 2025. Bias and Fairness in Conversational User Interfaces. Proceedings of the 2025 ACM Conference on Conversational User Interfaces (CUI’ 25), pages 1–8. find
  16. Guy Laban, Tomer Laban, and Hatice Gunes. 2024. find
  17. LEXI: Large Language Models Experimentation Interface. HAI 2024 - Proceedings of the 12th International Conference on Human-Agent Interaction, pages 250–259. find
  18. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th annual meeting of the…
  19. Digital Action. 2022. Digital action’s online harms taxonomy (draft). https://technologycoalition.org/. Working taxonomy cited in literature on data protection and online harms. ↗
  20. Jing-Jing Li, Valentina Pyatkin, Max Kleiman-Weiner, Liwei Jiang, Nouha Dziri, Anne Collins, Jana Schaich Borg, Maarten Sap, Yejin Choi, and Sydney Levine. 2025. Safetyanalyst: Interpretable, transparent, and steerable safety moderation for ai behavior. In Proceedings of the 42nd International Confe… find
  21. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462.
  22. Tharindu Liyanage and 1 others. 2025. Public health risk management, policy, and ethical imperatives in the use of AI tools for mental health therapy. Healthcare, 13(21):2721. find
  23. Xiaochen Luo, Smita Ghosh, Jacqueline L Tilley, Patrica Besada, Jinqiu Wang, and Yangyang Xiang. 2025. “shaping chatgpt into my digital therapist”: A thematic analysis of social media discourse on using generative artificial intelligence for mental health. Digital Health, 11:20552076251351088. find
  24. A Luke MacNeill, Shelley Doucet, and Alison Luke. 2024. Effectiveness of a mental health chatbot for people with chronic diseases: randomized controlled trial. JMIR Formative Research, 8:e50025. find
  25. Jared Moore, Declan Grabb, William Agnew, Kevin Klyman, Stevie Chancellor, Desmond C Ong, and Nick Haber. 2025. Expressing stigma and inappropriate responses prevents llms from safely replacing mental health providers. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transp… find
  26. Promptfoo. 2025. LLM red teaming guide. https://www.promptfoo.dev/docs/red-team/. Accessed: 2026-02-17. ↗
  27. Shaina Raza, Ranjan Sapkota, Manoj Karkee, and Christos Emmanouilidis. 2025. Trism for agentic ai: A review of trust, risk, and security management in llm-based agentic multi-agent systems. find
  28. Juan Ren, Mark Dras, and Usman Naseem. 2025. Agentic moderation: Multi-agent design for safer vision-language models. arXiv preprint arXiv:2510.25179. — [arXiv:2510.25179](https://arxiv.org/abs/2510.25179) find
  29. Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. 2023. Verbosity bias in preference labeling by large language models. arXiv preprint arXiv:2310.10076. — [arXiv:2310.10076](https://arxiv.org/abs/2310.10076) find
  30. World Economic Forum Global Coalition for Digital Safety. 2023. Toolkit for Digital Safety Design Interventions and Innovations: Typology of Online Harms. find
  31. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th international conference on world wide web, pages 1391–1399. find
  32. Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. 2024. Mentallama: Interpretable mental health analysis on social media with large language models. In WWW ’24: Proceedings of the ACM Web Conference 2024, pages 4489–4500, Singapore, Singapore. Association for… find
  33. Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, and 1 others. 2024. Justice or prejudice? quantifying biases in llm-as-ajudge. arXiv preprint arXiv:2410.02736.
  34. Xiaohan Yuan, Jinfeng Li, Dongxia Wang, Yuefeng Chen, Xiaofeng Mao, Longtao Huang, Hui Xue, Wenhai Wang, Kui Ren, and Jingyi Wang. 2024. S-eval: Automatic and adaptive test generation for benchmarking safety evaluation of large language models. arXiv preprint arXiv:2405.14191. — [arXiv:2405.14191](https://arxiv.org/abs/2405.14191) find
  35. Zaibin Zhang, Yongting Zhang, Lijun Li, Jing Shao, Hongzhi Gao, Yu Qiao, Lijun Wang, Huchuan Lu, and Feng Zhao. 2024. Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety. In Proceedings of the 62nd Annual Meeting of the ACL (Volume… find
  36. A Till Scholich, Maya Barr, Shannon Wiltsey Stirman, and Shriti Raj. 2025. A comparison of responses from human therapists and large language model–based chatbots to assess therapeutic communication: Mixed methods study. JMIR Mental Health, 12(1):e69709. find
  37. Lin Tian, Emily Booth, Francesco Bailo, Julian Droogan, and Marian-Andrei Rizoiu. 2025. Before it’s too late: A state space model for the early prediction of misinformation and disinformation engagement. In WWW ’25: Proceedings of the ACM Web Conference 2025, pages 5244–5254, Sydney, NSW, Australia.… find
  38. Han Wang, Rui Yang Tan, and Roy Ka-Wei Lee. 2025a. Cross-modal transfer from memes to videos: Addressing data scarcity in hateful video detection. In WWW ’25: Proceedings of the ACM Web Conference 2025, Sydney, NSW, Australia. Association for Computing Machinery. find
  39. Yixu Wang, Xin Wang, Yang Yao, Xinyuan Li, Yan Teng, Xingjun Ma, and Yingchun Wang. 2025b. Safeevalagent: Toward agentic and self-evolving safety evaluation of llms. arXiv preprint arXiv:2509.26100. — [arXiv:2509.26100](https://arxiv.org/abs/2509.26100) find
  40. Evaluation Mechanisms Operationalization Single-Agent Scoring. A single evaluation agent A(·; θ) reads the user–model conversation and outputs a discrete risk label per dimension using deterministic decoding (temperature 0): s = A(x; θ), (1) where θ denotes the parameters of the evaluation model. Du… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/luo-2026-dialogguard-multi-agent-psychosocial-safety/