Elamur Research⌕
☰
home / papers / 2026 / Danger Depends on the Mind: A Theory-of-Mind Grounded Dataset and Model for Context-Dependent Dangerous Speech

Danger Depends on the Mind: A Theory-of-Mind Grounded Dataset and Model for Context-Dependent Dangerous Speech

Paper · 2026 · Yuanchen Shi, Longyin Zhang, Guodong Zhou et al.

Summary

Most dangerous speech detectors judge a sentence on its wording alone and give a binary verdict. This ignores that the same words can be harmless from one speaker and risky from another. The authors reframe the task using Theory-of-Mind (ToM): the speaker's identity, emotion, intent and action decide whether an utterance is a threat.

They release ToM-DS, a Chinese benchmark built in five stages:

  • find sentences whose safety depends on context, using several LLMs (GPT, Grok, DeepSeek, Gemini);
  • generate further such sentences;
  • construct contrasting safe and dangerous contexts for each sentence, with explanatory reasoning;
  • annotate risk levels;
  • run cross-agent quality checks with human adjudication and deduplication.

They also propose ToMGuard, a compact model. It pairs a Chinese RoBERTa text encoder with an MLP encoder for structured context, then adds Dynamic ToM Attention, gated fusion and multi-task training.

With roughly 300M parameters, ToMGuard beats fine-tuned backbones and few-shot proprietary LLMs on both binary detection and severity prediction. Ablations show that removing ToM components consistently lowers results. The work suggests that moderation systems should consider who is speaking and in what state, not only what is said.

Key points

  • ToM-DS size and labels: about 79K text–context pairs. 55.7% are context-dependent, meaning one utterance can be labelled harmless under certain contexts but harmful under others. Each instance has a binary danger label, and dangerous cases also get one of four ordinal risk levels.
  • Context schema: speaker profile, ToM state and a two-level topic hierarchy.
    • The profile covers gender, WHO-based age groups and 25 ISCO/ILO-derived occupations.
    • The ToM state has 20 emotions, 23 intents and 22 actions.
  • ToMGuard design: text attends over five context dimensions (Personal Info, Emotion, Intent, Action, Topic). Training combines focal loss for imbalanced binary detection with CORAL loss for ordinal risk. Task weights are α = 0.4 (binary) and β = 0.6 (risk).
  • Comparison and case study: ToMGuard outperforms Qwen3, GPT-5.1, Grok-4.1 and XLR-Large baselines. In the case study, it is the only listed model that gets both the safe and the dangerous reading of an ambiguous threat right, including the correct risk level.
  • Limitations:
    • The dataset covers Chinese only.
    • Data generation and annotation rely heavily on LLMs.
    • Evaluation is in-distribution only.
    • The model does not capture interactions between context dimensions.
    • Data and code are released at github.com/FakerBoom/ToMDS-ToMGGuard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Soochow UniversitySuzhou · China63 works · 4,507 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Yuanchen Shi, Longyin Zhang, Guodong Zhou and Fang Kong. 2026. Danger Depends on the Mind: A Theory-of-Mind Grounded Dataset and Model for Context-Dependent Dangerous Speech. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.322

@inproceedings{shi2026danger,
  title = {{Danger Depends on the Mind: A Theory-of-Mind Grounded Dataset and Model for Context-Dependent Dangerous Speech}},
  author = {Yuanchen Shi, Longyin Zhang, Guodong Zhou and Fang Kong},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.322},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.322}
}

References

50 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marcel R Ackermann, Johannes Blömer, Daniel Kuntze, and Christian Sohler. 2014. Analysis of agglomerative clustering. Algorithmica, 69(1):184–215. find
  2. James Allen. 1995. Natural language understanding. Benjamin-Cummings Publishing Co., Inc. find
  3. Zewen Bai, Shengdi Yin, Junyu Lu, Jingjie Zeng, Haohao Zhu, Yuanyuan Sun, Liang Yang, and Hongfei Lin. 2025. State toxicn: A benchmark for span-level target-aware toxicity extraction in chinese hate speech detection. Preprint, arXiv:2501.15451. find
  4. Timothy Bickmore and Justine Cassell. 2005. Social dialogue with embodied conversational agents. Advances in natural multimodal dialogue systems, 30:23–54. find
  5. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019a. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491–500. find
  6. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019b. Nuanced metrics for measuring unintended bias with real data for text classification. CoRR, abs/1903.04561. — [arXiv:1903.04561](https://arxiv.org/abs/1903.04561) find
  7. Ruirui Chen, Weifeng Jiang, Chengwei Qin, and Cheston Tan. 2025. Theory of mind in large language models: Assessment and enhancement. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria. Association for Computational Lin…
  8. Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang. 2024. ToMBench: Benchmarking theory of mind in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Lingu…
  9. Herbert H Clark. 1996. Using language. Cambridge university press. find
  10. Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213. find
  11. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. find
  12. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, volume 11, pages 512–515.
  13. Jiawen Deng, Jingyan Zhou, Hao Sun, Fei Mi, and Minlie Huang. 2022. Cold: A benchmark for chinese offensive language detection. pages 11580–11599. find
  14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  15. P Ekman. 1992. An argumet for basic emotions. cognition and emotion. University of California, San Francisco. find
  16. World Professional Association for Transgender Health. 2011. Standards of care for the health of transsexual, transgender, and gender nonconforming people. World Professional Association for Transgender Health. find
  17. HB Ganzeboom. 2010. International standard classification of occupations isco-08 with isei-08 scores. Version of July, 27:2010. find
  18. Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, and 1 others. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793.
  19. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for implicit and adversarial hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. find
  20. Aiqi Jiang, Xiaohan Yang, Yang Liu, and Arkaitz Zubiaga. 2021. Swsr: A chinese dataset and lexicon for online sexism detection. Preprint, arXiv:2108.03070. — [arXiv:2108.03070](https://arxiv.org/abs/2108.03070) find
  21. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. Fantom: A benchmark for stress-testing machine theory of mind in interactions. In EMNLP. find
  22. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In EMNLP. find
  23. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988.
  24. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  25. Junyu Lu, Bo Xu, Xiaokun Zhang, Changrong Min, Liang Yang, and Hongfei Lin. 2023. Facilitating fine-grained detection of Chinese toxic language: Hierarchical taxonomy, resources, and benchmarks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon… find
  26. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. find
  27. Wolfgang Prinz. 1997. Perception and action planning. European journal of cognitive psychology, 9(2):129– 154. find
  28. Xiaojun Rao, Yangsen Zhang, and 1 others. 2023. Chinese hate speech detection method based on RoBERTa-WWM). In Proceedings of the 22nd Chinese National Conference on Computational Linguistics, Harbin, China. Chinese Information Processing Society of China. find
  29. James A Russell. 2003. Core affect and the psychological construction of emotion. Psychological review, 110(1):145. find
  30. Baochen Sun, Jiashi Feng, and Kate Saenko. 2016. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 30. find
  31. Hind Taud and Jean-Franccois Mas. 2017. Multilayer perceptron (mlp). In Geomatic approaches for modeling land change scenarios, pages 451–455. Springer. find
  32. Michael Tomasello. 2009. The cultural origins of human cognition. Harvard university press. find
  33. Eitan Wagner, Nitay Alon, Joseph M Barnby, and Omri Abend. 2025. Mind your theory: Theory of mind goes deeper than reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria. Association for Computational Linguistics. find
  34. JD Mayer and P Salovey. 1990. Emotional intelligence imagination cognition and personality. American Journal of Educational Research. find
  35. Chengyu Wang, Minghui Qiu, Taolin Zhang, Tingting Liu, Lei Li, Jianing Wang, Ming Wang, Jun Huang, and Wei Lin. 2022. Easynlp: A comprehensive and easy-to-use toolkit for natural language processing. find
  36. Michael F McTear. 2002. Spoken dialogue technology: enabling the conversational user interface. ACM Computing Surveys (CSUR), 34(1):90–169. find
  37. Yinghui Wu and 1 others. 2023. Hi-tom: A benchmark for evaluating higher-order theory of mind reasoning in large language models. In EMNLP. find
  38. US Dept of Labor. 2000. Occupational outlook handbook. Jist Works. find
  39. World Health Organization. 2004. International Statistical Classification of Diseases and related health problems: Alphabetical index, volume 3. World Health Organization. find
  40. World Health Organization. 2015. World report on ageing and health. World Health Organization. find
  41. John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. Toxicity detection: Does context really matter? arXiv preprint arXiv:2006.00998.
  42. James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of liwc2015. find
  43. Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and recent advances. In IEEE Access. find
  44. Yuheng Wu, Wentao Guo, Zirui Liu, Heng Ji, Zhaozhuo Xu, and Denghui Zhang. 2025. How large language models encode theory-of-mind: a study on sparse parameter patterns. npj Artificial Intelligence, 1(1):20. find
  45. Alexandros Xenos, John Pavlopoulos, and Ion Androutsopoulos. 2021. Context sensitivity estimation in toxicity detection. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 140–145. find
  46. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597.
  47. Zhuo Xu. 2021. Roberta-wwm-ext fine-tuning for chinese text classification. arXiv preprint arXiv:2103.00492. — [arXiv:2103.00492](https://arxiv.org/abs/2103.00492) find
  48. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  49. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In ACL.
  50. Task / Stage Model Pair Stage 1: Initial Filter Stage 1: Arbitration find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shi-2026-danger-depends-mind-theory-mind/