Elamur Research⌕
☰
home / papers / 2026 / Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

Paper · 2026 · Hongbin Na, Zimu Wang, Zhaoming Chen et al.

Summary

Emotional support dialogue systems have mostly modeled empathy, supporter strategies and affect. They have largely ignored how help-seekers' words reveal psychological defenses, which shape what people disclose and whether they accept help. The PsyDefDetect shared task at BioNLP@ACL 2026 addresses this gap. Systems must assign each seeker utterance, using only the preceding context, one of nine labels: the seven hierarchical levels of the Defense Mechanism Rating Scales (DMRS) plus two auxiliary classes for non-defensive turns and turns with insufficient context.

The data come from PsyDefConv, a corpus built by stratified sampling from ESConv and double-blind annotated by experts. Its label distribution is heavily skewed, so teams were ranked by macro-F1.

The top system reached a macro-F1 of 0.420, about 10 points above the best fine-tuned baseline. Systems commonly over-predicted the dominant High-Adaptive level, and many showed a large gap between accuracy and macro-F1. The organizers argue that theory-informed and LLM-based methods look promising. They see the task as a foundation for clinically informed dialogue systems.

Key points

  • PsyDefConv corpus: 200 dialogues, 4,709 utterances, and 2,336 annotated seeker turns.
    • Inter-annotator agreement was Cohen's κ = 0.639.
    • DMRS Co-Pilot, a four-stage LLM assistant, reduced annotation time by 24.0%.
  • Label imbalance: Level 7 (High-Adaptive) covers 51.8% of seeker turns, while Level 8 covers only 1.5%. The official metric is macro-F1 over labels 1–8.
  • Participation: the task ran on CodaBench and drew 172 participants and 563 submissions. Of these, 21 teams were officially ranked, and 15 teams from 12 countries wrote system papers.
  • Baselines from the dataset paper:
    • Zero-shot: Gemini 2.5 Pro reached macro-F1 0.2599, and DeepSeek-V3.2 reached 0.2617.
    • Fine-tuned: Ministral-8B reached macro-F1 0.3148 with accuracy 0.6483.
  • Future directions: modeling how defenses change across a dialogue, building multilingual versions of PsyDefConv, jointly modeling defenses and supporter strategies, and generating responses conditioned on predicted defense levels.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🎓The University of MelbourneMelbourne · Australia100 works · 22,618 citations🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🎓University of UtahSalt Lake City · United States38 works · 5,925 citations🎓University of Technology SydneySydney · Australia18 works · 12,833 citations🎓University of TurkuTurku · Finland17 works · 4,890 citations🎓Xi’an Jiaotong-Liverpool UniversitySuzhou · China10 works · 319 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Hongbin Na, Zimu Wang, Zhaoming Chen, Yining Hua, Rena Gao, Kailai Yang, Ling Chen, Wei Wang, Shaoxiong Ji, John Torous and Sophia Ananiadou. 2026. Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-1.75

@inproceedings{na2026overview,
  title = {{Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations}},
  author = {Hongbin Na, Zimu Wang, Zhaoming Chen, Yining Hua, Rena Gao, Kailai Yang, Ling Chen, Wei Wang, Shaoxiong Ji, John Torous and Sophia Ananiadou},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.75},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.75}
}

References

48 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jenny Chim, Adam Tsakalidis, Dimitris Gkoumas, Dana Atzil-Slonim, Yaakov Ophir, Ayah Zirikly, Philip Resnik, and Maria Liakata. 2024. Overview of the CLPsych 2024 shared task: Leveraging large language models to identify evidence of suicidality risk in online posts. In Proceedings of the 9th Worksho… find
  2. Shefayat E Shams Adib, Ahmed Alfey Sani, Md Hasibur Rahman Alif, and Ajwad Abrar. 2026. find
  3. LinguIU-Anh Phuong Chu, Luong Duc Tran, Dat Hoang Do, Phuong Tu Mai, Quynh Hoang Le, and Cat Duy Can. 2026. DAL team at PsyDefDetect: From supervised encoders to hierarchical LLM-RAG for psychological defense detection. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Ta…
  4. Brandon Colelough, Davis Bartels, and Dina Demner-Fushman. 2025. Overview of the ClinIQLink 2025 shared task on medical question-answering. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 378–387, Vienna, Austria. Association for Computational Linguistics. find
  5. Glen Coppersmith, Mark Dredze, Craig Harman, Kristy Hollingshead, and Margaret Mitchell. 2015. find
  6. CLPsych 2015 shared task: Depression and PTSD on Twitter. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, pages 31– 39, Denver, Colorado. Association for Computational Linguistics.
  7. Mariagrazia Di Giuseppe, Katie Aafjes-van Doorn, Vera Békés, Bernard S Gorman, Karl Stukenberg, and Sherwood Waldron. 2024. Therapists’ defense use impacts their patients’ defensive functioning: a systematic case study. Research in Psychotherapy: Psychopathology, Process, and Outcome, 27(2):797. find
  8. Tamjid Hasan Fahim, Syed Asif Johan, and Saad Bin Maksud. 2026. PerceptionLab at PsyDefDetect: Overcoming LLM polarization via rubric-grounded retrieval and supervised clinical reasoning distillation for fine-grained ordinal classification. In Proceedings of the 25th Workshop on Biomedical Language… find
  9. Sigmund Freud. 1936. Inhibitions, symptoms and anxiety. The Psychoanalytic Quarterly, 5(1):1–28. find
  10. Dima Galat and Marian-Andrei Rizoiu. 2026. UTS at PsyDefDetect: Multi-agent councils and absence-based reasoning for defense mechanism classification. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA. Association for Computational Linguistics. find
  11. Muhammad Abu Horaira, Mehreen Rahman, and Nahian Chowdhury. 2026. Eraserhead at PsyDefDetect: Prompt design and class rebalancing for psychological defense mechanism detection. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA. Association for C… find
  12. Yining Hua, Hongbin Na, Zehan Li, Fenglin Liu, Xiao Fang, David Clifton, and John Torous. 2025a. A scoping review of large language models for generative tasks in mental health care. npj Digital Medicine, 8(1):230. find
  13. Yining Hua, Steve Siddals, Zilin Ma, Isaac Galatzer-Levy, Winna Xia, Christine Hau, Hongbin Na, Matthew Flathers, Jake Linardon, Cyrus Ayubcha, and John Torous. 2025b. Charting the evolution of artificial intelligence mental health chatbots from rule-based systems to large language models: a systema… find
  14. Bin Huang, Liuyuan Su, Kaixuan Yuan, Guanghui Zhao, Shixin Zhang, and Kunli Zhang. 2026. zzucs at Psy-DefDetect: Bridging long-tail imbalance and clinical rubrics for DMRS defense-level detection. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, US… find
  15. Dongjin Kang, Sunghwan Kim, Taeyoon Kwon, Seungjun Moon, Hyunsouk Cho, Youngjae Yu, Dongha Lee, and Jinyoung Yeo. 2024. Can large language models be good emotional supporter? mitigating preference bias on emotional support conversation. In Proceedings of the 62nd Annual Meeting of the Association fo… find
  16. Siam Rahman Karip and Nahid Hossain. 2026. Alien-Annotators at PsyDefDetect: What lies between the lines: Probing lightweight open-source LLMs for psychological defense mechanism detection. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA. Asso… find
  17. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan…
  18. Veronica Lynn, Alissa Goodman, Kate Niederhoffer, Kate Loveys, Philip Resnik, and H. Andrew Schwartz. 2018. find
  19. CLPsych 2018 shared task: Predicting current and future psychological health from childhood essays. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, pages 37–46, New Orleans, LA. Association for Computational Linguistics. find
  20. Jiayuan Ma, Hongbin Na, Zimu Wang, Yining Hua, Yue Liu, Wei Wang, and Ling Chen. 2025. Detecting conversational mental manipulation with intent-aware prompting. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9176–9183, Abu Dhabi, UAE. Association for Computat… find
  21. David N. Milne, Glen Pink, Ben Hachey, and Rafael A. Calvo. 2016. find
  22. CLPsych 2016 shared task: Triaging content in online peer-support forums. In Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology, pages 118–127, San Diego, CA, USA. Association for Computational Linguistics. find
  23. Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, and Ling Chen. 2025. A survey of large language models in psychotherapy: Current landscape and future directions. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7362–7376, Vienna… find
  24. Hongbin Na, Zimu Wang, Zhaoming Chen, Peilin Zhou, Yining Hua, Grace Ziqi Zhou, Haiyang Zhang, Tao Shen, Wei Wang, John Torous, Shaoxiong Ji, and Ling Chen. 2026. You never know a person, you only know their defenses: Detecting levels of psychological defense mechanisms in supportive conversations.…
  25. Durjoy C. Paul, Arshitha Basavaraj, Callum Chan, Veronica Perez-Rosas, Diana Inkpen, Francisco Pereira, and Juan Antonio Lossio-Ventura. 2026. TONI-NLP at PsyDefDetect: Defense mechanism detection via LLM-based ensemble methods. In Proceedings of the 25th Workshop on Biomedical Language Processing (…
  26. J Christopher Perry and Melissa Henry. 2004. Studying defense mechanisms in psychotherapy using the defense mechanism rating scales. Advances in psychology, 136:165–192. find
  27. Oarisa Rebayet, Radiul Walee, Symom Hossain Shohan, Kawsar Ahmed, and Mohammed Moshiul Hoque. 2026. CS_Metro at PsyDefDetect: Detecting psychological defense mechanisms in mental health dialogues with summarization-enhanced transformer ensembles. In Proceedings of the 25th Workshop on Biomedical Lan…
  28. Pritha Saha, Shuvodwip Saha, and Anik Mahmud Shanto. 2026. transformer_1376 at PsyDefDetect: A QLoRA-based generative framework for context-aware psychological defense mechanism detection. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA. Assoc…
  29. Sarvesh Soni, Soumya Gayen, and Dina Demner-Fushman. 2025. Overview of the ArchEHR-QA 2025 shared task on grounded question answering from electronic health records. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 396–405, Vienna, Austria. Association for Computational L… find
  30. Vera Sorin, Dana Brin, Yiftach Barash, Eli Konen, Alexander Charney, Girish Nadkarni, and Eyal Klang. 2024. Large language models and empathy: Systematic review. J Med Internet Res, 26:e52597. find
  31. Philipp Steigerwald, Eric Rudolph, and Jens Albrecht. 2026. Nürnberg NLP at PsyDefDetect: Multi-axis voter ensembles for psychological defence mechanism classification. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA. Association for Computati…
  32. Adam Tsakalidis, Jenny Chim, Iman Munire Bilal, Ayah Zirikly, Dana Atzil-Slonim, Federico Nanni, Philip Resnik, Manas Gaur, Kaushik Roy, Becky Inkster, Jeff Leintz, and Maria Liakata. 2022. Overview of the CLPsych 2022 shared task: Capturing moments of change in longitudinal user posts. In Proceedin… find
  33. Talia Tseriotou, Jenny Chim, Ayal Klein, Aya Shamir, Guy Dvir, Iqra Ali, Cian Kennedy, Guneet Singh Kohli, Anthony Hills, Ayah Zirikly, Dana Atzil-Slonim, and Maria Liakata. 2025. Overview of the CLPsych 2025 shared task: Capturing mental health dynamics from social media timelines. In Proceedings o… find
  34. George E Vaillant. 2012. Adaptation to life. Harvard University Press. find
  35. Hoang-Thuy-Duong Vu, Quoc-Cuong Pham, and Huy-Hieu Pham. 2026. VISHC at PsyDefDetect: Mitigating data scarcity in psychological defense classification with context-aware synthetic augmentation. In Proceedings of the 25th Workshop on Biomedical Language Processing (Shared Tasks), San Diego, CA, USA.…
  36. Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, and Jie Zhou. 2022. MAVEN-ERE: A unified large-scale dataset for event coreference, temporal, causal, and subevent relation extraction. In Proceedings of the 2022 Conference on Emp…
  37. Yuqi Wang, Zimu Wang, Nijia Han, Wei Wang, Qi Chen, Haiyang Zhang, Yushan Pan, and Anh Nguyen. 2024a. Knowledge distillation from monolingual to multilingual models for intelligent and interpretable multilingual emotion detection. In Proceedings of the 14th Workshop on Computational Approaches to Su… find
  38. Zimu Wang, Hongbin Na, Rena Gao, Jiayuan Ma, Yining Hua, Ling Chen, and Wei Wang. 2025. From posts to timelines: Modeling mental health dynamics from social media timelines with hybrid LLMs. In Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025), page… find
  39. Zimu Wang, Lei Xia, Wei Wang, and Xinya Du. 2024b. Document-level causal relation extraction with knowledge-guided binary question answering. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16944–16955, Miami, Florida, USA. Association for Computational Linguistics. find
  40. Donald W Winnicott. 2018. Ego distortion in terms of true and false self. In The person who is me, pages 7–22. Routledge. find
  41. Chenghao Xiao, Kun Zhao, Xiao Wang, Siwei Wu, Sixing Yan, Tomas Goldsack, Sophia Ananiadou, Noura Al Moubayed, Liang Zhan, William K. Cheung, and Chenghua Lin. 2025. Overview of the BioLaySumm 2025 shared task on lay summarization of biomedical research articles and radiology reports. In Proceedings… find
  42. Qingyang Xu, Yaling Shen, Stephanie Fong, Zimu Wang, Yiwen Jiang, Xiangyu Zhao, Jiahe Liu, Zhongxing Xu, Vincent Lee, and Zongyuan Ge. 2026. Do no harm: Exposing hidden vulnerabilities of LLMs via persona-based client simulation attack in psychological counseling. Preprint, arXiv:2604.04842. — [arXiv:2604.04842](https://arxiv.org/abs/2604.04842) find
  43. Zhen Xu, Sergio Escalera, Adrien Pavão, Magali Richard, Wei-Wei Tu, Quanming Yao, Huan Zhao, and Isabelle Guyon. 2022. Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform. Patterns, 3(7):100543. find
  44. Tenggan Zhang, Xinjie Zhang, Jinming Zhao, Li Zhou, and Qin Jin. 2024. ESCoT: Towards interpretable emotional support dialogue systems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13395–13412, Bangkok, Thailand. Associatio… find
  45. Xiangyu Zhao, Yaling Shen, Yiwen Jiang, Zimu Wang, Jiahe Liu, Maxmartwell H Cheng, Guilherme C Oliveira, Robert Desimone, Dominic Dwyer, and Zongyuan Ge. 2025. It hears, it sees too: Multimodal LLM for depression detection by integrating visual understanding into audio language models. Preprint, arX… — [arXiv:2511.19877](https://arxiv.org/abs/2511.19877) find
  46. Chujie Zheng, Sahand Sabour, Jiaxin Wen, Zheng Zhang, and Minlie Huang. 2023. AugESC: Dialogue augmentation with large language models for emotional support conversation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1552–1568, Toronto, Canada. Association for Computa… find
  47. Ayah Zirikly, Philip Resnik, Özlem Uzuner, and Kristy Hollingshead. 2019. find
  48. CLPsych 2019 shared task: Predicting the degree of suicide risk in Reddit posts. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 24–33, Minneapolis, Minnesota. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/na-2026-overview-psydefdetect-shared-task-bionlp/