Elamur Research⌕
☰
home / papers / 2026 / transformer_1376 at PsyDefDetect: A QLoRA-Based Generative Framework for Context-Aware Psychological Defense Mechanism Detection

transformer_1376 at PsyDefDetect: A QLoRA-Based Generative Framework for Context-Aware Psychological Defense Mechanism Detection

Paper · 2026 · Pritha Saha, Shuvodwip Saha and Anik Mahmud Shanto

Summary

The paper presents a system for the PsyDefDetect shared task at BioNLP 2026. The task requires labeling a target utterance in a multi-turn emotional support dialogue with one of nine classes, which are derived from the Defense Mechanism Rating Scales (DMRS). These include a "no defense" label and a label for cases that lack enough context. The authors frame labeling as conditional generation rather than standard classification. Gemma-2-2B-IT receives a chat-formatted prompt with instructions, the preceding dialogue and the target turn, and learns to emit a single label token, with loss computed only on that token. Training combines 4-bit QLoRA (via Unsloth) on cloud T4 GPUs, random oversampling of rare classes, and left truncation of long contexts. On the blind test set the system reaches a Macro-F1 of 0.2475, ranking 12th of 21 teams. It clearly outperforms an in-house DeBERTa-v3-base classifier trained on the same data but falls short of the organizers' baseline (0.3148). The work shows that low-budget generative fine-tuning is workable for clinically grounded dialogue analysis, though with clear performance limits.

Key points

  • Data: PSYDEFCONV, built from a subset of ESConv. It has 200 dialogues and 4,709 utterances, of which 2,336 are annotated with 9 DMRS-based classes. Evaluation uses Macro-F1 over classes 1–8 and excludes the majority "No Defense" class.
  • Method: Gemma-2-2B-IT with 4-bit QLoRA, LoRA adapters on all attention and feed-forward projections, and prompt-token masking. Training uses AdamW with warmup plus cosine decay at an effective batch size of 16, and inference uses greedy decoding.
  • Results on the test set:
    • Proposed system: accuracy 0.5508, F1 0.2475.
    • DeBERTa-v3-base: accuracy 0.5500, F1 0.1376.
    • Official baseline: accuracy 0.6483, F1 0.3148.
  • Gemma's chat template has no system role, so the instruction is merged into the user turn.
  • Stated limitations include the small model size, possible overfitting from oversampling, reliance on fixed prompt templates, and the need for post-processing of generated labels.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh28 works · 29 citations🎓Southeast UniversityDhaka · Bangladesh9 works · 160 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksSocial Psychology · 330 works

Cite

Cite as Pritha Saha, Shuvodwip Saha and Anik Mahmud Shanto. 2026. transformer_1376 at PsyDefDetect: A QLoRA-Based Generative Framework for Context-Aware Psychological Defense Mechanism Detection. Proceedings of the BioNLP 2026 (Shared Tasks).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-2.15

@inproceedings{saha2026transformer,
  title = {{transformer\_1376 at PsyDefDetect: A QLoRA-Based Generative Framework for Context-Aware Psychological Defense Mechanism Detection}},
  author = {Pritha Saha, Shuvodwip Saha and Anik Mahmud Shanto},
  booktitle = {Proceedings of the BioNLP 2026 (Shared Tasks)},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-2.15},
  url = {https://doi.org/10.18653/v1/2026.bionlp-2.15}
}

References

13 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.
  3. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan…
  4. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  5. Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, and Ling Chen. 2025. A survey of large language models in psychotherapy: Current landscape and future directions. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7362–7376, Vienna… find
  6. Hongbin Na, Zimu Wang, Zhaoming Chen, Yining Hua, Rena Gao, Kailai Yang, Ling Chen, Wei Wang, Shaoxiong Ji, John Torous, and Sophia Ananiadou. 2026a. Overview of the psydefdetect shared task at bionlp 2026: Detecting levels of psychological defense mechanisms in supportive conversations. In Proceedi…
  7. Hongbin Na, Zimu Wang, Zhaoming Chen, Peilin Zhou, Yining Hua, Grace Ziqi Zhou, Haiyang Zhang, Tao Shen, Wei Wang, John Torous, Shaoxiong Ji, and Ling Chen. 2026b. You never know a person, you only know their defenses: Detecting levels of psychological defense mechanisms in supportive conversations.…
  8. J. Christopher Perry. 1990. Defense Mechanism Rating Scales (DMRS). find
  9. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Navonil Majumder, Amir Zadeh, and Louis-Philippe Morency. 2017. Context-dependent sentiment analysis in user-generated videos. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8…
  10. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  11. Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. Preprint, arXiv:1507.04808.
  12. Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. Transfertransfo: A transfer learning approach for neural network based conversational agents. Preprint, arXiv:1901.08149. find
  13. Lei Zhang, Shuai Wang, and Bing Liu. 2018. Deep learning for sentiment analysis : A survey. Preprint, arXiv:1801.07883. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saha-2026-transformer-1376-psydefdetect-qlora-based/