Elamur Research⌕
☰
home / papers / 2026 / PersonalityDBench: A Dataset for Personality Disorders - from Modeling to Controlled Generation

PersonalityDBench: A Dataset for Personality Disorders - from Modeling to Controlled Generation

Paper · 2026 · Federico Ravenda, Seyed Ali Bahrainian, Daniele Montagnani et al.

Summary

Most NLP mental-health research on social media has focused on depression and suicide risk. Personality disorders (PDs) have received little attention and have no clinically annotated resources. The authors introduce PersonalityDBench, a Reddit-based resource that covers all ten PDs across Clusters A, B and C. It has two parts. PRISMA collects posting histories of users who report a professional diagnosis and annotates them at two levels: individual posts are tagged with dimensional traits from the HiTOP framework, and whole users are assessed against SCID-5-PD diagnostic criteria. PersonaDSteering turns these clinical profiles into structured question-based tasks that measure how well LLMs can be steered toward disorder-consistent behaviour, for example for simulated patients.

The experiments show three main results. SCID-5-PD criteria can be detected in user histories, and a retrieval-augmented LLM setup reduces symptom overestimation. HiTOP traits are discriminative and reflect the hierarchy linking traits, spectra and diagnoses. Activation steering shifts outputs toward target PDs, often more so when combined with in-context examples, though controllability varies by disorder and model. The resource offers a standardized, clinically grounded testbed for both PD modeling and controlled generation.

Key points

  • Data come from 14 subreddits, including r/BPD, r/NPD, r/OCPD, r/AvPD, r/Schizoid, r/aspd and r/mentalillness. Users were split by self-reported formal diagnosis, which was detected with regular expressions.
  • There are two annotation layers: post-level HiTOP traits and user-level SCID-5-PD criteria. LLMs serve as annotators, with a RAG setup that brings predictions closer to clinical judgments.
  • PersonaDSteering compares four conditions: unsteered, ICL only, CAA steering only, and ICL+steering. The combination often gives the best results, and CAA sometimes beats ICL, notably for Llama-3.2-3B-Instruct.
  • OCPD, PPD and DPD are the easiest disorders to elicit, while NPD and ASPD are the hardest. Mistral-7B-Instruct is the least steerable model, plausibly because of safety alignment. The two Llama models perform best, and overly strong steering can produce incoherent text.
  • Access to the sensitive data is controlled via the GitHub repository Fede-stack/PersonalityDBench. Diagnoses are self-reported and unverifiable, and the authors flag the risk of stigma for disorders such as ASPD, NPD and BPD.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TübingenTübingen · Germany75 works · 41,081 citations🎓Università della Svizzera italianaLugano · Switzerland6 works · 647 citations🎓University of PaviaPavia · Italy6 works · 69 citations🎓University of InsubriaVarese · Italy3 works · 1 citation

Also on the byline: Organizzazione Sociopsichiatrica Cantonale.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 worksClinical Psychology · 20 works

Cite

Cite as Federico Ravenda, Seyed Ali Bahrainian, Daniele Montagnani, Antonietta Mira and Andrea Raballo. 2026. PersonalityDBench: A Dataset for Personality Disorders - from Modeling to Controlled Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1395

@inproceedings{ravenda2026personalitydbench,
  title = {{PersonalityDBench: A Dataset for Personality Disorders - from Modeling to Controlled Generation}},
  author = {Federico Ravenda, Seyed Ali Bahrainian, Daniele Montagnani, Antonietta Mira and Andrea Raballo},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1395},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1395}
}

References

70 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. APA. 2013. Diagnostic and statistical manual of mental disorders: DSM-5. American psychiatric association. find
  2. Yi Ji Bae, Midan Shim, and Won Hee Lee. 2021. Schizophrenia detection using machine learning approach from social media content. Sensors, 21(17):5924. find
  3. Seyed Ali Bahrainian, Jonathan Dou, and Carsten Eickhoff. 2024. Text simplification via adaptive teaching. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6574–6584, Bangkok, Thailand. Association for Computational Linguistics. find
  4. Seyed Ali Bahrainian, Marcus Liwicki, and Andreas Dengel. 2014. Fuzzy subjective sentiment phrases: A context sensitive and self-maintaining sentiment lexicon. In 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), volume 1, pages 361–… find
  5. Seyed Ali Bahrainian, Ida Mele, and Fabio Crestani. 2018. Predicting topics in scholarly papers. In Advances in Information Retrieval, pages 16–28, Cham. Springer International Publishing. find
  6. Intended use and misuse prevention. PersonalityDBench and its associated models are released strictly for research and educational purposes. The dataset and models must not be used for: Joschka Braun, Carsten Eickhoff, and Seyed Ali Bahrainian. Beyond multiple choice: Evaluating steering vectors for… find
  7. Systems that impersonate real individuals, simulate communities without transparency, or manipulate public opinion; Joschka Braun, Carsten Eickhoff, and Seyed Ali Bahrainian. 2026. Beyond multiple choice: Evaluating steering vectors for summarization. In Findings of the Association for Computational… find
  8. De-anonymization attempts or linking of users across platforms or datasets; • Commercial applications in hiring, law enforcement, insurance, or other high-stakes decision-making contexts where misuse could result in harm or discrimination. Any downstream application must adhere to stringent ethical… find
  9. Ana-Maria Bucur, Adrian Cosma, and Liviu P Dinu. 2021. Early risk detection of pathological gambling, self-harm and depression using bert. arXiv preprint arXiv:2106.16175. — [arXiv:2106.16175](https://arxiv.org/abs/2106.16175) find
  10. Ana-Maria Bucur, Andreea-Codrina Moldovan, Kru-tika Parvatikar, Marcos Zampieri, Ashiqur R KhudaBukhsh, and Liviu P Dinu. 2025. Datasets for depression modeling in social media: An overview. arXiv preprint arXiv:2503.21513. find
  11. Kai Chen, Zihao He, Taiwei Shi, and Kristina Lerman. 2025. Steer-bench: A benchmark for evaluating the steerability of large language models. arXiv preprint arXiv:2505.20645. find
  12. Arman Cohan, Bart Desmet, Andrew Yates, Luca Soldaini, Sean MacAvaney, and Nazli Goharian. 2018. Smhd: a large-scale resource for exploring online language usage for multiple mental health conditions. arXiv preprint arXiv:1806.05258.
  13. Alessandro De Grandi, Federico Ravenda, Andrea Raballo, and Fabio Crestani. 2024. The emotional spectrum of llms: Leveraging empathy and emotion-based markers for mental health support. arXiv preprint arXiv:2412.20068. find
  14. Oliver Delgaram-Nejad, Dawn Archer, Gerasimos Chatzidamianos, Louise Robinson, and Alex Bartha. 2023. The dais-c: A small, specialised, spoken, schizophrenia corpus. Applied Corpus Linguistics, 3(3):100069. find
  15. Antonio Di Noia, Iuri Macocco, Aldo Glielmo, Alessandro Laio, and Antonietta Mira. 2024. Beyond the noise: intrinsic dimension estimation with optimal neighbourhood identification. arXiv preprint arXiv:2405.15132. — [arXiv:2405.15132](https://arxiv.org/abs/2405.15132) find
  16. Zohar Elyoseph, Inbar Levkovich, and 1 others. 2024. Comparing the perspectives of generative ai, mental health experts, and the general public on schizophrenia recovery: case vignette study. JMIR Mental Health, 11(1):e53043. find
  17. MB First, JB Williams, LS Benjamin, and RL Spitzer. 2016. Structured clinical interview for dsm-5 personality disorders: Scid-5-pd. Testkatalog, page 30. find
  18. George Gkotsis, Anika Oellrich, Sumithra Velupillai, Maria Liakata, Tim JP Hubbard, Richard JB Dobson, and Rina Dutta. 2017. Characterisation of mental health conditions in social media using informed deep learning. Scientific reports, 7(1):1–11. find
  19. Jonathan Gratch, Ron Artstein, Gale M Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, and 1 others. 2014. The distress analysis interview corpus of human and computer interviews. In LREC, volume 14, pages 3123–3128. Reykjavik. find
  20. Shrey Gupta, Anmol Agarwal, Manas Gaur, Kaushik Roy, Vignesh Narayanan, Ponnurangam Kumaraguru, and Amit Sheth. 2022. Learning to automate follow-up question generation using process knowledge for depression triage on reddit posts. arXiv preprint arXiv:2205.13884. find
  21. Shawn Im and Yixuan Li. 2025. A unified understanding and evaluation of steering methods. arXiv preprint arXiv:2502.02716. — [arXiv:2502.02716](https://arxiv.org/abs/2502.02716) find
  22. Glorianna Jagfeld, Fiona Lobban, Paul Rayson, and Steven H Jones. 2021. Understanding who uses reddit: Profiling individuals with a self-reported bipolar disorder diagnosis. arXiv preprint arXiv:2104.11612. find
  23. Roman Kotov, Robert F Krueger, David Watson, Thomas M Achenbach, Robert R Althoff, R Michael Bagby, Timothy A Brown, William T Carpenter, Avshalom Caspi, Lee Anna Clark, and 1 others. 2017. The hierarchical taxonomy of psychopathology (hitop): A dimensional alternative to traditional nosologies. Jou… find
  24. Mark F Lenzenweger, Michael C Lane, Armand W Loranger, and Ronald C Kessler. 2007. Dsm-iv personality disorders in the national comorbidity survey replication. Biological psychiatry, 62(6):553–564. find
  25. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691.
  26. Inbar Levkovich, Eyal Rabin, Michal Brann, and Zohar Elyoseph. 2024. Large language models outperform general practitioners in identifying complex cases of childhood anxiety. Digital Health, 10:20552076241294182. find
  27. David E Losada, Fabio Crestani, and Javier Parapar. 2017. erisk 2017: Clef lab on early risk prediction on the internet: experimental foundations. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 8th International Conference of the CLEF Association, CLEF 2017, Dublin, Irelan… find
  28. David E Losada, Fabio Crestani, and Javier Parapar. 2019. Overview of erisk 2019 early risk prediction on the internet. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 340–357. Springer. find
  29. David E Losada, Fabio Crestani, and Javier Parapar. 2020. erisk 2020: Self-harm and depression challenges. In European conference on information retrieval, pages 557–563. Springer. find
  30. Daniel M Low, Laurie Rumker, John Torous, Guillermo Cecchi, Satrajit S Ghosh, and Tanya Talkar. 2020. Natural language processing reveals vulnerable mental health support groups and heightened health anxiety on reddit during covid-19: Observational study. Journal of medical Internet research, 22(10)… find
  31. Sean MacAvaney, Anjali Mittu, Glen Coppersmith, Jeff Leintz, and Philip Resnik. 2021. Community-level research on suicidality prediction in a secure environment: Overview of the CLPsych 2021 shared task. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Imp… find
  32. Heather McClelland, Seonaid Cleare, and Rory C O’Connor. 2023. Suicide risk in personality disorders: A systematic review. Current psychiatry reports, 25(9):405–417. find
  33. Sandra Mitrović, Oscar William Lithgow-Serrano, and Carlo Schillaci. 2024. Comparing panic and anxiety on a dataset collected from social media. In Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2024), pages 153–165. find
  34. Paul Moran, Helena Romaniuk, Carolyn Coffey, Andrew Chanen, Louisa Degenhardt, Rohan Borschmann, and George C Patton. 2016. The influence of personality disorder on the future mental health and social adjustment of young adults: a population-based, longitudinal cohort study. The Lancet Psychiatry, 3… find
  35. Usman Naseem, Adam G Dunn, Jinman Kim, and Matloob Khushi. 2022. Early identification of depression severity levels on reddit using ordinal classification. In Proceedings of the ACM web conference 2022, pages 2563–2572. find
  36. John A Naslund, Kelly A Aschbrenner, Lisa A Marsch, and Stephen J Bartels. 2016. The future of mental health care: peer-to-peer support and social media. Epidemiology and psychiatric sciences, 25(2):113– 122. find
  37. NIMH. Personality disorders. https://www.nimh.nih.gov/health/statistics/ personality-disorders. Accessed: 2025-10-06. ↗
  38. Yvette Oortwijn, Thijs Ossenkoppele, and Arianna Betti. 2021. Interrater disagreement resolution: A systematic procedure to reach consensus in annotation tasks. In Proceedings of the workshop on human evaluation of NLP systems (HumEval), pages 131–141. find
  39. World Health Organization and 1 others. 1992. Icd-11. (No Title). find
  40. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  41. Javier Parapar, Patricia Martín-Rodilla, David E Losada, and Fabio Crestani. 2021. erisk 2021: pathological gambling, self-harm and depression challenges. In European Conference on Information Retrieval, pages 650–656. Springer. find
  42. Anxo Pérez, Marcos Fernández-Pichel, Javier Parapar, and David E Losada. 2023. Depresym: A depression symptom annotated corpus and the role of llms as assessors of psychological markers. arXiv preprint arXiv:2308.10758. — [arXiv:2308.10758](https://arxiv.org/abs/2308.10758) find
  43. Paloma Piot, Patricia Martín-Rodilla, and Javier Parapar. 2024. Metahate: A dataset for unifying efforts on hate speech detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 2025–2039. find
  44. Shae E Quirk, Michael Berk, Andrew M Chanen, Heli Koivumaa-Honkanen, Sharon L Brennan-Olsen, Julie A Pasco, and Lana J Williams. 2016. Population prevalence of personality disorder and associations with physical health comorbidities and health care service utilization: A review. Personality Disorder… find
  45. Andrea Raballo, Federico Ravenda, and Antonietta Mira. Diagnosing schizophrenia spectrum disorders: Large language models (llms) vs. leading international psychiatrists (lips). Psychiatry and Clinical Neurosciences. find
  46. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  47. Nishat Raihan, Sadiya Sayara Chowdhury Puspo, Shafkat Farabi, Ana-Maria Bucur, Tharindu Ranasinghe, and Marcos Zampieri. 2024. Mentalhelp: A multi-task dataset for mental health in social media. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resource… find
  48. Saima Rani, Khandakar Ahmed, and Sudha Subramani. 2024. From posts to knowledge: Annotating a pandemic-era reddit dataset to navigate mental health narratives. Applied Sciences, 14(4):1547. find
  49. Arya S Rao and Anthony R Artino. 2025. Ai-driven osce preparation in medical education: Promise, pitfalls, and practical implications. find
  50. Federico Ravenda, Seyed Ali Bahrainian, Andrea Raballo, and Antonietta Mira. 2025a. Navigating through the hidden embedding space: steering llms to improve mental health assessment. arXiv preprint arXiv:2510.16373. — [arXiv:2510.16373](https://arxiv.org/abs/2510.16373) find
  51. Federico Ravenda, Seyed Ali Bahrainian, Andrea Raballo, Antonietta Mira, and Noriko Kando. 2025b. Are LLMs effective psychological assessors? leveraging adaptive RAG for interpretable mental health screening through psychometric practice. In Proceedings of the 63rd Annual Meeting of the Association… find
  52. Federico Ravenda, Antonio Preti, Michele Poletti, , Antonietta Mira, and Andrea Raballo. 2025c. Rethinking psychometrics though llms: How item semantics shape measurement and prediction in psychological questionnaires. Nature Scientific Reports. find
  53. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, Bangkok, Thaila… find
  54. Sonja Schmer-Galunder, Ruta Wheelock, Zaria Jalan, Alyssa Chvasta, Scott Friedman, and Emily Saltz. 2024. Annotator in the loop: A case study of in-depth rater engagement to create a prosocial benchmark dataset. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7, pages 13… find
  55. Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga-Alonso, and Robert Kirk. 2024. Analysing the generalisation and reliability of steering vectors. Advances in Neural Information Processing Systems, 37:139179– 139212. find
  56. Adam Tsakalidis, Jenny Chim, Iman Munire Bilal, Ayah Zirikly, Dana Atzil-Slonim, Federico Nanni, Philip Resnik, Manas Gaur, Kaushik Roy, Becky Inkster, and 1 others. 2022. Overview of the clpsych 2022 shared task: Capturing moments of change in longitudinal user posts. In Proceedings of the Eighth W… find
  57. Talia Tseriotou, Jenny Chim, Ayal Klein, Aya Shamir, Guy Dvir, Iqra Ali, Cian Kennedy, Guneet Singh Kohli, Anthony Hills, Ayah Zirikly, Dana Atzil-Slonim, and Maria Liakata. 2025. Overview of the clpsych 2025 shared task: Capturing mental health dynamics from social media timelines. In Proceedings o… find
  58. Peter Tyrer, Roger Mulder, Mike Crawford, Giles Newton-Howes, Erik Simonsen, David Ndetei, Nestor Koldobsky, Andrea Fossati, Joseph Mbatia, and Barbara Barrett. 2010. Personality disorder: a new global perspective. World Psychiatry, 9(1):56. find
  59. Bar Urkin, Josef Parnas, Andrea Raballo, and Danny Koren. 2024. Schizophrenia spectrum disorders: an empirical benchmark study of real-world diagnostic accuracy and reliability among leading international psychiatrists. Schizophrenia Bulletin Open, 5(1):sgae012. find
  60. Ruiyi Wang, Stephanie Milani, Jamie C Chiu, Jiayin Zhi, Shaun M Eack, Travis Labrum, Samuel M Murphy, Nev Jones, Kate Hardy, Hong Shen, and 1 others. 2024. Patient-{\\Psi}: Using large language models to simulate patients for training mental health professionals. arXiv preprint arXiv:2405.19660. find
  61. Xi Wang, Anxo Perez, Javier Parapar, and Fabio Crestani. 2025. Talkdep: clinically grounded llm personas for conversation-centric depression screening. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, pages 6554–6558. find
  62. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  63. Catherine Winsper, Ayten Bilgin, Andrew Thompson, Steven Marwaha, Andrew M Chanen, Swaran P Singh, Ariel Wang, and Vivek Furtado. 2020. The prevalence of personality disorders in the community: a global systematic review and meta-analysis. The British Journal of Psychiatry, 216(2):69–78. find
  64. Jia Xu, Tianyi Wei, Bojian Hou, Patryk Orzechowski, Shu Yang, Ruochen Jin, Rachael Paulbeck, Joost Wagenaar, George Demiris, and Li Shen. 2025. Mentalchat16k: A benchmark dataset for conversational mental health assistance. Preprint, arXiv:2503.13509. — [arXiv:2503.13509](https://arxiv.org/abs/2503.13509) find
  65. Andrew Yates, Arman Cohan, and Nazli Goharian. 2017. Depression and self-harm risk assessment in online forums. arXiv preprint arXiv:1709.01848.
  66. Jeewoo Yoon, Chaewon Kang, Seungbae Kim, and Jinyoung Han. 2022. D-vlog: Multimodal vlog dataset for depression detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12226–12234. find
  67. Junbo Zhang, Ran Chen, Qianli Zhou, Xinyang Deng, and Wen Jiang. 2025. Understanding and mitigating over-refusal for large language models via safety representation. arXiv preprint arXiv:2511.19009. — [arXiv:2511.19009](https://arxiv.org/abs/2511.19009) find
  68. Tianlin Zhang, Kailai Yang, Shaoxiong Ji, Boyang Liu, Qianqian Xie, and Sophia Ananiadou. 2024. Suicide-moji: Derived emoji dataset and tasks for suicide-related social content. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages… find
  69. Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Is in-context learning sufficient for instruction following in llms? arXiv preprint arXiv:2405.19874. — [arXiv:2405.19874](https://arxiv.org/abs/2405.19874) find
  70. Shengqi Zhu and Jeffrey Rzeszotarski. 2024. “get their hands dirty, not mine”: On researcher-annotator collaboration and the agency of annotators. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8773–8782. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ravenda-2026-personalitydbench-dataset-personality-disorders-modeling/