Elamur Research⌕
☰
home / papers / 2026 / Psycholinguistic Profiles of Cognitive Distortions in Reddit Data

Psycholinguistic Profiles of Cognitive Distortions in Reddit Data

Paper · 2026 · Neha Sharma, Navneet Agarwal and Kairit Sirts

Summary

Most computational work on cognitive distortions (CDs) aims to detect or classify them in individual texts. This paper instead asks what distorted language looks like at the population level. It poses two questions: does distorted language differ psycholinguistically from text without distortions, and does each type of distortion carry a distinct linguistic signature?

The authors used the RSDD Reddit corpus, which has no CD annotations, so they generated labels for it. They trained two mentalRoBERTa multi-label classifiers on the Therapist Q&A dataset. One used human annotations and the other used LLM-produced annotations. Only posts where the two classifiers' predicted labels fully or partly overlapped were kept. They then extracted a curated subset of LIWC-22 features and modelled them with a bootstrapped GLM.

Distorted text proved psycholinguistically distinct from non-distorted text. Each distortion type showed a profile that broadly matches its clinical description. Hierarchical clustering showed that the types overlap, with some sharing stable feature patterns.

The work offers an interpretable, theory-aligned picture of distorted thinking that can complement detection systems in mental health research.

Key points

  • Data: The final RSDD corpus covers 3,069 depression and 3,069 control users, with roughly 3.20M and 1.53M posts respectively.
  • Labeling: Labels were transferred from Therapist Q&A (2,530 exchanges) using two annotation sources. The human annotations had only 33.7% inter-annotator agreement, while the LLM annotations reached Fleiss's κ of 0.78. Posts where the two classifiers' predictions shared no label were discarded.
  • Model choice: The authors first fitted a GLMM, but its user-level random intercept collapsed to zero, so they used a simpler GLM. They acknowledge that per-user balancing may have caused this.
  • Clustering: Fortune Telling, Emotional Reasoning and Magnification consistently grouped together. The features driving this grouping relate to a negative view of the future: anxiety (emo_anx), future orientation (focusfuture), and death and illness vocabulary.
  • Limitations: Findings come from informal Reddit posts rather than clinical text. LIWC is not sensitive to context, and the authors selected the feature subset based on their own judgment.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TartuTartu · Estonia12 works · 2,874 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Psycholinguistics & language acquisition · 102 works

Cite

Cite as Neha Sharma, Navneet Agarwal and Kairit Sirts. 2026. Psycholinguistic Profiles of Cognitive Distortions in Reddit Data. Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.clpsych-1.25

@inproceedings{sharma2026psycholinguistic,
  title = {{Psycholinguistic Profiles of Cognitive Distortions in Reddit Data}},
  author = {Neha Sharma, Navneet Agarwal and Kairit Sirts},
  booktitle = {Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.clpsych-1.25},
  url = {https://doi.org/10.18653/v1/2026.clpsych-1.25}
}

References

41 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ana Abutara, Aline Kissimoto, Felipe Oliveira de Aguiar, Victor Otani, Ricardo Riyoiti Uchida, and Lucas Murrins Marques. 2025. Beyond words: understanding anxiety and depression in college applicants through liwc analysis of textual features. Frontiers in Psychology, Volume 16 - 2025. find
  2. Carla Agurto, Pat Pataranutaporn, Elif K. Eyigoz, Gustavo Stolovitzky, and Guillermo Cecchi. 2018. Predictive linguistic markers of suicidality in poets. In 2018 IEEE 12th International Conference on Semantic Computing (ICSC), pages 282–285. find
  3. Mohammed Al-Mosaiwi and Tom Johnstone. 2018. In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clinical psychological science, 6(4):529– 542. find
  4. Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1):1–48.
  5. Krishna C Bathina, Marijn Ten Thij, Lorenzo Lorenzo-Luaces, Lauren A Rutter, and Johan Bollen. 2021. Individuals with depression express more distorted thinking on social media. Nature human behaviour, 5(4):458–466. find
  6. Aaron T. Beck. 1963. Thinking and depression: I. Idiosyncratic content and cognitive distortions. Archives of General Psychiatry, 9:324–333. find
  7. Aaron T. Beck, A. John Rush, Brian F. Shaw, and Gary Emery. 1979. Cognitive Therapy of Depression. Guilford Press, New York. find
  8. Johan Bollen, Marijn ten Thij, Fritz Breithaupt, Alexander T. J. Barron, Lauren A. Rutter, Lorenzo Lorenzo-Luaces, and Marten Scheffer. 2021. Historical language records reveal a surge of cognitive distortions in recent decades. Proceedings of the National Academy of Sciences, 118(30). https://githu… find
  9. David D. Burns. 1980. Feeling Good: The New Mood Therapy. Signet, New York. find
  10. Zhiyu Chen, Yujie Lu, and William Wang. 2023. Empowering psychotherapy with large language models: Cognitive distortion detection through diagnosis of thought prompting. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4295–4304, Singapore. Association for Computationa… find
  11. The Therapist Q&A dataset and its annotations are available through Shreevastava and Foltz (2021) and Sharma et al. (2026) respectively. Access to the RSDD dataset may be requested directly from its original authors. Code to reproduce the analyses presented in this work is available on Github 3 . find
  12. Ryan L Boyd, Ashwini Ashokkumar, Sarah Seraj, and James W Pennebaker. 2022. The development and psychometric properties of liwc-22. Austin, TX: University of Texas at Austin, 10(1-47):6. find
  13. Glen Coppersmith, Craig Harman, and Mark Dredze. 2014. Measuring post traumatic stress disorder in twitter. Proceedings of the International AAAI Conference on Web and Social Media, 8(1):579–582. find
  14. Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2021. Predicting depression via social media. Proceedings of the International AAAI Conference on Web and Social Media, 7(1):128–137. find
  15. Matthew F. Dobbs, Alessia McGowan, Alexandria Selloni, Zarina Bilgrami, Cansu Sarac, Matthew Cotter, Shaynna N. Herrera, Guillermo A. Cecchi, Marianne Goodman, Cheryl M. Corcoran, and Agrima Srivastava. 2023. Linguistic correlates of suicidal ideation in youth at clinical high-risk for psychosis. Sc… find
  16. Johannes C. Eichstaedt, Robert J. Smith, Raina M. Merchant, Lyle H. Ungar, Patrick Crutchley, Daniel Preoţiuc-Pietro, David A. Asch, and H. Andrew Schwartz. 2018. Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences, 115(44):11203–11208. find
  17. Rachel Heyard. 2026. Introduction to regression methods for public health using r. The American Statistician, 80(1):188–190. find
  18. Stephanie Homan, Marion Gabi, Nina Klee, Sandro Bachmann, Ann-Marie Moser, Martina Duri’, Sofia Michel, Anna-Marie Bertram, Anke Maatz, Guido Seiler, Elisabeth Stark, and Birgit Kleim. 2022. Linguistic features of suicidal thoughts and behaviors: A systematic review. Clinical Psychology Review, 95:1… find
  19. Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2022. Mental-BERT: Publicly available pretrained language models for mental healthcare. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7184–7190, Marseille, France. European Langu… find
  20. Jutta Joormann and Colin H Stanton. 2016. Examining emotion regulation in depression: A review and future directions. Behaviour research and therapy, 86:35–49. find
  21. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431, Valencia, Spain. Associatio… find
  22. Clara Khuon, Gabriel Tillman, George Van Doorn, Jacob Dye, Kimberley A. McFarlane, Bridianne O’Dea, and Taylor A. Braund. 2026. Identifying structural linguistic markers of depression in written text: A narrative review of language analysis methods. Journal of Affective Disorders Reports, 23:101022. find
  23. Christopher Lalk, Lauren A. Rutter, Lorenzo Lorenzo-Luaces, and Johan Bollen. 2024. Depression symptoms are associated with frequency of cognitive distortions in psychotherapy transcripts. Cognitive Therapy and Research. find
  24. Sehee Lim, Yejin Kim, Chi-Hyun Choi, Jy-yong Sohn, and Byung-Hoon Kim. 2024. ERD: A framework for improving LLM reasoning for cognitive distortion classification. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 292–300, Mexico City, Mexico. Association for Computationa… find
  25. Kevin Lybarger, Justin Tauscher, Xiruo Ding, Dror Ben-zeev, and Trevor Cohen. 2022. Identifying distorted thinking in patient-therapist text message exchanges by leveraging dynamic multi-turn context. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 1… find
  26. Mai Mostafa, Alia El Bolock, and Slim Abdennadher. 2021. Automatic detection and classification of cognitive distortions in journaling text. In WEBIST, pages 444–452. find
  27. J. A. Nelder and R. W. M. Wedderburn. 1972. Generalized linear models. Journal of the Royal Statistical Society. Series A (General), 135(3):370–384. find
  28. Nawal Ouhmad, Romain Deperrois, Wissam El Hage, and Nicolas Combalbert. 2024. Cognitive distortions, anxiety, and depression in individuals suffering from ptsd. International Journal of Mental Health, 53(4):336–352. find
  29. Robin Quillivic, Yann Auxéméry, Frédérique Gayraud, Jacques Dayan, and Salma Mesmoudi. 2025. Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review. Journal of the American Medical Informatics Association, 32(8):1350–1363. find
  30. Lina Maria Rojas-Barahona, Bo-Hsiang Tseng, Yinpei Dai, Clare Mansfield, Osman Ramadan, Stefan Ultes, Michael Crawford, and Milica Gasic. 2018. Deep learning for language understanding of mental health concepts derived from cognitive behavioural therapy. CoRR, abs/1809.00640. find
  31. Stephanie Rude, Eva-Maria Gortner, and James Pennebaker. 2004. Language use of depressed and depression-vulnerable college students. Cognition & Emotion - COGNITION EMOTION, 18:1121–1133. find
  32. Archie Sage, Jeroen Keppens, and Helen Yannakoudakis. 2025. A survey of cognitive distortion detection and classification in NLP. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14884–14899, Suzhou, China. Association for Computational Linguistics. find
  33. Neha Sharma, Navneet Agarwal, and Kairit Sirts. 2026. Towards consistent detection of cognitive distortions: Llm-based annotation and dataset-agnostic evaluation. In Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), pages 10866–10882, Palma, Mallorca, Spain. Euro…
  34. Benjamin Shickel, Scott Siegel, Martin Heesacker, Sherry Benton, and Parisa Rashidi. 2019. Automatic detection and classification of cognitive distortions in mental health text. CoRR, abs/1909.07502. — [arXiv:1909.07502](https://arxiv.org/abs/1909.07502) find
  35. Sagarika Shreevastava and Peter Foltz. 2021. Detecting cognitive distortions from patient-therapist interactions. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 151–158, Online. Association for Computational Linguistics. find
  36. Taetem Simms, Clayton Ramstedt, Megan Rich, Michael Richards, Tony R. Martinez, and Christophe G. Giraud-Carrier. 2017. Detecting cognitive distortions through machine learning text analytics. 2017 IEEE International Conference on Healthcare Informatics (ICHI), pages 508–512. find
  37. Allison M Tackman, David A Sbarra, Angela L Carey, M Brent Donnellan, Andrea B Horn, Nicholas S Holtzman, To’Meisha S Edwards, James W Pennebaker, and Matthias R Mehl. 2019. Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research… find
  38. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54. find
  39. Joe H. Ward. 1963. Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301):236–244. find
  40. Katja Wiemer-Hastings, Adrian S Janit, Peter M Wiemer-Hastings, Steve Cromer, and Jennifer Kinser. 2004. Automatic classification of dysfunctional thoughts: a feasibility test. Behavior Research Methods, Instruments, & Computers, 36(2):203–212. find
  41. Andrew Yates, Arman Cohan, and Nazli Goharian. 2017. Depression and self-harm risk assessment in online forums. CoRR, abs/1709.01848.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharma-2026-psycholinguistic-profiles-cognitive-distortions-reddit/