Elamur Research
☰
home / papers / 2020 / A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support

A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.emnlp-main.425.pdf

Summary

Millions of people turn to text-based peer-support platforms such as TalkLife for mental health help. The volunteers who reply are usually untrained, and existing empathy measures were designed for spoken, face-to-face therapy. This paper adapts empathy measurement to short, asynchronous written exchanges.

Together with clinical psychologists, the authors build EPITOME, a framework that combines established psychotherapy scales into three mechanisms: Emotional Reactions, Interpretations, and Explorations. Each mechanism is rated as absent, weak, or strong. They release a corpus of 10k (post, response) pairs labeled by trained crowdworkers, who also marked the text spans (rationales) that justify each label. A multi-task RoBERTa-based bi-encoder both predicts empathy levels and extracts these rationales. It reaches roughly 80% accuracy and 70% macro-F1, beating standard baselines by about 4 macro-F1 points.

Running the model over 235k TalkLife conversations shows that empathic replies draw better reactions from seekers and more lasting connections. However, supporters do not become more empathic with experience. This motivates automated feedback tools, and a small pilot suggests such feedback can help.

Key points

  • EPITOME framework: three empathy communication mechanisms (Emotional Reactions, Interpretations, Explorations), each scored as no, weak, or strong communication. Responses that only give advice or facts, or that are abusive, count as non-empathic.
  • Dataset: 10k (post, response) pairs annotated with EPITOME labels plus rationale spans, with high inter-rater reliability.
  • Model: a multi-task RoBERTa-based bi-encoder for joint empathy classification and rationale extraction, at about 80% accuracy and 70% macro-F1 (+4 macro-F1 over popular baselines).
  • TalkLife analysis (235k interactions):
    • Empathy levels stay flat over time.
    • Seekers are 79% more likely to follow a supporter after an empathic exchange.
    • Strong explorations receive 47% more replies.
    • Female supporters are 32% more empathic toward female seekers than male supporters are toward male seekers.
  • Feedback pilot: with simple EPITOME-based feedback, three participants raised total empathy in their responses from 0.8 to 3.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

153 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 150 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.425

@inproceedings{sharma2020computational,
  title = {{A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support}},
  author = {Sharma, Ashish and Miner, Adam and Atkins, David and Althoff, Tim},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.425},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.425}
}

References

49 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tim Althoff, Kevin Clark, and Jure Leskovec. 2016. Large-scale analysis of counseling conversations: An application of natural language processing to mental health. TACL, 4:463–476. find
  2. Godfrey T Barrett-Lennard. 1981. The empathy cycle: Refinement of a nuclear concept. Journal of counseling psychology, 28(2):91. find
  3. The lancet psychiatry commission on psychological treatments research in tomorrow’s science. The Lancet Psychiatry, 5(3):237–286. find
  4. Arthur C Bohart, Robert Elliott, Leslie S Greenberg, and Jeanne C Watson. 2002. Empathy. J. C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. find
  5. Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems (TOIS), 38(3):1–32. find
  6. Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc. 2018. Modeling empathy and distress in reaction to news stories. In EMNLP. find
  7. Mark H Davis et al. 1980. A multidimensional approach to individual differences in empathy. Journal of Personality and Social Psychology. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. find
  9. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2020. Eraser: A benchmark to evaluate rationalized nlp models. In ACL. find
  10. Robert Elliott, Arthur C Bohart, Jeanne C Watson, and David Murphy. 2018. Therapist empathy and client outcome: An updated meta-analysis. Psychotherapy, 55(4):399. find
  11. Gunther Eysenbach, John Powell, Marina Englesakis, Carlos Rizo, and Anita Stern. 2004. Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. Bmj, 328(7449):1166. find
  12. James Gibson, Doğan Can, Bo Xiao, Zac E Imel, David C Atkins, Panayiotis Georgiou, and Shrikanth S Narayanan. 2016. A deep learning approach to modeling empathy in addiction counseling. Interspeech. find
  13. Simon B Goldberg, Tony Rousmaniere, Scott D Miller, Jason Whipple, Stevan Lars Nielsen, William T Hoyt, and Bruce E Wampold. 2016. Do psychotherapists improve with time and experience? a longitudinal analysis of outcomes in a clinical setting. Journal of counseling psychology, 63(1):1. find
  14. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In ACL. find
  15. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. find
  16. Emily A Holmes, Ata Ghaderi, Catherine J Harmer, Paul G Ramchandani, Pim Cuijpers, Anthony P Morrison, Jonathan P Roiser, Claudi LH Bockting, Rory C O’Connor, Roz Shafran, et al. 2018. find
  17. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. CoRR abs/1905.01969. External Links: Link Cited by, 2:2– 2. — [arXiv:1905.01969](https://arxiv.org/abs/1905.01969) find
  18. Zac E Imel, Mark Steyvers, and David C Atkins. 2015. Computational psychotherapy research: Scaling up the evaluation of patient–provider interactions. Psychotherapy, 52(1):19. find
  19. Alan E Kazdin and Sarah M Rabbitt. 2013. Novel models for delivering mental health services and reducing the burdens of mental illness. Clinical Psychological Science, 1(2):170–191. find
  20. Hamed Khanpour, Cornelia Caragea, and Prakhar Biyani. 2017. Identifying empathetic messages in online health communities. In IJCNLP), pages 246– 251. find
  21. Fei-Tzin Lee, Derrick Hull, Jacob Levine, Bonnie Ray, and Kathleen McKeown. 2019. Identifying therapist conversational actions across diverse psychotherapeutic approaches. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 12–23. find
  22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  23. Sarah Peregrine Lord, Elisa Sheng, Zac E Imel, John Baer, and David C Atkins. 2015. More than reflections: empathy in motivational interviewing includes language style synchrony between therapist and client. Behavior therapy, 46(3):296–303. find
  24. Tara Matthews, Kathleen O’Leary, Anna Turner, Manya Sleeper, Jill Palzkill Woelfer, Martin Shelton, Cori Manthorne, Elizabeth F Churchill, and Sunny Consolvo. 2017. Stories from survivors: Privacy & security practices when coping with intimate partner abuse. In CHI. find
  25. William R Miller, Theresa B Moyers, Denise Ernst, and Paul Amrhein. 2003. Manual for the motivational interviewing skill code (misc). Unpublished manuscript. Albuquerque: Center on Alcoholism, Substance Abuse and Addictions, University of New Mexico. find
  26. Adam S Miner, Nigam Shah, Kim D Bullock, Bruce A Arnow, Jeremy Bailenson, and Jeff Hancock. 2019. Key considerations for incorporating conversational ai in psychotherapy. Frontiers in psychiatry, 10. find
  27. CB Truax and RR Carkhuff. 1967. Modern applications in psychology. Toward effective counseling and psychotherapy: Training and practice. Hawthorne, NY, US: Aldine Publishing Co. find
  28. Robert R Morris and Rosalind Picard. 2012. Crowdsourcing collective emotional intelligence. arXiv preprint arXiv:1204.3481. — [arXiv:1204.3481](https://arxiv.org/abs/1204.3481) find
  29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. find
  30. Mark Olfson. 2016. Building the mental health workforce capacity needed to treat adults with serious mental illnesses. Health Affairs, 35(6):983–990. find
  31. Sundip Patel, Alexis Pelletier-Bui, Stephanie Smith, Michael B Roberts, Hope Kilgannon, Stephen Trze-ciak, and Brian W Roberts. 2019. Curricula for empathy and compassion training in medical education: A systematic review. PloS one, 14(8). find
  32. Verónica Pérez-Rosas, Rada Mihalcea, Kenneth Resnicow, Satinder Singh, and Lawrence An. 2017. Understanding and predicting empathic behavior in counseling therapy. In ACL. find
  33. Verónica Pérez-Rosas, Xinyi Wu, Kenneth Resnicow, and Rada Mihalcea. 2019. What makes a good counselor? learning to distinguish between high-quality and low-quality counseling conversations. In ACL. find
  34. Yada Pruksachatkun, Sachin R Pendse, and Amit Sharma. 2019. Moments of change: Analyzing peer-based cognitive support in online mental health forums. In CHI. find
  35. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9. find
  36. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In ACL. find
  37. Elliot Robert, Arthur C Bohart, JC Watson, and LS Greenberg. 2011. Empathy. Psychotherapy, 48(1):43–49. find
  38. Zijian Wang and David Jurgens. 2018. It’s going to be okay: Measuring access to support in online communities. In EMNLP. find
  39. Jeanne C Watson. 2007. Facilitating empathy. European Psychotherapy, 7(1):59–65. find
  40. Jeanne C Watson, Rhonda N Goldman, and Margaret S Warner. 2002. Client-centered and experiential psychotherapy in the 21st century: Advances in theory, research, and practice. PCCS Books. find
  41. Marsha White and Steve M Dorman. 2001. Receiving social support online: implications for health education. Health education research, 16(6):693–707. find
  42. Diyi Yang, Zheng Yao, Joseph Seering, and Robert Kraut. 2019. The channel matters: Self-disclosure, reciprocity and social support in online cancer support groups. In CHI. find
  43. Justine Zhang and Cristian Danescu-Niculescu-Mizil. 2020. Balancing objectives in counseling conversations: Advancing forwards or looking backwards. In ACL. find
  44. Justine Zhang, Robert Filbin, Christine Morrison, Jaclyn Weiser, and Cristian Danescu-Niculescu-Mizil. 2019. Finding your voice: The linguistic development of mental health counselors. In ACL. find
  45. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration. find our card
  46. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In ACL. find our card
  47. Robert L Selman. 1980. Growth of interpersonal understanding. Academic Press. find
  48. Eva Sharma and Munmun De Choudhury. 2018. Mental health support and its relationship to linguistic accommodation in online communities. In CHI. find
  49. Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In CIKM. find