Elamur Research⌕
☰
home / papers / 2018 / Modeling Empathy and Distress in Reaction to News Stories

Modeling Empathy and Distress in Reaction to News Stories

Summary

The paper addresses text-based empathy prediction, which earlier work handled with private datasets, labels assigned by outside annotators, and a single, simplified notion of empathy. The authors built a new public corpus by asking MTurk workers to read news articles chosen to provoke empathic reactions. After each article, participants first rated their own feelings using Batson's Empathic Concern – Personal Distress Scale and then wrote a short message about the story, as if addressed to friends. The labels therefore come from the writers themselves. They also separate two psychologically distinct states: other-oriented empathic concern and self-oriented personal distress. Because each text has only one rater, reliability comes from multi-item scales and is checked with split-half reliability, which turns out high. Three regressors built on FastText embeddings were compared, and a CNN performed best. The work matters because it provides the first shared gold standard for this task and models empathy closer to how psychology defines it.

Key points

  • Corpus construction: 418 news articles, 403 MTurk participants (each read 5 articles, paid 4 USD, median time 32 minutes). After manual filtering, 1860 messages remained, totaling 173,686 NLTK tokens (median 84 tokens per message).
  • Ratings use 6 empathy items and 8 distress items, each on a 7-point scale, averaged per construct.
  • Empathy and distress correlate only moderately (Pearson r=.451), which supports treating them as separate targets.
  • Split-half reliability is r=.875 for empathy and r=.924 for distress.
  • Results under 10-fold CV, in Pearson r:
    • Ridge: .385 (empathy) / .410 (distress).
    • FFN: .379 / .401.
    • CNN: .404 / .444, significantly better than the other two.
    • Recurrent models struggled during development because the texts are long.
  • Data and code: github.com/wwbp/empathic_reactions

Interesting ideas

  • I1. Writer-reported labels. Having authors rate their own emotional state right before writing gives a stronger ground truth than third-party judgments. This applies to any task where the label is an internal state, such as stress or mood.
  • I2. Multi-item scales instead of many annotators. Averaging several related questionnaire items can stand in for multiple raters when each text has only one annotator. This makes single-annotator datasets more trustworthy.
  • I3. Split-half reliability as an agreement metric. The scale items are randomly divided into two halves, and the correlation between the half-averages is repeated over many splits. This gives a reliability estimate when standard inter-annotator agreement cannot be computed.
  • I4. Splitting a vague construct into separate targets. Modeling empathy and distress separately exposed different behavior: the CNN's gain over the centroid models was larger for distress. The authors read this as a sign that word order and composition matter more for distress than for empathy. Breaking fuzzy labels into theory-based components could reveal similar differences in other tasks.
  • I5. A fair comparison setup. All models were evaluated on identical CV splits with a held-out dev set and paired t-tests across folds. This is a cheap, rigorous way to compare models on small datasets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

79 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1507

@inproceedings{buechel2018modeling,
  title = {{Modeling Empathy and Distress in Reaction to News Stories}},
  author = {Buechel, Sven and Buffone, Anneke and Slaff, Barry and Ungar, Lyle and Sedoc, João},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1507},
  url = {https://doi.org/10.18653/v1/d18-1507}
}

References

39 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Anneke Buffone, Hao Peng, Johannes C Eichstaedt, and Lyle H Ungar. 2017. Recognizing pathogenic empathy in social media. In ICWSM 2017 — Proceedings of the 11th International Conference on Web and Social Media, pages 448–451, Montreal, Canada, May 15–18, 2017. find
  2. Muhammad Abdul-Mageed and Lyle H. Ungar. 2017. EmoNet: Fine-grained emotion detection with gated recurrent neural networks. In ACL 2017 — Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1, long papers, pages 718–728, Vancouver, British Columbia, Canada… find our card
  3. Firoj Alam, Morena Danieli, and Giuseppe Riccardi. 2017. Annotating and modeling empathy in spoken conversations. Computer Speech & Language, pages 40–61. find
  4. C Daniel Batson, Jim Fultz, and Patricia A Schoenrade. 1987. Distress and empathy: Two qualitatively distinct vicarious emotions with different motivational consequences. Journal of personality, 55(1):19–39. find
  5. C Daniel Batson, Karen Sager, Eric Garst, Misook Kang, Kostia Rubchinsky, and Karen Dawson. 1997. Is empathy-induced helping due to self–other merging? Journal of personality and social psychology, 73(3):495. find
  6. C Daniel Batson and Laura L Shaw. 1991. Evidence for altruism: Toward a pluralism of prosocial motives. Psychological inquiry, 2(2):107–122. find
  7. Daniel Beck. 2017. Modelling representation noise in emotion analysis using gaussian processes. In IJC-NLP 2017 — Proceedings of the 8th International Joint Conference on Natural Language Processing, volume 2, short papers, pages 140–145, Taipei, Taiwan, November 27 – December 1, 2017. find
  8. Steven Bird. 2006. NLTK: The natural language toolkit. In COLING-ACL 2006 — Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, volume 4, interactive presentation sessions, pages 69–72, Sydney, Austra… find our card
  9. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomáš Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5(1):135–146. find
  10. Sven Buechel and Udo Hahn. 2017. EmoBank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis. In EACL 2017 — Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, volume 2, short papers, pages… find
  11. Sven Buechel and Udo Hahn. 2018a. Emotion representation mapping for automatic lexicon construction (mostly) performs on human level. In COL-ING 2018 — Proceedings of the 27th International Conference on Computational Linguistics, volume 1, technical papers, pages 2892–2904, Santa Fe, New Mexico, US… find
  12. Sven Buechel and Udo Hahn. 2018b. Word emotion induction for multiple languages as a deep multi-task learning problem. In NAACL-HLT 2018 — Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1, long pa… find
  13. Lee J Cronbach. 1947. Test reliability: Its meaning and determination. Psychometrika, 12(1):1–16. find
  14. Benjamin MP Cuff, Sarah J Brown, Laura Taylor, and Douglas J Howat. 2016. Empathy: a review of the concept. Emotion Review, 8(2):144–153. find
  15. Mark H Davis. 1980. Interpersonal Reactivity Index. Edwin Mellen Press. find
  16. ED Diener, Robert A Emmons, Randy J Larsen, and Sharon Griffin. 1985. The satisfaction with life scale. Journal of Personality Assessment, 49(1):71– 75. find
  17. Pascale Fung, Anik Dey, Farhad Bin Siddique, Ruixi Lin, Yang Yang, Yan Wan, and Ho Yin Ricky Chan. 2016. Zara the supergirl: An empathetic personality recognition system. In NAACL 2016 — Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics… find
  18. James Gibson, Nikolaos Malandrakis, Francisco Romero, David C Atkins, and Shrikanth S Narayanan. 2015. Predicting therapist empathy in motivational interviews using language features inspired by psycholinguistic norms. In Interspeech 2015 — Proceedings of the 16th Annual Conference of the Internatio… find
  19. Jennifer L Goetz, Dacher Keltner, and Emiliana Simon-Thomas. 2010. Compassion: an evolutionary analysis and empirical review. Psychological Bulletin, 136(3):351. find
  20. Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. 2003. A very brief measure of the big-five personality domains. Journal of Research in Personality, 37(6):504–528. find
  21. Hamed Khanpour, Cornelia Caragea, and Prakhar Biyani. 2017. Identifying empathetic messages in online health communities. In IJCNLP 2017 — Proceedings of the 8th International Joint Conference on Natural Language Processing, volume 2, short papers, pages 246–251, Taipei, Taiwan, November 27 – Decemb… find
  22. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR 2015 — Proceedings of the 3rd International Conference on Learning Representations, pages 1–15, San Diego, California, USA, May 7–9, 2015. find
  23. Bing Liu. 2015. Sentiment Analysis: Mining Opinions, Sentiments and Emotions. Cambridge University Press. find
  24. Scott W McQuiggan and James C Lester. 2007. Modeling and evaluating empathy in embodied companion agents. International Journal of Human-Computer Studies, 65(4):348–360. find
  25. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In LREC 2018 — Proceedings of the 11th International Conference on Language Resources and Evaluation, pages 52–55, Miyazaki, Japan, May 7-12, 2018. find our card
  26. M. Mikulincer and P. R. Shaver, editors. 2010. Prosocial motives, emotions, and behavior: The better angels of our nature. American Psychological Association. find
  27. Saif Mohammad and Felipe Bravo-Marquez. 2017a. WASSA-2017 shared task on emotion intensity. In WASSA 2017 — Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis @ EMNLP, pages 34–49, Copenhagen, Denmark, September 8, 2017. find our card
  28. Saif M. Mohammad and Felipe Bravo-Marquez. 2017b. Emotion intensities in tweets. In *SEM 2017 — Proceedings of the 6th Joint Conference on Lexical and Computational Semantics, pages 65–77, Vancouver, British Columbia, Canada, August 3–4, 2017. find our card
  29. Sylvia A Morelli, Matthew D Lieberman, and Jamil Zaki. 2015. The emerging study of positive empathy. Social and Personality Psychology Compass, 9(2):57–68. find
  30. Verónica Pérez-Rosas, Rada Mihalcea, Kenneth Resnicow, Satinder Singh, and Lawrence An. 2017. Understanding and predicting empathic behavior in counseling therapy. In ACL 2017 — Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1, long papers, pages 14… find
  31. R. W. Picard. 1997. Affective Computing. MIT Press. find
  32. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. SemEval-2017 Task 4: Sentiment analysis in Twitter. In SemEval 2017 — Proceedings of the 11th International Workshop on Semantic Evaluation @ ACL, pages 502–518, Vancouver, British Columbia, Canada, August 3–4, 2017. find our card
  33. H Andrew Schwartz, Salvatore Giorgi, Maarten Sap, Patrick Crutchley, Lyle Ungar, and Johannes Eichstaedt. 2017. DLATK: Differential language analysis toolkit. In EMNLP 2017 — Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, volume 2, system demonstrations, page… find
  34. Elliott Sober and David Sloan Wilson. 1997. Unto others: The evolution of altruism. Harvard University. find
  35. Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP 2013 — Proceedings of the 2013 Conference on Empirical Methods in Natural Language Proc… find
  36. Carlo Strapparava and Rada Mihalcea. 2007. find
  37. SemEval 2007 Task 14: Affective text. In SemEval 2007 — Proceedings of the 4th International Workshop on Semantic Evaluations @ ACL 2007, pages 70–74, Prague, Czech Republic, June 23–24, 2007. find our card
  38. Jin Wang, Liang-Chih Yu, K. Robert Lai, and Xuejie Zhang. 2016. Dimensional sentiment analysis using a regional CNN-LSTM model. In ACL 2016 — Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, volume 2, short papers, pages 225–230, Berlin, Germany, August 7–12,… find
  39. Bo Xiao, Dogan Can, Panayiotis G Georgiou, David Atkins, and Shrikanth S Narayanan. 2012. Analyzing the language of therapist empathy in motivational interview based psychotherapy. In APSIPA 2012 — Proceedings of the 2012 Asia-Pacific Signal and Information Processing Association Annual Summit and C… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/buechel-2018-modeling-empathy-distress-reaction-news/