Elamur Research⌕
☰
home / papers / 2018 / Representing Social Media Users for Sarcasm Detection

Representing Social Media Users for Sarcasm Detection

Summary

Sarcasm is hard to detect from text alone because the same words can be literal or ironic depending on context. This paper looks at one contextual factor, the identity of the author. Using the Self-Annotated Reddit Corpus (SARC), where users mark sarcastic comments with "/s", the authors build a text-only baseline: a bidirectional GRU (BiGRU) over FastText word vectors. They then extend it with two kinds of author information. The first is a Bayesian-style prior made of each author's counts of sarcastic and non-sarcastic training comments. The second is a learned 15-dimensional author embedding, initialized at random and updated during training. Both extensions outperform the baseline in most settings. The count prior works best within single subreddits (r/politics, r/AskReddit). The learned embeddings win on the full, more heterogeneous corpus, where more data supports richer author–text interactions. The results are competitive with CASCADE (Hazarika et al., 2018), even though the authors use no forum embeddings or engineered stylometric/personality features. This suggests that lightweight user representations are an effective, inexpensive way to add context to sarcasm classifiers.

Key points

  • SARC contains 533M Reddit comments. The experiments use five datasets: the full balanced set (257,082 comments) and balanced and unbalanced versions of r/politics and r/AskReddit. In the unbalanced versions, only 3.06% and 0.53% of comments are sarcastic.
  • Reported scores are mean macro-F1 over 10 runs:
    • Full balanced: 74.8 for the baseline, 74.0 for the Bayesian prior, and 75.3 for the 15d embeddings.
    • r/politics balanced: the Bayesian prior reaches 77.6.
    • r/politics unbalanced: the Bayesian prior reaches 64.7, compared with 58.7 for the baseline.
  • Compared with CASCADE, the models score slightly lower on the full balanced set (77.0 for CASCADE) but higher on r/politics balanced (75.0 for CASCADE).
  • Qualitative analysis shows that the embedding model sometimes pushes non-sarcastic comments toward sarcasm. It also tends to overfit when an author has few training examples.
  • Code is released at github.com/kolchinski/reddit-sarc.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1140 · arXiv 1808.08470

@inproceedings{kolchinski2018representing,
  title = {{Representing Social Media Users for Sarcasm Detection}},
  author = {Kolchinski, Y. Alex and Potts, Christopher},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1140},
  eprint = {1808.08470},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1140}
}

References

26 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Silvio Amir, Byron C Wallace, Hao Lyu, and Paula Carvalho Mário J Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. arXiv preprint arXiv:1607.00976. ↗ our card
  2. David Bamman and Noah A Smith. 2015. Contextualized sarcasm detection on twitter. In ICWSM, pages 574–577. find
  3. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724– 1734. Association for Computational Linguistics. ↗
  4. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. CASCADE: Contextual sarcasm detection in online discussion forums. ArXiv:1805.06413. ↗ our card
  5. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodra-halli. 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579. ↗ our card
  6. Roger J Kreuz and Gina M Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on computational approaches to Figurative Language, pages 1–4. Association for Computational Linguistics. find our card
  7. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning, pages 1188–1196. ↗ our card
  8. Herbert H. Clark and Richard J. Gerrig. 1984. On the pretense theory of irony. Journal of Experimental Psychology: General, 113(1):121–126. ↗
  9. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018). find our card
  10. Herbert L. Colston and Sabrina Y. Lee. 2004. Gender differences in verbal irony use. Metaphor and Symbol, 19(4):289–306. ↗
  11. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. arXiv preprint arXiv:1610.08815. ↗ our card
  12. Megan L. Dress, Roger J. Kreuz, Kristen E. Link, and Gina M. Caucci. 2008. Regional variation in the use of sarcasm. Journal of Language and Social Psychology, 27(1):71–85. ↗
  13. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pages 97–106. ACM. find
  14. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. arXiv preprint arXiv:1708.00524. ↗ our card
  15. Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. ArXiv:1707.09861. ↗ our card
  16. Herbert H. Clark. 1996. Using Language. ’Using’ Linguistic Books. Cambridge University Press. find
  17. Aniruddha Ghosh and Tony Veale. 2017. Magnets for sarcasm: Making sarcasm detection timely, contextual and very personal. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 482–491. find our card
  18. Debanjan Ghosh and Smaranda Muresan. 2018. ” with 1 follower i must be awesome: P”. exploring the role of irony markers in irony recognition. arXiv preprint arXiv:1804.05253. ↗
  19. Raymond W. Gibbs. 2000. Irony in talk among friends. Metaphor and Symbol, 15(1-2):5–27. ↗
  20. Roberto González-Ibánez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  21. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
  22. Yi Tay, Luu Anh Tuan, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. arXiv preprint arXiv:1805.02856. ↗ our card
  23. Byron C Wallace, Eugene Charniak, et al. 2015. Sparse, contextually informed models for irony detection: Exploiting user communities, entities and sentiment. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1035–1044. find
  24. Byron C Wallace, Laura Kertz, Eugene Charniak, et al. 2014. Humans require context to infer ironic intent (so computers probably do, too). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 512–516. find
  25. Robert West, Hristo S. Paskov, Jure Leskovec, and Christopher Potts. 2014. Exploiting social network structure for person-to-person sentiment analysis. Transactions of the Association for Computational Linguistics, 2(2):297–310. ↗
  26. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/kolchinski-2018-representing-social-media-users-sarcasm/