Elamur Research
☰
home / papers / 2016 / Are Word Embedding-based Features Useful for Sarcasm Detection?

Are Word Embedding-based Features Useful for Sarcasm Detection?

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D16-1104.pdf

Summary

Sarcasm often depends on context incongruity, but earlier detectors mostly found it through sentiment-bearing words. They miss subtle cases that contain no sentiment word, such as the well-known quip likening a woman's need for a man to a fish's need for a bicycle. Joshi et al. test whether semantic similarity or discordance between word vectors can serve as an extra signal for this incongruity.

For every sentence, they compute cosine similarities between all pairs of non-stop words. From these they derive the maximum and minimum scores of the most similar and most dissimilar pairs, both unweighted (S) and divided by the squared distance between the words (WS). These features are added to four published feature sets and tested with four pre-trained embeddings: LSA, GloVe, dependency-based and Word2Vec. The data is a GoodReads quote corpus, classified with SVMperf.

Adding the embedding features generally raises F-score, regardless of which embedding or base feature set is used. Word2Vec and dependency-based vectors help the most. The paper presents itself as the first use of embedding similarity features for sarcasm detection.

Key points

  • Dataset: 3,629 GoodReads quotes. 759 tagged "sarcastic" are positives, and quotes tagged "philosophy" are negatives. Results use five-fold cross-validation with SVMperf (c=20, w=3, F-score-optimizing loss).
  • Base feature sets: Liebrecht et al. (2013), González-Ibánez et al. (2011a, including LIWC words), Buschmeier et al. (2014) and Joshi et al. (2015). Each is tested alone and with +S, +WS and +S+WS.
  • Embeddings and vocabulary sizes: LSA (100,000), GloVe (2,195,904), dependency-based vectors from Levy and Goldberg (2014) (174,015), and Google News Word2Vec (3,000,000).
  • Results with Word2Vec: used alone, the similarity features score below unigrams (F 69.49 for S and 58.26 for WS, versus 72.53 for unigrams). As add-ons, they raise F-score by about 4% for three of the four prior feature sets and slightly lower it for the fourth.
  • Fair comparison on shared vocabulary: on the 60,252 words common to all embeddings, average F-score gains are 1.143 (Word2Vec), 1.048 (dependency), 0.651 (GloVe) and 0.452 (LSA). The error analysis points to word-sense ambiguity, sarcasm that needs conversational context, and metaphors in non-sarcastic text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

40 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 32 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1104 · arXiv 1610.00883

@inproceedings{joshi2016are,
  title = {{Are Word Embedding-based Features Useful for Sarcasm Detection?}},
  author = {Joshi, Aditya and Tripathi, Vaibhav and Patel, Kevin and Bhattacharyya, Pushpak and Carman, Mark},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1104},
  eprint = {1610.00883},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d16-1104}
}

References

24 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Konstantin Buschmeier, Philipp Cimiano, and Roman Klinger. 2014. An impact analysis of features in a classification approach to irony detection in product reviews. In Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 42–49. find
  2. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  3. Debanjan Ghosh, Weiwei Guo, and Smaranda Muresan. 2015. Sarcastic or not: Word embeddings to predict the literal or sarcastic meaning of words. In EMNLP. find
  4. Roberto González-Ibánez, Smaranda Muresan, and Nina Wa-cholder. 2011a. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  5. Roberto González-Ibánez, Smaranda Muresan, and Nina Wa-cholder. 2011b. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  6. Stacey L Ivanko and Penny M Pexman. 2003. Context incongruity and irony processing. Discourse Processes, 35(3):241–279. find
  7. Thorsten Joachims. 2006. Training linear svms in linear time. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 217–226. ACM. find
  8. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 2, pages 757–762. find our card
  9. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2016. Automatic sarcasm detection: A survey. arXiv preprint arXiv:1602.03426. ↗ our card
  10. Anupam Khattri, Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2015. Your sentiment precedes you: Using an authors historical tweets to predict sarcasm. In 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA), page 25. find
  11. Roger J Kreuz and Gina M Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on computational approaches to Figurative Language, pages 1–4. Association for Computational Linguistics. find
  12. Thomas K Landauer and Susan T. Dumais. 1997. A solution to platos problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. PSY-CHOLOGICAL REVIEW, 104(2):211–240. find
  13. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 2: Short Papers, pages 302–308. find
  14. CC Liebrecht, FA Kunneman, and APJ van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets# not. find our card
  15. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. The Journal of Machine Learning Research, 12:2825–2830. find
  16. Rachel Rakov and Andrew Rosenberg. 2013. ” sure, i did the right thing”: a system for sarcasm detection in speech. In INTERSPEECH, pages 842–846. find
  17. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta, May. ELRA. http://is.muni.cz/publication/884893/en. ↗
  18. Antonio Reyes, Paolo Rosso, and Davide Buscaldi. 2012. From humor recognition to irony detection: The figurative language of social media. Data & Knowledge Engineering, 74:1–12. find
  19. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, pages 704–714. find our card
  20. Joseph Tepperman, David R Traum, and Shrikanth Narayanan. 2006. ” yeah right”: sarcasm recognition for spoken dialogue systems. In INTERSPEECH. Citeseer. find
  21. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM. find
  22. Tony Veale and Yanfen Hao. 2010. Detecting ironic intent in creative comparisons. In ECAI, volume 215, pages 765–770. find
  23. Byron C Wallace, Laura Kertz Do Kook Choe, and Eugene Charniak. 2014. Humans require context to infer ironic intent (so computers probably do, too). In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 512–516. find
  24. Byron C Wallace. 2015. Sparse, contextually informed models for irony detection: Exploiting user communities,entities and sentiment. In ACL. find