Elamur Research
☰
home / papers / 2016 / A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks

A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/C16-1151.pdf

Summary

The paper addresses sarcasm detection in tweets. Sarcasm matters for sentiment analysis because it can reverse the apparent polarity of a message. Earlier work mostly treated it as text categorization with hand-built cues such as n-grams, emoticons and punctuation. The authors instead let a convolutional neural network learn features from a sarcasm corpus. They add features from three separate CNNs, each pre-trained on its own benchmark data, for sentiment, emotion and personality. Their reasoning is that sarcasm often shows up as a sentiment shift, and some users are more prone to sarcasm than others. The baseline CNN features and the pre-trained features are combined and fed to a classifier, with word2vec embeddings as the input representation. Across benchmark Twitter datasets the combined model beats prior approaches (Joshi et al., 2015; Ptácek et al., 2014), especially on the hardest dataset. The authors also run cross-dataset tests, a generalization issue earlier studies rarely examined. The work matters as an early application of deep learning and pre-trained NLP models to sarcasm, and it shows that affective and personality cues help on unseen data.

Key points

  • The authors describe this as the first deep-learning approach to sarcasm detection and the first use of personality-based features for the task.
  • On Dataset 3 the proposed model scores 93.30%. Ptácek et al. reach 62.37%, and Joshi et al. reach a TP rate of only 10.07%.
  • Generalizability is mixed:
    • Training on Dataset 1 and testing on Dataset 3 drops the F1-score to 33.05%, which PCA attributes to Dataset 3 not being linearly separable.
    • Training on Dataset 3 and testing on Dataset 1 gives an F1-score of 76.78%, versus 47.32% (Joshi et al.) and 53.02% (Ptácek et al.).
  • Baseline CNN features outperform the pre-trained features alone, which reach 64.25% F1 in the cross-dataset test. Combining the two adds 4.19% F1 over the baseline.
  • Sentiment and emotion features help most after the baseline features. Spearman tests show significant correlations among sentiment, emotion and personality features, e.g. 0.83 between sentiment and conscientiousness.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

39 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 33 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 1610.08815

@inproceedings{poria2016deeper,
  title = {{A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks}},
  author = {Poria, Soujanya and Cambria, Erik and Hazarika, Devamanyu and Vij, Prateek},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  eprint = {1610.08815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1610.08815}
}

References

31 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saima Aman and Stan Szpakowicz. 2007. Identifying expressions of emotion in text. In International Conference on Text, Speech and Dialogue, pages 196–205. Springer. find
  2. Francesco Barbieri, Horacio Saggion, and Francesco Ronzano. 2014. Modelling sarcasm in Twitter, a novel approach. ACL 2014, pages 50–58. find our card
  3. Cristina Bosco, Viviana Patti, and Andrea Bolioli. 2013. Developing corpora for sentiment analysis and opinion mining: A survey and the Senti-TUT case study. IEEE Intelligent Systems, 28(2):55–63. find
  4. Konstantin Buschmeier, Philipp Cimiano, and Roman Klinger. 2014. An impact analysis of features in a classification approach to irony detection in product reviews. In Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 42–49. find
  5. Erik Cambria and Amir Hussain. 2015. Sentic Computing: A Common-Sense-Based Framework for Concept-Level Sentiment Analysis. Springer, Cham, Switzerland. find
  6. Erik Cambria and Bebo White. 2014. Jumping NLP curves: A review of natural language processing research. IEEE Computational Intelligence Magazine, 9(2):48–57. find
  7. Erik Cambria, Soujanya Poria, Federica Bisio, Rajiv Bajpai, and Iti Chaturvedi. 2015. The CLSA model: A novel framework for concept-level sentiment analysis. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 3–22. Springer. find
  8. Erik Cambria, Soujanya Poria, Rajiv Bajpai, and Björn Schuller. 2016. SenticNet 4: A semantic resource for sentiment analysis based on conceptual primitives. In COLING. find
  9. Erik Cambria. 2016. Affective computing and sentiment analysis. IEEE Intelligent Systems, 31(2):102–107. find
  10. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio De Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In International CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  11. Iti Chaturvedi, Yew-Soon Ong, Ivor Tsang, Roy Welsch, and Erik Cambria. 2016. Learning word dependencies in text by means of a deep recurrent belief network. Knowledge-Based Systems, 108:144–154. find
  12. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in Twitter and Amazon. In Conference on computational natural language learning, pages 107–116. Association for Computational Linguistics. find our card
  13. Delia Farı́as, Viviana Patti, and Paolo Rosso. 2016. Irony detection in Twitter: The role of affective content. ACM Transactions on Internet Technology, 16(3). find
  14. Roberto González-Ibánez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in Twitter: A closer look. In Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  15. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 2, pages 757–762. find our card
  16. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2016. Automatic sarcasm detection: A survey. arXiv preprint arXiv:1602.03426. ↗ our card
  17. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. CoRR, abs/1404.2188. ↗ our card
  18. Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets # not. In Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA), pages 41–47. find our card
  19. Gerald Matthews and Kirby Gilliland. 1999. The personality theories of H.J. Eysenck and J.A. Gray: A comparative review. Personality and Individual differences, 26(4):583–626. find
  20. Diana Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In LREC, pages 4238–4243. find our card
  21. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  22. Luca Oneto, Federica Bisio, Erik Cambria, and Davide Anguita. 2016. Statistical learning theory and ELM for big social data analysis. IEEE Computational Intelligence Magazine, 11(3):45–55. find
  23. Soujanya Poria, Erik Cambria, and Alexander Gelbukh. 2015a. Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In EMNLP, pages 2539–2544. find
  24. Soujanya Poria, Erik Cambria, Alexander Gelbukh, Federica Bisio, and Amir Hussain. 2015b. Sentiment data flow analysis by means of dynamic linguistic patterns. IEEE Computational Intelligence Magazine, 10(4):26–36. find
  25. Soujanya Poria, Erik Cambria, and Alexander Gelbukh. 2016a. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 108:42–49. find
  26. Soujanya Poria, Iti Chaturvedi, Erik Cambria, and Amir Hussain. 2016b. Convolutional MKL based multimodal emotion recognition and sentiment analysis. In ICDM, Barcelona. find
  27. Tomás Ptácek, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on Czech and English Twitter. In COLING, pages 213–223. find our card
  28. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in Twitter. Language resources and evaluation, 47(1):239–268. find
  29. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, volume 13, pages 704–714. find our card
  30. Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. Semeval-2014 task 9: Sentiment analysis in Twitter. In International workshop on semantic evaluation (SemEval 2014), pages 73–80. find
  31. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. ICWSM-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM, pages 162–169. find