Elamur Research
☰
home / papers / 2016 / Tweet Sarcasm Detection Using Deep Neural Network

Tweet Sarcasm Detection Using Deep Neural Network

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/C16-1231.pdf

Summary

The paper asks whether learned neural representations can replace hand-crafted discrete features for spotting sarcasm in tweets, treated as a binary classification problem. The authors first build a strong discrete baseline. It uses word uni-, bi- and trigram templates over the target tweet, plus tf-idf-ranked salient words taken from the author's earlier tweets, and feeds both into a logistic regression classifier.

They then design a neural counterpart with the same two-part layout. A bi-directional gated recurrent neural network (GRNN) with gated pooling encodes the target tweet from word embeddings. A separate pooling network summarises embeddings of the contextual words drawn from the author's history tweets.

On the tweet corpus of Rajadesingan et al. (2015), tested in both balanced and imbalanced settings, the neural model significantly outperforms the discrete baseline. That baseline is itself competitive with SCUBA++. The neural model makes more confident predictions, stays stable on longer tweets and gains as much from author history as the discrete one. Its practical value is that it needs no feature engineering and no external tools such as POS taggers or sentiment lexicons.

Key points

  • Data: the corpus shared by Rajadesingan et al. (2015), with up to 80 recent history tweets per author collected through the Twitter API. Evaluation covers balanced and imbalanced versions.
  • Baselines compared include the lexicon-based contrast method of Riloff et al. (2013) and SCUBA++. Switching the baseline's regularization from l2 to l1 shifts accuracy by as much as 1%.
  • On the imbalanced data, accuracy is higher but micro F-measure is lower, mainly because sarcasm-class F drops. Neural and contextual features help close this gap.
  • In the analysis by tweet length, discrete-model accuracy degrades on longer tweets while the neural model holds steady. The 75.4% reported by Bamman and Smith (2015) comes from different data and is not directly comparable.
  • Source code is released under GPL at github.com/zhangmeishan/SarcasmDetection.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

40 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 32 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zhang2016tweet,
  title = {{Tweet Sarcasm Detection Using Deep Neural Network}},
  author = {Zhang, Meishan and Zhang, Yue and Fu, Guohong},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1231/}
}

References

43 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho, and Mario J. Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. In CONLL 2016. find our card
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ↗ our card
  3. David Bamman and Noah A Smith. 2015. Contextualized sarcasm detection on twitter. In Ninth International AAAI Conference on Web and Social Media. find
  4. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio De Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. find
  5. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 740–750, Doha, Qatar, October. Association for Computational Linguistics. find our card
  6. Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014a. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259. ↗ our card
  7. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014b. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014EMNLP, pages 1724–1734. find
  8. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  9. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. find our card
  10. Cicero dos Santos and Maira Gatti. 2014. Deep convolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING 2014, pages 69–78. find
  11. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121–2159. find
  12. Elena Filatova. 2012. Irony and sarcasm: Corpus generation and analysis using crowdsourcing. In LREC, pages 392–398. find our card
  13. Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In Proceedings of the 7th WASSA, pages 161–169. find our card
  14. Raymond W Gibbs and Herbert L Colston. 2007. Irony in language and thought: A cognitive science reader. Psychology Press. find
  15. Raymond W Gibbs. 1986. On the psycholinguistics of sarcasm. Journal of Experimental Psychology: General, 115(1):3. find
  16. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, pages 1–12. find
  17. Roberto González-Ibánez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th ACL, pages 581–586. find our card
  18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  19. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd ACL, pages 757–762. find our card
  20. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2016a. Automatic sarcasm detection: A survey. arXiv preprint arXiv:1602.03426. ↗ our card
  21. Aditya Joshi, Vaibhav Tripathi, Kevin Patel, Pushpak Bhattacharyya, and Mark Carman. 2016b. Are word embedding-based features useful for sarcasm detection? In EMNLP. find our card
  22. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd ACL, pages 655–665. Association for Computational Linguistics. find our card
  23. Jihen Karoui, Benamara Farah, Véronique MORICEAU, Nathalie Aussenac-Gilles, and Lamia Hadrich-Belguith. 2015. Towards a contextual pragmatic model to detect irony in tweets. In Proceedings of the 53rd ACL, pages 644–650. find
  24. Roger J Kreuz and Gina M Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on computational approaches to Figurative Language, pages 1–4. find
  25. Roger J Kreuz and Sam Glucksberg. 1989. How to be sarcastic: The echoic reminder theory of verbal irony. Journal of Experimental Psychology: General, 118(4):374. find
  26. Stephanie Lukin and Marilyn Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. In Proceedings of the Workshop on Language Analysis in Social Media, pages 30–40. find
  27. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proceedings of ICML, pages 1310–1318. find
  28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 EMNLP, pages 1532–1543. find
  29. Tomáš Ptáček, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on czech and english twitter. In Proceedings of COLING 2014, pages 213–223, Dublin, Ireland, August. Dublin City University and Association for Computational Linguistics. find our card
  30. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM ’15, pages 97–106. find
  31. Antonio Reyes, Paolo Rosso, and Davide Buscaldi. 2012. From humor recognition to irony detection: The figurative language of social media. Data & Knowledge Engineering, 74:1–12. find
  32. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in twitter. Language resources and evaluation, 47(1):239–268. find
  33. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, pages 704–714. find our card
  34. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the EMNLP. find
  35. Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1014–1023, July. find
  36. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM. find
  37. Akira Utsumi. 2000. Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics, 32(12):1777–1806. find
  38. Duy-Tin Vo and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI, BueNos Aires, Argentina, August. find
  39. Byron C. Wallace, Do Kook Choe, and Eugene Charniak. 2015. Sparse, contextually informed models for irony detection: Exploiting user communities, entities and sentiment. In Proceedings of the 53rd ACL, pages 1035– 1044. find
  40. Kaisheng Yao, Trevor Cohn, Katerina Vylomova, Kevin Duh, and Chris Dyer. 2015. Depth-gated recurrent neural networks. arXiv preprint arXiv:1508.03790. ↗
  41. Meishan Zhang, Yue Zhang, and Duy Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of the EMNLP, pages 612–621. find
  42. Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated neural networks for targeted sentiment analysis. In Thirtieth AAAI Conference on Artificial Intelligence. find
  43. Shusen Zhou, Qingcai Chen, Xiaolong Wang, and Xiaoling Li. 2014. Hybrid deep belief networks for semi-supervised sentiment classification. In Proceedings of COLING 2014, pages 1341–1349. Dublin City University and Association for Computational Linguistics. find