Elamur Research
☰
home / papers / 2016 / Fracking Sarcasm using Neural Network

Fracking Sarcasm using Neural Network

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W16-0425.pdf

Summary

Ghosh and Veale address automatic sarcasm detection on Twitter. Most earlier systems relied on surface lexical cues, and parsing noisy tweets to get richer meaning tends to be unreliable. The authors built a training corpus of about 39K tweets. Sarcastic examples were gathered through self-labelling hashtags such as #sarcasm, extended with related tags found via an LSA-based method, and the marker hashtags were removed from the text. A 2,000-tweet test set was annotated in-house.

They compared two approaches:

  • a recursive SVM over Stanford constituency parse trees, enriched with sentiment, POS and hashtag information;
  • a layered neural model that feeds a CNN into an LSTM, followed by a fully connected DNN.

The neural architecture clearly beat the parse-based SVM and reached an F-score of roughly .92. It also outperformed earlier systems on two public sarcasm datasets. This suggests that learned semantic representations can replace fragile parsing for figurative language in social media. The authors additionally launched a bot that retweets detected sarcasm, so the data does not disappear when tweets are deleted.

Key points

  • Data: about 39K training tweets (18K sarcastic, 21K non-sarcastic) and a 2,000-tweet test set with 3 human annotations per tweet. Multi-word hashtags were split, and their words were appended to the tweet text.
  • Model comparison: the best recursive SVM (BOW + POS + Sentiment + HT-splitter) scored F ≈ .73. CNN + LSTM + DNN without dropout scored F ≈ .92, and dropout hurt performance.
  • Riloff et al. (2013) dataset: the model achieved F1 ≈ .87–.88, compared with .51 for the original method.
  • Tsur et al. (2010) dataset: the model achieved F1 ≈ .89–.90 versus .83 for SASI, although SASI kept higher precision.
  • Bot and limitations: the bot @onlinesarcasm retweets predictions with softmax probability above 0.75, where disagreement with human labels was negligible. A known weakness is generalising across conceptually similar phrasings, for example "Thank God It's Monday!".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

60 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 51 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w16-0425

@inproceedings{ghosh2016fracking,
  title = {{Fracking Sarcasm using Neural Network}},
  author = {Ghosh, Aniruddha and Veale, Tony},
  booktitle = {Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2016},
  doi = {10.18653/v1/w16-0425},
  url = {https://doi.org/10.18653/v1/w16-0425}
}

References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1183–1193. Association for Computational Linguistics. find our card
  2. Penelope Brown and Stephen C Levinson. 1978. Universals in language usage: Politeness phenomena. In Questions and politeness: Strategies in social interaction. find
  3. William Chan and Ian Lane. 2015. Deep convolutional neural networks for acoustic modeling in low resource languages. In IEEE International Conference on Acoustics, Speech and Signal Processing. find
  4. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  5. Shelly Dews and Ellen Winner. 1995. Muting the meaning a social function of irony. Metaphor and Symbol, 10(1):3–19. find
  6. Shelly Dews and Ellen Winner. 1999. Obligatory processing of literal and nonliteral meanings in verbal irony. Journal of pragmatics, 31(12):1579–1599. find
  7. Cıcero Nogueira Dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 1, pages 626–634. find
  8. Jacob Eisenstein. 2013. What to do about bad language on the internet. In HLT-NAACL, pages 359–369. 2007. Irony in language and thought: A cognitive science reader. Psychology Press. find
  9. Deanna W. Gibbs and Herbert H. Clark. 1992. Coordinating beliefs in conversation. Journal of Memory and Language. find
  10. Roberto González-Ibánez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  11. Emiliano Guevara. 2010. A regression model of adjective-noun compositionality in distributional semantics. In Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics, pages 33–37. Association for Computational Linguistics. find
  12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. ↗
  13. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188. ↗ our card
  14. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  15. Roger J. Kreuz and Gina M. Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on Computational Approaches to Figurative Language. find
  16. Roger J Kreuz and Sam Glucksberg. 1989. How to be sarcastic: The echoic reminder theory of verbal irony. Journal of Experimental Psychology: General. find
  17. CC Liebrecht, FA Kunneman, and APJ van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets #not. find our card
  18. Stephanie Lukin and Marilyn Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. In Proceedings of the Workshop on Language Analysis in Social Media, pages 30–40. find
  19. Prodromos Malakasiotis. 2011. Paraphrase and Textual Entailment Recognition and Generation. Ph.D. thesis, Ph. D. thesis, Department of Informatics, Athens University of Economics and Business, Greece. find
  20. Jeff Mitchell and Mirella Lapata. 2008. Vector-based models of semantic composition. In ACL, pages 236– 244. find
  21. Jeff Mitchell and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive science, 34(8):1388–1429. find
  22. Tetsuji Nakagawa, Kentaro Inui, and Sadao Kurohashi. 2010. Dependency tree-based sentiment classification using crfs with hidden variables. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 786–794. Association for Computational Linguistics. find
  23. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, pages 704–714. find our card
  24. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 151–161. Association for Computational Linguistics. find our card
  25. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM. find
  26. A. Utsumi. 2000. Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics. find
  27. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103. ACM. find