Elamur Research⌕
☰
home / papers / 2014 / Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis.

Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis.

Summary

Sentiment tools for social media usually ignore sarcasm. It is hard to handle, and it can reverse or distort the polarity a tweet expresses. Earlier work mostly stopped at labelling a text as sarcastic or not. This paper asks what to do with sentiment after sarcasm has been detected. The authors look at the scope of sarcasm: which part of a tweet the sarcastic marker actually changes, especially when hashtags are involved. They built a hashtag tokeniser for GATE that splits multi-word hashtags into separate words. It uses dictionary and gazetteer lookups and a Viterbi-like search. They also wrote rules that use the sentiment and sarcasm signals inside hashtags to decide when and where to flip polarity. The tokeniser reaches about 98% Precision. On a sample of general tweets, sarcasm detection reaches about 91% and polarity detection about 80%. The work shows that detecting sarcasm helps but is not enough on its own. Reliable opinion mining also needs scope handling.

Key points

  • The hashtag retokeniser was tested on 2010 hashtags (4538 tokens) and scored 98.12% Precision, 96.41% Recall and 97.25% F1. Most errors come from unknown named entities and ambiguous splits, such as #greatstart being split into "greats" + "tart".
  • Sarcasm indicators came from a hand-built list of hashtags such as #sarcasm. The authors extended it automatically with hashtags that co-occur with known ones, such as #notreally, using the GazetteerListCollector GATE plugin.
  • The scope rules treat several adjacent sarcasm hashtags as one indicator. When a positive or negative hashtag comes before a sarcasm marker, they turn positive or neutral tweets negative.
  • The evaluation set was 400 TrendMiner tweets, double-annotated. Inter-annotator agreement was 87.5% for polarity and 91.07% for sarcasm. The set contained 91 sarcastic sentences, and many of them had no explicit marker.
  • In a check of 257 #irony tweets, only 2 showed verbal irony. Most referred to situational irony or to context outside the tweet. The authors therefore do not separate irony from sarcasm.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

85 papers of our library cite it (of the 140673 whose bibliographies we read); 32 of them have a page here, 53 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{maynard2014who,
  title = {{Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis.}},
  author = {Maynard, Diana and Greenwood, Mark},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/2ios4bdd5xas},
  url = {https://doi.org/10.63317/2ios4bdd5xas}
}

References

11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Giacomo Berardi, Andrea Esuli, Diego Marcheggiani, and Fabrizio Sebastiani. 2011. ISTI@ TREC Microblog Track 2011: Exploring the Use of Hashtag Segmentation and Text Quality Ranking. In TREC. find
  2. Kalina Bontcheva, Leon Derczynski, Adam Funk, Mark A. Greenwood, Diana Maynard, and Niraj Aswani. 2013. TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text. In Proceedings of the International Conference on Recent Advances in Natural Language Processing. Association for Computational Linguistics. ↗
  3. D. Bousfield. 2007. Never a truer word said in jest: A prag-mastylistic analysis of impoliteness as banter in Henry IV, Part I. Contemporary Stylistics, pages 195–208. find
  4. Hamish Cunningham, Diana Maynard, Kalina Bontcheva, and Valentin Tablan. 2002. GATE: an Architecture for Development of Robust HLT Applications. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 7–12 July 2002, ACL ’02, pages 168–175, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  5. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in Twitter and Amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. http://www.arcomem.eu http://www.decarbonet.eu ↗ our card
  6. Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets# not. WASSA 2013, page 29. find our card
  7. Diana Maynard, Kalina Bontcheva, and Dominic Rout. 2012. Challenges in developing opinion mining tools for social media. In Proceedings of @NLP can u tag #usergeneratedcontent?! Workshop at LREC 2012, Turkey. find
  8. D. Preotiuc-Pietro, S. Samangooei, T. Cohn, N. Gibbins, and M. Niranjan. 2012. Trendminer: An architecture for real time analysis of social media text. In Proceedings of the workshop on Real-Time Analysis and Mining of Social Streams. find
  9. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, pages 1–30. ↗
  10. M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede. 2011. Lexicon-based methods for sentiment analysis. Computational Linguistics, 1(September 2010):1–41. ↗ our card
  11. O. Tsur, D. Davidov, and A. Rappoport. 2010. Icwsm–a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, pages 162–169. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maynard-2014-who-cares-about-sarcastic-tweets/