Elamur Research
☰
home / papers / 2014 / Modelling Sarcasm in Twitter, a Novel Approach

Modelling Sarcasm in Twitter, a Novel Approach

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W14-2609.pdf

Summary

The paper addresses automatic sarcasm detection on Twitter, a key challenge for sentiment analysis because sarcastic tweets often use positive wording to convey negative meaning. Earlier systems relied on typical sarcastic words and phrases. The authors instead propose a model that uses no word patterns as features, which keeps the feature space small and avoids dependence on culture-specific expressions. Each tweet is represented by seven groups of lexical and structural features. These capture signals such as rare-versus-common word use, mixing of written and spoken registers, adverb and adjective intensity, sentiment contrast, synonym choice, ambiguity, and surface structure such as punctuation, emoticons and links. A Weka Decision Tree classifies tweets as sarcastic or not. The negative class is drawn from one of five other topical domains at a time. Sarcasm is separated well from news, politics, education and humour tweets. Distinguishing it from irony remains close to chance, which suggests the two phenomena look structurally alike under this model.

Key points

  • Corpus: 60,000 tweets in six topics of 10,000 each: Sarcasm, Education, Humour, Irony, Politics and Newspaper.
    • Five topics were collected via user hashtags, which were then removed from the text. The Newspaper tweets were drawn from three outlets: The Guardian, The Economist and the New York Times.
    • The tweet IDs were published so others can compare results.
  • Feature groups: Frequency, Written-Spoken, Intensity, Structure, Sentiments, Synonyms and Ambiguity. The authors claim four of these are new to this task: Frequency, Written-Spoken, Intensity and Synonyms.
  • Tools and resources:
    • TwitIE (GATE) for tokenisation, POS tagging (90.54% token accuracy) and named entities.
    • American National Corpus frequency data for word frequencies.
    • WordNet APIs for synonym and ambiguity features.
  • Classifier choice: Decision Tree was preferred over Random Forest. Random Forest scored higher in cross-validation, but Decision Tree generalised better across domains.
  • Results: Each experiment was balanced, with 10k sarcastic and 10k non-sarcastic tweets. F1 under 10-fold cross-validation was:
    • Newspaper: 0.97
    • Politics: 0.90
    • Education: 0.88
    • Humour: 0.88
    • Irony: 0.62
    • Scores on the held-out test set were slightly lower, ranging from 0.60 to 0.95.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

42 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 34 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-2609

@inproceedings{barbieri2014modelling,
  title = {{Modelling Sarcasm in Twitter, a Novel Approach}},
  author = {Barbieri, Francesco and Saggion, Horacio and Ronzano, Francesco},
  booktitle = {Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2014},
  doi = {10.3115/v1/w14-2609},
  url = {https://doi.org/10.3115/v1/w14-2609}
}

References

27 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francesco Barbieri and Horacio Saggion. 2014. Modelling Irony in Twitter. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 56–64, Gothenburg, Sweden, April. Association for Computational Linguistics. find
  2. Kalina Bontcheva, Leon Derczynski, Adam Funk, Mark A. Greenwood, Diana Maynard, and Niraj Aswani. 2013. TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text. In Proceedings of the International Conference on Recent Advances in Natural Language Processing. Association for Computational Linguistics. find
  3. Cristina Bosco, Viviana Patti, and Andrea Bolioli. 2013. Developing corpora for sentiment analysis and opinion mining: the case of irony and senti-tut. Intelligent Systems, IEEE. find
  4. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio de Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  5. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  6. Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. In Proceedings of Language Resources and Evaluation Conference, volume 6, pages 417–422. find
  7. Elena Filatova. 2012. Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing. In Proceedings of Language Resources and Evaluation Conference, pages 392–398. find our card
  8. Rachel Giora. 1995. On irony and negation. Discourse processes, 19(2):239–264. find
  9. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying Sarcasm in Twitter: A Closer Look. In ACL (Short Papers), pages 581–586. Citeseer. find our card
  10. H Paul Grice. 1975. Logic and conversation. 1975, pages 41–58. find
  11. Daniel C Howe. 2009. Rita wordnet. Java based API to access Wordnet. find
  12. Nancy Ide and Keith Suderman. 2004. The American National Corpus First Release. In Proceedings of the Language Resources and Evaluation Conference. find
  13. Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets# not. WASSA 2013, page 29. find our card
  14. Joan Lucariello. 1994. Situational irony: A concept of events gone awry. Journal of Experimental Psychology: General, 123(2):129. find
  15. Stephanie Lukin and Marilyn Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. NAACL 2013, page 30. find
  16. George A Miller. 1995. WordNet: a lexical database for English. Communications of the ACM, 38(11):39–41. ↗ our card
  17. Christopher Potts. 2011. Developing adjective scales from user-supplied textual metadata. NSF Workshop on Restructuring Adjectives in WordNet. Arlington,VA. find
  18. Quintilien and Harold Edgeworth Butler. 1953. The Institutio Oratoria of Quintilian. With an English Translation by HE Butler. W. Heinemann. find
  19. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, pages 1–30. find
  20. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. find our card
  21. Brett Spell. 2009. Java API for WordNet Searching (JAWS). find
  22. Kristina Toutanova, Dan Klein, Christopher D Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 173–180. Association for Computational Linguistics. find
  23. Akira Utsumi. 2000. Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics, 32(12):1777–1806. find
  24. Tony Veale and Yanfen Hao. 2010a. An ironic fist in a velvet glove: Creative mis-representation in the construction of ironic similes. Minds and Machines, 20(4):635–650. find
  25. Tony Veale and Yanfen Hao. 2010b. Detecting Ironic Intent in Creative Comparisons. In ECAI, volume 215, pages 765–770. find
  26. Deirdre Wilson and Dan Sperber. 2002. Relevance theory. Handbook of pragmatics. find
  27. Ian H Witten and Eibe Frank. 2005. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. find