Elamur Research⌕
☰
home / papers / 2014 / Modelling Irony in Twitter: Feature Analysis and Evaluation

Modelling Irony in Twitter: Feature Analysis and Evaluation

Summary

Automatic irony detection is difficult because ironic statements convey something other than their literal content. Tweets add more difficulty: they are short, noisy and often ungrammatical. Building on their earlier model (Barbieri and Saggion, 2014), the authors treat irony detection as binary classification. They use the Twitter corpus of Reyes et al. (2013), in which tweets tagged #irony are contrasted with tweets from three non-ironic topics: Education, Humour and Politics.

The model avoids word or n-gram features. Instead it describes each tweet with seven feature groups: frequency, written/spoken style, structure, intensity, sentiment, synonymy and ambiguity. Frequencies come from the American National Corpus, and tweets are processed with TwitIE. A Weka Decision Tree classifies the tweets.

The new experiments examine features as groups, test redundancy through feature selection, and compare feature relevance across topics. The most discriminative cues are rare-word frequency, synonym choice and punctuation. However, feature relevance depends on the topic, and Humour behaves differently from Education and Politics. This argues for keeping a broad feature set if irony is to be detected against varied non-ironic text.

Key points

  • The dataset contains 40,000 tweets split evenly across #irony, #education, #humour and #politics, with the hashtags removed. Each experiment pairs Irony against one negative topic.
  • Preprocessing uses the TwitIE tokeniser and POS tagger (90.54% token accuracy), WordNet APIs and ANC frequency data. TwitIE brought no significant performance gain, since the key features are structural rather than POS-based.
  • The Experiment 1 confusion matrices imply about 72% accuracy for Irony vs Education and about 75% for both Humour and Politics. Education is the hardest topic.
  • Information-gain correlations between topics are 0.80 for Education–Politics, 0.44 for Humour–Politics and 0.24 for Humour–Education.
  • Correlation-based Feature Subset Selection (Best First) produced reduced models whose performance was comparable to the full model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{barbieri2014modelling,
  title = {{Modelling Irony in Twitter: Feature Analysis and Evaluation}},
  author = {Barbieri, Francesco and Saggion, Horacio},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/5j4gfhonpq9n},
  url = {https://doi.org/10.63317/5j4gfhonpq9n}
}

References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Barbieri, Francesco and Saggion, Horacio. (2014). Irony detection in Twitter. In Proceedings of the European Chapter of the Association for Computational Linguistics (Student Workshop). find
  2. Bontcheva, Kalina, Derczynski, Leon, Funk, Adam, Greenwood, Mark A., Maynard, Diana, and Aswani, Niraj. (2013). TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text. In Proceedings of the International Conference on Recent Advances in Natural Language Processing. Association for Computational Linguistics. ↗
  3. Bosco, Cristina, Patti, Viviana, and Bolioli, Andrea. (2013). Developing corpora for sentiment analysis and opinion mining: the case of irony and senti-tut. find
  4. Carvalho, Paula, Sarmento, Luı́s, Silva, Mário J, and de Oliveira, Eugénio. (2009). Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  5. Davidov, Dmitry, Tsur, Oren, and Rappoport, Ari. (2010). Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  6. Esuli, Andrea and Sebastiani, Fabrizio. (2006). Sentiwordnet: A publicly available lexical resource for opinion mining. In Proceedings of Language Resources and Evaluation Conference, volume 6, pages 417–422. find our card
  7. Filatova, Elena. (2012). Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing. In Proceedings of Language Resources and Evaluation Conference, pages 392–398. find our card
  8. Giora, Rachel. (1995). On irony and negation. Discourse processes, 19(2):239–264. ↗
  9. González-Ibáñez, Roberto, Muresan, Smaranda, and Wacholder, Nina. (2011). Identifying Sarcasm in Twitter: A Closer Look. In ACL (Short Papers), pages 581–586. Citeseer. find our card
  10. Grice, H Paul. (1975). Logic and conversation. 1975, pages 41–58. ↗
  11. Hall, Mark A and Smith, Lloyd A. (1998). Practical feature subset selection for machine learning. find
  12. Howe, Daniel C. (2009). Rita wordnet. Java based API to access Wordnet. find
  13. Ide, Nancy and Suderman, Keith. (2004). The American National Corpus First Release. In Proceedings of the Language Resources and Evaluation Conference. find
  14. Liebrecht, Christine, Kunneman, Florian, and van den Bosch, Antal. (2013). The perfect solution for detecting sarcasm in tweets# not. WASSA 2013, page 29. find our card
  15. Lukin, Stephanie and Walker, Marilyn. (2013). Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. NAACL 2013, page 30. find our card
  16. Miller, George A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11):39–41. find our card
  17. Potts, Christopher. (2011). Developing adjective scales from user-supplied textual metadata. NSF Workshop on Restructuring Adjectives in WordNet. Arlington,VA. find
  18. Quintilien and Butler, Harold Edgeworth. (1953). The Institutio Oratoria of Quintilian. With an English Translation by HE Butler. W. Heinemann. find
  19. Reyes, Antonio, Rosso, Paolo, and Veale, Tony. (2013). A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, pages 1–30. ↗
  20. Spell, Brett. (2009). Java API for WordNet Searching (JAWS). find
  21. Toutanova, Kristina, Klein, Dan, Manning, Christopher D, and Singer, Yoram. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 173–180. Association for Computational Linguistics. find
  22. Utsumi, Akira. (2000). Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics, 32(12):1777– 1806. ↗
  23. Veale, Tony and Hao, Yanfen. (2010a). An ironic fist in a velvet glove: Creative mis-representation in the construction of ironic similes. Minds and Machines, 20(4):635–650. ↗
  24. Veale, Tony and Hao, Yanfen. (2010b). Detecting Ironic Intent in Creative Comparisons. In ECAI, volume 215, pages 765–770. ↗
  25. Veale, Tony. (2004). The challenge of creative information retrieval. In Computational Linguistics and Intelligent Text Processing, pages 457–467. Springer. find
  26. Wilson, Deirdre and Sperber, Dan. (2002). Relevance theory. Handbook of pragmatics. find
  27. Witten, Ian H and Frank, Eibe. (2005). Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/barbieri-2014-modelling-irony-twitter-feature-analysis/