Elamur Research⌕
☰
home / papers / 2015 / Sparse, Contextually Informed Models for Irony Detection: Exploiting User Communities, Entities and Sentiment

Sparse, Contextually Informed Models for Irony Detection: Exploiting User Communities, Entities and Sentiment

Paper · 2015 · Byron C. Wallace, Do Kook Choe and Eugene Charniak

Summary

Sarcasm on the web is hard to spot from wording alone, yet earlier detectors mostly relied on cues inside the text, such as word counts and grammatical signals. Wallace, Choe and Charniak argue that the setting of a comment matters: the same praise of a politician may be sincere in one community and mocking in another. They use reddit comments from two polarized subreddit pairs, progressive vs conservative and atheism vs Christianity. The subreddit stands in for the author's community. They cross it with automatically extracted NNPs and inferred sentence sentiment, and they also take NNPs from the surrounding thread. These crossed features are very numerous, but only a few should be informative. To handle this, the authors train a log-loss linear model with a squared ℓ2 penalty on all weights plus separate ℓ1 penalties on the sentence-level and thread-level interaction features. The contextual features consistently raised recall with little or no loss of precision. The mixed penalty gave sparser models whose performance varied less. The authors suggest the approach applies wherever group membership shapes what a text means.

Key points

  • Data: reddit comments from two pairs of opposed communities, the political subreddits (progressive/conservative) and the religion subreddits (atheism/Christianity).
  • Feature types:
    • Sentiment, from the Stanford Sentiment Analysis tool, reduced to positive vs negative/neutral.
    • Subreddit, which is observed directly.
    • NNP, taken from the Stanford Part of Speech tagger.
    • NNP+, which adds NNPs from the thread, including its title.
    • Two-way interactions (NNP × subreddit) and three-way interactions (NNP × subreddit × sentiment).
  • Regularization: bag-of-words features get only the squared ℓ2 term. Interaction features from the sentence (set I) and from the thread (set T) each get their own ℓ1 term, with separate α weights. Replacing ℓ2 with ℓ1 over all weights performed very poorly.
  • Results: context features improved recall at little or no precision cost. The composite penalty reduced variance in performance.
  • Interpretation: weights were estimated on the full training set over 100 SGD runs and reported as means with standard deviations.
    • Political subreddits: 'obamacare' and 'president' signal irony in conservative posts, while 'freedom' and 'kenya' do so in progressive posts.
    • Religion subreddits: 'god' (0.324) signals irony in atheism and 'professor' (0.297) in Christianity.
    • Three-way features: ostensibly positive sentiment toward 'Cruz' signals irony in the conservative subreddit.

Interesting ideas

  • I1. Irony becomes visible only when the speaker's group is crossed with the entity mentioned, since the same sentence flips meaning across communities; this frames irony as a relation between speaker stance and target rather than a lexical property.
  • I2. Seemingly positive polarity aimed at a specific entity is a strong irony cue, because sarcasm typically wraps a negative view in positive wording.
  • I3. Entities from the thread title and surrounding discussion, not just the sentence itself, carry the context needed to interpret a comment.
  • I4. A split penalty keeps dense, reliable base features under ℓ2 while forcing a huge space of combinatorial interaction features under ℓ1, so only a few survive; this yields compact, interpretable models with lower variance.
  • I5. Feature importance was reported as mean and standard deviation over 100 repeated SGD runs, with an explicit warning that rare three-way features should not be over-read.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the irony module in prag could test whether the surface praise of an entity conflicts with the stance of the speaker's group recorded in the world, and cite that conflict as its explanation.
  • I2 (frames): implicature handling could flag a positive evaluative frame whose target the speaker is known to oppose as a candidate for ironic reversal, rather than relying on polarity words alone.
  • I3 (world): when reading a fable or a text passage, entities from the title and earlier discourse could be kept active as the backdrop for interpreting evaluative or ironic statements later on.
  • I4 (learn): when learning weights for self-induced repair rules or crossed feature templates, an ℓ1 penalty on the combinatorial part could prune most candidates and keep the surviving rules inspectable.
  • I5 (learn): evaluations of learned rules or features could be repeated across seeds, and any rule whose weight is unstable or rarely triggered could be marked as tentative.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Brown UniversityProvidence · United States58 works · 25,310 citations🎓The University of Texas at AustinAustin · United States72 works · 55,722 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Sentiment & emotion · 608 works

Cite

Cite as Byron C. Wallace, Do Kook Choe and Eugene Charniak. 2015. Sparse, Contextually Informed Models for Irony Detection: Exploiting User Communities, Entities and Sentiment. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1100

@inproceedings{wallace2015sparse,
  title = {{Sparse, Contextually Informed Models for Irony Detection: Exploiting User Communities, Entities and Sentiment}},
  author = {Byron C. Wallace, Do Kook Choe and Eugene Charniak},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1100},
  url = {https://doi.org/10.3115/v1/p15-1100}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C Burfoot and T Baldwin. 2009. Automatic satire detection: are you having a laugh? In ACL-IJCNLP, pages 161–164. ACL. find
  2. P Carvalho, L Sarmento, MJ Silva, and E de Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  3. HH Clark and RJ Gerrig. 1984. On the pretense theory of irony. Journal of Experimental Psychology, 113:121–126. ↗
  4. D Davidov, O Tsur, and A Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. Conference on Natural Language Learning (CoNLL), page 107. our card
  5. J Eisenstein, A Ahmed, and EP Xing. 2011a. Sparse additive generative models of text. In International Conference on Machine Learning (ICML). find
  6. J Eisenstein, NA Smith, and EP Xing. 2011b. Discovering sociolinguistic associations with structured sparsity. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1365–1374. find
  7. E Filatova. 2012. Irony and sarcasm: Corpus generation and analysis using crowdsourcing. In LREC, volume 12, pages 392–398. our card
  8. R González-Ibáñez, S Muresan, and N Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In ACL, volume 2, pages 581–586. Citeseer. our card
  9. HP Grice. 1975. Logic and conversation. 1975, pages 41–58. ↗
  10. T Joachims. 1998. Text categorization with support vector machines: Learning with many relevant features. Springer. ↗ our card
  11. S Lukin and M Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. NAACL, pages 30–40. our card
  12. F Pedregosa, G Varoquaux, A Gramfort, V Michel, B Thirion, O Grisel, M Blondel, P Prettenhofer, R Weiss, V Dubourg, J Vanderplas, A Passos, D Cournapeau, M Brucher, M Perrot, and E Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  13. A Reyes, P Rosso, and T Veale. 2012. A multidimensional approach for detecting irony in twitter. LREC, pages 1–30. ↗
  14. E Riloff, A Qadir, P Surve, LD Silva, N Gilbert, and R Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, pages 704–714. our card
  15. R Socher, A Perelygin, JY Wu, J Chuang, CD Manning, AY Ng, and C Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1631–1642. Citeseer. find
  16. D Sperber and D Wilson. 1981. Irony and the use-mention distinction. 1981. find
  17. J Tepperman, D Traum, and S Narayanan. 2006. ”Yeah Right”: Sarcasm Recognition for Spoken Dialogue Systems. find
  18. K Toutanova, D Klein, CD Manning, and Y Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 173– 180. Association for Computational Linguistics. our card
  19. O Tsur, D Davidov, and A Rappoport. 2010. ICWSM-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In AAAI Conference on Weblogs and Social Media. find
  20. Y Tsuruoka, J Tsujii, and S Ananiadou. 2009. Stochastic gradient descent training for l1-regularized log-linear models with cumulative penalty. In Proceedings of the Joint Conference of the Annual Meeting of the ACL and the International Joint Conference on Natural Language Processing of the AFNLP, pages 477–485. Association for Computational Linguistics. find
  21. BC Wallace, DK Choe, L Kertz, and E Charniak. 2014. Humans require context to infer ironic intent (so computers probably do, too). Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 512–516. our card
  22. BC Wallace. 2013. Computational irony: A survey and new perspectives. Artificial Intelligence Review, pages 1–17. ↗
  23. D Yogatama and NA Smith. 2014a. Linguistic structured sparsity in text categorization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 786–796. find
  24. D Yogatama and NA Smith. 2014b. Making the most of bag of words: Sentence regularization with alternating direction method of multipliers. In Proceedings of The 31st International Conference on Machine Learning, pages 656–664. find
  25. H Zou and T Hastie. 2005. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2):301–320. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wallace-2015-sparse-contextually-informed-models-irony/