Elamur Research
☰
home / papers / 2017 / Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog

Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog

What it is about

Rhetorical questions (questions not meant to get an answer) are collected with an intent-based heuristic: a question in the middle of a turn that the same speaker answers at once, without giving up the turn. This yields 10,270 rhetorical questions from debate forums (Internet Argument Corpus) and Twitter. An SVM on word embeddings separates them from sincere information-seeking questions at 0.76 F1. Rhetorical questions are then classified as sarcastic or other (argumentative structuring in forums; attention-grabbing and calls to action in tweets): adding LIWC category features lifts forum results to 0.76/0.77 F1, an LSTM is sensitive to how much context is given (the sarcasm is often sharpened by the remark after the question), and tweets are easier (higher scores). The paper also recalls linguistic cues from earlier work: after all at the start and strong negative polarity items (lift a finger, budge an inch) mark rhetorical questions, by any chance marks sincere ones.

Ideas

For Mova

languages/en/34-irony-and-figurative-language.md §6 (rhetorical questions), 26-questions.md, 31-speech-acts.md §5. Code: prag/src/frames.rs, prag/src/agree.rs, prag/data/cues.tsv, world/src/qud.rs. Measures: act accuracy on question-form lines in prag eval; QUD stack tests.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W17-5537.pdf

Summary

Rhetorical questions (RQs) look like ordinary questions but are not meant to get an answer. They serve pragmatic goals such as arguing, persuading, emphasizing, or mocking. Despite this, computational work on them has been scarce. The authors extend their earlier debate-forum dataset and add a new Twitter collection, producing 10,270 RQs paired with the statements that follow them. To find RQs, they use a conservative heuristic: a question asked mid-turn that the same speaker answers right away. Forum RQs from IAC 2.0 were labeled as sarcastic or not by Mechanical Turk workers. An SVM with Word2Vec features separates RQs from sincere, information-seeking questions at 0.76 F1. The authors then compare SVM and LSTM models with linguistic (LIWC) features and different amounts of post-level context to tell sarcastic RQs from other uses. Those other uses tend to be argumentative on forums and persuasive (ads, calls-to-action) on Twitter. Tweets prove easier to classify than forum posts. This is the first detailed study of how RQs relate to sarcasm in social media dialog.

Key points

  • Corpus: 2,496 balanced forum RQs from IAC 2.0 plus 7,774 new Twitter RQs, for 10,270 in total. It is released as the Sarcasm RQ corpus.
  • Forum annotation used five Turkers per item with a majority vote. 1,248 of 4,840 posts (25.8%) were judged sarcastic, about twice the roughly 12% sarcasm rate estimated for debate forums.
  • Distinguishing RQs from information-seeking questions reaches 0.76 F1 using an SVM with GoogleNews Word2Vec features.
  • Sarcastic vs. other RQs: forums reach 0.76 F1 (SARCASTIC) and 0.77 F1 (OTHER), beating the earlier n-gram baseline of about 0.70/0.71. Twitter reaches 0.83 F1 for both classes.
  • Adding LIWC features clearly helps, and the LSTM reacts more strongly to context changes than the SVM.
  • Second-person references behave differently by domain: they signal sarcasm in forums but persuasion in non-sarcastic tweets.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-5537 · arXiv 1709.05305

@inproceedings{oraby2017are,
  title = {{Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog}},
  author = {Oraby, Shereen and Harrison, Vrindavan and Misra, Amita and Riloff, Ellen and Walker, Marilyn},
  booktitle = {Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2017},
  doi = {10.18653/v1/w17-5537},
  eprint = {1709.05305},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/w17-5537}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fréderic Godin, Baptist Vandersmissen, Wesley De Neve, and Rik Van de Walle. 2015. Multimedia lab@ acl w-nut ner shared task: named entity recognition for twitter microposts using distributed word representations. ACL-IJCNLP 2015:146–153. find
  2. Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. find
  3. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers. Citeseer, volume 2, pages 581–586. find our card
  4. Chung-hye Han. 1997. Deriving the interpretation of rhetorical questions. In The Proc. of the Sixteenth West Coast Conference on Formal Linguistics, WC-CFL16. find
  5. Robert Abbott, Brian Ecker, Pranav Anand, and Marilyn Walker. 2016. Internet argument corpus 2.0: An sql schema for dialogic social media and the corpora to go with it. In Language Resources and Evaluation Conference, LREC2016. find
  6. Chung-hye Han. 2002. Interpreting interrogatives as rhetorical questions. Lingua 112(3):201–229. find
  7. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Cornelia Ilie. 1994. What else can I tell you?: a and Prateek Vij. 2016. A Deeper Look into Sarcastic pragmatic study of English rhetorical questions Tweets Using Deep Convolutional Neural Networks. as discursive and argumentative acts. Acta In 26th International Conference on Computational Universitatis Stockholmiensis: Stockholm stud-Linguistics (COLING2016) ies in English. Almqvist & Wiksell International. https://books.google.com/books?id=T2wiAQAAIAAJ. Ashequl Qadir, Ellen Riloff, and Marilyn A. Walker. Aditya Joshi, Vinita Sharma, and Pushpak Bhat-2015. Learning to recognize affective polarity in tacharyya. 2015. Harnessing context incongruity for similes. In Conferenece on Empirical Methods in sarcasm detection. In Proceedings of the 53rd An-NLP, EMNLP-2015. nual Meeting of the Association for Computational Ashequl Qadir, Ellen Riloff, and Marilyn A Walker. Linguistics and the 7th International Joint Confer-2016. Automatically inferring implicit properties ence on Natural Language Processing. volume 2, in similes. In Proceedings of NAACL-HLT. pages pages 757–762. 1223–1232. Dan Jurafsky, Liz Shriberg, and Debra Biasca. ↗
  8. Antonio Reyes, Paolo Rosso, and Davide Buscaldi. 1997. Swbd-damsl labeling project coder’s 2012. From humor recognition to irony detection: manual. Technical report, University of Col-The figurative language of social media. Data & orado. Available as http://stripe.colorado.edu/ juraf-Knowledge Engineering . sky/manual.august1.html. ↗
  9. Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets #not. In Proc. of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. WASSA 2013. find our card
  10. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. find our card
  11. Michael McCarthy and Ronald Carter. 2004. ’There’s millions of them’: hyperbole in everyday conversation Journal of Pragmatics 36:149–184. find
  12. Richard M Roberts and Roger J Kreuz. 1994. Why do people use figurative language? Psychological Science 5(3):159–163. find
  13. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. pages 3111–3119. find
  14. Hannah Rohde. 2006. Rhetorical questions as redundant interrogatives. Department of Linguistics, UCSD . find
  15. Jerrold M Sadock. 1971. Queclaratives. In Seventh Regional Meeting of the Chicago Linguistic Society. volume 7, pages 223–232. find
  16. Shereen Oraby, Vrindavan Harrison, Ernesto Hernandez, Lena Reed, Ellen Riloff, and Marilyn Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In Proc. of the SIGDIAL 2015 Conference: The 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. find our card
  17. Deborah Schaffer. 2005. Can rhetorical questions function as retorts? : Is the pope catholic? Journal of Pragmatics 37:433–600. find
  18. Reid Swanson, Stephanie Lukin, Luke Eisenberg, Thomas Chase Corcoran, and Marilyn A Walker. 2014. Getting reliable annotations for sarcasm in online dialogues. In Language Resources and Evaluation Conference, LREC 2014. find
  19. Shereen Oraby, Lena Reed, Ryan Compton, Ellen Riloff, Marilyn Walker, and Steve Whittaker. 2015. And thats a fact: Distinguishing factual and emotional argumentation in online dialogue. find
  20. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm–a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In Proceedings of the fourth international AAAI conference on weblogs and social media. pages 162–169. find
  21. Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincet Michel, Bertrand Thirion, Oliver Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12:2825–2830. find
  22. Tony Veale and Yanfen Hao. 2007. Learning to understand figurative language: from similes to metaphors to irony. In Proceedings of the Cognitive Science Society. volume 29. find
  23. James Pennebaker, Martha Francis, and Rojer Booth. 2001. LIWC: Linguistic Inquiry and Word Count. find
  24. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet Sarcasm Detection Using Deep Neural Network. In 26th International Conference on Computational Linguistics (COLING2016) find our card
  25. Richard Petty, John Cacioppo, and Martin Heesacker. 1981. Effects of rhetorical questions on persuasion: A cognitive response analysis. Journal of personality and social psychology 40(3):432. find