Elamur Research⌕
☰
home / papers / 2021 / “Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space

“Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space

Paper · 2021 · Debanjan Ghosh, Ritvik Shrivastava and Smaranda Muresan

Summary

In online debates, people often signal agreement or disagreement indirectly. Sarcasm can carry the move without any explicit cue word, which makes classifying argumentative relations (agree/disagree/none) difficult. The authors test whether explicitly modeling sarcasm improves this task. They use the IAC orig subset of the Internet Argument Corpus: about 10K prior-turn/current-turn pairs from forums on controversial topics, crowd-annotated for both argumentative relation and sarcasm.

They compare three families of models:

  • a Logistic Regression that combines argument features with sarcasm features (pragmatic cues, sarcasm markers, incongruity signals);
  • a dual LSTM with hierarchical attention;
  • BERT.

The two neural families are run as single-task and multitask variants. One multitask variant learns the task loss weights from uncertainty. Sarcasm modeling helps in every setup. Sarcasm features raise accuracy by roughly 3%, and multitask models outperform their single-task counterparts by a statistically significant margin, with dynamic loss weighting working best. This gives empirical support to argumentation theory's view of sarcasm as a device for disagreement and ad hominem attacks.

Key points

  • Data. IAC orig (Abbott et al., 2011) has about 10K <pt, ct> pairs drawn from debates on subjects like Evolution, Abortion, Gun Control and Gay Marriage, among others. Annotation was done via Mechanical Turk, with a median Cohen's κ of 0.5.
  • Labels. Relations were originally rated on a scale of −5 to 5 and binned as follows: [−5, −2] → disagree, [−1, 1] → none, [2, 5] → agree.
  • Sarcasm rates. 33% of agree turns, 57% of disagree turns and 44% of none turns are sarcastic.
  • Splits. The data is split 80/10/10 into train, test and dev (7,982 / 999 / 999 pairs), stratified by both relation and sarcasm label.
  • Models. LR ArgF+SarcF, a dual LSTM with hierarchical attention (LSTM MT uncert), BERT Orig, BERT ALT and BERT MT uncert, which uses the Kendall et al. (2018) uncertainty weighting.
  • Error analysis.
    • Concessions and hedged, uncommitted beliefs are often mistaken for none.
    • Locally sarcastic but non-argumentative turns get labeled disagree.
    • Code is released on GitHub (multitask transformers).

Interesting ideas

  • I1. Turning graded crowd judgments into three classes, with a wide neutral band around zero, reduces noise from annotators who only lean weakly one way.
  • I2. Treating sarcasm as an auxiliary signal for detecting the argumentative move shows that irony and stance should be modeled together rather than in isolation.
  • I3. Cheap surface sarcasm markers, such as runs of punctuation like "???", tag questions and all-caps negations, received the highest weights in the interpretable model.
  • I4. Many errors come from concessions ("it's a consideration, but…") and hedged commitment ("it is possible that"), where the stance is present but softened.
  • I5. Sarcasm through echoing, where the reply repeats the prior turn's word and the repeated word resolves an earlier "it", links coreference to the humorous attack.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when witness parsers or LLM teacher tiers give graded confidence, map scores into agree/abstain/disagree bands before building silver data, rather than thresholding at a single point.
  • I2 (frames): the prag crate could compute an irony flag first and then let it flip or reinforce the speech-act reading of a reply, for example a sarcastic question read as disagreement.
  • I3 (tokens): the tokenizer/tagger could emit explicit attributes for punctuation runs, casing emphasis and tag questions, so downstream irony rules have transparent evidence.
  • I4 (frames): add implicature rules that treat a "but"-clause after a concession as the real stance, and record hedges as reduced commitment rather than neutrality.
  • I5 (frames): cross-turn coreference could flag a lexical echo of an antecedent as a possible mocking reformulation and pass that cue to the irony detector.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States149 works · 48,852 citations🎓Massachusetts Institute of TechnologyCambridge · United States159 works · 65,157 citations🤝Educational Testing ServicePrinceton · United States14 works · 1,879 citations🏢Cisco SystemsSan Jose · United States2 works · 246 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPhilosophy · 211 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 608 works

Cite

Cite as Debanjan Ghosh, Ritvik Shrivastava and Smaranda Muresan. 2021. “Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.eacl-main.171 · arXiv 2101.10952

@inproceedings{ghosh2021laughing,
  title = {{“Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space}},
  author = {Debanjan Ghosh, Ritvik Shrivastava and Smaranda Muresan},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-main.171},
  eprint = {2101.10952},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.eacl-main.171}
}

References

59 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rob Abbott, Brian Ecker, Pranav Anand, and Marilyn Walker. 2016. Internet argument corpus 2.0: An sql schema for dialogic social media and the corpora to go with it. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4445–4452. find
  2. Rob Abbott, Marilyn Walker, Pranav Anand, Jean E Fox Tree, Robeson Bowmani, and Joseph King. 2011. How can you say such things?!?: Recognizing disagreement in informal political argument. In Proceedings of the Workshop on Languages in Social Media, pages 2–11. Association for Computational Linguistics. find
  3. Salvatore Attardo. 2000. Irony markers and functions: Towards a goal-oriented theory of irony and its processing. Rask, 12(1):3–20. find
  4. Joshua M. Averbeck. 2013. Comparisons of ironic and sarcastic arguments in terms of appropriateness and effectiveness in personal relationships. Argumentation and Advocacy, 50(1):47–57. ↗
  5. Elena Cabrio and Serena Villata. 2012. Combining textual entailment and argumentation theory for supporting online debates interactions. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 208–212. find
  6. Rich Caruana. 1997. Multitask learning. Machine learning, 28(1):41–75. find
  7. Tuhin Chakrabarty, Christopher Hidey, Smaranda Muresan, Kathy McKeown, and Alyssa Hwang. 2019. AMPERSAND: Argument mining for PER-SuAsive oNline discussions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2933–2943, Hong Kong, China. Association for Computational Linguistics. find
  8. Dushyant Singh Chauhan, SR Dhanush, Asif Ekbal, and Pushpak Bhattacharyya. 2020. Sentiment and emotion help sarcasm? a multi-task learning framework for multi-modal sarcasm, sentiment and emotion analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4351–4360.
  9. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286, Florence, Italy. Association for Computational Linguistics. ↗
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  11. Frans H. van Eemeren and Rob Grootendorst. 1992. Argumentation, communication, and fallacies: a pragma-dialectical perspective. Lawrence Erlbaum Associates, Inc. find
  12. Frans Hendrik van Eemeren, Rob Grootendorst, Sally Jackson, Scott Jacobs, et al. 1993. Reconstructing argumentative discourse. University of Alabama Press. find
  13. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. Allennlp: A deep semantic natural language processing platform.
  14. Aniruddha Ghosh and Tony Veale. 2017. Magnets for sarcasm: Making sarcasm detection timely, contextual and very personal. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 482–491. ↗
  15. Debanjan Ghosh, Alexander R Fabbri, and Smaranda Muresan. 2018. Sarcasm analysis using conversation context. Computational Linguistics, 44(4):755– 792. ↗
  16. Debanjan Ghosh, Alexander Richard Fabbri, and Smaranda Muresan. 2017. The role of conversation context for sarcasm detection in online interactions. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 186– 196, Saarbrücken, Germany. Association for Computational Linguistics.
  17. Debanjan Ghosh, Aquila Khanam, Yubo Han, and Smaranda Muresan. 2016. Coarse-grained argumentation features for scoring persuasive essays. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 549–554, Berlin, Germany. Association for Computational Linguistics. find
  18. Debanjan Ghosh and Smaranda Muresan. 2018. “with 1 follower i must be awesome :p”. exploring the role of irony markers in irony recognition. Proceedings of ICWSM.
  19. Debanjan Ghosh, Smaranda Muresan, Nina Wacholder, Mark Aakhus, and Matthew Mitsui. 2014. Analyzing argumentative discourse units in online interactions. In Proceedings of the First Workshop on Argumentation Mining, pages 39–48. ↗
  20. Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. 2020. A report on the 2020 sarcasm detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, pages 1– 11, Online. Association for Computational Linguistics. ↗
  21. Raymond W Gibbs and Christin Izett. 2005. Irony as persuasive communication. Figurative language comprehension: Social and cultural influences, pages 131–151. ↗
  22. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: A closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 581–586, Portland, Oregon, USA. Association for Computational Linguistics.
  23. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. Association for Computational Linguistics. ↗
  24. Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name-calling: Dynamics and triggers of ad hominem fallacies in web argumentation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 386–396, New Orleans, Louisiana. Association for Computational Linguistics.
  25. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  26. Yufang Hou and Charles Jochim. 2017. Argument relation classification using a joint inference model. In Proceedings of the 4th Workshop on Argument Mining, pages 60–66, Copenhagen, Denmark. Association for Computational Linguistics. find
  27. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177. ACM. find
  28. S Jackson. 1992. Virtual standpoints’ and the pragmatics of conversational argument. Argumentation illuminated, page 260–269. find
  29. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757–762, Beijing, China. Association for Computational Linguistics. ↗
  30. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651. ↗
  31. Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491. find
  32. Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4487–4496, Florence, Italy. Association for Computational Linguistics.
  33. Navonil Majumder, Soujanya Poria, Haiyun Peng, Niyati Chhaya, Erik Cambria, and Alexander Gelbukh. 2019. Sentiment and sarcasm classification with multitask learning. IEEE Intelligent Systems, 34(3):38–43. ↗
  34. Stefano Menini and Sara Tonelli. 2016. Agreement and disagreement: Comparison of points of view in the political domain. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2461–2470. find
  35. Amita Misra and Marilyn A Walker. 2013. Topic independent identification of agreement and disagreement in social media dialogue. In Proceedings of the SIGDIAL 2013 Conference, pages 41–50. Association for Computational Linguistics. find
  36. Smaranda Muresan, Roberto Gonzalez-Ibanez, Debanjan Ghosh, and Nina Wacholder. 2016. Identification of nonliteral language in social media: A case study on sarcasm. Journal of the Association for Information Science and Technology. ↗
  37. Huy Nguyen and Diane Litman. 2016. Context-aware argumentative relation mining. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1127–1137. find
  38. Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, page 31.
  39. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. ↗
  40. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: Liwc 2001. Mahway: Lawrence Erlbaum Associates, 71:2001. find
  41. Isaac Persing and Vincent Ng. 2016. Modeling stance in student essays. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2174– 2184, Berlin, Germany. Association for Computational Linguistics. find
  42. Peter Potash, Alexey Romanov, and Anna Rumshisky. 2017. Here’s my point: Joint pointer architecture for argument mining. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1364–1373, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  43. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 704–714. Association for Computational Linguistics.
  44. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2015. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664. ↗
  45. Sara Rosenthal and Kathleen McKeown. 2015. I couldn’t agree more: The role of conversational structure in agreement and disagreement detection in online discussions. In 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, page 168. find
  46. Swapna Somasundaran and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 116–124. Association for Computational Linguistics.
  47. Christian Stab and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive essays. In EMNLP, pages 46–56.
  48. Christian Stab and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619–659. the Association for Computational Linguistics: Human Language Technologies, pages 592–596. Association for Computational Linguistics. ↗
  49. Marilyn A Walker, Jean E Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012b. A corpus for research on deliberation and debate. In LREC, pages 812–817.
  50. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing, pages 347–354. Association for Computational Linguistics.
  51. Diyi Yang, Alon Lavie, Chris Dyer, and Eduard Hovy. 2015. Humor recognition and humor anchor extraction. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2367–2376.
  52. Jie Yang and Yue Zhang. 2018. Ncrf++: An open-source neural sequence labeling toolkit. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.
  53. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of NAACL-HLT, pages 1480–1489.
  54. Manfred Stede and Jodi Schneider. 2018. Argumentation mining. Synthesis Lectures on Human Language Technologies, 11(2):1–191. find
  55. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of WWW. ↗
  56. Christopher W Tindale and James Gough. 1987. The use of irony in argumentation. Philosophy & rhetoric, pages 1–17. find
  57. Jesse Vig. 2019. A multiscale visualization of attention in the transformer model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 37–42, Florence, Italy. Association for Computational Linguistics.
  58. Nina Wacholder, Smaranda Muresan, Debanjan Ghosh, and Mark Aakhus. 2014. Annotating multiparty discourse: Challenges for agreement metrics. LAW VIII, page 120. find
  59. Marilyn A Walker, Pranav Anand, Robert Abbott, and Ricky Grant. 2012a. Stance classification using dialogic properties of persuasion. In Proceedings of the 2012 Conference of the North American Chapter of

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ghosh-2021-laughing-you-or-you-role/