Elamur Research⌕
☰
home / papers / 2019 / Wait Signals Predict Sarcasm in Online Debates

Wait Signals Predict Sarcasm in Online Debates

Paper · 2019 · J. Trevor D'Arcey, Shereen Oraby and Jean E. Fox Tree

Summary

The paper asks whether markers of hesitation, which the authors call "wait signals," help readers recognize sarcasm in written online arguments. In speech, fillers like um and uh warn the listener that a delay is coming. In asynchronous writing no real delay exists, so these markers look out of place, and the authors argue that this mismatch points readers toward non-literal meaning. Study 1 counted um/uh (and British er/erm) across six spoken and written corpora. Studies 2 and 3 asked Amazon Mechanical Turk workers to label debate-forum posts as sarcastic or not. The posts came from material such as the Internet Argument Corpus, and the studies varied whether posts contained fillers at the start or fillers and ellipses in the middle. Opening fillers doubled the chance of a sarcastic label. Fillers and ellipses inside a post raised it roughly three- to fivefold, which exceeded other cues such as obviously and quotation marks. This suggests that cheap, explainable surface signals carry real weight for detecting irony in text.

Key points

  • Study 1 compared filler frequency in six corpora. Fillers were far more frequent in spoken data than in written data.
  • The um : uh ratio depends on corpus type:
    • um dominates in conversational American English.
    • er/uh dominates in British conversation (e.g., the British National Corpus), American TV/radio, and all written corpora.
  • Study 2: debate posts that began with um or uh were about twice as likely to be rated sarcastic by AMT workers.
  • Study 3: posts with fillers or ellipses mid-text were about 3–5 times more likely to be rated sarcastic. This was a stronger effect than for the word obviously or for quotation marks.
  • Raters received an explicit two-part definition of sarcasm plus one example with sarcasm and one without. The authors note that sarcasm is sparse in corpora (around 10%), which makes precision, recall and F1 the relevant metrics for classifiers.

Interesting ideas

  • I1. A cue carries meaning because it clashes with its medium: hesitation is pointless in asynchronous text, so its presence signals that something non-literal is going on.
  • I2. Wait signals are grouped by function (announcing a delay) rather than by surface form. That one group covers fillers, ellipses, and possibly elongations (thiiiis), spaced letters (t h i s), em-dashes and words like hang on.
  • I3. Listeners interpret um through its basic meaning plus the task: they ask why the speaker needs to delay right here. Judgments of honesty or difficulty come from that reasoning, not from the filler leaking an inner state.
  • I4. The experiments vary a single surface element and measure how human judgments shift. This minimal-pair design isolates the contribution of one cue.
  • I5. Cue strengths are compared against each other (fillers versus obviously versus quotation marks) instead of only checking whether each cue is significant.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the irony detector in prag could score a token as "medium-incongruent" by checking whether it is typical of speech but unexpected in edited writing, and record that clash as an explicit reason for an irony reading.
  • I2 (tokens): the en tokenizer should not normalize these away. It should keep them as tagged tokens (e.g., a FILLER or PAUSE feature) so that later stages can see them.
  • I3 (frames): implicature rules could pair a wait signal's core meaning with the current speech act and derive candidate readings (sarcasm, evasion, politeness), each with a traceable chain of reasoning.
  • I4 (learn): evaluation could insert or remove fillers and ellipses in otherwise identical sentences, as controlled perturbations that show whether the irony component reacts only to the intended cue.
  • I5 (learn): self-induced irony rules could be ranked by how much each cue raises the odds of an irony label on silver data, which keeps the strongest signals first and explains their order.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Santa CruzSanta Cruz · United States24 works · 3,112 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Human-Computer Interaction · 61 works

🧭 Topics 1 Sentiment & emotion · 608 works

Cite

Cite as J. Trevor D'Arcey, Shereen Oraby and Jean E. Fox Tree. 2019. Wait Signals Predict Sarcasm in Online Debates. Dialogue & Discourse.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2019.203

@inproceedings{darcey2019wait,
  title = {{Wait Signals Predict Sarcasm in Online Debates}},
  author = {J. Trevor D'Arcey, Shereen Oraby and Jean E. Fox Tree},
  booktitle = {Dialogue \& Discourse},
  year = {2019},
  doi = {10.5087/dad.2019.203},
  url = {https://doi.org/10.5087/dad.2019.203}
}

References

76 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbott, R., Walker, M., Anand, P., Fox Tree, J. E., Bowmani, R., & King, J. (2011, June). How can you say such things?!?: Recognizing disagreement in informal political argument. In Proceedings of the Workshop on Languages in Social Media (pp. 2-11). Association for Computational Linguistics. find
  2. Abercrombie, G., & Hovy, D. (2016). Putting sarcasm detection into context: The effects of class imbalance and manual labelling on supervised machine classification of twitter conversations. In Proceedings of the ACL 2016 Student Research Workshop (pp. 107-113). our card
  3. Akimoto, Y., & Miyazawa, S. (2017). Individual Differences in Irony Use Depend on Context. Journal of Language and Social Psychology, 36(6), 675-693. https://doi.org/10.1177/0261927X17706937 ↗
  4. Attardo, S., Eisterhold, J., Hay, J., & Poggi, I. (2003). Multimodal markers of irony and sarcasm. Humor, 16(2), 243-260. https://doi.org/10.1515/humr.2003.012 ↗
  5. Blackwell, N. & Fox Tree, J. E. (2012). Social factors affect quotative choice. Journal of Pragmatics, 44, 1150-1162. https://doi.org/10.1016/j.pragma.2012.05.001 ↗
  6. Blackwell, N. L., Perlman, M., & Fox Tree, J. E. (2015). Quotation as multimodal construction. Journal of Pragmatics, 81, 1-7. https://doi.org/10.1016/j.pragma.2015.03.004 ↗
  7. Blakemore, D. 2002. Relevance and Linguistic Meaning: The Semantics and Pragmatics of Discourse Markers. Cambridge: Cambridge University Press. find
  8. British National Corpus Consortium. (2007). British National Corpus version 3 (BNC XML edition). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. Retrieved February 13, 2012. find
  9. Bryant, G. & Fox Tree, J. E. (2002). Recognizing verbal irony in spontaneous speech. Metaphor and Symbol, 17(2), 99-117. https://doi.org/10.1207/S15327868MS1702_2 ↗
  10. Bryant, G. A., & Fox Tree, J. E. (2005). Is there an ironic tone of voice? Language and Speech, 48(3), 257-277. https://doi.org/10.1177/00238309050480030101 ↗
  11. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977-990. https://doi.org/10.3758/BRM.41.4.977 ↗
  12. Burfoot, C., & Baldwin, T. (2009, August). Automatic satire detection: Are you having a laugh?. In Proceedings of the ACL-IJCNLP 2009 Conference (pp. 161-164). Association for Computational Linguistics. find
  13. Burgers, C., Van Mulken, M., & Schellens, P. J. (2011). Finding irony: An introduction of the verbal irony procedure (VIP). Metaphor and Symbol, 26(3), 186-205. https://doi.org/10.1080/10926488.2011.583194 ↗
  14. Burgers, C., van Mulken, M., & Schellens, P. J. (2012). Type of evaluation and marking of irony: The role of perceived complexity and comprehension. Journal of Pragmatics, 44(3), 231-242. ↗
  15. Campbell, J. D., & Katz, A. N. (2012). Are there necessary conditions for inducing a sense of sarcastic irony? Discourse Processes, 49(6), 459-480. ↗
  16. Carberry, S. (1989). A pragmatics-based approach to ellipsis resolution. Computational Linguistics, 15(2), 75-96. find
  17. Carvalho, P., Sarmento, L., Silva, M. J., & De Oliveira, E. (2009, November). Clues for detecting irony in user-generated contents: oh...!! it's “so easy” ;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion (pp. 53-56). ACM. find
  18. Caucci, G. M., & Kreuz, R. J. (2012). Social and paralinguistic cues to sarcasm. Humor, 25(1), 1– 22. https://doi.org/10.1515/humor-2012-0001 ↗
  19. Clark, H. H. (1996). Using language. 1996. Cambridge University Press: Cambridge. find
  20. Clark, H. H., & Fox Tree, J. E. (2002). Using uh and um in spontaneous speaking. Cognition, 84(1), 73-111. https://doi.org/10.1016/S0010-0277(02)00017-3 ↗
  21. Clark, H. H., & Gerrig, R. J. (1984). http://dx.doi.org/10.1037/0096-3445.113.1.121 On the pretense theory of irony. ↗
  22. Davidov, D., Tsur, O., & Rappoport, A. (2010, July). Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (pp. 107-116). Association for Computational Linguistics. our card
  23. Davies, M. (2008-). The Corpus of Contemporary American English (COCA): 520 million words, 1990-present. Available online at http://corpus.byu.edu/coca/. ↗
  24. Davies, M. (2009). The 385+ million word Corpus of Contemporary American English (1990– 2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14(2), 159-190. http://dx.doi.org/10.1075/ijcl.14.2.02dav ↗
  25. Derks, D., Bos, A. E., & Von Grumbkow, J. (2008). Emoticons and online message interpretation. Social Science Computer Review, 26(3), 379-388. https://doi.org/10.1177/0894439307311611 ↗
  26. Dress, M. L., Kreuz, R. J., Link, K. E., & Caucci, G. M. (2008). Regional variation in the use of sarcasm. Journal of Language and Social Psychology, 27(1), 71-85. https://doi.org/10.1177/0261927X07309512 ↗
  27. Felbo, B., Mislove, A., Søgaard, A., Rahwan, I., & Lehmann, S. (2017). Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. ArXiv Preprint ArXiv:1708.00524. ↗ our card
  28. Fox Tree, J. E. (2001). Listeners’ uses of um and uh in speech comprehension. Memory and Cognition, 29(2), 320-326. find
  29. Fox Tree, J. E. (2002). Interpreting pauses and ums at turn exchanges. Discourse Processes, 34(1), 37-55. https://doi.org/10.1207/S15326950DP3401_2 ↗
  30. Fox Tree, J. E. (2007). Folk notions of um and uh, you know, and like. Text & Talk, 27(3), 297-314. https://doi.org/10.1515/TEXT.2007.012 ↗
  31. Fox Tree, J. E. (2010) Discourse markers across speakers and settings. Language and Linguistics Compass, 3(1), 1–13. https://doi.org/10.1111/j.1749-818X.2010.00195.x ↗
  32. Fox Tree, J. E. (2015). Discourse markers in writing. Discourse Studies, 17(1), 64-82. ↗
  33. Fox Tree, J. E., & Clark, H. H. (1997). Pronouncing “the” as “thee” to signal problems in speaking. Cognition, 62(2), 151-167. https://doi.org/10.1016/S0010-0277(96)00781-0 ↗
  34. Fox Tree, J. E., Mayer, S. A., & Betts, T. E. (2011). Grounding in instant messaging. Journal of Educational Computing Research, 45(4), 455-475. https://doi.org/10.2190/EC.45.4.e ↗
  35. Fox Tree, J. E., & Schrock, J. C. (1999). Discourse markers in spontaneous speech: Oh what a difference an oh makes. Journal of Memory and Language, 40(2), 280-295. https://doi.org/10.1006/jmla.1998.2613 ↗
  36. Fox Tree, J. E. & Tomlinson, J. M., Jr. (2008). The rise of like in spontaneous quotations. Discourse Processes, 45, 85-102. https://doi.org/10.1080/01638530701739280 ↗
  37. Ghosh, A., & Veale, T. (2016). Fracking sarcasm using neural network. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (pp. 161-169). ↗ our card
  38. Gibbs, R. W. (2000). Irony in talk among friends. Metaphor and symbol, 15(1-2), 5-27. https://doi.org/10.1080/10926488.2000.9678862 ↗
  39. González-Ibánez, R., Muresan, S., & Wacholder, N. (2011, June). Identifying sarcasm in Twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers-Volume 2 (pp. 581-586). Association for Computational Linguistics. our card
  40. Ivanko, S. L., Pexman, P. M., & Olineck, K. M. (2004). How sarcastic are you? Individual differences and verbal irony. Journal of Language and Social Psychology, 23(3), 244-271. https://doi.org/10.1177/0261927X04266809 ↗
  41. Hancock, J. T. (2004). Verbal irony use in face-to-face and computer-mediated conversations. Journal of Language and Social Psychology, 23(4), 447-463. https://doi.org/10.1177/0261927X04269587 ↗
  42. Heritage, J., & Atkinson, J. M. (1984). Structures of social action. Studies in Conversation Analysis. find
  43. Holtgraves, T. (2000). Preference organization and reply comprehension. Discourse Processes, 30(2), 87–106. https://doi.org/10.1207/S15326950DP3002_01 ↗
  44. Johansson, V. (2009). Lexical diversity and lexical density in speech and writing: A developmental perspective. Working Papers in Linguistics, 53, 61-79. find
  45. Jorgensen, J. (1996). The functions of sarcastic irony in speech. Journal of pragmatics, 26(5), 613-634. https://doi.org/10.1016/0378-2166(95)00067-4 ↗
  46. Jucker, A. H. (1993). The discourse marker ‘well’: a relevance theoretical account. Journal of Pragmatics, 19(5), 435–52. https://doi.org/10.1016/0378-2166(93)90004-9 ↗
  47. Kaland, N., Møller-Nielsen, A., Callesen, K., Mortensen, E. L., Gottlieb, D., & Smith, L. (2002). A new ‘advanced’ test of theory of mind: evidence from children and adolescents with Asperger syndrome. Journal of Child Psychology and Psychiatry, 43(4), 517-528. https://doi.org/10.1111/1469-7610.00042 Koehrsen, W. Beyond Accuracy: Precision and Recall - Choosing the right metrics for classification tasks [Web log message]. Retrieved from https://towardsdatascience.com/beyond-accuracy-precision-and-recall-3da06bea9f6c ↗
  48. Kovaz, D., Kreuz, R. J., & Riordan, M. A. (2013). Distinguishing sarcasm from literal language: Evidence from books and blogging. Discourse Processes, 50(8), 598-615. ↗
  49. Kreuz, R. J., Long, D. L., & Church, M. B. (1991). On being ironic: Pragmatic and mnemonic implications. Metaphor and symbol, 6(3), 149-162. https://doi.org/10.1080/0163853X.2013.849525 ↗
  50. Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human communication research, 30(3), 411-433. ↗
  51. LaMarre, H. L., Landreville, K. D., & Beam, M. A. (2009). The irony of satire: Political ideology and the motivation to see what you want to see in The Colbert Report. The International Journal of Press/Politics, 14(2), 212-231. https://doi.org/10.1177/1940161208330904 ↗
  52. Lampert, M. (2013). Say, be like, quote (unquote), and the air-quotes: interactive quotatives and their multimodal implications. English Today, 29(04), 45-56. https://doi.org/10.1017/S026607841300045X ↗
  53. Lee, D. (2006). Humor in spoken academic discourse. NUCB Journal of Language Culture and Communication, 8(1), 49-68. find
  54. Leech, G., & Rayson, P. (2014). Word frequencies in written and spoken English: Based on the British National Corpus. Routledge. ↗
  55. Liebrecht, C. C., Kunneman, F. A., & van Den Bosch, A. P. J. (2013). The perfect solution for detecting sarcasm in tweets# not. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 29-37. http://hdl.handle.net/2066/112949 ↗ our card
  56. Liu, K., Fox Tree, J. E., & Walker, L. (2016). Coordinating communication in the wild: The Artwalk dialogue corpus of pedestrian navigation and mobile referential communication. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (pp. 3159-3166). find
  57. Masten, A. S. (1986). Humor and competence in school-aged children. Child development, 461-473. http://dx.doi.org/10.2307/1130601 ↗
  58. Nunberg, G. (2001). The Way We Talk Now: Commentaries on Language and Culture from NPR's" Fresh Air". Houghton Mifflin Harcourt. find
  59. O’Reilly, K., Peterson, C. C., & Wellman, H. M. (2014). Sarcasm and advanced theory of mind understanding in children and adults with prelingual deafness. Developmental Psychology, 50(7), 1862. https://doi.org/10.1037/a0036654 ↗
  60. Oraby, S., Harrison, V., Reed, L., Hernandez, E., Riloff, E., & Walker, M. (2016, September). Creating and Characterizing a Diverse Corpus of Sarcasm in Dialogue. In 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue (p. 31). our card
  61. Overholser, J. C. (1992). Sense of humor when coping with life stress. Personality and Individual Differences, 13(7), 799-804. https://doi.org/10.1016/0191-8869(92)90053-R ↗
  62. Peng, C., Lakis, M., & Pan, J.W. (2015). Detecting Sarcasm in Text An Obvious Solution to a Trivial Problem. Foundations and trends in information retrieval. find
  63. Peterson, C. C., Wellman, H. M., & Slaughter, V. (2012). The mind behind the message: Advancing theory-of-mind scales for typically developing children, and those with deafness, autism, or Asperger syndrome. Child Development, 83(2), 469-485. https://doi.org/10.1111/j.1467-8624.2011.01728.x ↗
  64. Poria, S., Cambria, E., Hazarika, D., & Vij, P. (2016). A deeper look into sarcastic tweets using deep convolutional neural networks. arXiv preprint arXiv:1610.08815. ↗ our card
  65. Prichard, C., & Rucynski Jr, J. (2019). Second language learners’ ability to detect satirical news and the effect of humor competency training. TESOL Journal, 10(1), e00366. ↗
  66. Reyes Pérez, A.; Rosso, P. (2014). On the difficulty of automatically detecting irony: beyond a simple case of negation. Knowledge and Information Systems. 40(3):595-614. doi:10.1007/s10115-013-0652-8. ↗
  67. Reyes, A., Rosso, P., & Veale, T. (2013). A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, 47(1), 239–268. https://doi.org/10.1007/s10579-012-9196-x ↗
  68. Riloff, E., Qadir, A., Surve, P., De Silva, L., Gilbert, N., & Huang, R. (2013). Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP 2013 - 2013 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (pp. 704-714). Association for Computational Linguistics (ACL). our card
  69. Rockwell, P., & Theriot, E. M. (2001). Culture, gender, and gender mix in encoders of sarcasm: A self-assessment analysis. Communication Research Reports, 18(1), 44-52. https://doi.org/10.1080/08824090109384781 ↗
  70. Simpson, R. C., Briggs, S. L., Ovens, J., & Swales, J. M. (2002). The Michigan Corpus of Academic Spoken English. Ann Arbor, MI: The Regents of the University of Michigan. find
  71. Smith, V. L., & Clark, H.H. (1993). On the course of answering questions. Journal of Memory & Language, 32, 25-38. ↗
  72. Swanson, R., Lukin, S. M., Eisenberg, L., Corcoran, T., & Walker, M. A. (2017). Getting Reliable Annotations for Sarcasm in Online Dialogues. In LREC (pp. 4250-4257). https://arxiv.org/abs/1709.01042 ↗ our card
  73. Walker, M. A., Fox Tree, J. E., Anand, P., Abbott, R., & King, J. (2012). A Corpus for Research on Deliberation and Debate. In LREC (pp. 812-817). our card
  74. Whalen, J. M., Pexman, P. M., & Gill, A. J. (2009). “Should Be Fun—Not!”: Incidence and Marking of Nonliteral Language in E-Mail. Journal of Language and Social Psychology. https://doi.org/10.1177/0261927X09335253 ↗
  75. Williams, J. A., Burns, E. L., & Harmon, E. A. (2009). Insincere utterances and gaze: eye contact during sarcastic statements. Perceptual and Motor Skills, 108(2), 565-572. ↗
  76. Wit. (2019). In OxfordDictionaries.com. Retrieved from https://en.oxforddictionaries.com/definition/wit ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/arcey-2019-wait-signals-predict-sarcasm-online/