Elamur Research⌕
☰
home / papers / 2024 / Was that Sarcasm?: A Literature Survey on Sarcasm Detection

Was that Sarcasm?: A Literature Survey on Sarcasm Detection

Paper · 2024 · Harleen Kaur Bagga, Jasmine Bernard, Sahil Shaheen et al.

Summary

This survey asks why sarcasm is still hard for machines, given that it often states the opposite of what the speaker means and even people misjudge it fairly often. The authors give no new model. They organise earlier work into families. Linguistic and context-based systems look for incongruity between sentiment and situation, sometimes adding facts about the author and audience. Embedding methods enrich word vectors with topics or affect. Multimodal systems combine text with video, audio or images. Graph-based methods use GCNs to model inconsistency across modalities. The survey also profiles the main corpora: MUStARD built from sitcoms, SARC built from self-tagged Reddit comments, and a Twitter image–text set. It points out practical traps, such as models picking up speaker identity instead of sarcasm. Benchmark tables collect reported accuracy and F1 for each approach. The closing section points to fine-tuned LLMs, multilingual data, synthetic examples and metaphor cues as next steps. The value of the work is a compact map of datasets, methods and their pitfalls for anyone starting on irony detection.

Key points

  • The paper cites average human accuracy on sarcasm of about 81.6%. Common metrics in the field are accuracy, precision, recall and F1.
  • SARC is labelled through the Reddit "/s" marker:
    • It has 533M comments, of which 1.34M (0.25%) are sarcastic.
    • Reported false positive and false negative rates are about 1% and 2%.
    • N-grams such as "obviously", "clearly" and "so fun" signal sarcasm.
  • MUStARD draws its clips from four shows (Sarcasmaholics Anonymous, Friends, The Big Bang Theory and The Golden Girls), pairing each with video, audio, a transcript and the surrounding dialogue as context.
    • Using several modalities cut error by 12.9% in F-score.
    • Much of that gain came from speaker identity. In a speaker-independent split the improvement was small.
  • Embedding results:
    • LDA2vec topic-enriched embeddings beat GloVe, fastText and word2vec, in that order.
    • In AWES, binary sentiment embeddings suited tweets, while Ekman six-emotion embeddings suited longer reviews and forum posts.
  • Other resources:
    • The Cai et al. (2019) Twitter image+text set has 24,635 items, 43% of them sarcastic.
    • On FLUTE, a fine-tuned T5 reached 91.6% accuracy on entail/contradict figurative pairs with explanations.
    • WITS covers Hindi-English code-mixed sitcom dialogue.

Interesting ideas

  • I1. Splitting train and test so that no speaker appears in both exposed a model that had learned who is sarcastic instead of what sarcasm looks like; such identity-leak controls fit any classifier evaluation.
  • I2. SARC trusted a self-applied tag only when it ended the comment and the author had used it before, a cheap way to clean noisy weak labels.
  • I3. Treating sarcasm as incongruity, for example a positive evaluative verb applied to a situation that is normally unpleasant, makes detection a structured contrast check rather than a word-count signal.
  • I4. AWES found that coarse sentiment works for short texts and fine-grained emotions help on longer ones, so the depth of affect features should match text length.
  • I5. FLUTE pairs each figurative sentence with an entail/contradict label and a natural-language explanation, which turns irony detection into an explainable inference task.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Add held-out-author or held-out-source splits as a negative control, so irony and speech-act gains are not just memorised voices.
  • I2 (learn): Silver irony labels could likewise require that markers are positionally consistent and come from repeat users before entering training.
  • I3 (frames): The irony module could compare the polarity of the experiencer's stated attitude in a frame against the default valence of the event filling its stimulus role.
  • I4 (world): Store event valence at two grains (polarity and an Ekman-style emotion) and let longer narratives use the finer grain when tracking characters' states.
  • I5 (answer): Readers could justify an "it was sarcastic" answer by citing the literal frame, the contradicting world fact, and the resulting inverted meaning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 972 works

Cite

Cite as Harleen Kaur Bagga, Jasmine Bernard, Sahil Shaheen and Sarthak Arora. 2024. Was that Sarcasm?: A Literature Survey on Sarcasm Detection. arXiv:2412.00425.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2412.00425 · arXiv 2412.00425

@misc{bagga2024was,
  title = {{Was that Sarcasm?: A Literature Survey on Sarcasm Detection}},
  author = {Harleen Kaur Bagga, Jasmine Bernard, Sahil Shaheen and Sarthak Arora},
  year = {2024},
  doi = {10.48550/arxiv.2412.00425},
  eprint = {2412.00425},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.00425}
}

References

26 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Deirdre Wilson. 2006. ”The pragmatics of verbal irony: Echo or pretence?” Lingua 116, 10 (2006), 1722–1743. find
  2. Rachel Giora. 1995. ”On irony and negation”. Discourse processes 19, 2 (1995), 239–264. ↗
  3. Joshi, Aditya, Vinita Sharma, and Pushpak Bhattacharyya. ”Harnessing context incongruity for sarcasm detection.” Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2015.
  4. Khodak, Mikhail, Nikunj Saunshi, and Kiran Vodrahalli. ”A large self-annotated corpus for sarcasm.” arXiv preprint arXiv:1704.05579 (2017). ↗
  5. Joshi, Aditya, Pushpak Bhattacharyya, and Mark J. Carman. ”Automatic sarcasm detection: A survey.” ACM Computing Surveys (CSUR) 50.5 (2017): 1-22.
  6. Bamman, David, and Noah Smith. ”Contextualized sarcasm detection on twitter.” Proceedings of the International AAAI Conference on Web and Social Media. Vol. 9. No. 1. 2015. find
  7. Onan, Aytuğ. ”Topic-enriched word embeddings for sarcasm identification.” Computer Science On-line Conference. Springer, Cham, 2019. find
  8. Agrawal, Ameeta and Aijun An. ”Affective Representations for Sarcasm Detection.”The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (2018): n. pag. find
  9. Castro, Santiago, et al. ”Towards multimodal sarcasm detection (an obviously perfect paper).” arXiv preprint arXiv:1906.01815 (2019). ↗
  10. Pan, Hongliang et al. “Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection.” FINDINGS (2020).
  11. Liang, Bin, et al. ”Multi-modal sarcasm detection with interactive in-modal and cross-modal graphs.” Proceedings of the 29th ACM international conference on multimedia. 2021. find
  12. Liang, Bin, et al. ”Multi-modal sarcasm detection via cross-modal graph convolutional network.” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vol. 1. Association for Computational Linguistics, 2022.
  13. Onan, Aytuğ and Mansur Alp Toçoğlu. “A Term Weighted Neural Language Model and Stacked Bidirectional LSTM Based Framework for Sarcasm Identification.” IEEE Access 9 (2021): 7701-7722. ↗
  14. Tay, Yi et al. “Reasoning with Sarcasm by Reading In-Between.” ACL (2018).
  15. Potamias, Rolandos Alexandros, Georgios Siolas, and Andreas-Georgios Stafylopatis. ”A transformer-based approach to irony and sarcasm detection.” Neural Computing and Applications 32.23 (2020): 17309-17320.
  16. Yue, Tan, et al. ”KnowleNet: Knowledge fusion network for multimodal sarcasm detection.” Information Fusion 100 (2023): 101921. find
  17. Speer, Robyn, Joshua Chin, and Catherine Havasi. ”Conceptnet 5.5: An open multilingual graph of general knowledge.” Proceedings of the AAAI conference on artificial intelligence. Vol. 31. No. 1. 2017. find
  18. Cai, Yitao, Huiyu Cai, and Xiaojun Wan. ”Multi-modal sarcasm detection in twitter with hierarchical fusion model.” Proceedings of the 57th annual meeting of the association for computational linguistics. 2019.
  19. Xu, Kelvin, et al. ”Neural image caption generation with visual attention.” Proc. ICML. Vol. 37. 2015. find
  20. Chakrabarty, Tuhin, et al. ”FLUTE: Figurative language understanding through textual explanations.” arXiv preprint arXiv:2205.12404 (2022). ↗
  21. Kumar, Shivani, et al. ”When did you become so smart, oh wise one?! sarcasm explanation in multi-modal multi-party dialogues.” arXiv preprint arXiv:2203.06419 (2022). ↗
  22. Oraby, Shereen, et al. ”Creating and characterizing a diverse corpus of sarcasm in dialogue.” arXiv preprint arXiv:1709.05404 (2017). ↗
  23. Misra, Rishabh. ”News headlines dataset for sarcasm detection.” arXiv preprint arXiv:2212.06035 (2022). ↗
  24. Ptáek, Tomáš, Ivan Habernal, and Jun Hong. ”Sarcasm detection on czech and english twitter.” COLING 2014, the 25th International Conference on Computational Linguistics. 2014.
  25. Lukin, Stephanie M., et al. ”Argument strength is in the eye of the beholder: Audience effects in persuasion.” arXiv preprint arXiv:1708.09085 (2017). ↗
  26. Riloff, Ellen, et al. ”Sarcasm as contrast between a positive sentiment and negative situation.” Proceedings of the 2013 conference on empirical methods in natural language processing. 2013.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bagga-2024-was-that-sarcasm-literature-survey/