Elamur Research⌕
☰
home / papers / 2021 / Explainable Detection of Sarcasm in Social Media

Explainable Detection of Sarcasm in Social Media

Paper · 2021 · Ramya Akula and Ivan Garibay

Summary

Sarcasm is hard to detect in written text. Online posts lack the voice, face and gesture cues that reveal it in speech, and the literal words often say the opposite of what is meant. Akula and Garibay propose a neural classifier for English sarcasm that also shows which words drove each decision. Contextual token embeddings from the pre-trained "bert-base-uncased" model feed a stack of multi-head self-attention layers (3 layers, 8 heads). A bidirectional GRU then condenses the attended sequence into a single vector, and a sigmoid output layer gives the probability of sarcasm. They test the model on three corpora from different platforms: tweets (Riloff et al., 2013), political-debate dialogues (Sarcasm Corpus V2, Oraby et al., 2016) and Reddit (SARC 2.0, Khodak et al., 2018). The model beats earlier systems on every corpus, by a wide margin on Twitter. Ablations measure how much attention depth and head count contribute. Visualised attention maps show that the model focuses on typical sarcasm markers. The work matters because it pairs strong accuracy with a built-in, inspectable explanation instead of a post-hoc visualisation tool.

Key points

  • Architecture and training:
    • BERT embeddings → 3 self-attention layers × 8 heads → one BiGRU (hidden size 512) → fully-connected layer with sigmoid, trained with BCE loss.
    • Adam optimiser, learning rate 1e-4, batch size 64, dropout 0.2.
    • About 13M parameters, implemented in PyTorch.
  • Data:
    • Twitter: 1,368 train / 588 test.
    • Sarcasm Corpus V2 Dialogues: 3,754 / 938.
    • Reddit SARC 2.0: 154,702 / 64,666, split into Main and Political subsets.
  • Results:
    • Twitter: F1 98.7 and AUC 99.6, against F1 90.0 for A2Text-Net.
    • Dialogues: F1 77.2, against 76.0 for ELMo-BiLSTM FULL.
    • SARC 2.0: accuracy and F1 of 81/81 (Main) and 80/80 (Political) with user personality features, beating CASCADE.
    • Without personality features the model still leads: 70/70 on Main, 71/72 on Political.
  • Ablations (Dialogues, F1):
    • Attention depth: GRU only 75.6, 1 layer 76.1, 3 layers 77.2, 5 layers 77.6, so gains flatten after 3 layers.
    • Heads: 1 head 74.4, 4 heads 76.8, 8 heads 77.2.
  • Attention analysis:
    • Highly attended cue words include "just", "again", "totally", "ever" and "!"; different heads pick up different cues in the early layers.
    • In a 5-layer model, the last two layers spread attention evenly over all words, which suggests those layers are redundant.

Interesting ideas

  • I1. Score each word by its peak attention across layers, average over heads, then normalise over the sentence, giving one readable highlight per token instead of a stack of maps
  • I2. Sarcasm cues are often small adverbs and punctuation ("just", "again", "totally", "!") that the model singles out, while non-sarcastic sentences get uniformly low weights
  • I3. When attention in deeper layers flattens over every token, treat it as evidence that those layers add nothing, and confirm this by checking where the ablation curve saturates
  • I4. Always report a stripped-down baseline (GRU alone, zero attention layers) next to the full model, so the gain from the key component is measured directly
  • I5. Extra speaker information (user personality features) is appended late, just before classification, and results are reported with and without it

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): the irony detector in prag could report a per-word "cue strength" built the same way, so every irony verdict points to the exact words that triggered it.
  • I2 (frames): these markers could seed an explainable list of irony-cue features over tagged tokens, combined with sentiment clashes in the frame instead of a bag-of-words score.
  • I3 (learn): when training gparse or other models, monitor attention entropy per layer as a cheap test for depth that can be cut.
  • I4 (learn): every new module in evaluation should ship with a "component removed" run as a built-in negative control.
  • I5 (world): speaker or author traits kept in the world of the text could be fed as a separate late signal to irony judgements, with results always reported both ways.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.English · 1,373 worksSentiment & emotion · 608 works

Cite

Cite as Ramya Akula and Ivan Garibay. 2021. Explainable Detection of Sarcasm in Social Media. Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{akula2021explainable,
  title = {{Explainable Detection of Sarcasm in Social Media}},
  author = {Ramya Akula and Ivan Garibay},
  booktitle = {Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2021},
  url = {https://aclanthology.org/2021.wassa-1.4/}
}

References

18 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What does bert look at? an analysis of bert’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286. find
  2. Rishabh Misra and Prahal Arora. 2019. Sarcasm detection using hybrid neural network. arXiv preprint arXiv:1908.07414. ↗
  3. Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 31–41.
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of NAACL: Human Language Technologies, pages 4171–4186.
  5. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32. ↗
  6. Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In Proceedings of the 7th workshop on computational approaches to subjectivity, sentiment and social media analysis, pages 161–169. ↗
  7. Aniruddha Ghosh and Tony Veale. 2017. Magnets for sarcasm: making sarcasm detection timely, contextual and very personal. In Proceedings of the 2017 Conference on EMNLP, pages 482–491.
  8. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on EMNLP, pages 704–714.
  9. Debanjan Ghosh, Alexander R Fabbri, and Smaranda Muresan. 2018. Sarcasm analysis using conversation context. Computational Linguistics, pages 755–792. ↗
  10. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. Cascade: Contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1837–1848.
  11. Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626.
  12. Suzana Ilic, Edison Marrese-Taylor, Jorge Balazs, and Yutaka Matsuo. 2018. Deep contextualized word representations for detecting sarcasm and irony. In Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 2–7.
  13. Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. In Proceedings of the 56th Annual Meeting of the ACL, pages 1010–1020.
  14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
  15. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv. ↗
  16. Tao Xiong, Peiran Zhang, Hongbo Zhu, and Yihui Yang. 2019. Sarcasm detection with self-matching networks and low-rank bilinear pooling. In The World Wide Web Conference, pages 2115–2124. find
  17. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, The 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460.
  18. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/akula-2021-explainable-detection-sarcasm-social-media/