Elamur Research⌕
☰
home / papers / 2020 / Context-Aware Sarcasm Detection Using BERT

Context-Aware Sarcasm Detection Using BERT

Summary

The paper examines whether conversational context helps detect sarcasm. It describes the authors' submission to the sarcasm detection shared task held at the second FigLang workshop, a workshop devoted to processing figurative language, at ACL 2020. The task asks whether the final reply in a dialogue thread is sarcastic, using two balanced corpora: a Twitter dataset and a Reddit dataset.

The authors compared three classifiers:

  • a fine-tuned uncased large BERT;
  • a BiLSTM over 300-dimensional fastText vectors;
  • a linear SVM on character n-gram TF-IDF features.

Context was added by joining the reply with zero, one, two, three, or all previous turns. The turns were placed newest-first and separated by a special token.

On Twitter, adding only the immediately preceding turn gave the best BERT score, though the gain over the reply alone was marginal. On Reddit, the reply without any context worked best. Runs using two or three prior turns collapsed, which the authors attribute to excessive padding at longer sequence lengths.

The study suggests that naive concatenation of context brings little benefit. It also shows that a strong pretrained encoder alone already beats the official baselines.

Key points

  • Data: the Twitter set has 5,000 training and 1,800 test instances, and the Reddit set has 4,400 training and 1,800 test instances. Both training sets are split 50/50 between sarcastic and non-sarcastic.
  • Twitter results: BERT reached F1 0.743 with the reply plus the last turn (max sequence length 140), compared with 0.741 for the reply alone. BiLSTM scored 0.672 and SVM 0.676, against a baseline of 0.670.
  • Reddit results: the best BERT F1 was 0.658 using the reply alone, against a baseline of 0.600. Using all turns gave 0.585.
  • Context failures: with two or three prior turns, F1 dropped to about 0.50 and 0.33 on both datasets.
  • Ranking: the submitted runs placed 14/36 (Twitter) and 21/37 (Reddit) on the CodaLab leaderboard. The best runs arrived after the deadline and were removed from the leaderboard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.figlang-1.12

@inproceedings{baruah2020context,
  title = {{Context-Aware Sarcasm Detection Using BERT}},
  author = {Baruah, Arup and Das, Kaushik and Barbhuiya, Ferdous and Dey, Kuntal},
  booktitle = {Proceedings of the Second Workshop on Figurative Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.figlang-1.12},
  url = {https://doi.org/10.18653/v1/2020.figlang-1.12}
}

References

7 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Bamman and Noah A. Smith. 2015. Contextualized sarcasm detection on twitter. In Proceedings of the Ninth International Conference on Web and Social Media, ICWSM 2015, University of Oxford, Oxford, UK, May 26-29, 2015, pages 574–577. AAAI Press. find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. find
  3. Debanjan Ghosh, Alexander R Fabbri, and Smaranda Muresan. 2018. Sarcasm analysis using conversation context. Computational Linguistics, 44(4):755– 792. ↗ our card
  4. Debanjan Ghosh, Alexander Richard Fabbri, and Smaranda Muresan. 2017. The role of conversation context for sarcasm detection in online interactions. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, Saarbrücken, Germany, August 15-17, 2017, pages 186–196. Association for Computational Linguistics. find our card
  5. Sam Glucksberg. 2001. Understanding Figurative Language: From Metaphor to Idioms. Oxford University Press, New York. find
  6. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. CASCADE: contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 1837–1848. Association for Computational Linguistics. find our card
  7. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗

This page in Markdown → https://md.research.elamur.ai/free/papers/baruah-2020-context-aware-sarcasm-detection-bert/