📄 Sarcasm Detection using Context Separators in Online Discourse

Summary

This paper tackles automatic sarcasm detection in social media replies, where the intended meaning often depends on earlier turns of the conversation. Working with the two balanced datasets from the FigLang 2020 shared task (Twitter and Reddit threads), the authors fine-tune RoBERTa large and compare three input formats: the reply alone, the reply concatenated with preceding turns, and the same concatenation with a separation token placed between the context and the reply. The average number of context turns differs considerably between the training and test splits, so they keep only the two most recent dialogue turns as context. Adding context helps on both platforms. On Twitter, the separator makes almost no difference. On Reddit, it gives a clear gain, raising F1 by roughly 5.1% relative to plain concatenation. The study shows that a pretrained transformer can benefit from conversational context. It also shows that explicitly marking the boundary between that context and the reply can matter, depending on the platform.

Key points

• Data: The FigLang 2020 shared task provides 5,000 English tweets and 4,400 Reddit posts, each set with equal sarcastic and non-sarcastic counts. Each reply comes with an ordered list of earlier dialogue turns.
• Context mismatch between splits: The average number of contexts per post varies by split.
  • Reddit: 2.491 (train) vs 4.254 (test).
  • Twitter: 3.867 (train) vs 3.164 (test).
  • Because of this mismatch, only the last two turns are used as context.
• Model and setup: RoBERTa large (355M parameters, 50,265-token vocabulary), fine-tuned with these settings:
  • learning rate 1e-5;
  • 3 epochs;
  • 90/10 train-validation split;
  • maximum sequence length 50 for the reply alone and 256 for the context-based inputs.
• Twitter test F1:
  • Response-only: 0.752
  • Context-Response: 0.772
  • Context-Response (Separated): 0.771
• Reddit test F1:
  • Response-only: 0.679
  • Context-Response: 0.681
  • Context-Response (Separated): 0.716, about a 5.13% improvement from the separator.