📄 Code-Switching Patterns Can Be an Effective Route to Improve Performance of Downstream NLP Applications: A Case Study of Humour, Sarcasm and Hate Speech Detection

Summary

Hindi-English code-mixed tweets are common on social media. Linguists have suggested that the act of switching languages can itself carry meaning, for example as a vehicle for jokes. Bansal et al. test whether switching patterns help classifiers detect humour, sarcasm and hate speech in such tweets. They use three existing Twitter corpora: Humour (Khandelwal et al., 2018), Sarcasm (Swami et al., 2018) and Hate (Bohra et al., 2018). The authors language-tagged the Hate corpus themselves. Statistical analysis shows that switching is associated positively with humour but negatively with sarcasm and hate. The authors encode this signal as nine simple features, such as switch counts, language fractions and run-length statistics. They add these features first to the original machine-learning baselines and then to a Hierarchical Attention Network (HAN), where the features join the final fully connected layer. Switching features improved macro-F1 for every model on all three tasks. The best variant, HAN plus features, reached relative gains of up to 17.7% over the baselines. The study suggests that cheap, linguistically motivated cues can strengthen classifiers for mixed-language text.

Key points

• Data: All three datasets are Hindi-English code-mixed tweets:

  • Humour: 3,453 tweets, 1,755 positive.
  • Sarcasm: 5,250 tweets, only 504 positive.
  • Hate: 4,575 tweets, 1,661 positive.

• Hate corpus tagging: Two authors added word-level language tags independently and agreed on 98.1% of tokens.

• Correlation with switching: The Pearson correlation between a label and the presence of an English word in Hindi context was +0.04 for humour, −0.17 for sarcasm and −0.04 for hate.

• Macro-F1 results (baseline → HAN + features):

  • Humour: 69.34 → 72.71.
  • Sarcasm: 78.4 → 82.07.
  • Hate: 33.60 → 39.54.

  These correspond to relative gains of 4.9%, 4.7% and 17.7%. Adding the features to the baselines alone gave relative gains of 2.62%, 1.85% and 3.36%.

• Setup:

  • The HAN uses a GRU implementation.
  • GloVe embeddings were trained on 725,173 code-mixed tweets from Patro et al. (2017) plus 13,278 tweets from the three datasets.
  • The sarcasm data was rebalanced by removing easy negative samples, those with very low predicted probability of being sarcastic.