Elamur Research⌕
☰
home / papers / 2020 / Applying Transformers and Aspect-based Sentiment Analysis approaches on Sarcasm Detection

Applying Transformers and Aspect-based Sentiment Analysis approaches on Sarcasm Detection

Summary

This paper addresses sarcasm detection in social media conversations, where an ironic reply can reverse the apparent sentiment of a message and mislead opinion-mining systems. The authors took part in the FigLang 2020 Sarcasm Detection shared task. In this task, a system receives a response together with its preceding dialogue turns and must decide whether the response is sarcastic. They used the balanced Twitter and Reddit corpora provided by the organisers and, for Reddit, added roughly one million comments from the Khodak et al. (2017) corpus. They compared several families of models:

  • a character n-gram NBSVM baseline,
  • a GloVe+fastText recurrent-convolutional network,
  • BERT and XLNet variants,
  • aspect-based sentiment analysis (ABSA) architectures (IAN, LCF-BERT, BERT-AEN), repurposed so that the last context turn acts as the "context" and the response acts as the "target."

LCF-BERT performed best on Twitter (F1 0.730), and plain BERT performed best on Reddit (F1 0.734). The study suggests that pretrained transformers combined with two-segment interactive modelling transfer well to sarcasm detection. It also shows that simple character-level baselines remain competitive on noisy text.

Key points

  • Data: Twitter has 5000 training and 1800 test samples, and Reddit has 4400 training and 1800 test samples. Each sample has an ID, a Context (ordered dialogue turns), a Response and a Label.
  • On Twitter, LCF-BERT scored 0.730, BERT-base-cased 0.726 and NBSVM 0.691. For BERT and NBSVM, using only the response worked better than adding context.
  • On Reddit, BERT scored 0.734, XLNet 0.698 and NBSVM 0.675, while IAN scored only 0.502. The extra ~1M Reddit comments improved NBSVM by about 7%.
  • Preprocessing consisted of hashtag segmentation and removal of @USER and <URL> tokens. It had no notable effect on the results.
  • The team placed 5th of 37 on Reddit and 25th of 36 on Twitter. Future work points to combining contextual and character embeddings and to normalising social-media text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.figlang-1.9

@inproceedings{ataei2020applying,
  title = {{Applying Transformers and Aspect-based Sentiment Analysis approaches on Sarcasm Detection}},
  author = {ataei, Taha Shangipour and Javdan, Soroush and Minaei-Bidgoli, Behrouz},
  booktitle = {Proceedings of the Second Workshop on Figurative Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.figlang-1.9},
  url = {https://doi.org/10.18653/v1/2020.figlang-1.9}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Silvio Amir, Byron C Wallace, Hao Lyu, and Paula Carvalho Mário J Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. arXiv preprint arXiv:1607.00976. ↗ our card
  2. Abhijit Mishra, Diptesh Kanojia, Seema Nagar, Kuntal Dey, and Pushpak Bhattacharyya. 2016. Harnessing cognitive features for sarcasm detection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1095–1104, Berlin, Germany. Association for Computational Linguistics. ↗ our card
  3. David Bamman and Noah A Smith. 2015. Contextualized sarcasm detection on twitter. In Ninth International AAAI Conference on Web and Social Media. find
  4. Shubhadeep Mukherjee and Pradip Kumar Bala. 2017. Sarcasm detection in microblogs using naı̈ve bayes and fuzzy clustering. Technology in Society, 48:19– 27. find
  5. Santosh Kumar Bharti, Korra Sathya Babu, and Sanjay Kumar Jena. 2015. Parsing-based sarcasm sentiment recognition in twitter data. In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 1373– 1380. IEEE. find
  6. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  8. Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314. ↗
  9. Yufeng Diao, Hongfei Lin, Liang Yang, Xiaochao Fan, Yonghe Chu, Kan Xu, and Di Wu. 2020. A multi-dimension question answering network for sarcasm detection. IEEE Access. ↗
  10. Sida Wang and Christopher Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 90–94, Jeju Island, Korea. Association for Computational Linguistics. find
  11. Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In Proceedings of the 7th workshop on computational approaches to subjectivity, sentiment and social media analysis, pages 161–169. ↗ our card
  12. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pages 5754–5764. ↗ our card
  13. Debanjan Ghosh, Alexander R Fabbri, and Smaranda Muresan. 2018. Sarcasm analysis using conversation context. Computational Linguistics, 44(4):755– 792. ↗ our card
  14. Biqing Zeng, Heng Yang, Ruyang Xu, Wu Zhou, and Xuli Han. 2019. Lcf: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences, 9(16):3389. ↗
  15. Aditya Joshi, Vaibhav Tripathi, Pushpak Bhattacharyya, and Mark Carman. 2016. Harnessing sequence labeling for sarcasm detection in dialogue from tv series ‘friends’. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 146–155. find our card
  16. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579. ↗ our card
  17. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893. ↗
  18. Diana G Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In LREC 2014 Proceedings. ELRA. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/ataei-2020-applying-transformers-aspect-based-sentiment/