Elamur Research⌕
☰
home / papers / 2020 / Affective and Contextual Embedding for Sarcasm Detection

Affective and Contextual Embedding for Sarcasm Detection

Summary

Sarcasm is hard to detect in writing because tone of voice and facial cues are missing. Earlier detectors relied either on hand-crafted affect features or on static embeddings like Word2vec, which tend to place opposites such as "good" and "bad" near each other. The authors propose two BERT-based classifiers, ACE 1 and ACE 2, that decide whether a document is sarcastic. Both combine two parts:

  • an affective feature embedding (AFE), built from a Bi-LSTM with multi-head attention over sentence-level affect representations;
  • a contextual feature embedding (CFE) taken from a transformer.

ACE 1 changes BERT's architecture directly. It adds affective embeddings to the input sequence and trains the model from scratch on a corpus rich in sarcastic and emotional language (WikiSarc). ACE 2 is simpler: it merges affective features with SBERT-based contextual vectors in a fully connected softmax layer. On five sarcasm datasets, both models clearly beat many affective, contextual and hybrid baselines in F1, and ACE 1 performs best. The work suggests that building task-specific signals into transformer training beats reusing generic pre-trained embeddings.

Key points

  • The paper describes itself as the first attempt to modify BERT's architecture and pre-train it from the ground up for sarcasm detection, rather than fine-tuning existing checkpoints.
  • Two affective feature representations, EMoSi and EAISe, feed the AFE module. Stage 2 of ACE 1 produces a reusable embedding model called WikiSarcA-BERT.
  • Best F1 scores across the five datasets:
    • ACE 1 (WikiSarc + EMoSi): 92.21, 89.22, 80.71, 84.57 and 93.14.
    • ACE 2 (WikiSarcA-BERT + EAISe): second at 90.31, 86.50, 80.39, 84.33 and 88.19.
  • Ablations show three things:
    • Combining affective and contextual features beats either one alone.
    • Fusing affect deeply during embedding training (ACE 1) beats concatenating features (ACE 2).
    • Pre-training on sarcasm-rich text helps.
  • Code and data are available at github.com/NastaranBa/ACE-for-Sarcasm-Detection.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

24 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 17 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.20

@inproceedings{babanejad2020affective,
  title = {{Affective and Contextual Embedding for Sarcasm Detection}},
  author = {Babanejad, Nastaran and Davoudi, Heidar and An, Aijun and Papagelis, Manos},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.20},
  url = {https://doi.org/10.18653/v1/2020.coling-main.20}
}

References

66 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ameeta Agrawal and Aijun An. 2018. Affective Representations for Sarcasm Detection. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1029–1032, Ann Arbor MI USA, June. ACM. find
  2. Ameeta Agrawal, Aijun An, and Manos Papagelis. 2018. Learning emotion-enriched word representations. In Proceedings of the 27th International Conference on Computational Linguistics, pages 950–961. find
  3. Ameeta Agrawal, Aijun An, and Manos Papagelis. 2020. Leveraging transitions of emotions for sarcasm detection. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1505–1508. find
  4. Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho, and Mário J. Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 167–177, Berlin, Germany, August. Association for Computational Linguistics. find our card
  5. Nastaran Babanejad, Ameeta Agrawal, Aijun An, and Manos Papagelis. 2020. A comprehensive analysis of preprocessing for word representation learning in affective tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5799–5810, Online, July. Association for Computational Linguistics. find
  6. Paula Carvalho, Luı́s Sarmento, Mário J. Silva, and Eugénio de Oliveira. 2009. Clues for detecting irony in user-generated contents: Oh...!! it’s “so easy”FIX ME!!!!;-). In Proceedings of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion, TSA ’09, page 53–56, New York, NY, USA. Association for Computing Machinery. find
  7. Paula Carvalho, Luı́s Sarmento, Jorge Teixeira, and Mário J. Silva. 2011. Liars and saviors in a sentiment annotated corpus of comments to political debates. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 564–568, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  8. Santiago Castro, Devamanyu Hazarika, Verónica Pérez-Rosas, Roger Zimmermann, Rada Mihalcea, and Soujanya Poria. 2019. Towards multimodal sarcasm detection (an-obviously-perfect paper). Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find our card
  9. Shelley Channon, Asa Pellijeff, and Andrea Rule. 2005. Social cognition after head injury: Sarcasm and theory of mind. Brain and Language, 93(2):123–134, May. ↗
  10. Joshua Coates and Danushka Bollegala. 2018. Frustratingly easy meta-embedding – computing meta-embeddings by averaging source word embeddings. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). find
  11. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcasm in twitter and Amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116, Uppsala, Sweden, July. Association for Computational Linguistics. find our card
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  13. Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th International Conference on World Wide Web, WWW ’15 Companion, page 29–30, New York, NY, USA. Association for Computing Machinery. find
  14. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 International Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
  15. Erik Forslid and Niklas Wikén. 2015. Automatic irony-and sarcasm detection in Social media. Ph.D. thesis, UPPSALA UNIVERSITET. find
  16. Debanjan Ghosh, Alexander Richard Fabbri, and Smaranda Muresan. 2017. The Role of Conversation Context for Sarcasm Detection in Online Interactions. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 186–196, Saarbrücken, Germany. Association for Computational Linguistics. find our card
  17. Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. 2020. A report on the 2020 sarcasm detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, pages 1–11, Online, July. Association for Computational Linguistics. find our card
  18. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: A closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 581–586, Portland, Oregon, USA, June. Association for Computational Linguistics. find our card
  19. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional lstm and other neural network architectures. NEURAL NETWORKS, pages 5–6. ↗
  20. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. CASCADE: Contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1837–1848, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics. find our card
  21. Delia Irazú Hernández Farı́as, Manuel Montes-y Gómez, Hugo Jair Escalante, Paolo Rosso, and Viviana Patti. 2018. A knowledge-based weighted KNN for detecting Irony in Twitter. In Proceedings17th Mexican International Conference on Artificial Intelligence, MICAI 2018, volume 11289, pages 194–206. Springer Verlag. Accepted: 2019-04-14T18:17:25Z. find
  22. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. ↗
  23. Li Huang, Francesca Gino, and Adam D. Galinsky. 2015. The highest form of intelligence: Sarcasm increases creativity for both expressers and recipients. Organizational Behavior and Human Decision Processes, 131(C):162–177. ↗
  24. Suzana Ilić, Edison Marrese-Taylor, Jorge Balazs, and Yutaka Matsuo. 2018. Deep contextualized word representations for detecting sarcasm and irony. In Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 2–7, Brussels, Belgium, October. Association for Computational Linguistics. find our card
  25. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757–762, Beijing, China, July. Association for Computational Linguistics. find our card
  26. Aditya Joshi, Vaibhav Tripathi, Pushpak Bhattacharyya, and Mark J. Carman. 2016. Harnessing sequence labeling for sarcasm detection in dialogue from TV series ‘Friends’. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 146–155, Berlin, Germany, August. Association for Computational Linguistics. find our card
  27. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2017. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579. ↗ our card
  28. Douwe Kiela, Changhan Wang, and Kyunghyun Cho. 2018. Dynamic meta-embeddings for improved sentence representations. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find
  29. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  30. Avinash Kumar, Vishnu Teja Narapareddy, Veerubhotla Aditya Srikanth, Aruna Malapati, and Lalita Bhanu Murthy Neti. 2020. Sarcasm Detection Using Multi-Head Attention Based Bidirectional LSTM. IEEE Access, 8:6388– 6397. Conference Name: IEEE Access. ↗
  31. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. find
  32. Edwin Lunando and Ayu Purwarianti. 2013. Indonesian social media sentiment analysis with sarcasm detection. 2013 International Conference on Advanced Computer Science and Information Systems (ICACSIS), September. find
  33. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. Proceedings of the 2019 Conference of the North. find our card
  34. Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. ↗
  35. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b. Distributed Representations of Words and Phrases and their Compositionality. arXiv:1310.4546 [cs, stat], October. arXiv: 1310.4546. Rishabh Misra and Prahal Arora. arXiv:1908.07414. 2019. Sarcasm detection using hybrid neural network. arXiv preprint Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a Word–Emotion Association Lexicon. Computational Intelligence, 29(3):436–465, August. ↗
  36. Saif M. Mohammad. 2017. Word affect intensities. ArXiv, abs/1704.08798. ↗ our card
  37. Marzieh Mozafari, Reza Farahbakhsh, and Noël Crespi. 2019. A bert-based transfer learning approach for hate speech detection in online social media. Studies in Computational Intelligence, page 928–940, Nov. find
  38. Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn Walker. 2016. Creating and characterizing a diverse corpus of sarcasm in dialogue. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 31–41, Los Angeles, September. Association for Computational Linguistics. find our card
  39. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  40. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). find
  41. Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ↗
  42. Bethany Pickering, Dominic Thompson, and Ruth Filik. 2018. Examining the emotional impact of sarcasm using a virtual environment. Metaphor and Symbol, 33(3):185–197, July. Publisher: Routledge eprint: https://doi.org/10.1080/10926488.2018.1481261. ↗
  43. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. ArXiv, abs/1610.08815. ↗ our card
  44. Rolandos Alexandros Potamias, Georgios Siolas, and Andreas Georgios Stafylopatis. 2019. A transformer-based approach to irony and sarcasm detection. ↗ our card
  45. Tomás Ptácek, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on czech and english twitter. In COLING. find our card
  46. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM ’15, page 97–106, New York, NY, USA. Association for Computing Machinery. find
  47. Rachel Rakov and Andrew Rosenberg. 2013. ”sure, i did the right thing”: a system for sarcasm detection in speech. In INTERSPEECH. find
  48. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11. find
  49. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 704–714, Seattle, Washington, USA, October. Association for Computational Linguistics. find our card
  50. Shiv Naresh Shivhare and Sri Khetwat Saritha. 2014. Emotion Detection From Text Documents. International Journal of Data Mining & Knowledge Management Process, 4(6):51–57, November. ↗
  51. Chi Sun, Luyao Huang, and Xipeng Qiu. 2019. Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 380–385, Minneapolis, Minnesota, June. Association for Computational Linguistics. ↗ our card
  52. Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. Proceedings of 56th Annual Meeting of the Association for Computational Linguistics. find our card
  53. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. ICWSM-A Great Catchy Name: Semi-Supervised Recognition of Sarcastic Sentences in Online Product Reviews. In In Fourth International AAAI Conference on Weblogs and Social Media. find
  54. Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018. SemEval-2018 task 3: Irony detection in English tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 39–50, New Orleans, Louisiana, June. Association for Computational Linguistics. find our card
  55. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA. Curran Associates Inc. find
  56. Bin Wang and C. C. Jay Kuo. 2020. Sbert-wk: A sentence embedding method by dissecting bert-based word models. ↗
  57. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144. ↗
  58. Chuhan Wu, Fangzhao Wu, Sixing Wu, Junxin Liu, Zhigang Yuan, and Yongfeng Huang. 2018. THU NGN at SemEval-2018 task 3: Tweet irony detection with densely connected LSTM and multi-task learning. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 51–56, New Orleans, Louisiana, June. Association for Computational Linguistics. find
  59. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489, San Diego, California, June. Association for Computational Linguistics. find our card
  60. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. ↗ our card
  61. Zhengchen Zhang, Minghui Dong, and Shuzhi Sam Ge. 2014. Emotion analysis of children’s stories with context information. Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific, pages 1–7. find
  62. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460, Osaka, Japan, December. find our card
  63. The COLING 2016 Organizing Committee. find
  64. Shiwei Zhang, Xiuzhen Zhang, Jeffrey Chan, and Paolo Rosso. 2019a. Irony detection via sentiment-based transfer learning. Information Processing & Management, 56(5):1633–1644, September. ↗
  65. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019b. Bertscore: Evaluating text generation with bert. ↗
  66. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019. Moverscore: Text generation evaluating with contextualized embeddings and earth mover distance. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/babanejad-2020-affective-contextual-embedding-sarcasm-detection/