Elamur Research⌕
☰
home / papers / 2019 / A deep-learning framework to detect sarcasm targets

A deep-learning framework to detect sarcasm targets

Summary

The paper addresses sarcasm target identification: given a text already known to be sarcastic, find the words naming the entity or situation being mocked, or return the fall-back label "outside" if no such words exist. Before this work, Joshi et al. (2018) was the only system for the task, and it combined rule-based and statistical extractors. The authors use that study's dataset of book snippets and tweets. First, they test which socio-linguistic properties statistically separate target words from other words. These include NER types, POS tags, and LIWC and Empath categories. Then they build a neural classifier that judges each word separately, using its left context, right context and its own embedding. These pass through LSTM, Bi-LSTM or TD-LSTM layers, and the significant socio-linguistic features are optionally added before a sigmoid output. Every variant clearly beats the baseline. The Bi-LSTM model with socio-linguistic features scores best, setting a new state of the art. The result matters for aspect-based sentiment analysis, opinion mining and response generation.

Key points

  • Data: the Joshi et al. (2018) corpus of 224 book snippets and 506 tweets, with targets annotated by three experienced linguists. Average target length is 1.6 words for snippets and 2.08 for tweets.
  • LOC and ORG entities, many POS tags, and numerous LIWC and Empath categories differ significantly between target and non-target words, with p-values between 0.05 and 0.001.
  • Setup: 3-fold cross-validation, Adam with a learning rate of 1e-5, and batch size 64. ELMo initialization worked better than GloVe, fastText and BERT.
  • Bi-LSTM + socio-linguistic features reached the following scores, compared with the baseline's best:
    • Exact match: 30.12 on tweets (baseline 13.45) and 31.17 on snippets (baseline 16.51).
    • Dice score: 84.11 on tweets (baseline 39.63) and 88.16 on snippets (baseline 32.68).
  • The code is publicly released on GitHub (Srijanb97/Sarcasm_Target_Detection-EMNLP-).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1663

@inproceedings{patro2019deep,
  title = {{A deep-learning framework to detect sarcasm targets}},
  author = {Patro, Jasabanta and Bansal, Srijan and Mukherjee, Animesh},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1663},
  url = {https://doi.org/10.18653/v1/d19-1663}
}

References

25 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gavin Abercrombie and Dirk Hovy. 2016. Putting sarcasm detection into context: The effects of class imbalance and manual labelling on supervised machine classification of twitter conversations. In Proceedings of the ACL 2016 Student Research Workshop, pages 107–113. find our card
  2. Mondher Bouazizi and Tomoaki Ohtsuki. 2015. Sarcasm detection in twitter:” all your products are incredibly amazing!!!”-are they really? In 2015 IEEE Global Communications Conference (GLOBECOM), pages 1–6. IEEE. find
  3. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the fourteenth conference on computational natural language learning, pages 107–116. Association for Computational Linguistics. Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), volume 2, pages 757–762. find our card
  4. Y. Alex Kolchinski and Christopher Potts. 2018. Representing social media users for sarcasm detection. In EMNLP. ↗ our card
  5. Suman Kalyan Maity, Aishik Chakraborty, Pawan Goyal, and Animesh Mukherjee. 2018. Opinion conflicts: An effective route to detect incivility in twitter. In ACM CSCW. ↗
  6. Suman Kalyan Maity, Ritvik Saraf, and Animesh Mukherjee. 2016. #bieber + #blast = #bieberblast: Early prediction of popular hashtag compounds. In CSCW ’16, pages 50–63. ↗
  7. Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019a. Spread of hate speech in online social media. In ACM WebSci. find
  8. Ethan Fast, Binbin Chen, and Michael S Bernstein. 2016. Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 4647–4657. ACM. find
  9. Binny Mathew, Punyajoy Saha, Hardik Tharad, Sub-ham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, and Animesh Mukherjee. 2019b. Thou shalt not hate: Countering online hate-speech. In ICWSM. ↗
  10. Clay Fink, Jonathon Kopecky, and Maksym Morawski. 2012. Inferring gender from the content of tweets: A region specific example. In ICWSM ’12, pages 459–462. find
  11. Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends R in Information Retrieval, 2(1–2):1–135. ↗
  12. Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of twitter abusive behavior. In ICWSM. ↗
  13. Debanjan Ghosh and Smaranda Muresan. 2018. ”with 1 follower i must be awesome : P”. exploring the role of irony markers in irony recognition. In ICWSM. find
  14. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. Cascade: Contextual sarcasm detection in online discussion forums. In COLING. find our card
  15. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2017. Automatic sarcasm detection: A survey. ACM Comput. Surv., 50:73:1–73:22. ↗ our card
  16. Aditya Joshi, Pranav Goel, Pushpak Bhattacharyya, and Mark Carman. 2018. Sarcasm target identification: Dataset and an introductory approach. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find our card
  17. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Tomáš Ptáček, Ivan Habernal, and Jun Hong. 2014. Sarcasm detection on czech and english twitter. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 213–223. find our card
  18. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pages 97–106. ACM. find
  19. Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgı́lio A. F. Almeida, and Wagner Meira. 2018. Characterizing and detecting hateful users on twitter. In ICWSM. ↗
  20. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 704–714. find our card
  21. Joni O. Salminen, Hind Almerekhi, Milica Milenkovic, Soon-Gyo Jung, Jisun An, Haewoon Kwak, and Bernard J. Jansen. 2018. Anatomy of online hate: Developing a taxonomy and machine learning models for identifying and classifying hate in online news media. In ICWSM. find
  22. Hansen Andrew Schwartz, Johannes C. Eich-staedt, Margaret L. Kern, Lukasz Dziurzynski, Richard E. Lucas, Megha Agrawal, Gregory J. Park, Shrinidhi K. Lakshmikanth, Sneha Jha, Martin E. P. Seligman, and Lyle Ungar. 2013. Characterizing geographic variation in well-being using tweets. In ICWSM ’13, pages 583–591. find
  23. Yi Tay, Anh Tuan Luu, Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. In ACL. find our card
  24. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsma great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In Fourth International AAAI Conference on Weblogs and Social Media. find
  25. Byron C Wallace, Laura Kertz, Eugene Charniak, et al. 2014. Humans require context to infer ironic intent (so computers probably do, too). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 512–516. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/patro-2019-deep-learning-framework-detect-sarcasm/