Elamur Research⌕
☰
home / papers / 2016 / Harnessing Cognitive Features for Sarcasm Detection

Harnessing Cognitive Features for Sarcasm Detection

Summary

The paper investigates whether the way people read can help machines recognize sarcasm in short, non-contextual texts such as reviews and tweets. Textual cues alone often fail here. The authors argue that sarcasm usually involves incongruity, which forces readers to re-process the text and leaves traces in their eye movements. They use their earlier freely released eye-tracking dataset of 1,000 short texts, read by seven participants with an SR-Research Eyelink-1000 tracker. They show that sarcastic texts receive markedly longer fixations per word. They then design two kinds of cognitive features: simple gaze measures (fixations, forward and backward saccades) and features derived from a graph whose nodes are words and whose edges are saccades. These are combined with the linguistic and stylistic features of the strongest prior detector (Joshi et al., 2015) and fed to several classifiers, including a multi-instance MILR model. The enriched feature set raises F-score by 3.7% over the best reported system. This suggests that cognitive signals, which could eventually come from cheap mobile eye-trackers, can complement text-based NLP.

Key points

  • Dataset composition: 350 sarcastic texts (103 from quote websites, 76 Amazon Movie Corpus reviews, 171 #sarcasm tweets) and 650 non-sarcastic texts, each 10–40 words long.
  • Avoiding the priming effect: participants only labeled polarity and were never told to look for sarcasm. For all seven readers, the difference in average fixation duration between sarcastic and literal texts was highly significant (t-test).
  • Statistical significance: a McNemar test comparing the MILR model with all features against an SVM using only the Joshi et al. (2015) features gave p = 0.02 and an odds ratio of 1.43.
  • Feature importance: in rankings by Information Gain and Chi-squared tests, 16 of the top 20 features were gaze-based. The two highest-ranked features were the mean time readers fixated on each word and the position of the biggest backward eye movement.
  • Reading time alone is not enough: substituting plain reading time for the gaze features produced no statistically significant gain. A key limitation is that the method needs eye-tracking data at test time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

22 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 11 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1104 · arXiv 1701.05574

@inproceedings{mishra2016harnessing,
  title = {{Harnessing Cognitive Features for Sarcasm Detection}},
  author = {Mishra, Abhijit and Kanojia, Diptesh and Nagar, Seema and Dey, Kuntal and Bhattacharyya, Pushpak},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1104},
  eprint = {1701.05574},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p16-1104}
}

References

28 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francesco Barbieri, Horacio Saggion, and Francesco Ronzano. 2014. Modelling sarcasm in twitter, a novel approach. ACL 2014, page 50. find our card
  2. Dale J Barr, Roger Levy, Christoph Scheepers, and Harry J Tily. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language, 68(3):255–278. ↗
  3. C. Christine Camblin, Peter C. Gordon, and Tamara Y. Swaab. 2007. The interplay of discourse congruence and lexical association during sentence processing: Evidence from {ERPs} and eye tracking. Journal of Memory and Language, 56(1):103 – 128. ↗
  4. Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. Oxford University Press. find
  5. John D Campbell and Albert N Katz. 2012. Are there necessary conditions for inducing a sense of sarcastic irony? Discourse Processes, 49(6):459–480. ↗
  6. Stacey L Ivanko and Penny M Pexman. 2003. Context incongruity and irony processing. Discourse Processes, 35(3):241–279. ↗
  7. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio De Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  8. Julia Jorgensen, George A Miller, and Dan Sperber. 1984. Test of the mention theory of irony. Journal of Experimental Psychology: General, 113(1):112. ↗
  9. Chih-Chung Chang and Chih-Jen Lin. 2011. LIB-SVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27. Software available at http://www.csie.ntu.edu.tw/ ∼ cjlin/libsvm. ↗
  10. Herbert H Clark and Richard J Gerrig. 1984. On the pretense theory of irony. ↗
  11. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20(3):273–297. ↗
  12. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  13. Hartmut; Wallington Katie; Page Jemma Filik, Ruth; Leuthold. 2014. Testing theories of irony processing using eye-tracking and erps. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(3):811–828. ↗
  14. Rudolph Flesch. 1948. A new readability yardstick. Journal of applied psychology, 32(3):221. ↗
  15. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. Proceedings of 53rd Annual Meeting of the Association for Computational Linguistics, Beijing, China, page 757. find our card
  16. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2016. Automatic sarcasm detection: A survey. CoRR, abs/1602.03426. ↗ our card
  17. Marta Kutas and Steven A Hillyard. 1980. Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427):203–205. ↗
  18. Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets# not. WASSA 2013, page 29. find our card
  19. Diana Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In Proceedings of LREC. find our card
  20. Abhijit Mishra, Diptesh Kanojia, and Pushpak Bhattacharyya. 2016. Predicting readers’ sarcasm understandability by modeling gaze behavior. In Proceedings of AAAI. find
  21. Raymond W. Gibbs. 1986. Comprehension and memory for nonliteral utterances: The problem of sarcastic indirect requests. Acta Psychologica, 62(1):41 – 57. ↗
  22. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd annual meeting on Association for Computational Linguistics, page 271. Association for Computational Linguistics. ↗
  23. Rachel Giora. 1995. On irony and negation. Discourse processes, 19(2):239–264. ↗
  24. Keith Rayner and Sara C Sereno. 1994. Eye movements in reading: Psycholinguistic studies. find
  25. Roberto González-Ibánez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  26. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, pages 704– 714. find our card
  27. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. 2009. The weka data mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10– 18. ↗
  28. Xin Xu and Eibe Frank. 2004. Logistic regression and boosting for labeled bags of instances. In Advances in knowledge discovery and data mining, pages 272– 281. Springer. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/mishra-2016-harnessing-cognitive-features-sarcasm-detection/