Elamur Research⌕
☰
home / papers / 2019 / “When Numbers Matter!”: Detecting Sarcasm in Numerical Portions of Text

“When Numbers Matter!”: Detecting Sarcasm in Numerical Portions of Text

Summary

The paper addresses a sarcasm subtype that earlier detectors largely missed: tweets whose irony depends on a numerical value, such as praising a phone battery that lasts only two hours. The authors estimate this subtype accounts for roughly 11% of sarcastic tweets in their data. They define a binary task, labelling a tweet with numbers as either sarcastic because of the number or non-sarcastic, and release a labelled collection of 60,949 such tweets. They explore three NLP paradigms. A rule-based system shows that the core issue is incongruity between the context and the number. Statistical classifiers reveal which handcrafted cues matter. Two deep models serve as the final detectors. The deep models reach F-scores of about 0.9. The detector is meant to work as a second-stage filter, re-examining tweets that a general sarcasm detector has marked non-sarcastic. Added on top of three earlier systems, it improved their F-scores. The work is presented as the first study of numerical sarcasm, and it sets baselines for later research.

Key points

  • Dataset: 60,949 tweets containing numbers, with two labels: sarcastic due to number, and non-sarcastic.
  • Rule-based method:
    • Each tweet is stored in a sarcastic or non-sarcastic repository as a tuple: index, noun-phrase vector, number, and unit. The vectors are averaged 200-D GloVe embeddings fine-tuned on the dataset.
    • A test tweet is matched to its most similar entry by cosine similarity.
    • It is labelled sarcastic if its number falls within ±2.58 standard deviations of the mean for that unit.
  • Statistical ML: SVM and Random Forest, using punctuation counts, the numerical value, and a one-hot encoding of the unit, among other features.
  • Deep learning: a CNN-FF model scores F = 0.93 and an attention network scores F = 0.91.
  • Error analysis:
    • Error types include sarcasm not caused by numbers, numbers that only amplify sarcasm, comparisons between several numbers, unseen situations, "special" numeric tokens (e.g., "2" for "to"), and tweets that need extra context.
    • Planned future work includes a language model for numbers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-1309

@inproceedings{dubey2019when,
  title = {{“When Numbers Matter!”: Detecting Sarcasm in Numerical Portions of Text}},
  author = {Dubey, Abhijeet and Kumar, Lakshya and Somani, Arpan and Joshi, Aditya and Bhattacharyya, Pushpak},
  booktitle = {Proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2019},
  doi = {10.18653/v1/w19-1309},
  url = {https://doi.org/10.18653/v1/w19-1309}
}

References

22 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho, and Mario J. Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 167–177. Association for Computational Linguistics. find our card
  2. Raymond W Gibbs. 1986. On the psycholinguistics of sarcasm. Journal of Experimental Psychology: General, 115(1):3–15. ↗
  3. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: A closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 581–586, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  4. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10), Valletta, Malta. European Languages Resources Association (ELRA). find our card
  5. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. Cascade: Contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1837–1848. Association for Computational Linguistics. find our card
  6. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 704–714. Association for Computational Linguistics. find our card
  7. Stacey L. Ivanko and Penny M. Pexman. 2003. Context incongruity and irony processing. Discourse Processes, 35(3):241–279. ↗
  8. Emilio Sulis, Delia Iraz Hernndez Faras, Paolo Rosso, Viviana Patti, and Giancarlo Ruffo. 2016. Figurative messages and affect in twitter: Differences between irony, sarcasm and not. Knowledge-Based Systems, 108:132 – 143. New Avenues in Knowledge Bases for Natural Language Processing. ↗
  9. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2017. Automatic sarcasm detection: A survey. ACM Comput. Surv., 50:73:1–73:22. ↗ our card
  10. Joseph Tepperman, David R. Traum, and Shrikanth Narayanan. 2006. ”yeah right”: sarcasm recognition for spoken dialogue systems. In INTERSPEECH 2006 - ICSLP, Ninth International Conference on Spoken Language Processing, Pittsburgh, PA, USA, September 17-21, 2006. find
  11. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757–762, Beijing, China. Association for Computational Linguistics. find our card
  12. Akira Utsumi. 2000. Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics, 32(12):1777 – 1806. ↗
  13. Marvin Karson. 1968. Handbook of methods of applied statistics. volume i: Techniques of computation descriptive methods, and statistical inference. volume ii: Planning of surveys and experiments. i. m. chakravarti, r. g. laha, and j. roy, new york, john wiley; 1967, $9.00. Journal of the American Statistical Association, 63(323):1047–1049. ↗
  14. Cynthia Van Hee, Els Lefever, and Veronique Hoste. 2018. Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 39– 50, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card
  15. Christine Liebrecht, Florian Kunneman, and Antal Van den Bosch. 2013. The perfect solution for detecting sarcasm in tweets #not. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 29–37. Association for Computational Linguistics. find our card
  16. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489, San Diego, California. Association for Computational Linguistics. find our card
  17. Lotem Peled and Roi Reichart. 2017. Sarcasm sign: Interpreting sarcasm with sentiment based monolingual machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1690–1700. Association for Computational Linguistics. find our card
  18. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460. find our card
  19. The COLING 2016 Organizing Committee. find
  20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. ↗
  21. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1601–1612. ↗ our card
  22. The COLING 2016 Organizing Committee. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/dubey-2019-when-numbers-matter-detecting-sarcasm/