Elamur Research⌕
☰
home / papers / 2021 / Sarcasm Detection and Building an English Language Corpus in Real Time

Sarcasm Detection and Building an English Language Corpus in Real Time

Paper · 2021 · Oliver Cakebread-Andrews

Summary

This short paper is a doctoral research proposal, not an experimental report, so it presents no new results. It addresses sarcasm detection in English, where sentiment can flip because of cues such as hyperbole or a clash between positive wording and a negative situation. The author reviews prior work and identifies a gap. Sarcasm corpora tend to be small, are often assembled for a single study, and stay frozen once built. The plan has two parts. First, harvest Reddit comments that authors have marked with the /s tag to build a large English corpus that keeps growing. A detector would flag new sarcastic posts, add them to the data, and retrain on the expanded set. Second, apply LSTM, Bi-LSTM, CNN or BERT models, possibly in combination, to raise accuracy, especially on natural text unlike the training data. Later stages may add images or audio and attempt transfer to Japanese. The proposal matters because it frames the corpus as a living, self-extending resource rather than a fixed snapshot.

Key points

  • Self-labelled sarcasm is easy to collect on social media: Reddit users mark it with /s and Twitter users with hashtags. The proposal starts with Reddit.
  • Most cited corpora are small, for example 900, 4000, 5000 and 39,000 Tweets. The author attributes this partly to Twitter API limits. An exception is the 406-thousand Dutch Tweet corpus of Kunneman et al. (2015).
  • The 2020 Sarcasm Detection Shared Task reported scores from 59 to 83 on Reddit and from 68 to 93 on Twitter. Twitter results are consistently higher, and the reason is unclear. Reddit was chosen because it leaves more room for improvement.
  • The planned models are LSTM, Bi-LSTM, CNN and BERT, or ensembles of them. Recent Bi-LSTM work reaches F-scores in the high 70s on Reddit and the low 90s on Twitter.
  • Further directions include multimodal sarcasm detection (images, possibly audio) and cross-lingual transfer to a non-European language such as Japanese.

Interesting ideas

  • I1. Use explicit author markers such as Reddit's /s as free, naturally occurring labels, which avoids expensive annotation and yields authentic examples.
  • I2. Build the corpus as a self-growing loop: the classifier labels new posts, accepted ones join the data, and the model retrains on them.
  • I3. Treat a clash between positive evaluative wording and a clearly unpleasant described situation as a core sarcasm cue, as in loving the experience of waking up sick.
  • I4. Pick the harder domain on purpose, here Reddit with its lower scores, and judge success on natural data unrelated to the training set rather than on in-distribution splits.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Harvest /s-tagged comments as a silver test set for the irony detector in prag, checking whether its explainable cues fire on texts their authors labelled sarcastic.
  • I2 (learn): A similar loop could extend silver data, but only items agreed by several witness parsers should enter, and negative controls are needed to stop the loop amplifying its own errors.
  • I3 (frames): prag could detect irony explicitly by comparing the polarity of the speaker's evaluation frame with the valence of the event state in the world of the text, giving a traceable reason for each irony label.
  • I4 (learn): Evaluations should include held-out text from sources unseen during training, so that gains reflect real generalisation and not memorised domain quirks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.English · 1,373 worksSentiment & emotion · 608 works

Cite

Cite as Oliver Cakebread-Andrews. 2021. Sarcasm Detection and Building an English Language Corpus in Real Time. Proceedings of the Student Research Workshop Associated with RANLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{cakebreadandrews2021sarcasm,
  title = {{Sarcasm Detection and Building an English Language Corpus in Real Time}},
  author = {Oliver Cakebread-Andrews},
  booktitle = {Proceedings of the Student Research Workshop Associated with RANLP 2021},
  year = {2021},
  doi = {10.26615/issn.2603-2821.2021\_005},
  url = {https://doi.org/10.26615/issn.2603-2821.2021_005}
}

References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chronopoulou, A., Baziotis, C., & Potamianos, A. (2019). An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models. Proceedings of the 2019 Conference of the North. http://dx.doi.org/10.18653/v1/N19-1213 ↗
  2. Aboobaker, J., & Illavarasan, E. (2020). A Survey on Sarcasm Detection and Challenges. 2020 6th International Conference on Advanced Computing and Communication Systtems (ICACCS). https://doi.org/10.1109/ICACCS48705.2020. 9074163 ↗
  3. Eke, C. I., Norman, A. A., & Shuib, L. (2021). Multi-feature fusion framework for sarcasm identification on twitter data: A machine learning based approach. PLOS ONE, 16(6). https://doi.org/10.1371/journal.pone.025291 8 ↗
  4. Eke, C. I., Norman, A. A., Shuib, L., & Nweke, H. F. (2019). Sarcasm identification in textual data: systematic review, research challenges and open directions. Artificial Intelligence Review, 53(6), 4215-4258. https://doi.org/10.1007/s10462-019-09791-8 ↗
  5. Avvaru, A., Vobilisetty, S., & Mamidi, R. (2020). Detection Sarcasm in Conversation Context Using Transformer-Based Models. Proceedings of the Second Workshop on Figurative Language Processing. http://dx.doi.org/10.18653/v1/2020.figlang-1.15 ↗
  6. Frenda, S., & Patti, V. (2019). Computational models for irony detection in three spanish variants. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019), co-located with 34th Conference of the Spanish Society for Natural Language Processing (SEPLN 2019). CEUR. http://hdl.handle.net/2318/1795396 ↗
  7. Baruah, A., Das, K., Barbhuiya, F., & Dey, K. (2020). Context-Aware Sarcasm Detection Using BERT. Proceedings of the Second Workshop on Figurative Language Processing. http://dx.doi.org/10.18653/v1/2020.figlang-1.12 ↗
  8. Bharti, S. K. (2019). Sarcasm Detection in Textual Data: A Supervised Approach. Doctoral Dissertation. https://doi.org/10.4018/978-1-5225-7186-5.CH010 ↗
  9. Ghosh, A., & Veale, D. T. (2016). Fracking Sarcasm using Neural Network. Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. http://dx.doi.org/10.18653/v1/W16-0425 Processing & Management, 51(4), 500-509. https://doi.org/10.1016/j.ipm.2014.07.006 ↗
  10. Ghosh, D., Fabbri, A. R., & Muresan, S. (2018). Sarcasm Analysis Using Conversation Context. Computational Linguistics, 44(4), 755-792. https://doi.org/10.1162/coli_a_00336 ↗
  11. Mishra, P., Kaushik, S., & Dey, K. (2020). Bi-ISCA: Bidirectional Inter-Sentence Contextual Attention Mechanism for Detecting Sarcasm in User Generated Noisy Short Text. arXiv. https://arxiv.org/abs/2011.11465 ↗
  12. Ghosh, D., Vajpayee, A., & Muresan, S. (2020). A Report on the 2020 Sarcasm Detection Shared Task. Proceedings of the Second Workshop on Figurative Language Processing. http://dx.doi.org/10.18653/v1/2020.figlang-1.1 ↗
  13. Ortega-Bueno, R., Rangel, F., Hernández Farıas, D., Rosso, P., Montes-y-Gómez, M., & Medina Pagola, J. E. (2019). Overview of the task on irony detection in Spanish variants. Proceedings of the Iberian languages evaluation forum (IberLEF 2019), co-located with 34th conference of the Spanish Society for natural language processing (SEPLN 2019). CEUR. find
  14. González-Ibánez, R., Muresan, S., & Wacholder, N. (2011). Identifying sarcasm in twitter: a closer look. roceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, (pp. 581-596). https://aclanthology.org/P11-2102 ↗
  15. Pelser, D., & Murrell, H. (2019). Deep and Dense Sarcasm Detection. ArXiv. https://arxiv.org/abs/1911.07474 ↗
  16. Ranasinghe, T., Saadany, H., Plum, A., Mandhari, S., Mohamed, E. O., & Mitkov, R. (2019). RGCL at IDAT: Deep Learning models for Irony Detection in Arabic Language. Proceedings of the IDAT@FIRE2019. CEUR Workshop Proceedings. http://hdl.handle.net/2436/622828 ↗
  17. Hiai, S., & Shimada, K. (2018). Sarcasm Detection Using Features Based on Indicator and Roles. Advances in Intelligent Systems and Computing, 418-428. https://doi.org/10.1007/978-3-319-72550-5_40 ↗
  18. Ren, Y., Ji, D., & Ren, H. (2018). Context-augmented convolutional neural networks for twitter sarcasm detection. Neurocomputing, 308, 1-7. https://doi.org/10.1016/j.neucom.2018.03.04 7 ↗
  19. Jain, D., Kumar, A., & Garg, G. (2020). Sarcasm detection in mash-up language using soft-attention based bi-directional LSTM and feature-rich CNN. Applied Soft Computing, 91. https://doi.org/10.1016/j.asoc.2020.106198 ↗
  20. Tavan, E., Rahmati, A., & Ali Keyvanrad, M. (2020). find
  21. Persian Emoji Prediction Using Deep Learning and Emoji Embedding. 2020 10th International Conference on Computer and Knowledge Engineering (ICCKE). https://doi.org/10.1109/ICCKE50421.2020.9 303639 ↗
  22. Jaiswal, N. (2020). Neural Sarcasm Detection using Conversation Context. Proceedings of the Second Workshop on Figurative Language Processing. http://dx.doi.org/10.18653/v1/2020.figlang-1.11 ↗
  23. Khodak, M., Saunshi, N., & Vodrahalli, K. (2018). A Large Self-Annotated Corpus for Sarcasm. ArXiv. https://aclanthology.org/L18-1102 ↗
  24. Vyas, V., & Uma, V. (2019). Approaches to Sentiment Analysis on Product Reviews. Advances in Business Information Systems and Analytics, 15-30. https://doi.org/10.4018/978-1-5225-4999-4.CH002 ↗
  25. Kumar, A., & Garg, G. (2019). Sarcasm Detection Using Feature-Variant Learning Models. Proceedings of ICETIT 2019, (pp. 683-693). https://doi.org/10.1007/978-3-030-30577-2_61 ↗
  26. Wang, X., Sun, X., Yang, T., & Wang, H. (2020). Building a Bridge: A Method for Image-Text Sarcasm Detection Without Pretraining on Image-Text Data. Proceedings of the First International Workshop on Natural Kunneman, F., Liebrecht, C., Mulken, M. V., & Bosch, A. V. (2015). Signalling sarcasm: From hyperbole to hashtag. Information Language Processing Beyond Text. http://dx.doi.org/10.18653/v1/2020.nlpbt-1.3 ↗
  27. Wiguna, B. S., Hudiyanti, C. V., Alqis Rausanfita, A., & Arifin, A. Z. (2021). Sarcasm Detection Engine for Twitter Sentiment Analysis using Textual and Emoji Feature. Jurnal Ilmu Komputer Dan Informasi, 41(1), 1-8. https://doi.org/10.21609/jiki.v14i1.812 ↗
  28. Xu, N., Zeng, Z., & Mao, W. (2020). Reasoning with Multimodal Sarcastic Tweets via Modeling Cross-Modality Contrast and Semantic Association. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. http://dx.doi.org/10.18653/v1/2020.acl-main.349 ↗
  29. Zhang, S., Zhang, X., Chan, J., & Rosso, P. (2019). Irony detection via sentiment-based transfer learning. Information Processing & Management, 56(5), 1633-1644. https://doi.org/10.1016/j.ipm.2019.04.006 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/andrews-2021-sarcasm-detection-building-english-language/