Elamur Research⌕
☰
home / papers / 2021 / sarcasm detection and quantification in arabic tweets

sarcasm detection and quantification in arabic tweets

Summary

Sarcasm in Arabic social media is hard to detect automatically. Arabic has many dialects and rich morphology, and it has far fewer labeled resources than English. People also disagree about what counts as sarcastic. The authors respond to this disagreement by treating sarcasm as a matter of degree rather than a yes/no property. They gathered Arabic tweets through the Twitter API, covering MSA and several dialects in four topic areas, and had 11 native speakers judge each one independently. The share of annotators who called a tweet sarcastic becomes its target score between 0 and 1. For modeling, tweets go unpreprocessed through the ArabicBERT tokenizer and a Multi-dialect Arabic BERT, which had been fine-tuned for dialect identification (NADI). The resulting sentence embeddings feed a small feed-forward regressor with a sigmoid output, trained with MSE. Under 10-fold cross-validation the average validation loss is about 0.0116. The authors present this as a first benchmark for graded sarcasm and promise to release the corpus publicly.

Key points

  • The corpus has 1554 tweets, 1165 of them labeled sarcastic, in four categories: entertainment, politics, products and services, and sports. Politics holds most of the sarcastic tweets.
  • Each tweet was judged by 11 native Arabic speakers, and its label is the fraction of "sarcastic" votes (e.g., 2/11, 9/11, 1, 0).
  • The encoder is Multi-dialect Arabic BERT. It starts from ArabicBERT (about 93 GB of Arabic text) and was fine-tuned on NADI 26-way dialect identification using 10M tweets (Adam, lr 3.75×10⁻⁵, batch 16, 3 epochs).
  • The regressor has 2 hidden layers of 128 units, sigmoid activations, dropout 0.2, and a single sigmoid output unit. It was trained with MSE loss, Adam, batch size 8, for 10 epochs.
  • 10-fold CV losses fall from about 0.040 in fold 1 to about 0.004–0.005 in the later folds, with a mean of 0.0116. No classification metrics and no comparison against prior systems are reported.

Interesting ideas

  • I1. The label stores the fraction of annotators who saw sarcasm, so disagreement is kept as information rather than averaged into one majority vote. — For Mova (frames): The irony detector in prag could output a graded irony strength calibrated against several human or LLM judgments, so later steps can weigh a pragmatic reading by how contested it is.
  • I2. The authors frame a subjective judgment as regression with a bounded output (sigmoid unit plus MSE), which avoids forcing a hard yes/no decision. — For Mova (learn): When several witness parsers produce silver data, their agreement ratio on an arc or frame could serve as a soft training target or a confidence weight, instead of dropping every non-unanimous item.
  • I3. The paper reports only the regression loss, with no constant-prediction baseline. That makes it impossible to tell whether 0.0116 MSE beats always guessing the mean score. — For Mova (learn): Every graded-score evaluation should include a trivial predictor as a negative control, such as the mean score or the label prior, so that low losses are shown to reflect real learning.
  • I4. They reuse an encoder already adapted to a neighboring task (dialect identification on tweets) as a feature extractor for a small downstream head, which suits a dataset of only about 1.5k items. — For Mova (tree): A gparse encoder already tuned on a related signal could be shared across small, specialized treebanks or UD dialect conversions, with only light task heads trained on top.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Sentiment & emotion · Semitic languages

Institutions

🎓Jordan University of Science and TechnologyIrbid · Jordan9 works · 1 785 citations

Also on the byline: Al-Buraimi University College.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2108.01425 · arXiv 2108.01425

@misc{talafha2021sarcasm,
  title = {{sarcasm detection and quantification in arabic tweets}},
  author = {Talafha, Bashar and Za'ter, Muhy Eddin and Suleiman, Samer and Al-Ayyoub, Mahmoud and Al-Kabi, Mohammed N.},
  year = {2021},
  doi = {10.48550/arxiv.2108.01425},
  eprint = {2108.01425},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2108.01425}
}

References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Uraz Yavanoglu, Taha Yasin Ibisoglu, and Setra Genyang Wıcana. Sarcasm detection algorithms. International Journal of Semantic Computing, 12(03):457–478, 2018. find
  2. Setra Genyang Wicana, Taha Yasin İbisoglu, and Uraz Yavanoglu. A review on sarcasm detection from machine-learning perspective. In 2017 IEEE 11th International Conference on Semantic Computing (ICSC), pages 469–476. IEEE, 2017. find
  3. Aditya Joshi, Pushpak Bhattacharyya, and Mark J Carman. Automatic sarcasm detection: A survey. ACM Computing Surveys (CSUR), 50(5):1–22, 2017. find our card
  4. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. Sarcasm detection on twitter: A behavioral modeling approach. In Proceedings of the eighth ACM international conference on web search and data mining, pages 97–106, 2015. find
  5. Muhammad Khalifa and Noura Hussein. Ensemble learning for irony detection in arabic tweets. In FIRE (Working Notes), pages 433–438, 2019. find
  6. Samer Muthana Sarsam, Hosam Al-Samarraie, Ahmed Ibrahim Alzahrani, and Bianca Wright. Sarcasm detection using machine learning algorithms in twitter: A systematic review. International Journal of Market Research, 62(5):578–598, 2020. find
  7. Hamada Nayel, Eslam Amer, Aya Allam, and Hanya Abdallah. Machine learning-based model for sentiment and sarcasm detection. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 386–389, 2021. find our card
  8. Dana Al-Ghadhban, Eman Alnkhilan, Lamma Tatwany, and Muna Alrazgan. Arabic sarcasm detection in twitter. In 2017 International Conference on Engineering & MIS (ICEMIS), pages 1–7. IEEE, 2017. find
  9. Nawaf A Abdulla, Nizar A Ahmed, Mohammed A Shehab, and Mahmoud Al-Ayyoub. Arabic sentiment analysis: Lexicon-based and corpus-based. In 2013 IEEE Jordan conference on applied electrical engineering and computing technologies (AEECT), pages 1–6. IEEE, 2013. find
  10. Ali Farghaly and Khaled Shaalan. Arabic natural language processing: Challenges and solutions. ACM Transactions on Asian Language Information Processing (TALIP), 8(4):1–22, 2009. find
  11. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. Arabic natural language processing: An overview. Journal of King Saud University-Computer and Information Sciences, 2019. find our card
  12. Bilal Ghanem, Jihen Karoui, Farah Benamara, Paolo Rosso, and Véronique Moriceau. Irony detection in a multilingual context. In European Conference on Information Retrieval, pages 141–149. Springer, 2020. find
  13. Jihen Karoui, Farah Banamara Zitoune, and Veronique Moriceau. Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168, 2017. find
  14. Bilal Ghanem, Jihen Karoui, Farah Benamara, Véronique Moriceau, and Paolo Rosso. Idat at fire2019: Overview of the track on irony detection in arabic tweets. In Proceedings of the 11th Forum for Information Retrieval Evaluation, pages 10–13, 2019. find
  15. Ibrahim Abu Farha and Walid Magdy. From arabic sentiment analysis to sarcasm detection: The arsarcasm dataset. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39, 2020. find our card
  16. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. find
  17. Ines Abbes, Wajdi Zaghouani, Omaima El-Hardlo, and Faten Ashour. DAICT: A dialectal Arabic irony corpus extracted from Twitter. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6265–6271, Marseille, France, May 2020. European Language Resources Association. find our card
  18. Abdelkader El Mahdaouy, Abdellah El Mekki, Kabil Essefar, Nabil El Mamoun, Ismail Berrada, and Ahmed Khoumsi. Deep multi-task model for sarcasm detection and sentiment analysis in arabic language. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 334–339, 2021. find our card
  19. Abeer Abuzayed and Hend Al-Khalifa. Sarcasm and sentiment detection in arabic tweets using bert-based models and data augmentation. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 312–317, 2021. find our card
  20. Hazem Elgabry, Shimaa Attia, Ahmed Abdel-Rahman, Ahmed Abdel-Ate, and Sandra Girgis. A contextual word embedding for Arabic sarcasm detection with random forests. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 340–344, Kyiv, Ukraine (Virtual), April 2021. Association for Computational Linguistics. find our card
  21. Fatemah Husain and Ozlem Uzuner. Leveraging offensive language for sarcasm and sentiment detection in Arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 364–369, Kyiv, Ukraine (Virtual), April 2021. Association for Computational Linguistics. find our card
  22. Abdullah I Alharbi and Mark Lee. Multi-task learning using a combination of contextualised and static word embeddings for arabic sarcasm detection and sentiment analysis. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 318–322, 2021. find our card
  23. Khalid N Elmadani, Mukhtar Elgezouli, and Anas Showk. Bert fine-tuning for arabic text summarization. arXiv preprint arXiv:2004.14135, 2020. ↗ our card
  24. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. ↗ our card
  25. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint arXiv:1706.03762, 2017. ↗ our card
  26. Bashar Talafha, Mohammad Ali, Muhy Eddin Za’ter, Haitham Seelawi, Ibraheem Tuffaha, Mostafa Samir, Wael Farhan, and Hussein T Al-Natsheh. Multi-dialect arabic bert for country-level dialect identification. arXiv preprint arXiv:2007.05612, 2020. ↗
  27. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. Nadi 2021: The second nuanced arabic dialect identification shared task. arXiv preprint arXiv:2103.08466, 2021. ↗
  28. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/talafha-2021-sarcasm-detection-quantification-arabic-tweets/