Elamur Research⌕
☰
home / papers / 2021 / Sarcasm and Sentiment Detection in Arabic language A Hybrid Approach Combining Embeddings and Rule-based Features

Sarcasm and Sentiment Detection in Arabic language A Hybrid Approach Combining Embeddings and Rule-based Features

Summary

The paper describes the ArabicProcessors entry to the WANLP 2021 shared task on Arabic sarcasm detection (subtask 1) and three-way sentiment classification (subtask 2). Both subtasks used the ArSarcasm-v2 tweet corpus, which is small, skewed toward non-sarcastic tweets, and mixes MSA with several dialects. The authors first studied the corpus and found surface cues that separate sarcastic from ordinary tweets, such as particular emojis, laughter interjections, and word-length differences. They turned these cues into a handcrafted feature set covering surface form, lexicon-based sentiment, and intensifiers. Three models were then combined by weighted soft voting:

  • a Gaussian Naive Bayes classifier on the features;
  • MarBERT fed the tweet text with its dialect label attached;
  • a Bi-LSTM over Mazajak embeddings that also receives the features.

On the hidden test set the ensemble scored about 51% F1 on the sarcastic class and about 71% F1-PN for sentiment. Sentiment results generalized well, but sarcasm scores fell noticeably from cross-validation to test. The work shows how cheap rule-derived cues can sit beside contextual embeddings in a low-resource, dialect-rich setting.

Key points

  • Data. ArSarcasm-v2 has 12,548 training tweets, of which 2,168 (17.3%) are sarcastic. Sentiment is roughly half neutral, 37% negative and 17% positive. About 68% of tweets are MSA, and Egyptian is the leading dialect.
  • Corpus cues.
    • The "face with tears of joy" emoji appears in 32% of sarcastic tweets versus 7% of the rest.
    • A repeated-letter "haha" interjection appears in 4% versus under 0.5%.
    • Sarcastic tweets have more words (18 vs 15) but shorter words (5.6 vs 6.6 characters).
  • Features and resources.
    • Sentiment scores come from the Arabic Emoticon Lexicon, the Arabic Hashtag Lexicon (dialectal) and Emoji Sentiment Ranking 1.0.
    • A diacritics share above 45% flags Quranic text.
    • Hand lists cover sarcasm words, opposition words, political entities and intensifiers.
  • Cross-validation (5-fold, stratified; sarcasm / sentiment). Gaussian NB 43/46, MarBERT 57/71, Bi-LSTM-Mazajak 46/58, weighted ensemble 59/73. MarBERT beat AraBERT and Arabic-BERT, and Gaussian NB beat SVM, XGBoost, LightGBM, random forest and logistic regression.
  • Test set (3,000 tweets).
    • The ensemble scored 50.86 on sarcasm and 71.45 on sentiment.
    • MarBERT alone scored 54.17 and 73.32, i.e. slightly better than the ensemble.
    • The Bi-LSTM collapsed to 23.15 on sarcasm.
    • The test set had 27% sarcastic tweets versus 17% in training, which the authors cite as a likely cause of the drop.

Interesting ideas

  • I1. A Boolean feature checks whether the sentiment of the opening part of a text contradicts the sentiment of its closing part, a cheap and inspectable proxy for the incongruity behind sarcasm. — For Mova (frames): the irony module in prag could compare evaluative polarity across clauses or frames of one utterance and flag a reversal as an explainable irony cue.
  • I2. The authors propose a cascade in which sentiment is predicted first and then fed as an input to the sarcasm decision, because sarcasm correlates strongly with negative polarity. — For Mova (frames): implicature and irony detection could take an already computed speaker-attitude value as an explicit upstream input rather than re-deriving it.
  • I3. A diacritic-density threshold is used to recognize embedded quoted scripture, i.e. a register switch detected from simple character statistics. — For Mova (tokens): a character-level density check could mark quoted verse, code or foreign-script spans so the tagger and lemmatizer treat them as a separate register.
  • I4. Metadata (the dialect label) is joined to the raw text before it enters the transformer, letting the model condition on variety without any architecture change. — For Mova (tree): gparse could receive a prepended genre or source-treebank tag so one parser adapts to UD dialect and domain differences.
  • I5. Ensemble weights are tuned by Bayesian optimization on cross-validation, and each member's drop from cross-validation to test is then inspected. This exposed the Bi-LSTM as fragile and showed that MarBERT alone beat the blend. — For Mova (learn): when combining witness parsers or teacher tiers, compare train and evaluation label distributions and track each member's generalization gap so weak voters can be down-weighted or dropped.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

2 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gaanoun2021sarcasm,
  title = {{Sarcasm and Sentiment Detection in Arabic language A Hybrid Approach Combining Embeddings and Rule-based Features}},
  author = {Gaanoun, Kamel and Benelallam, Imade},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.45/}
}

References

18 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. Arbert & marbert: Deep bidirectional transformers for arabic. arXiv preprint arXiv:2101.01785. ↗ our card
  2. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics. find
  3. Ibrahim Abu Farha and Walid Magdy. 2020. From Arabic sentiment analysis to sarcasm detection: The ArSarcasm dataset. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39, Marseille, France. European Language Resource Association. find our card
  4. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop. find our card
  5. Ameeta Agrawal, Aijun An, and Manos Papagelis. 2020. Leveraging transitions of emotions for sarcasm detection. pages 1505–1508. find
  6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. find our card
  7. Bilal Ghanem, Jihen Karoui, Farah Benamara, Véronique Moriceau, and Paolo Rosso. 2019. Idat at fire2019: Overview of the track on irony detection in arabic tweets. In Proceedings of the 11th Forum for Information Retrieval Evaluation, FIRE ’19, page 10–13, New York, NY, USA. Association for Computing Machinery. find
  8. Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. 2018. CASCADE: Contextual sarcasm detection in online discussion forums. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1837–1848, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  9. Aditya Joshi, Pushpak Bhattacharyya, and Mark J Carman. 2017. Automatic sarcasm detection: A survey. ACM Computing Surveys (CSUR), 50(5):1–22. find our card
  10. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757–762, Beijing, China. Association for Computational Linguistics. ↗ our card
  11. Jaeyong Kang and Jeonghwan Gwak. 2020. Ensemble learning of lightweight deep learning models using knowledge distillation for image classification. Mathematics, 8(10). find
  12. Jihen Karoui, Farah Banamara Zitoune, and Veronique Moriceau. 2017. Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168. ↗
  13. Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. 2015. Sentiment of emojis. PLoS ONE, 10(12):e0144296. find
  14. Roger Kreuz and Gina Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on Computational Approaches to Figurative Language, pages 1–4, Rochester, New York. Association for Computational Linguistics. find our card
  15. Kamal Nigam and Matthew Hurst. 2006. Towards a robust metric of polarity. In Computing Attitude and Affect in Text: Theory and Applications, pages 265– 279. Springer. find
  16. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2:1–135. find
  17. Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015. Sarcasm detection on twitter: A behavioral modeling approach. WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining, pages 97–106. find
  18. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media. In Proceedings of the International Workshop on Semantic Evaluation (SemEval). find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gaanoun-2021-sarcasm-sentiment-detection-arabic-language/