Elamur Research⌕
☰
home / papers / 2021 / WANLP 2021 Shared-Task: Towards Irony and Sentiment Detection in Arabic Tweets using Multi-headed-LSTM-CNN-GRU and MaRBERT

WANLP 2021 Shared-Task: Towards Irony and Sentiment Detection in Arabic Tweets using Multi-headed-LSTM-CNN-GRU and MaRBERT

Summary

This paper presents one team's system for the two WANLP 2021 shared tasks on Arabic tweets. Task one is binary sarcasm detection, and task two is three-way sentiment classification (positive, negative, neutral). Both tasks use the ArSarcasm-v2 dataset, which is mostly MSA and Egyptian and has a strong label imbalance.

The author compared two main approaches:

  • MARBERT, fine-tuned: the pre-trained Arabic transformer adapted to the task data.
  • Multi-headed-LSTM-CNN-GRU: a custom network that takes cleaned text and emojis as separate inputs. Text uses Arabic news embeddings and goes through parallel BiLSTM-CNN and BiGRU-CNN branches. Emojis use emoji2vec vectors and go through a CNN-LSTM branch.

SVM and logistic regression baselines were also tested.

On the author's internal split, the custom network looked competitive for sarcasm. On the official test set, however, MARBERT was clearly stronger. MARBERT placed 3rd of 22 in sentiment and 10th of 27 in sarcasm.

The work shows that large pre-trained Arabic transformers generalize better than smaller task-specific networks. It also shows that internal validation can be misleading when the test distribution differs, for example in emoji frequency.

Key points

  • Data: ArSarcasm-v2 has 12,548 training tweets and 3,000 test tweets. Only about 17.3% are sarcastic. Sentiment labels split as roughly 36.8% negative, 45.8% neutral and 17.4% positive.
  • Official sentiment result: MARBERT reached F1-PN 0.7321, ranking 3rd of 22.
  • Official sarcasm result: MARBERT reached F1-Sarcastic 0.5662, ranking 10th of 27. The Multi-headed-LSTM-CNN-GRU model reached only 0.1657, possibly because test tweets contain far fewer emojis than training tweets.
  • MARBERT training setup: fine-tuned for 5 epochs with a learning rate of 2e-06, batch size 32 and an Adam weight-decay optimizer.
  • Code: the source code is available on GitHub under the user "rematchka".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abdelsalam2021wanlp,
  title = {{WANLP 2021 Shared-Task: Towards Irony and Sentiment Detection in Arabic Tweets using Multi-headed-LSTM-CNN-GRU and MaRBERT}},
  author = {Abdel-Salam, Reem},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.37/}
}

References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. Arbert & marbert: Deep bidirectional transformers for arabic. ArXiv, abs/2101.01785. ↗ our card
  2. Nawaf A. Abdulla, N. A. Ahmed, Mohammed A. She-hab, and Mahmoud Al-Ayyoub. 2013. Arabic sentiment analysis: Lexicon-based and corpus-based. 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), pages 1–6. find
  3. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop. find our card
  4. Mohammad Al-Smadi, Bashar Talafha, Mahmoud Al-Ayyoub, and Y. Jararweh. 2019. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics, pages 1–13. ↗
  5. Abdulaziz M. Alayba, V. Palade, Matthew England, and R. Iqbal. 2018. A combined cnn and lstm model for arabic sentiment analysis. ArXiv, abs/1807.02911. ↗
  6. Abdulaziz M. Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2017. Arabic language sentiment analysis on health services. CoRR, abs/1702.03197. ↗
  7. A. Aziz Altowayan and L. Tao. 2016. Word embeddings for arabic sentiment analysis. 2016 IEEE International Conference on Big Data (Big Data), pages 3820–3825. find
  8. Steven Bird. 2006. Nltk: The natural language toolkit. ArXiv, cs.CL/0205028. find
  9. Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bosnjak, and S. Riedel. 2016. emoji2vec: Learning emoji representations from their description. ArXiv, abs/1609.08359. ↗
  10. Ashraf Elnagar, Yasmin S. Khalifa, and Anas Einea. 2018. Hotel arabic-reviews dataset construction for sentiment analysis applications. ↗
  11. Kariman Elshakankery and M. Ahmed. 2019. Hilatsa: A hybrid incremental learning approach for arabic tweets sentiment analysis. Egyptian Informatics Journal, 20:163–171. ↗
  12. Jihen Karoui, F. Benamara, and Véronique Moriceau. 2017. Soukhria: Towards an irony detection system for arabic in social media. In ACLING. find
  13. M. Khalifa and Noura Hussein. 2019. Ensemble learning for irony detection in arabic tweets. In FIRE. find
  14. Mahmoud Nabil, Mohamed A. Aly, and A. F. Atiya. 2015. Astd: Arabic sentiment tweets dataset. In EMNLP. ↗

This page in Markdown → https://md.research.elamur.ai/free/en/papers/salam-2021-wanlp-2021-shared-task-irony/