Elamur Research⌕
☰
home / papers / 2022 / A Benchmark Study of Contrastive Learning for Arabic Social Meaning

A Benchmark Study of Contrastive Learning for Arabic Social Meaning

Paper · 2022 · Md Tawkat Islam Khondaker, El Moatez Billah Nagoudi, AbdelRahim Elmadany et al.

Summary

The paper asks whether supervised contrastive learning (CL), which has helped many NLP tasks, also helps Arabic social meaning tasks such as dialect identification, emotion recognition, abusive and hate speech detection, and demographic attribute prediction. The authors say CL had not been tried for Arabic before. On top of a MARBERT encoder, they compare plain cross-entropy (CE) finetuning with five supervised CL objectives:

  • SCL, which uses dropout views and NT-Xent loss.
  • CAT, which uses adversarial perturbation of the word embedding matrix and InfoNCE loss.
  • TACT, a token-level adversarial variant of CAT.
  • LCL, a label-aware loss with a weighting network.
  • TLCL, which combines token-level adversarial training with LCL.

CL beats vanilla finetuning on most tasks, but the size of the gain depends on the task. The authors relate this to label granularity, meaning how many classes a task has and how fine they are. CL is also more data efficient: with only a fraction of the training data, several CL variants match or exceed CE trained on the full set. This makes CL attractive for low-resource Arabic settings.

Key points

  • Six training regimes are compared on MARBERT: CE, SCL, CAT, TACT, LCL and TLCL. The tasks are Arabic social meaning datasets, including Dialect at Country and Region level, AraNeT emo and an Abusive dataset (normal / abusive / hate).
  • In the data-size study (10%, 25%, 50%, 100% of training data), CE-MARBERT never beats the best CL method at any fraction.
  • Some CL variants overtake full-data CE using only part of the data:
    • Dialect-Region: CAT at 50% scores 65.42, against 65.29 for CE at 100%.
    • Dialect-Country: SCL at 50% scores 32.89, against 32.84 for CE at 100%.
    • AraNeT emo: TLCL at 50% scores 62.98, against 62.47 for CE at 100%. TLCL reaches 64.13 with all the data.
  • Performance does not always rise steadily with more data. Tested batch sizes were 4, 8 and 16, and larger batches usually help because each batch contains more contrasting classes.
  • In t-SNE plots of the Abusive data, CL methods pull normal and abusive examples further apart than CE does. LCL and TLCL also separate the easily confused abusive and hate classes better.
  • The stated limitations are that CL is sensitive to hyperparameters (batch size, τ, λ) and that larger batches cost more compute.

Interesting ideas

  • I1. Running the same input through the encoder twice with different dropout gives a free "positive" pair for contrastive training, with no external augmentation needed.
  • I2. LCL adds a small weighting network that learns how confusable each label is with the others, then scales the contrastive penalty accordingly; this especially helped neighbouring classes like abusive vs hate.
  • I3. Adversarial positives can be made by nudging embeddings along the loss gradient, either on the whole vocabulary matrix or on the tokens of one sentence, which toughens the model against small input noise.
  • I4. Training efficiency is measured as the smallest fraction of training data at which a method catches up with the full-data baseline, rather than only by final scores.
  • I5. Coloured t-SNE plots of internal representations by gold label serve as a cheap check of whether a training objective really separates classes.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): In gparse training, two dropout passes over one sentence could be required to agree on arc and label scores, adding a consistency signal on top of the treebank loss.
  • I2 (frames): In prag, confusable speech-act or irony labels (e.g. sarcasm vs plain negative assertion) could get confusion-aware weights, so training concentrates on the boundaries that actually get mixed up.
  • I3 (tree): Gradient-based perturbations of token embeddings could test and harden gparse on noisy or informal English, with the size of the arc changes serving as a robustness metric.
  • I4 (learn): When adding silver data from the witness parsers, the same 10/25/50/100% curves would show how much silver data is really needed and whether gains level off.
  • I5 (learn): Plotting parser or frame-classifier hidden states by gold relation or role would reveal which UD labels or VerbNet roles stay tangled, pointing to where repair rules are needed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of British ColumbiaVancouver · Canada98 works · 34,805 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Md Tawkat Islam Khondaker, El Moatez Billah Nagoudi, AbdelRahim Elmadany, Muhammad Abdul-Mageed and Laks Lakshmanan V.S. 2022. A Benchmark Study of Contrastive Learning for Arabic Social Meaning. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.wanlp-1.7

@inproceedings{khondaker2022benchmark,
  title = {{A Benchmark Study of Contrastive Learning for Arabic Social Meaning}},
  author = {Md Tawkat Islam Khondaker, El Moatez Billah Nagoudi, AbdelRahim Elmadany, Muhammad Abdul-Mageed and Laks Lakshmanan V.S.},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.7},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.7}
}

References

94 references, 52 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ines Abbes, Wajdi Zaghouani, Omaima El-Hardlo, and Faten Ashour. 2020. Daict: A dialectal Arabic irony corpus extracted from twitter. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 6265–6271.
  2. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, and Lyle Ungar. 2020b. Toward micro-dialect identification in diaglossic and code-switched environments. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5855–5876, Online. Association for Computational Linguistics. find
  3. Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish, and Younes Samih. 2021. Pre-training bert on arabic tweets: Practical considerations. arXiv preprint arXiv:2102.10684. ↗
  4. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2020. Arabic Dialect Identification in the Wild. Proceedings of the Sixth Arabic Natural Language Processing Workshop. ↗
  5. Muhammad Abdul-Mageed, Chiyu Zhang, Azadeh Hashemi, et al. 2020c. AraNet: A deep learning toolkit for arabic social media. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 16–23. ↗
  6. Muhammad Abdul-Mageed, Hassan AlHuzli, and Mona Diab DuaaAbu Elhija. 2016. Dina: A multi-dialect dataset for arabic emotion analysis. In The 2nd workshop on Arabic corpora and processing tools, page 29. find
  7. Malak Abdullah, Dalya Alnore, Safa Swedat, Jumana Khrais, and Mahmoud Al-Ayyoub. 2022. Sarcasmdet at semeval-2022 task 6: Detecting sarcasm using pre-trained transformers in english and arabic languages. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1025– 1030.
  8. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021a. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  9. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the WANLP 2021 shared task on sarcasm and sentiment detection in Arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 296–305, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  10. Muhammad Abdul-Mageed, Mohammed Korayem, and Ahmed YoussefAgha. 2011. “Yes we can?": Subjectivity annotation and tagging for the health domain. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 666–671. find
  11. Ftoon Abu Shaqra, Rehab Duwairi, and Mahmoud Al-Ayyoub. 2022. A multi-modal deep learning system for arabic emotion recognition. International Journal of Speech Technology, pages 1–17. ↗
  12. Muhammad Abdul-Mageed, Sandra Kübler, and Mona Diab. 2012. SAMAR: A system for subjectivity and sentiment analysis of arabic social media. In Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis, pages 19–28. Association for Computational Linguistics.
  13. Abeer Abuzayed and Hend Al-Khalifa. 2021. Sarcasm and sentiment detection in arabic tweets using bert-based models and data augmentation. In Proceedings of the sixth Arabic natural language processing workshop, pages 312–317.
  14. Mahmoud Al-Ayyoub, Abed Allah Khamaiseh, Yaser Jararweh, and Mohammed N Al-Kabi. 2019. A comprehensive survey of arabic sentiment analysis. Information Processing & Management, 56(2):320–342. ↗
  15. Muhammad Abdul-Mageed and Lyle Ungar. 2017. EmoNet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 718–728.
  16. Tareq Al-Moslmi, Mohammed Albared, Adel Al-Shabi, Nazlia Omar, and Salwani Abdullah. 2018. Arabic senti-lexicon: Constructing publicly available language resources for arabic sentiment analysis. Journal of Information Science, 44(3):345–362. ↗
  17. Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  18. Ahmad Al Sallab, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim El Hajj, and Khaled Bashir Shaban. 2015. Deep learning models for sentiment analysis in arabic. In Proceedings of the second workshop on Arabic natural language processing, pages 9–17. ↗
  19. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The madar shared task on arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199– 207.
  20. Mohammad Al-Smadi, Bashar Talafha, Mahmoud Al-Ayyoub, and Yaser Jararweh. 2019. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics, 10(8):2163–2175. ↗
  21. Rui Cao, Yihao Wang, Yuxin Liang, Ling Gao, Jie Zheng, Jie Ren, and Zheng Wang. 2022. Exploring the impact of negative samples of contrastive learning: A case study of sentence embedding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3138–3152, Dublin, Ireland. Association for Computational Linguistics. find
  22. Muath Alali, Nurfadhlina Mohd Sharef, Masrah Azrifah Azmi Murad, Hazlina Hamdan, and Nor Azura Husin. 2022. Multitasking learning model based on hierarchical attention network for arabic sentiment analysis classification. Electronics, 11(8):1193. ↗
  23. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020a. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR. ↗
  24. Hassan Alhuzali, Muhammad Abdul-Mageed, and Lyle Ungar. 2018. Enabling deep learning of emotion with first-person seed expressions. In Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, pages 25–35.
  25. Ting Chen, Yizhou Sun, Yue Shi, and Liangjie Hong. 2017. On sampling strategies for neural network-based collaborative filtering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, page 767–776, New York, NY, USA. Association for Computing Machinery. find
  26. Badr AlKhamissi and Mona Diab. 2022. Meta AI at Arabic hate speech 2022: MultiTask learning with self-correction for hate speech classification. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 186–193, Marseille, France. European Language Resources Association. ↗
  27. Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020b. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297. ↗
  28. Latifah Almuqren and Alexandra Cristea. 2021. Aracust: a saudi telecom tweets corpus for sentiment analysis. PeerJ Computer Science, 7:e510. ↗
  29. Ryan Cotterell and Chris Callison-Burch. 2014. A multi-dialect, multi-genre corpus of informal written arabic. In LREC, pages 241–245.
  30. Ali Alshehri, Muhammad Abdul-Mageed, et al. 2020. Understanding and detecting dangerous speech in social media. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 40–47. ↗
  31. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  32. Mahmoud El-Haj. 2020. Habibi-a multi dialect multi national arabic song lyrics corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 1318–1326.
  33. Salaheddin Alzu’bi, Thiago Castro Ferreira, Lucas Pavanelli, and Mohamed Al-Badrashiny. 2022. aixplain at arabic hate speech 2022: An ensemble based approach to detecting offensive tweets.
  34. Heba Elfardy and Mona T Diab. 2013. Sentence level dialect identification in arabic. In ACL (2), pages 456–461.
  35. David Bamman and Noah Smith. 2015. Contextualized sarcasm detection on twitter. In proceedings of the international AAAI conference on web and social media, volume 9, pages 574–577. find
  36. AbdelRahim Elmadany, Chiyu Zhang, Muhammad Abdul-Mageed, and Azadeh Hashemi. 2020a. Leveraging affective bidirectional transformers for offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 102–108. ↗
  37. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on AbdelRahim Elmadany, Chiyu Zhang, Muhammad Abdul-Mageed, and Azadeh Hashemi. 2020b. Leveraging affective bidirectional transformers for offensive language detection. In The 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT4), LREC.
  38. Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In International Conference on Learning Representations. ↗
  39. Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE. find
  40. Hady ElSahar and Samhaa R El-Beltagy. 2015. Building large arabic multi-domain resources for sentiment analysis. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 23–34. Springer. find
  41. Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-hsuan Sung, Laszlo Lukacs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. find
  42. Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. 2020. Cert: Contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005.12766. ↗
  43. Fatemah Husain and Ozlem Uzuner. 2022. Investigating the effect of preprocessing arabic text on offensive language and hate speech detection. Transactions on Asian and Low-Resource Language Information Processing, 21(4):1–20. ↗
  44. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198.
  45. Ibrahim Abu Farha and Walid Magdy. 2020. From Arabic Sentiment Analysis to Sarcasm Detection: The ArSarcasm Dataset. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39.
  46. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  47. Atefeh Farzindar and Diana Inkpen. 2015. Natural language processing for social media. Synthesis Lectures on Human Language Technologies, 8(2):1–166. ↗
  48. Sverker Janson, Evangelina Gogoulou, Erik Ylipää, Amaru Cuba Gyllensten, and Magnus Sahlgren. 2021. Semantic re-tuning with contrastive tension. In International Conference on Learning Representations, 2021. find
  49. Chayma Fourati, Abir Messaoudi, and Hatem Haddad. 2020. Tunizi: a tunisian arabizi sentiment analysis dataset. In AfricaNLP Workshop, Putting Africa on the NLP Map. ICLR 2020, Virtual Event, volume arXiv:3091079. ↗
  50. Jihen Karoui, Farah Banamara Zitoune, and Veronique Moriceau. 2017. Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168. ↗
  51. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  52. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online. Association for Computational Linguistics. ↗
  53. Bilal Ghanem, Jihen Karoui, Farah Benamara, Véronique Moriceau, and Paolo Rosso. 2019. Idat@fire2019: Overview of the track on irony detection in arabic tweets. In Mehta P., Rosso P., Majumder P., Mitra M. (Eds.) Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2019). CEUR Workshop Proceedings. In: CEUR-WS.org, Kolkata, India, December 12-15. find
  54. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661–18673. Curran Associates, Inc. ↗
  55. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019. Learning dense representations for entity retrieval. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 528–537, Hong Kong, China. Association for Computational Linguistics.
  56. Tassilo Klein and Moin Nabi. 2022. SCD: Self-contrastive decorrelation of sentence embeddings. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 394–400, Dublin, Ireland. Association for Computational Linguistics. ↗
  57. Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. In International Conference on Learning Representations, volume abs/1803.02893. ↗
  58. Hamdy Mubarak, Sabit Hassan, and Shammur Absar Chowdhury. 2022c. Emojis as anchors to detect arabic offensive language and hate speech. arXiv preprint arXiv:2201.06723. ↗
  59. Asmaa Mansour Khoudja, Mourad Loukam, and Fatma Zohra Belkredim. 2022. Towards author profiling from modern standard arabic texts: A review. In Proceedings of Sixth International Congress on Information and Communication Technology, pages 745–753. Springer. ↗
  60. Mahmoud Nabil, Mohamed Aly, and Amir F Atiya. 2015. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2515– 2519. ↗
  61. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2022. AraT5: Text-to-text transformers for Arabic language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 628–647, Dublin, Ireland. Association for Computational Linguistics.
  62. Alaa Mansy, Sherine Rady, and Tarek Gharib. 2022. An ensemble deep learning approach for emotion detection in arabic tweets. International Journal of Advanced Computer Science and Applications, 13(4). ↗
  63. Yu Meng, Chenyan Xiong, Payal Bajaj, Paul Bennett, Jiawei Han, Xia Song, et al. 2021. Coco-lm: Correcting and contrasting text sequences for language model pretraining. Advances in Neural Information Processing Systems, 34:23102–23114. ↗
  64. Mohamed Aziz Ben Nessir, Malek Rhouma, Hatem Haddad, and Chayma Fourati. 2022. icompass at arabic hate speech 2022: Detect hate speech using qrnn and transformers.
  65. Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval-2018 task 1: Affect in tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 1–17, New Orleans, Louisiana. Association for Computational Linguistics.
  66. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. ↗
  67. Lin Pan, Chung-Wei Hang, Avirup Sil, and Saloni Potdar. 2022. Improved text classification via contrastive adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11130–11138. find
  68. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pages 31–41.
  69. Yao Qiu, Jinchao Zhang, and Jie Zhou. 2021. Improving gradient-based adversarial training for text classification by contrastive learning and auto-encoder. arXiv preprint arXiv:2109.06536. ↗
  70. Hamdy Mubarak, Hend Al-Khalifa, and AbdulMohsen Al-Thubaity. 2022a. Overview of osact5 shared task on arabic offensive language and hate speech detection.
  71. Hamdy Mubarak, Shammur Absar Chowdhury, and Firoj Alam. 2022b. Arabgend: Gender analysis and inference on arabic twitter. arXiv preprint arXiv:2203.00271. ↗
  72. Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, and Weizhu Chen. 2020. Coda: Contrast-enhanced and diversity-promoting data augmentation for natural language understanding. arXiv preprint arXiv:2010.08670. ↗
  73. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on Arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56, Vancouver, BC, Canada. Association for Computational Linguistics.
  74. Francisco Rangel, Paolo Rosso, Anis Charfi, Wajdi Zaghouani, Bilal Ghanem, and Javier Sánchez-Junquera. 2019. Overview of the track on author profiling and deception detection in arabic. Working Notes of FIRE 2019. CEUR-WS. org, vol. 2517, pages 70–83. find
  75. Hamdy Mubarak, Kareem Darwish, Walid Magdy, Tamer Elsayed, and Hend Al-Khalifa. 2020. Overview of OSACT4 Arabic offensive language detection shared task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 48–52, Marseille, France. European Language Resource Association.
  76. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3). ↗
  77. Hager Saleh, Sherif Mostafa, Abdullah Alharbi, Shaker El-Sappagh, and Tamim Alkhalifah. 2022. Heterogeneous ensemble deep learning model for enhanced arabic sentiment analysis. Sensors, 22(10):3707. ↗
  78. Hamdy Mubarak, Sabit Hassan, and Ahmed Abdelali. 2021. Adult content detection on Arabic Twitter: Analysis and experiments. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 136–144, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. find
  79. Ahmad Shapiro, Ayman Khalafallah, and Marwan Torki. 2022. Alexu-aic at arabic hate speech 2022: Contrast to classify. arXiv preprint arXiv:2207.08557. ↗
  80. Yixuan Su, Fangyu Liu, Zaiqiao Meng, Tian Lan, Lei Shu, Ehsan Shareghi, and Nigel Collier. 2022. TaCL: Improving BERT pre-training with token-aware contrastive learning. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2497–2507, Seattle, United States. Association for Computational Linguistics. find
  81. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗
  82. Chiyu Zhang and Muhammad Abdul-Mageed. 2019. No army, no navy: Bert semi-supervised learning of arabic dialects. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 279– 284.
  83. Varsha Suresh and Desmond Ong. 2021. Not all negatives are equal: Label-aware contrastive loss for fine-grained text classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4381–4394, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  84. Chiyu Zhang and Muhammad Abdul-Mageed. 2022. Improving social meaning detection with pragmatic masking and surrogate fine-tuning. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 141–156. find
  85. Jenny A Thomas. 2014. Meaning in interaction: An introduction to pragmatics. Routledge. find
  86. Chiyu Zhang, Muhammad Abdul-Mageed, and Ganesh Jawahar. 2022a. Infodcl: A distantly supervised contrastive learning framework for social meaning. arXiv preprint arXiv:2203.07648. ↗
  87. Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018. Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 39– 50.
  88. Chiyu Zhang, Muhammad Abdul-Mageed, and El Moatez Billah Nagoudi. 2022b. Decay no more: A persistent twitter dataset for learning social meaning. Workshop Proceedings of the 16th International AAAI Conference on Web and Social Media. ↗
  89. Al-Jamal Wafa’Q, Ahmad M Mustafa, and Mostafa Z Ali. 2022. Sarcasm detection in arabic short text using deep learning. In 2022 13th International Conference on Information and Communication Systems (ICICS), pages 362–366. IEEE. find
  90. Kun Zhou, Beichen Zhang, Xin Zhao, and Ji-Rong Wen. 2022. Debiased contrastive learning of unsupervised sentence representations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6120– 6130, Dublin, Ireland. Association for Computational Linguistics. find
  91. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88–93.
  92. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  93. Wajdi Zaghouani and Anis Charfi. 2018. Arap-tweet: A large multi-dialect twitter corpus for gender, age and language variety identification. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find
  94. Omar F Zaidan and Chris Callison-Burch. 2011. The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37–41. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khondaker-2022-benchmark-study-contrastive-learning-arabic/