Elamur Research⌕
☰
home / papers / 2020 / AraNet: A Deep Learning Toolkit for Arabic Social Media

AraNet: A Deep Learning Toolkit for Arabic Social Media

Paper · 2020 · Muhammad Abdul-Mageed, Chiyu Zhang, Azadeh Hashemi et al.

Summary

Arabic has about 400 million native speakers and many dialects, but it had no single toolkit for inferring social attributes from social media posts. Earlier work focused mostly on sentiment and dialect identification, so tasks such as age and gender detection had few tools. The authors built AraNet, a publicly released Python library that covers six prediction tasks: dialect, age, gender, sentiment, emotion and irony.

Every task uses the same approach. BERT-Base Multilingual Cased is fine-tuned on a task-specific labeled set with one shared hyper-parameter recipe and no hand-crafted features. Only dialect identification adds self-training on confident predictions over unlabeled data. The training data are mostly tweets, from public sets such as Arap-Tweet and a new in-house UBC Twitter Gender corpus, plus some cleaner news-style data for sentiment.

On Arab-Tweet, fine-tuned BERT beats a small GRU baseline by roughly 15 accuracy points for age and 12 for gender. The main contribution is a uniform set of strong, reproducible baselines. Shared tweet collections lose posts over time, so comparing against released models is a practical substitute for stable benchmark data.

Key points

  • Backbone: BERT-Base Multilingual Cased, covering 104 languages, with 12 layers, 768 hidden units, 12 heads, 110M parameters and a 119,547-entry WordPiece vocabulary.
  • Shared fine-tuning settings:
    • maximum length of 50 tokens, batch size 32, learning rate 2e-5;
    • 15 epochs, keeping the checkpoint that scores best on the development set;
    • the sentiment model alone used PyTorch with learning rate 2e-6.
  • Arab-Tweet results, using an 80/10/10 split:
    • the small-GRU baseline reached 36.29% accuracy for age and 53.37% for gender;
    • BERT added 15.13 points for age and 11.93 points for gender.
  • UBC Twitter Gender corpus:
    • 1,989 users from all 21 Arab countries were labeled by hand;
    • "unknown" users were dropped and the classes balanced to 528 male and 528 female users;
    • the split was made by user, and BERT scored 62.42% on DEV and 60.54% on TEST.
  • Preprocessing removes usernames, URLs and diacritics, and shortens any run of a repeated character to two. Dialect and irony models come from the authors' earlier work, including their winning MADAR shared-task system.

Interesting ideas

  • I1. When shared datasets decay (tweets that can only be redistributed as IDs gradually disappear), releasing trained models allows comparisons without the original data.
  • I2. Applying one fixed fine-tuning recipe to every task means score differences reflect the data and the task rather than per-task tuning effort.
  • I3. The gender corpus is split by user rather than by individual post, which stops one author's writing style from appearing in both train and test.
  • I4. The authors self-train with several confidence thresholds and vary how much predicted data to add, keeping only the setting that scores best.
  • I5. Shortening elongated characters to two copies cleans noisy informal text without fully erasing that emphasis was present.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Keep frozen, versioned checkpoints of past parsers and evaluate new ones against them on fresh text, so progress can be measured even if an old silver corpus can no longer be rebuilt.
  • I2 (tree): Train gparse with a single locked configuration across all treebanks and UD dialect conversions, so that gaps between treebanks point to annotation or conversion problems rather than hyper-parameter luck.
  • I3 (learn): Split silver data and school-test material by source document or author, so evaluation scores are not inflated by near-duplicates.
  • I4 (learn): Tune how much witness-agreed silver data to add, and at what agreement level, by sweeping these settings and checking each against a negative control rather than fixing one cutoff.
  • I5 (tokens): Normalize forms like "sooooo" to "soo" before lemma lookup and record the original form, so the lemmatizer stays robust and prag can still read the emphasis as a cue for affect or irony.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of British ColumbiaVancouver · Canada98 works · 34,805 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Muhammad Abdul-Mageed, Chiyu Zhang, Azadeh Hashemi and El Moatez Billah Nagoudi. 2020. AraNet: A Deep Learning Toolkit for Arabic Social Media. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abdulmageed2020aranet,
  title = {{AraNet: A Deep Learning Toolkit for Arabic Social Media}},
  author = {Muhammad Abdul-Mageed, Chiyu Zhang, Azadeh Hashemi and El Moatez Billah Nagoudi},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection},
  year = {2020},
  url = {https://aclanthology.org/2020.osact-1.3/}
}

References

48 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  2. Abdul-Mageed, M. and Diab, M. T. (2012). Awatif: A multi-genre corpus for modern standard arabic subjectivity and sentiment analysis. In LREC, volume 515, pages 3907–3914. Citeseer.
  3. Abdul-Mageed, M., Korayem, M., and YoussefAgha, A. (2011). “yes we can?”: Subjectivity annotation and tagging for the health domain. In Proceedings of RANLP2011. find
  4. Abdul-Mageed, M., Diab, M., and Kübler, S. (2013). Asma: A system for automatic segmentation and morpho-syntactic disambiguation of modern standard arabic. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, pages 1–8.
  5. Abdul-Mageed, M., Diab, M., and Kübler, S. (2014a). Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language, 28(1):20–37. ↗
  6. Abdul-Mageed, M., Diab, M., and Kübler, S. (2014b). Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language, 28(1):20–37. ↗
  7. Abdul-Mageed, M., AlHuzli, H., and Duaa’Abu Elhija, M. D. (2016). Dina: A multidialect dataset for arabic emotion analysis. In The 2nd Workshop on Arabic Corpora and Processing Tools, page 29. find
  8. Abdul-Mageed, M. (2019). Modeling arabic subjectivity and sentiment in lexical space. Information Processing & Management, 56(2):291–307. find
  9. Abdulla, N., Mahyoub, N., Shehab, M., and Al-Ayyoub, M. (2013). Arabic sentiment analysis: Corpus-based and lexicon-based. In Proceedings of The IEEE conference on Applied Electrical Engineering and Computing Technologies (AEECT). find
  10. Abumalloh, R. A., Al-Sarhan, H. M., Ibrahim, O., and Abu-Ulbeh, W. (2016). Arabic part-of-speech tagging. Journal of Soft Computing and Decision Support Systems, 3(2):45–52. find
  11. Al-Ayyoub, M., Khamaiseh, A. A., Jararweh, Y., and Al-Kabi, M. N. (2019). A comprehensive survey of arabic sentiment analysis. Information Processing & Management, 56(2):320– 342. ↗
  12. Al-Moslmi, T., Albared, M., Al-Shabi, A., Omar, N., and Abdullah, S. (2018). Arabic senti-lexicon: Constructing publicly available language resources for arabic sentiment analysis. Journal of Information Science, 44(3):345–362. ↗
  13. Al Sallab, A., Hajj, H., Badaro, G., Baly, R., El Hajj, W., and Shaban, K. B. (2015). Deep learning models for sentiment analysis in arabic. In Proceedings of the second workshop on Arabic natural language processing, pages 9–17. ↗
  14. Al-Smadi, M., Talafha, B., Al-Ayyoub, M., and Jararweh, Y. (2019). Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics, 10(8):2163–2175. ↗
  15. Alhuzali, H., Abdul-Mageed, M., and Ungar, L. (2018). Enabling deep learning of emotion with first-person seed expressions. In Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, pages 25–35.
  16. Alomari, K. M., ElSherif, H. M., and Shaalan, K. (2017). Arabic tweets sentimental analysis using machine learning. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pages 602–610. Springer. find
  17. Alshehri, A., Nagoudi, E. M. B., and Abdul-Mageed, M. (2020). Understanding and detecting dangerous speech in social media. In The 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT4), LREC. ↗
  18. Aly, M. and Atiya, A. (2013). Labr: A large scale arabic book reviews dataset. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 494–498. ↗
  19. Baly, R., Khaddaj, A., Hajj, H., El-Hajj, W., and Shaban, K. B. (2019). Arsentd-lev: A multi-topic corpus for target-based sentiment analysis in arabic levantine tweets. arXiv preprint arXiv:1906.01830. ↗
  20. Bouamor, H., Habash, N., Salameh, M., Zaghouani, W., Rambow, O., Abdulrahim, D., Obeid, O., Khalifa, S., Eryani, F., Erdmann, A., et al. (2018). The madar arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find
  21. Bouamor, H., Hassan, S., and Habash, N. (2019a). The madar shared task on arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP19), Florence, Italy.
  22. Bouamor, H., Hassan, S., and Habash, N. (2019b). The MADAR Shared Task on Arabic Fine-Grained Dialect Identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP19), Florence, Italy.
  23. Buckwalter, T. (2002). Buckwalter arabic morphological analyzer version 1.0. Linguistic Data Consortium, University of Pennsylvania. find
  24. Cotterell, R. and Callison-Burch, C. (2014). A multi-dialect, multi-genre corpus of informal written arabic. In LREC, pages 241–245.
  25. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  26. Diab, M., Hacioglu, K., and Jurafsky, D. (2004). Automatic tagging of arabic text: From raw text to base phrase chunks. In Proceedings of HLT-NAACL 2004: Short papers, pages 149– 152. Association for Computational Linguistics.
  27. Elfardy, H. and Diab, M. T. (2013). Sentence level dialect identification in arabic. In ACL (2), pages 456–461.
  28. Elmadany, A., Zhang, C., Abdul-Mageed, M., and Hashemi, A. (2020). Leveraging affective bidirectional transformers for offensive language detection. In The 4th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT4), LREC. ↗
  29. ElSahar, H. and El-Beltagy, S. R. (2015). Building large arabic multi-domain resources for sentiment analysis. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 23–34. Springer. find
  30. Farha, I. A. and Magdy, W. (2019). Mazajak: An online arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198.
  31. Ghanem, B., Karoui, J., Benamara, F., Moriceau, V., and Rosso, P. (2019). Idat@fire2019: Overview of the track on irony detection in arabic tweets. In Mehta P., Rosso P., Majumder P., Mitra M. (Eds.) Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2019). CEUR Workshop Proceedings. In: CEUR-WS.org, Kolkata, India, December 12-15. find
  32. Karoui, J., Zitoune, F. B., and Moriceau, V. (2017). Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168. find
  33. Kiritchenko, S., Mohammad, S., and Salameh, M. (2016). Semeval-2016 task 7: Determining sentiment intensity of english and arabic phrases. In Proceedings of the 10th international workshop on semantic evaluation (SEMEVAL-2016), pages 42–51.
  34. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  35. Mohammad, S., Bravo-Marquez, F., Salameh, M., and Kiritchenko, S. (2018). Semeval-2018 task 1: Affect in tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 1–17.
  36. Nabil, M., Aly, M., and Atiya, A. (2015). Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2515–2519. ↗
  37. Pasha, A., Al-Badrashiny, M., Diab, M. T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In LREC, volume 14, pages 1094–1101.
  38. Pavlinek, M. and Podgorelec, V. (2017). Text classification method based on self-training and lda topic models. Expert Systems with Applications, 80:83–93. ↗
  39. Rangel, F., Rosso, P., Charfi, A., Zaghouani, W., Ghanem, B., and Sánchez-Junquera, J. (2019). Overview of the track on author profiling and deception detection in arabic. In Mehta P., Rosso P., Majumder P., Mitra M. (Eds.) Working Notes of the Forum for Information Retrieval Evaluation (FIRE 2019). CEUR Workshop Proceedings. In: CEUR-WS.org, Kolkata, India, December 12-15. find
  40. Rosenthal, S., Farra, N., and Nakov, P. (2017). Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017), pages 502–518.
  41. Salameh, M., Mohammad, S., and Kiritchenko, S. (2015). Sentiment after translation: A case-study on arabic social media posts. In Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: Human language technologies, pages 767–777.
  42. Triguero, I., Garcı́a, S., and Herrera, F. (2015). Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study. Knowledge and Information Systems, 42(2):245–284, Feb. find
  43. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, pages 6000–6010.
  44. Zaghouani, W. and Charfi, A. (2018). Arap-tweet: A large multi-dialect twitter corpus for gender, age and language variety identification. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find
  45. Zaidan, O. F. and Callison-Burch, C. (2011). The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37–41. Association for Computational Linguistics.
  46. Zaidan, O. F. and Callison-Burch, C. (2014). Arabic dialect identification. Computational Linguistics, 40(1):171–202. ↗
  47. Zhang, C. and Abdul-Mageed, M. (2019a). Multi-task bidirectional transformer representations for irony detection. In The 11th meeting of the Forum for Information Retrieval Evaluation 2019. ↗
  48. Zhang, C. and Abdul-Mageed, M. (2019b). No army, no navy: Bert semi-supervised learning of arabic dialects. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 279–284.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mageed-2020-aranet-deep-learning-toolkit-arabic/