Elamur Research⌕
☰
home / papers / 2021 / ASAD: Arabic Social media Analytics and unDerstanding

ASAD: Arabic Social media Analytics and unDerstanding

Paper · 2021 · Sabit Hassan, Hamdy Mubarak, Ahmed Abdelali et al.

Summary

Arabic Twitter content is rich in dialects and informal language, yet few public tools analyse it, and those that exist are scattered or trained on non-Twitter data. ASAD bundles seven classifiers for Arabic tweets into one service: dialect identification, sentiment, news category, offensive language, hate speech, adult content and spam. Each module is trained on an existing or in-house tweet or news dataset. The authors prioritised cheap deployment. They compared linear SVMs on tf-idf-weighted character n-grams against fine-tuned AraBERT and found the SVMs equal or within roughly 1–2 F1 points on six tasks. These SVMs need no GPU and stay only a few megabytes in size. Dialect identification was the exception: BERT led by a wider margin there, so it is the one module deployed with AraBERT. The toolkit is available through a browser interface, which supports small file uploads and shows a map heatmap for dialects, and through POST-based web APIs. It fills a gap in openly accessible Arabic social-media analytics.

Key points

  • Datasets used:
    • Dialect ID: QADI, covering 18 countries, with 540K automatically labelled training tweets and 3.3K tweets annotated by native speakers for testing.
    • Sentiment: ArSAS, 21K tweets, with Mixed and Neutral merged into one class.
    • Offensive language: the OffensEval 2020 data, 8K training and 2K test tweets.
    • Other modules: 6.9K tweets for hate speech, 50K for adult content and about 30K Aljazeera news items for news category.
  • Effect of n-gram range on offensiveness detection:
    • Character n-grams C[1-3] give a 3.9 MB model with F1 85.0.
    • C[2-7] grows the model to 120.5 MB for F1 87.4.
    • The small setting was chosen for deployment.
  • SVM versus AraBERT F1:
    • Dialect ID: 54.1 vs 60.6, so AraBERT was deployed.
    • Spam: 98.3 vs 98.9.
    • Adult content: 88.5 vs 88.1.
    • Hate speech: 75.2 vs 75.1.
  • The authors suggest the SVMs stay competitive because they were trained on tweets, while AraBERT was pretrained on more formal news and Wikipedia text.
  • Implementation details:
    • The backend uses Flask and scikit-learn, with class probabilities produced by Platt calibration.
    • File uploads are capped at 100 lines and protected by reCaptcha.
    • Dialect probabilities are drawn on a Leaflet/OpenStreetMap heatmap.

Interesting ideas

  • I1. For every module, sweep the feature configuration while recording both model size and score, then ship the smallest setup that is close to the best rather than the absolute top scorer.
  • I2. Train on large noisy labels but test only on a small human-checked set. Examples are QADI's automatic dialect tags, spam labels inherited from the posting account, and random tweets assumed to be negatives.
  • I3. Character n-grams cope well with spelling variation in informal text, which can let a light linear model match a large pretrained model trained on mismatched, formal data.
  • I4. Return calibrated class probabilities (via Platt scaling) along with every label instead of a bare decision.
  • I5. Choose the model per module: keep the cheap model wherever it suffices, and pay for the heavier one only where the measured gap is large.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation reports could pair each parser or tagger variant's accuracy with its size and latency, making the trade-off explicit when choosing what to deploy.
  • I2 (learn): Silver data agreed by witness parsers can be used freely for training, as long as scores are always reported on a separate gold set so silver noise does not inflate them.
  • I3 (tokens): For unknown or misspelled words, character n-gram features in the tagger and lemmatiser could give a robust fallback without hiding the reason for each decision.
  • I4 (tree): Calibrating arc and label scores from the perceptron parser and gparse would make their confidences comparable, which helps when merging parses and when explaining weak attachments.
  • I5 (tree): The perceptron parser could serve by default, with gparse called only for sentence types or constructions where evaluation shows a clear accuracy gap.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar141 works · 14,175 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Sabit Hassan, Hamdy Mubarak, Ahmed Abdelali and Kareem Darwish. 2021. ASAD: Arabic Social media Analytics and unDerstanding. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.eacl-demos.14

@inproceedings{hassan2021asad,
  title = {{ASAD: Arabic Social media Analytics and unDerstanding}},
  author = {Sabit Hassan, Hamdy Mubarak, Ahmed Abdelali and Kareem Darwish},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-demos.14},
  url = {https://doi.org/10.18653/v1/2021.eacl-demos.14}
}

References

24 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California. Association for Computational Linguistics.
  2. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2020. Arabic dialect identification in the wild. ArXiv, abs/2005.06557. ↗
  3. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. Nadi 2020: The first nuanced arabic dialect identification shared task. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110. ↗
  4. Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy. Association for Computational Linguistics.
  5. Nora Al Twairesh, Mawaheb Al Tuwaijri, Afnan Al Moammar, and Sarah Al Humoud. 2016. Arabic spam detection in twitter. In The 2nd Workshop on Arabic Corpora and Processing Tools 2016 Theme: Social Media, page 38. find
  6. Ali Alshehri, El Moatez Billah Nagoudi, Hassan Alhuzali, and Muhammad Abdul-Mageed. 2018. Think before your click: Data and models for adult content in arabic twitter. find
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of The 4th Workshop on Open-Source Arabic Corpora and Processing Tools, pages 9–15. ↗
  8. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics.
  9. Shammur Absar Chowdhury, Ahmed Abdelali, Kareem Darwish, Jung Soon-Gyo, Joni Salminen, and Bernard J. Jansen. 2020. Improving Arabic text categorization using transformer training diversification. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 226–236, Barcelona, Spain (Online). Association for Computational Linguistics.
  10. Kareem Darwish, Ahmed Abdelali, and Hamdy Mubarak. 2014. Using stem-templates to improve arabic pos and gender/number tagging. In LREC, pages 2926–2931. Citeseer.
  11. Kareem Darwish and Hamdy Mubarak. 2016. Farasa: A new fast and accurate arabic word segmenter. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074.
  12. AbdelRahim A. Elmadany, Hamdy Mubarak, and Walid Magdy. 2018. An arabic speech-act and sentiment corpus of tweets. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA). The 3rd Workshop on Open-Source Arabic Corpora and Processing Tools, OSACT3 ; Conference date: 08-05-2018. find
  13. Sabit Hassan, Younes Samih, Hamdy Mubarak, and Ahmed Abdelali. 2020a. ALT at SemEval-2020 task 12: Arabic and English offensive language identification in social media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1891–1897, Barcelona (online). International Committee for Computational Linguistics.
  14. Sabit Hassan, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, Ammar Rashed, and Shammur Absar Chowdhury. 2020b. ALT submission for OSACT shared task on offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 61–65, Marseille, France. European Language Resource Association.
  15. Hamdy Mubarak, Ahmed Abdelali, Sabit Hassan, and Kareem Darwish. 2020a. Spam detection on arabic twitter. In Social Informatics, pages 237–251, Cham. Springer International Publishing. find
  16. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. Abusive language detection on arabic social media. In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
  17. Hamdy Mubarak, Kareem Darwish, Walid Magdy, Tamer Elsayed, and Hend Al-Khalifa. 2020b. Overview of osact4 arabic offensive language detection shared task. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection.
  18. Hamdy Mubarak, Sabit Hassan, and Ahmed Abdelali. 2021. Adult content detection on arabic twitter: Analysis and experiments. In Proceedings of the Sixth Arabic Natural Language Processing Workshop. find
  19. Ossama Obeid, Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2019. ADIDA: Automatic dialect identification for Arabic. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 6–11, Minneapolis, Minnesota. Association for Computational Linguistics.
  20. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  21. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), pages 1094–1101, Reykjavik, Iceland. European Languages Resources Association (ELRA).
  22. John C. Platt. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In ADVANCES IN LARGE MAR-GIN CLASSIFIERS, pages 61–74. MIT Press. find
  23. Omar F Zaidan and Chris Callison-Burch. 2011. The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic With High Dialectal Content. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), pages 37–41.
  24. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hassan-2021-asad-arabic-social-media-analytics/