Elamur Research⌕
☰
home / papers / 2020 / NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets

NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets

Paper · 2020 · Hamada Nayel

Summary

This system paper describes a submission to OffensEval 2020 (SemEval-2020 Task 12) for the Arabic track. The task asks whether a tweet contains offensive language, treated as a two-class problem with the labels NOT and OFF. The approach is deliberately simple and classical. Tweets are first cleaned by removing placeholder tokens, punctuation and digits, and runs of repeated letters. Each tweet then becomes a TF/IDF vector over word unigrams, bigrams and trigrams. Several classifiers are trained on these vectors, and the one that scores best on the development set is submitted, since the task allowed only a single run. The winner is a linear classifier with hinge loss optimized by Stochastic Gradient Descent (SGD). It reaches an F1 of 84.21% on dev and 81.82% on test. For comparison, the strongest and weakest systems in the competition scored 90.17% and 44.51% on test. The work sets a useful mid-table baseline and shows how far a light pipeline with no external resources can go. The author attributes the remaining gap to local sparse features and the absence of embeddings or extra data.

Key points

  • Data come from the organizers' Arabic offensive tweet dataset (Mubarak et al., 2020), split into train, dev and test; the test portion has 2000 unlabeled tweets.
  • Preprocessing removes @USER, URL and <LF>, drops punctuation and both Western and Arabic-Indic digits, and shrinks any letter repeated more than twice.
  • Features are TF/IDF weights over word n-grams with n from 1 to 3.
  • Dev/test F1 by classifier:
    • Linear SGD with hinge loss: 0.8421 / 0.8182
    • Linear-kernel SVM: 0.8115 / 0.8043
    • MLP: 0.8033 / 0.7831
    • Hard-voting ensemble: 0.8265 / 0.8129
  • The paper is inconsistent about the MLP: the text gives 20 hidden neurons with logistic activation, while the table labels it n = 60. Also, the training/dev statistics table is empty in the extracted text.

Interesting ideas

  • I1. Collapsing expressive letter stretching (anything repeated more than twice) is a cheap normalization that shrinks the vocabulary of noisy user text without needing a dictionary.
  • I2. When only one submission is allowed, several model families are compared on dev and the single best is chosen, and the dev ranking largely held on test.
  • I3. A hard-voting ensemble of all classifiers did not beat the best single member, because adding weaker voters can dilute a strong one.
  • I4. A plain TF/IDF n-gram classifier with no external resources gives a transparent reference point that more structured systems must clearly exceed.
  • I5. Platform placeholders such as user mentions, links and line-break markers are stripped as content-free noise before features are built.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A similar rule could map English forms like "soooo" or "nooooo" to a canonical spelling before lemma lookup, while keeping a flag that the word was stretched.
  • I2 (learn): This supports a strict rule of picking parser or rule variants on held-out dev data only, then reporting the test score once.
  • I3 (learn): When combining witness parsers for silver data, members should be weighted by measured reliability rather than given equal votes, and the agreed output should be checked against the best single witness.
  • I4 (learn): Such a bag-of-n-grams model could serve as a negative-control baseline on Mova's classification-like evaluations, showing that frame- and tree-based reasoning adds value beyond surface word statistics.
  • I5 (tokens): Mova's tokenizer could recognize these placeholders as dedicated token types, so they do not distort tagging or parsing of the real sentence.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Benha UniversityBanhā · Egypt14 works · 89 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksCommunication · 97 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

💡 Concepts 1Key ideas and methods explained on our concept pages.TF-IDF · 26 works

Cite

Cite as Hamada Nayel. 2020. NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets. Proceedings of the Fourteenth Workshop on Semantic Evaluation.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.semeval-1.276

@inproceedings{nayel2020nayel,
  title = {{NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets}},
  author = {Hamada Nayel},
  booktitle = {Proceedings of the Fourteenth Workshop on Semantic Evaluation},
  year = {2020},
  doi = {10.18653/v1/2020.semeval-1.276},
  url = {https://doi.org/10.18653/v1/2020.semeval-1.276}
}

References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Naaima Boudad, Rdouan Faizi, Rachid [Oulad Haj Thami], and Raddouane Chiheb. 2018. Sentiment analysis in arabic: A review of the literature. Ain Shams Engineering Journal, 9(4):2479 – 2490. ↗
  2. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of ICWSM. ↗
  3. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2019. Arabic natural language processing: An overview. Journal of King Saud University - Computer and Information Sciences. Ritesh Kumar, Atul Kr. ↗
  4. Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking Aggression Identification in Social Media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbulling (TRAC), Santa Fe, USA.
  5. Irene Kwok and Yuzhou Wang. 2013. Locate the hate: Detecting tweets against blacks. In AAAI Conference on Artificial Intelligence. find
  6. Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1–16. ↗
  7. Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of the 11th Forum for Information Retrieval Evaluation, pages 14–17. find
  8. Hamdy Mubarak, Darwish Kareem, and Magdy Walid. 2017. Abusive Language Detection on Arabic Social Media. In Proceedings of the Workshop on Abusive Language Online (ALW), Vancouver, Canada.
  9. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192. ↗
  10. Hamada A. Nayel and Shashirekha H. L. 2019. DEEP at HASOC2019: A machine learning framework for hate speech and offensive language detection. In Parth Mehta, Paolo Rosso, Prasenjit Majumder, and Mandar Mitra, editors, Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 336–343. CEUR-WS.org. find
  11. Hamada A. Nayel and H. L. Shashirekha. 2017. Mangalore-University@INLI-FIRE-2017: Indian Native Language Identification using Support Vector Machines and Ensemble Approach. In Prasenjit Majumder, Mandar Mitra, Parth Mehta, and Jainisha Sankhavara, editors, Working notes of FIRE 2017 - Forum for Information Retrieval Evaluation, Bangalore, India, December 8-10, 2017., volume 2036 of CEUR Workshop Proceedings, pages 106–109. CEUR-WS.org. find
  12. Hamada A. Nayel, Walaa Medhat, and Metwally Rashad. 2019. BENHA@IDAT: Improving Irony Detection in Arabic Tweets using Ensemble Approach. In Parth Mehta, Paolo Rosso, Prasenjit Majumder, and Mandar Mitra, editors, Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 401–408. CEUR-WS.org, December. find
  13. Hamada A. Nayel. 2019. NAYEL@APDA: Machine Learning Approach for Author Profiling and Deception Detection in Arabic Texts. In Parth Mehta, Paolo Rosso, Prasenjit Majumder, and Mandar Mitra, editors, Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 92–99. CEUR-WS.org, December. find
  14. Lorenzo Rosasco, Ernesto De Vito, Andrea Caponnetto, Michele Piana, and Alessandro Verri. 2004. Are loss functions all the same? Neural Comput., 16(5):1063–1076, May. ↗
  15. Sergios Theodoridis and Konstantinos Koutroumbas. 2009. Chapter 3 - Linear Classifiers. In Pattern Recognition (Fourth Edition), pages 91 – 150. Academic Press, Boston, fourth edition edition. find
  16. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. In Proceedings of the First Workshop on Abusive Langauge Online.
  17. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019a. Predicting the Type and Target of Offensive Posts in Social Media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 1415–1420.
  18. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019b. SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval).
  19. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nayel-2020-nayel-semeval-2020-task-12/