Elamur Research⌕
☰
home / papers / 2023 / USTHB at NADI 2023 shared task: Exploring Preprocessing and Feature Engineering Strategies for Arabic Dialect Identification

USTHB at NADI 2023 shared task: Exploring Preprocessing and Feature Engineering Strategies for Arabic Dialect Identification

Paper · 2023 · Mohamed Lichouri, Khaled Lounnas, Aicha Zitouni et al.

Summary

The supplied text does not match the title. The title names a NADI 2023 dialect identification paper, but the extracted PDF is the same group's ArAIEval 2023 entry on Arabic disinformation detection in tweets. This summary covers the text actually provided.

The authors tackle two subtasks: Task2A (binary disinformation vs. not) and Task2B (fine-grained multiclass labels). Their system is a classical pipeline with four modular phases:

  • surface cleanup;
  • morphological reduction;
  • feature extraction with FastText or TF-IDF over word, char and char_wb analyzers;
  • a weighted fusion of the three TF-IDF views.

All features feed a Linear SVC. They ran four experiment families, each centred on one phase, and tuned n-gram ranges, TF-IDF weights and vocabulary sizes on the development set.

On dev, surface preprocessing scored best, with weighted fusion and morphological preprocessing close behind. FastText embeddings lagged far behind. Official test scores were 76.70% (binary) and 50.46% (multiclass) F1-micro, against task averages of 77.96% and 64.85%. The work shows how far a tuned sparse baseline goes on noisy Arabic social-media text, and where class imbalance hurts it.

Key points

  • Data size: about 14,147 training tweets for Task2A and 2,656 for Task2B. Dev has 2,115/397 tweets and test has 3,729/876. The majority classes are "no-disinformation" and "HS"; "Rumor" and "Spam" are rare.

  • Surface steps: Arabic letter normalization and removal of punctuation, emojis, stop words, diacritics and non-Arabic content.

  • Morphological steps: lemmatization and stemming.

  • Dev F1-micro, binary:

    ConfigurationF1-micro
    Surface preprocessing81.23
    Weighted fusion81.13
    Morphological preprocessing81.08
    FastText48.45

  • Dev F1-micro, multiclass: the best score was about 57.4, and FastText dropped to roughly 27.6.

  • Search space:

    • n-gram ranges (m, n) with m from 1 to 3 and n from 1 to 10;
    • TF-IDF weights from 0.5 to 1;
    • max features from 1,000 to 25,000;
    • SVM with C=100;
    • FastText with 100 epochs (supervised 'ova' loss; skipgram, window 6, dimension 1000).

  • Test results: 76.70% binary and 50.46% multiclass, below the participant averages. The authors attribute the large multiclass drop to class imbalance.

Interesting ideas

  • I1. Rather than fixing one cleanup recipe, the authors try every ordering and subset of the surface normalization steps and let dev scores choose, turning preprocessing into a measured hyperparameter.
  • I2. Word, character and within-word-boundary character n-grams are kept as separate views, then merged with an explicit weight per view, so each view's contribution stays visible and tunable.
  • I3. A tuned sparse linear model clearly beat dense FastText features on this small, noisy dataset.
  • I4. The paper links the much weaker multiclass scores to skewed label distribution rather than to model choice. It also shows a visible gap between dev and test results.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): treat each normalization step as a toggle and ablate it against tagger and lemma accuracy, keeping only the steps that measurably help.
  • I2 (tokens): char_wb-style suffix and prefix features could be a separately weighted, inspectable feature group in the perceptron tagger for unknown English words.
  • I3 (learn): keep a simple transparent baseline as a standing control in every evaluation, so a heavier model must prove it beats it.
  • I4 (learn): report per-class results and the dev-vs-test gap alongside headline scores, so imbalance and overfitting to dev are flagged rather than hidden.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Sciences and Technology Houari BoumedieneAlgiers · Algeria5 works · 30 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Mohamed Lichouri, Khaled Lounnas, Aicha Zitouni, Houda Latrache and Rachida Djeradi. 2023. USTHB at NADI 2023 shared task: Exploring Preprocessing and Feature Engineering Strategies for Arabic Dialect Identification. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.69

@inproceedings{lichouri2023usthb,
  title = {{USTHB at NADI 2023 shared task: Exploring Preprocessing and Feature Engineering Strategies for Arabic Dialect Identification}},
  author = {Mohamed Lichouri, Khaled Lounnas, Aicha Zitouni, Houda Latrache and Rachida Djeradi},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.69},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.69}
}

References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mourad Abbas, Mohamed Lichouri, and Abed Alhakim Freihat. 2019. St madar 2019 shared task: Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 269–273. ↗
  2. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Giovanni Da San Martino, and Preslav Nakov. 2022. Overview of the WANLP 2022 shared task on propaganda detection in Arabic. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 108–118, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  3. Jaffar Atwan, Mohammad Wedyan, Qusay Bsoul, Ahmad Hamadeen, Ryan Alturki, and Mohammed Ikram. 2021. The effect of using light stemming for arabic text classification. International Journal of Advanced Computer Science and Applications, 12(5). ↗
  4. Anoual El Kah and Imad Zeroual. 2021. The effects of pre-processing techniques on arabic text classification. Int. J, 10(1):1–12. find
  5. Khaled M Fouad, Sahar F Sabbeh, and Walaa Medhat. 2022. Arabic fake news detection using deep learning. Computers, Materials & Continua, 71(2). ↗
  6. Fouzi Harrag and Mohamed Khalil Djahli. 2022. Arabic fake news detection: A fact checking based deep learning approach. Transactions on Asian and Low-Resource Language Information Processing, 21(4):1– 34. ↗
  7. Maram Hasanain, Firoj Alam, Hamdy Mubarak, Samir Abdaljalil, Wajdi Zaghouani, Preslav Nakov, Giovanni Da San Martino, and Abed Alhakim Freihat. 2023. ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text. In Proceedings of the First Arabic Natural Language Processing Conference (ArabicNLP 2023), Singapore. Association for Computational Linguistics.
  8. Hanen Himdi, George Weir, Fatmah Assiri, and Hassanin Al-Barhamtoshy. 2022. Arabic fake news detection based on textual analysis. Arabian Journal for Science and Engineering, 47(8):10453–10469. ↗
  9. Khitam Mahmoud Abdalla Jbara, Azzam T Sleit, and Bassam H Hammo. 2009. Knowledge discovery in Al-Hadith using text classification algorithm. University of Jordan. find
  10. Mohamed Lichouri and Mourad Abbas. 2020a. Simple vs oversampling-based classification methods for fine grained arabic dialect identification in twitter. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 250–256.
  11. Mohamed Lichouri, Mourad Abbas, Khaled Lounnas, Besma Benaziz, and Aicha Zitouni. 2021b. Arabic dialect identification based on a weighted concatenation of TF-IDF features. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 282–286, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  12. Hamdy Mubarak, Samir Abdaljalil, Azza Nassar, and Firoj Alam. 2023. Detecting and identifying the reasons for deleted tweets before they are posted. Frontiers in Artificial Intelligence, 6. ↗
  13. El Moatez Billah Nagoudi, AbdelRahim Elmadany, Muhammad Abdul-Mageed, Tariq Alhindi, and Hasan Cavusoglu. 2020. Machine generation and detection of arabic manipulated and fake news. arXiv preprint arXiv:2011.03092. ↗
  14. Ahmed Omar, Tarek M Mahmoud, Tarek Abd-El-Hafeez, and Ahmed Mahfouz. 2021. Multi-label arabic text classification in online social networks. Information Systems, 100:101785. ↗
  15. Rouhia M Sallam, Hamdy M Mousa, and Mahmoud Hussein. 2016. Improving arabic text categorization using normalization and stemming techniques. Int. J. Comput. Appl, 135(2):38–43. ↗
  16. Mohammed Shiha and Serkan Ayvaz. 2017. The effects of emoji in sentiment analysis. Int. J. Comput. Electr. Eng.(IJCEE.), 9(1):360–369. ↗
  17. Kai Shu, Amrita Bhattacharjee, Faisal Alatawi, Tahora H Nazer, Kaize Ding, Mansooreh Karami, and Huan Liu. 2020. Combating disinformation in a social media age. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(6):e1385. ↗
  18. Mohamed Lichouri and Mourad Abbas. 2020b. Speechtrans@ smm4h’20: Impact of preprocessing and ngrams on automatic classification of tweets that mention medications. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 118–120. find
  19. Mohamed Lichouri, Mourad Abbas, and Besma Benaziz. 2020. Profiling fake news spreaders on twitter based on tfidf features and morphological process. find
  20. Mohamed Lichouri, Mourad Abbas, Besma Benaziz, Aicha Zitouni, and Khaled Lounnas. 2021a. Preprocessing solutions for detection of sarcasm and sentiment for Arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 376–380, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  21. Mohamed Lichouri, Mourad Abbas, Abed Alhakim Freihat, and Dhiya El Hak Megtouf. 2018. Word-level vs sentence-level language identification: Application to algerian and arabic dialects. Procedia Computer Science, 142:246–253. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lichouri-2023-usthb-nadi-2023-shared-task/