Elamur Research⌕
☰
home / papers / 2026 / Uslub at AbjadAuthorID Shared Task: A Comparative Analysis of Traditional Machine Learning and Transformer-Based Models for Authorship Attribution in Arabic and Urdu

Uslub at AbjadAuthorID Shared Task: A Comparative Analysis of Traditional Machine Learning and Transformer-Based Models for Authorship Attribution in Arabic and Urdu

Paper · 2026 · Shahad Alsuhaibani and Mohamed Alkaoud

Summary

This paper presents a King Saud University team's entry to the AbjadNLP shared task on authorship attribution. The task is to identify which known author wrote a paragraph, for two languages that share the Arabic script but are structurally unrelated: Arabic (Semitic) and Urdu (Indo-Aryan). The organizers supplied paragraph-segmented book data. Arabic covers 21 authors with 10 books each, and Urdu covers 10 authors. Both sets are clearly imbalanced across authors.

The authors compare three families of methods:

  • a linear SVM over unigram+bigram TF-IDF features;
  • fine-tuned AraBERT (bert-base-arabertv02);
  • gpt-5-chat, used both zero-shot and as a few-shot reranker over the top-3 candidates from the strongest classifier.

AraBERT works best for Arabic. For Urdu, the Arabic-pretrained encoder transfers poorly, and the simple lexical SVM is the most reliable option. Zero-shot prompting is near chance, but the reranking setup almost matches the best Arabic model. The submitted systems placed 6th (Arabic) and 1st (Urdu). The main lesson is that a shared script does not imply shared models, and cheap sparse baselines remain competitive.

Key points

  • Data: Arabic has 21 authors (10 public books each, split into paragraphs). Urdu has 10 authors. Both have train/validation/test splits with strong class imbalance.
  • Arabic validation results: AraBERT reached 87.75 accuracy and 81.95 macro F1. SVM+TF-IDF had higher accuracy (89.25) but lower macro F1 (79.12). GPT-5 zero-shot scored 13.70 / 10.75, and GPT-5 few-shot reranking scored 87.02 / 80.55.
  • Urdu validation results: SVM+TF-IDF reached 45.29 / 45.17. AraBERT collapsed to 22.94 / 17.11. GPT-5 scored 17.65 / 16.50 zero-shot and 33.82 / 38.14 few-shot.
  • Top-3 accuracy: the best retriever's top-3 accuracy was 95.26 (Arabic, AraBERT) and 67.35 (Urdu, SVM). Conditioned on the gold author being in the top 3, the LLM chose correctly 0.914 of the time in Arabic but only 0.502 in Urdu.
  • Blind test and setup: on the test set, Arabic AraBERT scored 0.869 accuracy / 0.836 F1, and Urdu SVM scored 0.355 / 0.395. AraBERT was trained with lr 5×10⁻⁶, batch 8 and max length 512, for 7 epochs (Arabic) or 3 epochs (Urdu). LLM prompts were written in English for both languages.

Interesting ideas

  • I1. LLM as a reranker: a cheap classifier first narrows the label space to three candidates, and the LLM then picks among them using two examples per candidate. This keeps the prompt short and avoids burying evidence in long contexts.
  • I2. Conditional accuracy to separate failures: dividing final accuracy by the retriever's top-k accuracy shows whether errors come from candidate retrieval or from the final decision step.
  • I3. Macro F1 alongside accuracy: in Arabic, the SVM won on accuracy but lost on macro F1, and the per-class confusion matrix exposed small authors being absorbed by larger ones.
  • I4. Shared surface form does not mean transfer: an Arabic-pretrained encoder failed on Urdu even though both use the same script, while a plain lexical baseline held up.
  • I5. Prompt in English, keep data native: the instructions stayed in English while the input text stayed in its original language, a choice the authors back with prior findings.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers could adjudicate only among the top-3 analyses proposed by witness parsers, with a couple of exemplar sentences per option, instead of producing full analyses from scratch.
  • I2 (answer): QA readers could report how often the correct entity or answer was in the candidate set versus how often it was then chosen, pinpointing which pipeline step to fix.
  • I3 (tree): parser evaluation should break LAS down per dependency label and inspect confusions, so rare relations are not hidden by frequent ones.
  • I4 (tree): when converting or mixing UD dialects and treebanks, gparse transfer should not be assumed from matching tokenization; a simple perceptron baseline is worth keeping as a sanity check.
  • I5 (learn): when using LLM teachers on non-English material for translation through the meta-world, keeping instructions in English and only the payload in the source language may give steadier teacher labels.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓King Saud UniversityRiyadh · Saudi Arabia37 works · 527 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Shahad Alsuhaibani and Mohamed Alkaoud. 2026. Uslub at AbjadAuthorID Shared Task: A Comparative Analysis of Traditional Machine Learning and Transformer-Based Models for Authorship Attribution in Arabic and Urdu. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.abjadnlp-1.65

@inproceedings{alsuhaibani2026uslub,
  title = {{Uslub at AbjadAuthorID Shared Task: A Comparative Analysis of Traditional Machine Learning and Transformer-Based Models for Authorship Attribution in Arabic and Urdu}},
  author = {Shahad Alsuhaibani and Mohamed Alkaoud},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.65},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.65}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Samir Abdaljalil, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Youssef Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, and Firoj Alam. 2024. LAraBench: Benchmarking Arabic AI with large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 487–520, St. Julian’s, Malta. Association for Computational Linguistics. ↗
  2. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. The AraGenEval shared task on Arabic authorship style transfer and AI generated text detection. In Proceedings of The Third Arabic Natural Language Processing Conference at EMNLP 2025, Suzhou, China. Association for Computational Linguistics.
  3. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Mustafa Jarrar, Mo El-Haj, Nadir Durrani, Hassan Sajjad, Farah Adeeba, and Sina Ahmadi. 2026. Abjadauthorid: Authorship identification for arabic-script languages at abjadnlp 2026. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2026), co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), Rabat, Morocco. find
  4. Mohamed Alkaoud. 2024. A bilingual benchmark for evaluating large language models. PeerJ Computer Science, 10:e1893. ↗
  5. Fetoun Mansour AlZahrani and Maha Al-Yahya. 2023. A transformer-based approach to authorship attribution in classical arabic texts. Applied Sciences, 13(12):7255. ↗
  6. Wissam Antoun and 1 others. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools.
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. Krishno Dey, Prerona Tarannum, Md.
  8. Arid Hasan, Imran Razzak, and Usman Naseem. 2024. Better to ask in english: Evaluation of large language models on english, low-resource and cross-lingual settings. arXiv preprint.
  9. Ahmed Elmadany and 1 others. 2021. Araelectra: Pre-training text discriminators for arabic language understanding. In Proceedings of the 3rd Arabic Natural Language Processing Workshop.
  10. Svetlana Gorovaia, Gleb Schmidt, and Ivan P. Yamshchikov. 2024. Sui generis: Large language models for authorship attribution and verification in latin. In Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities, pages 398–412. Association for Computational Linguistics. ↗
  11. Zhengmian Hu, Tong Zheng, and Heng Huang. 2024. A bayesian approach to harnessing the power of llms in authorship attribution. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13216–13227. Association for Computational Linguistics. find
  12. Chia-Yu Hung, Zhiqiang Hu, Yujia Hu, and Roy Lee. 2023. Who wrote it and why? prompting large-language models for authorship verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14078–14084. Association for Computational Linguistics. find
  13. Mike Kestemont. 2014. Function words in authorship attribution: From black magic to theory? In Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLfL), pages 59–66. ACL. find
  14. Viet Dac Lai, Nghia Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023. ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13171–13189, Singapore. Association for Computational Linguistics.
  15. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173. ↗
  16. Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 28(1):11–21. ↗
  17. Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. Journal of the American Society for Information Science and Technology, 60(3):538–556. ↗
  18. Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A. Huerta, and Hao Peng. 2025. Context length alone hurts llm performance despite perfect retrieval. In Findings of EMNLP 2025. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alsuhaibani-2026-uslub-abjadauthorid-shared-task-comparative/