Elamur Research⌕
☰
home / papers / 2026 / AyahVerse at AbjadGenEval Shared Task: Monolingual Precision and Cross-Lingual Analysis in Perso-Arabic AI Detection

AyahVerse at AbjadGenEval Shared Task: Monolingual Precision and Cross-Lingual Analysis in Perso-Arabic AI Detection

Paper · 2026 · Fizza Nawaz, Ibad-ur-Rehman Rashid, Uswa Abid et al.

Summary

This system paper describes the AyahVerse team's entry to the AbjadGenEval shared task, which asks whether news text in Arabic or Urdu was written by a person or produced by an LLM. The authors fine-tuned monolingual encoders (AraBERTv2, CAMeLBERT-DA, ArabicBERT) and multilingual ones (mBERT, XLM-R). Each was trained on two kinds of input: text that keeps punctuation, diacritics and formatting, and text that strips punctuation and unifies orthography. Comparing the two shows whether detectors depend on surface cues or on content.

Keeping the surface cues worked better in every comparison. On the official test sets, AraBERTv2 reached an F1 of 0.75 for Arabic and mBERT about 0.87 for Urdu. The authors also tested zero-shot transfer between the two languages with XLM-R. Transfer worked better from Urdu to Arabic than the reverse, and it broke down when lower encoder layers were frozen. This suggests some signals of machine authorship carry over across Perso-Arabic-script languages. However, the training sets differ greatly in size, so this conclusion is tentative.

Key points

  • Data: The official training data has 9,289 Arabic and 20,776 Urdu items; the test sets have 200 Arabic and 2,630 Urdu items. Robustness was checked on 2,000 out-of-domain scientific abstracts from KFUPM-JRCAI, split evenly between human-written and ALLaM-generated.
  • Leaderboard: Arabic scored 0.75 F1 with AraBERTv2 (Pipeline B), placing 6th of 12. Urdu scored 0.87 F1 with mBERT (Pipeline D), placing 6th of 7. The abstract reports 0.86 for Urdu. Development F1 was about 0.99, so there is a clear generalization gap.
  • Surface cues vs. normalized text: The artifact-preserving pipelines (B/D) beat the normalized ones (A/C). On the KFUPM Arabic set the scores were 0.95 vs. 0.90, and on the Urdu test set 0.87 vs. 0.70. AraBERTv2, pre-trained on formal MSA, outperformed CAMeLBERT-DA, which is dialect-oriented (0.64 test F1).
  • Cross-lingual transfer (XLM-R):
    • Urdu→Arabic: 0.75 macro F1, with machine-class recall of 0.91.
    • Arabic→Urdu: 0.61 macro F1, with machine-class recall of 0.71.
    • mBERT recognized human texts better (human recall 0.76) but missed more machine texts (machine recall 0.48).
  • Fine-tuning setup: Freezing the bottom six encoder layers wrecked cross-lingual transfer. Training used AdamW with a learning rate of 2e-5, batch size 32, and 2–6 epochs. The roughly 2:1 Urdu/Arabic data imbalance may account for the difference between transfer directions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Fizza Nawaz, Ibad-ur-Rehman Rashid, Uswa Abid and Junaid Hussain. 2026. AyahVerse at AbjadGenEval Shared Task: Monolingual Precision and Cross-Lingual Analysis in Perso-Arabic AI Detection. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.abjadnlp-1.63

@inproceedings{nawaz2026ayahverse,
  title = {{AyahVerse at AbjadGenEval Shared Task: Monolingual Precision and Cross-Lingual Analysis in Perso-Arabic AI Detection}},
  author = {Fizza Nawaz, Ibad-ur-Rehman Rashid, Uswa Abid and Junaid Hussain},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.63},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.63}
}

References

11 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saad Ezzini, Irfan Ahmed, Salmane Chafik, Shadi Abudalfa, Mo El-Haj, Ahmed Abdelali, Mustafa Jarrar, Nadir Durrani, Hassan Sajjad, and Farah Adeeba. 2026. Abjadgeneval: Abjad ai generated text detection shared task for languages using arabic script at abjadnlp 2026. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2026), co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), Rabat, Morocco.
  2. Shadi Abudalfa, Saad Ezzini, Ahmed Abdelali, Hamza Alami, Abdessamad Benlahbib, Salmane Chafik, Mo El-Haj, Abdelkader El Mahdaouy, Mustafa Jarrar, Salima Lamsiyah, and Hamzah Luqman. 2025. The AraGenEval shared task on Arabic authorship style transfer and AI generated text detection. In Proceedings of The Third Arabic Natural Language Processing Conference at EMNLP 2025, Suzhou, China. Association for Computational Linguistics.
  3. Maged S Al-Shaibani and Moataz Ahmed. 2025. The arabic ai fingerprint: Stylometric analysis and detection of large language models text. arXiv preprint arXiv:2505.23276. ↗
  4. Iqra Ali, Jesse Atuhurra, Hidetaka Kamigaito, and Taro Watanabe. 2025. HLU: Human vs LLM generated text detection dataset for Urdu at multiple granularities. In Proceedings of the 31st International Conference on Computational Linguistics, page 3495–3510, Abu Dhabi, UAE. Association for Computational Linguistics. find
  5. Hamed Alshammari and Khaled Elleithy. 2024. Toward robust arabic ai-generated text detection: Tackling diacritics challenges. Information, 15(7). ↗
  6. Muhammad Ammar, Hadiya Murad Hadi, and Usman Majeed Butt. 2025. AI-Generated Text Detection in Low-Resource Languages: A Case Study on Urdu. Preprint, arXiv:2510.16573. ↗
  7. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, page 111–116, Florence, Italy. Association for Computational Linguistics.
  8. Salima Lamsiyah, Saad Ezzini, Abdelkader El Mahdaouy, Hamza Alami, Abdessamad Benlahbib, Samir El amrany, Salmane Chafik, and Hicham Hammouchi. 2025. M-DAIGT: A Shared Task on Multi-Domain Detection of AI-Generated Text. In Proceedings of the Shared Task on Multi-Domain Detection of AI-Generated Text, page 1–9, Varna, Bulgaria. IN-COMA Ltd., Shoumen, Bulgaria. find
  9. Ibad-ur-Rehman Rashid and Muhammad AyahVerse at MA-Hashir Khalil. 2025. HED shared task: Fine-tuning ArabicBERT with preprocessing for hope and hate detection. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, page 670–676, Suzhou, China. Association for Computational Linguistics. find
  10. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, page Irene Solaiman, Miles Brundage, Jack Clark, 9–15, Marseille, France. European Language Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Resources Association. Alec Radford, Gretchen Krueger, Jong Wook 501 ↗
  11. Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. 2019. Release strategies and the social impacts of language models. Preprint, arXiv:1908.09203. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nawaz-2026-ayahverse-abjadgeneval-shared-task-monolingual/