Elamur Research⌕
☰
home / papers / 2026 / Benchmarking Arabic Authorship Attribution and Style Transfer with Large Language Models

Benchmarking Arabic Authorship Attribution and Style Transfer with Large Language Models

Summary

The paper tackles two understudied problems for Arabic: identifying who wrote a text (authorship attribution, AA) and rewriting text in a given author's style (authorship style transfer). Earlier Arabic AA work relied mostly on small, Classical or formal-domain collections and older classifiers. It rarely accounted for dialect and seldom included human evaluation. The authors build A3D, a contemporary literary dataset that mixes Modern Standard Arabic novels from the Hindawi collection with Gulf Arabic forum novels from the Gumar corpus. They fine-tune transformer AA models with a combined cross-entropy and contrastive loss, then check these models against human judgments on a style-similarity ranking task. Finally, they use the trained AA model as an automatic judge to benchmark LLMs on two tasks: ranking texts by stylistic closeness and generating author-styled text. The results show clear weaknesses in how current LLMs handle Arabic writing style. The dataset, annotations, model generations, AA models, and code are released, which gives future Arabic style research a shared foundation.

Key points

  • A3D covers 40 authors, each with at least four documents. MSA comes from Hindawi books (novel and literature genres, collected by Elmadani et al.), and Gulf Arabic comes from Gumar forum novels.
  • Training snippets are drawn from several novels per author. Test data mixes snippets from documents seen in training with snippets from unseen documents, so generalization can be measured.
  • AA models are transformer-based and trained with a dual objective that combines cross-entropy and contrastive learning.
  • In the human study, annotators rank 7 candidate snippets (40 words each) by stylistic similarity to a reference. Their guidelines cover surface, lexical, syntactic, dialectness, and aesthetic features, including Arabic-specific cues such as diacritics, elongation, and hamzas.
  • LLMs receive essentially the same rubric as a prompt and must output only comma-separated ranks. They are also evaluated on authorship style transfer, with the AA model scoring the outputs.

Interesting ideas

  • I1. Spreading each author's training samples across several source documents, and testing on both seen and unseen documents, keeps a model from learning to recognize books instead of styles. — For Mova (learn): When training gparse or the perceptron parser on silver data, split by source document or genre and report seen-versus-unseen scores as a negative control against memorizing corpus quirks.
  • I2. A trained attribution classifier can act as an automatic referee that checks whether generated text really sounds like its target, so style evaluation does not depend on manual reading alone. — For Mova (back): A style or register classifier could score text regenerated by the text → graph → text compressor, flagging cases where meaning survives but the original voice is lost.
  • I3. Humans and LLMs get the same feature checklist (surface, lexical, syntactic, dialect level, aesthetics), which makes their rankings directly comparable and their disagreements easier to interpret. — For Mova (learn): Give the LLM teacher tiers the same explicit rubric as human checkers and measure agreement per rubric item, so teacher errors can be traced to specific criteria.
  • I4. Ranking seven candidates against a reference yields a finer-grained signal than a binary same/different-author decision, and it reveals partial understanding. — For Mova (answer): School-test and fable-QA readers could be evaluated by having them rank candidate answers or paraphrases, which exposes near-misses that a plain right/wrong score hides.
  • I5. Treating the degree of dialect or colloquialism as a graded style dimension, rather than a binary language label, captures variation that standard and informal text share. — For Mova (tokens): The tagger could attach a graded informality score to each sentence, so the tokenizer and lemmatizer can switch to colloquial-spelling handling without forcing a hard dialect decision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/22xohncktzx8

@inproceedings{hamed2026benchmarking,
  title = {{Benchmarking Arabic Authorship Attribution and Style Transfer with Large Language Models}},
  author = {Hamed, Injy and Alhafni, Bashar and Habash, Nizar and Solorio, Thamar},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/22xohncktzx8},
  url = {https://doi.org/10.63317/22xohncktzx8}
}

References

78 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Opeyemi Aborisade and Mohd Anwar. 2018. Classification for authorship of tweets by comparing logistic regression and naive bayes classifiers. In Proceedings of IRI, pages 269–276. find
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. ↗ our card
  3. Tosin Adewumi, Nudrat Habib, Lama Alkhaled, and Elisa Barney. 2025. On the limitations of large language models (LLMs): False attribution. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing-Natural Language Processing in the Generative AI Era, pages 11–21. find
  4. Bo Ai, Yuchen Wang, Yugin Tan, and Samson Tan. 2022. Whodunit? learning to contrast for authorship attribution. In Proceedings of AACL-IJCNLP, pages 1142–1157. find
  5. Mohammed Al-Sarem and Abdel-Hamid Emara. 2019. The effect of training set size in authorship attribution: application on short Arabic texts. International Journal of Electrical and Computer Engineering, 9(1):652. find
  6. Mohammed Al-Sarem, Faisal Saeed, Abdullah Alsaeedi, Wadii Boulila, and Tawfik Al-Hadhrami. 2020. Ensemble methods for instance-based Arabic language authorship attribution. IEEE Access, 8:17331–17345. find
  7. Saad Alanazi. 2015. Classical Arabic authorship attribution using simple features. Natural Language Processing and Cognitive Science, 4551. find
  8. Jafar Albadarneh, Bashar Talafha, Mahmoud Al-Ayyoub, Belal Zaqaibeh, Mohammad Al-Smadi, Yaser Jararweh, and Elhadj Benkhelifa. 2015. Using big data analytics for authorship authentication of Arabic tweets. In Proccedings of the IEEE/ACM 8th International Conference on Utility and Cloud Computing (UCC), pages 448–452. find
  9. Bashar Alhafni, Vivek Kulkarni, Dhruv Kumar, and Vipul Raheja. 2024. Personalized text generation with fine-grained linguistic control. In Proceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024), pages 88–101. find
  10. Fatimah Alqahtani and Mischa Dohler. 2023. Survey of authorship identification tasks on Arabic texts. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4):1–24. find
  11. Lama Alqurashi, Serge Sharoff, Janet Watson, and Jacob Blakesley. 2025. BERT-based classical Arabic poetry authorship attribution. In Proceedings of COLING, pages 6105–6119. find
  12. Haroon Nasser Alsager. 2020. Towards a stylometric authorship recognition model for the social media texts in Arabic. Arab World English Journal, 11(4):490–507. find
  13. Malik H Altakrori, Farkhund Iqbal, Benjamin CM Fung, Steven HH Ding, and Abdallah Tubaishat. 2018. Arabic authorship attribution: An extensive study on twitter posts. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 18(1):1–51. find
  14. Alaa Saleh Altheneyan and Mohamed El Bachir Menai. 2014. Naïve bayes classifiers for authorship attribution of Arabic texts. Journal of King Saud University-Computer and Information Sciences, 26(4):473–484. find
  15. Fetoun Mansour AlZahrani and Maha Al-Yahya. 2023. A transformer-based approach to authorship attribution in classical Arabic texts. Applied Sciences, 13(12):7255. find
  16. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools. find our card
  17. Irene Baucells, Javier Aula-Blasco, Iria de Dios-Flores, Silvia Paniagua Suárez, Naiara Perez, Anna Salles, Susana Sotelo Docio, Júlia Falcão, José Javier Saiz, Robiert Sepúlveda-Torres, et al. 2025. Iberobench: A benchmark for LLM evaluation in Iberian languages. In Proceedings of COLING, pages 10491–10519. find
  18. Victor Benjamin, Wingyan Chung, Ahmed Abbasi, Joshua Chuang, Catherine A Larson, and Hsinchun Chen. 2013. Evaluating text visualization: An experiment in authorship analysis. In Proceedings of the International Conference on Intelligence and Security Informatics, pages 16–20. find
  19. Sebastian Bischoff, Niklas Deckers, Marcel Schliebs, Ben Thies, Matthias Hagen, Efstathios Stamatatos, Benno Stein, and Martin Potthast. 2020. The importance of suppressing domain style in authorship analysis. arXiv preprint arXiv:2005.14714. ↗
  20. Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213. find
  21. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. ↗ our card
  22. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186. find
  23. Joachim Diederich, Jörg Kindermann, Edda Leopold, and Gerhard Paass. 2003. Authorship attribution with support vector machines. Applied intelligence, 19(1):109–123. find
  24. Alaa M El-Halees. 2022. Arabic poetry authorship attribution and verification using transfer learning. Egyptian Computer Science Journal, 46(1). find
  25. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
  26. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. find
  27. Nudrat Habib, Tosin Adewumi, Marcus Liwicki, and Elisa Barney. 2025. Trends and challenges in authorship analysis: A review of ML, DL, and LLM approaches. arXiv preprint arXiv:2505.15422. ↗
  28. Skyler Hallinan, Faeze Brahman, Ximing Lu, Jaehun Jung, Sean Welleck, and Yejin Choi. 2023. STEER: Unified style transfer with expert reinforcement. In Findings of EMNLP. find
  29. Injy Hamed, Caroline Sabty, Slim Abdennadher, Ngoc Thang Vu, Thamar Solorio, and Nizar Habash. 2025. A survey of code-switched Arabic NLP: Progress, challenges, and future directions. In Proceedings of COLING. find
  30. Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. A probabilistic formulation of unsupervised text style transfer. In Proceedings of ICLR. find
  31. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In Proceedings of ICLR. find our card
  32. Xie He, Arash Habibi Lashkari, Nikhill Vombatkere, and Dilli Prasad Sharma. 2024. Authorship attribution methods, challenges, and future research directions: A comprehensive survey. Information, 15(3):131. find
  33. David I Holmes. 1994. Authorship attribution. Computers and the Humanities, 28(2):87–106. find
  34. Zachary Horvitz, Ajay Patel, Kanishk Singh, Chris Callison-Burch, Kathleen Mckeown, and Zhou Yu. 2024. TinyStyler: Efficient few-shot text style transfer with authorship embeddings. In Findings of EMNLP, pages 13376–13390. find
  35. Fatma Howedi and Masnizah Mohd. 2014. Text classification for authorship attribution using naive bayes classifier with limited training data. computer engineering and intelligent systems, 5(4):48–56. find
  36. Baixiang Huang, Canyu Chen, and Kai Shu. 2025. Authorship attribution in the era of LLMs: Problems, methodologies, and challenges. ACM SIGKDD Explorations Newsletter, 26(2):21–43. find
  37. Weihang Huang, Akira Murakami, and Jack Grieve. 2024. ALMS: Authorial language models for authorship attribution. arXiv preprint arXiv:2401.12005. ↗
  38. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021a. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Arabic Natural Language Processing Workshop, pages 92–104. find our card
  39. Go Inoue, Salam Khalifa, and Nizar Habash. 2021b. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. In Findings of ACL, pages 1708–1719. find
  40. Fereshteh Jafariakinabad, Sansiri Tarnpradab, and Kien A Hua. 2020. Syntactic neural model for authorship attribution. In FLAIRS, pages 234– 239. find
  41. Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155–205. find
  42. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. ALDi: Quantifying the Arabic level of dialectness of text. In Proceedings of EMNLP, pages 10597–10611. find
  43. Heba M Khalil, Taha Ahmed, and Tarek El-Shistawy. 2020. Authorship authentication of political Arabic articles based on modified TF-IGF algorithm. Journal of Theoretical and Applied Information Technology, 98(17):3575–3583. find
  44. Moshe Koppel, Jonathan Schler, Shlomo Argamon, and Eran Messeri. 2006. Authorship attribution with thousands of candidate authors. In Proceedings of the annual international ACM SIGIR conference on Research and development in information retrieval, pages 659–660. find
  45. Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating unsupervised style transfer as paraphrase generation. In Proceedings of EMNLP, pages 737–762. find our card
  46. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024a. DeepSeek-V3 technical report. arXiv preprint arXiv:2412.19437. ↗
  47. Pusheng Liu, Lianwei Wu, Linyong Wang, Sensen Guo, and Yang Liu. 2024b. Step-by-step: Controlling arbitrary style in text with large language models. In Proceedings of LREC-COLING, pages 15285–15295. find
  48. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of ICLR. find
  49. Smriti Priya Medh and Shikhar Kumar Sarma. 2025. Automatic author attribution of different languages: A review. Edelweiss Applied Science and Technology, 9(2):1245–1259. find
  50. Basel Mousi, Nadir Durrani, Fatema Ahmad, Md Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, and Firoj Alam. 2025. AraDiCE: Benchmarks for dialectal and cultural capabilities in LLMs. In Proceedings of COLING, pages 4186–4218. find
  51. Sourabrata Mukherjee, Atul Kr Ojha, and Ondřej Dušek. 2024. Are large language models actually good at text style transfer? In Proceedings of the International Natural Language Generation Conference, pages 523–539. find
  52. Sourabrata Mukherjee, Atul Kr Ojha, John Philip McCrae, and Ondřej Dušek. 2025. Evaluating text style transfer evaluation: Are there any reliable metrics? In Proceedings of the Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 418–434. find
  53. Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of EMNLP-IJCNLP, pages 188–197. find our card
  54. Melania Nitu and Mihai Dascalu. 2024. Authorship attribution in less-resourced languages: A hybrid transformer approach for romanian. Applied Sciences, 14(7):2700. find
  55. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of LREC, pages 7022– 7032. find our card
  56. Siham Ouamour and Halim Sayoud. 2012. Authorship attribution of ancient texts written by ten Arabic travelers using a smo-svm classifier. In Proceedings of the International Conference on Communications and Information Technology (ICCIT), pages 44–47. find
  57. Siham Ouamour and Halim Sayoud. 2013. Authorship attribution of ancient texts written by ten Arabic travelers using character n-grams. In Proceedings of the International Conference on Computer, Information and Telecommunication Systems (CITS), pages 1–5. find
  58. Ajay Patel, Nicholas Andrews, and Chris Callison-Burch. 2022. Low-resource authorship style transfer: Can non-famous authors be imitated? arXiv preprint arXiv:2212.08986. ↗ our card
  59. Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, and Chris Callison-Burch. 2024. StyleDistance: Stronger content-independent style embeddings with synthetic parallel examples. arXiv preprint arXiv:2410.12757. ↗
  60. Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, and Jason Wei. 2022. A recipe for arbitrary text style transfer with large language models. In Proceedings of ACL, pages 837–848. find our card
  61. Rafael A Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021. Learning universal authorship representations. In Proceedings of EMNLP, pages 913–919. find
  62. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of COLING, pages 1332– 1344. find our card
  63. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, et al. 2023. Jais and Jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv preprint arXiv:2308.16149. ↗ our card
  64. Yanir Seroussi, Ingrid Zukerman, and Fabian Bohnert. 2014. Authorship attribution with topic models. Computational Linguistics, 40(2):269–310. find
  65. Kareem Shaker and David Corne. 2010. Authorship attribution in Arabic using a hybrid of evolutionary search and linear discriminant analysis. In Proceedings of the UK Workshop on Computational Intelligence (UKCI), pages 1–6. find
  66. Kanishka Silva, Burcu Can, Frédéric Blain, Raheem Sarwar, Laura Ugolini, and Ruslan Mitkov. 2023. Authorship attribution of late 19th century novels using gan-bert. In Proceedings of ACL (Volume 4: Student Research Workshop), pages 310–320. find
  67. Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, and Partha Talukdar. 2024. IndicGenBench: A multilingual benchmark to evaluate generation capabilities of LLMs on Indic languages. In Proceedings of ACL. find
  68. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, et al. 2025. Fanar: An Arabic-centric multimodal generative AI platform. arXiv preprint arXiv:2501.13944. ↗ our card
  69. Enzo Terreau, Antoine Gourru, and Julien Velcin. 2021. Writing style author embedding evaluation. In Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems. find
  70. Jacob Tyo, Bhuwan Dhingra, and Zachary C Lipton. 2023. Valla: Standardizing and benchmarking authorship attribution and verification through empirical evaluation and comparative analysis. In Proceedings of IJCNLP-AACL. find
  71. Xinchen Yang and Marine Carpuat. 2025. Steering large language models with register analysis for arbitrary style transfer. arXiv preprint arXiv:2505.00679. ↗
  72. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In Proceedings of ICLR. 10. find
  73. Language Resource References Muhamed Al-Khalil, Nizar Habash, and Zhengyang Jiang. 2020. A large-scale leveled readability lexicon for standard Arabic. In Proceedings of LREC, pages 3053–3062. find our card
  74. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, pages 830–839. find
  75. Khalid Elmadani, Nizar Habash, and Hanada Taha. 2025a. A large and balanced corpus for fine-grained Arabic readability assessment. In Findings of ACL, pages 16376–16400. find
  76. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025b. BAREC shared task 2025 on Arabic readability assessment. In Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pages 239–252. find
  77. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016. A large scale corpus of Gulf Arabic. In Proceedings of LREC, pages 4282–4289. find
  78. Bryan Klimt and Yiming Yang. 2004. The Enron corpus: A new dataset for email classification research. In Proceedings of the European Conference on Machine Learning, pages 217–226. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamed-2026-benchmarking-arabic-authorship-attribution-style/