Elamur Research⌕
☰
home / papers / 2025 / Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification

Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification

Summary

This paper reports one team's entry to the AraGenEval shared task, which covers three problems in Modern Standard Arabic. The first is rewriting a passage in the voice of a chosen author (Author Style Transfer). The second is identifying which of 17 authors wrote a paragraph. The third is separating human-written text from LLM output. For style transfer, the team adapted small decoder models (Gemma3-1B, Qwen2.5-1.5B) with LoRA. The model receives the target author's name as a control signal, and a Sentence-BERT cosine penalty discourages drift in meaning. For the two classification tasks, AraBERTv2 and AraModernBert were fully fine-tuned. The team also trained a simple probe on each layer's hidden states to locate stylistic information. Gemma clearly outperformed Qwen at style transfer. Detection on the new ARATECT data was near ceiling (about 99% accuracy). Author attribution reached about 0.89 macro-F1, although the abstract's claim of >0.99 for classification conflicts with the results table. Middle layers proved more informative than the top layer. The work matters as an early multi-task benchmark for stylistic Arabic NLP.

Key points

  • AST corpus: 39,279 paired samples from 17 authors, averaging about 335 words each (about 13.1M words in total). A disproportionate share of the samples comes from three writers: Mohammad Hussein Heikal, Ahmad Amin, and Hassan Hanafi. The author-classification set is the same corpus relabelled for multi-class prediction.
  • Style transfer results (100 unseen articles):
    • Gemma r=32 scored best, with BLEU 19.04 and chrF 55.14.
    • Gemma r=64 scored 18.85/55.00 and Gemma r=16 scored 18.13/54.75.
    • Qwen r=16 lagged far behind at 10.18/44.42.
  • Author classification:
    • AraBERTv2 reached 89.71% accuracy and 0.89 macro-F1.
    • AraModernBert reached 87.1% accuracy and 0.87 macro-F1.
    • The reported best layers are 7 and 20 respectively, even though probing is described only over layers 1–12.
  • ARATECT detection data: Human texts come from news and literary sources. Machine texts come from GPT-4, Mistral, LLaMA and similar models. AraModernBert achieved 99.4% accuracy (F1 0.9923) and AraBERTv2 achieved 99.3% (F1 0.9932).
  • Setup:
    • Splits were stratified: 70/15/15 for style transfer and classification, 80/10/10 for detection.
    • Preprocessing removed diacritics and normalized characters, with a 512-token maximum length.
    • Encoders were trained with AdamW (lr 2e-5, batch 4, 3 epochs, 5% warmup).

Interesting ideas

  • I1. Train a cheap logistic-regression probe on each layer's hidden states instead of trusting only the final representation; the most useful signal may sit mid-network. — For Mova (tree): probe each layer of gparse to choose which layer, or mix of layers, should feed the arc and label scorers, rather than defaulting to the top layer.
  • I2. While training a rewriter, penalize the distance between source and output meaning so that changing style does not silently change content. — For Mova (back): in the text → graph → text compressor, re-parse the generated text and penalize divergence from the source graph, an explainable analogue of the embedding check.
  • I3. Condition a single generator on a target identity (here an author name) passed as an input token, rather than training separate models per style. — For Mova (back): give the generator a register or style tag (plain, formal, child-level) so one realizer can produce answers for different readers.
  • I4. Error analysis showed fluent machine text passing as human and messy human social-media text flagged as machine. Surface cues mislead, so evaluation needs adversarial slices. — For Mova (learn): include noisy human text and polished LLM text as negative controls when evaluating silver-data filters, so the filters are not merely rewarding fluency.
  • I5. Paraphrase human samples to widen stylistic variance, so a classifier cannot rely on lexical overlap alone. — For Mova (answer): paraphrase school-test questions and fable QA items to verify that readers answer from the meaning graph, not from word matches with the text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

1 paper of our library cites it (of the 140884 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-sharedtasks.17

@inproceedings{malhis2025jenin,
  title = {{Jenin at AraGenEval Shared Task: Parameter-Efficient Fine-Tuning and Layer-Wise Analysis of Arabic LLMs for Authorship Style Transfer and Classification}},
  author = {Malhis, Huthayfa and Tami, Mohammad and Ashqar, Huthaifa I.},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.17},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.17}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abudalfa, S., Ezzini, S., Abdelali, A., Alami, H., Benlahbib, A., Chafik, S., El-Haj, M., Mahdaouy, A. El, Jarrar, M., Lamsiyah, S., & Luqman, H. (2025). The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI-Generated Text Detection. Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025). find our card
  2. Almanea, M. M. (2021). Automatic methods and neural networks in Arabic texts diacritization: a comprehensive survey. IEEE Access, 9, 145012–145032. find
  3. Almarwani, N., & Aloufi, S. (2023). SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification. Proceedings of ArabicNLP 2023, 625– 630. find
  4. Alqahtani, F., & Dohler, M. (2023). Survey of authorship identification tasks on Arabic texts. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4), 1–24. find
  5. Al-Sarem, M., Saeed, F., Alsaeedi, A., Boulila, W., & Al-Hadhrami, T. (2020). Ensemble methods for instance-based Arabic language authorship attribution. IEEE Access, 8, 17331–17345. find
  6. Ashqar, H. I., & Tami, M. (2025). Translation with LLMs through Prompting with Long-Form Context. Authorea Preprints. find
  7. Han, Z., Gao, C., Liu, J., Zhang, J., & Zhang, S. Q. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. ArXiv Preprint ArXiv:2403.14608. ↗ our card
  8. Joshi, S., Khan, M. S., Dafe, A., Singh, K., Zope, V., & Jhamtani, T. (2024). Fine tuning LLMs for low resource languages. 2024 5th International Conference on Image Processing and Capsule Networks (ICIPCN), 511–519. find
  9. Khoboko, P. W., Marivate, V., & Sefara, J. (2025). Optimizing translation for low-resource languages: Efficient fine-tuning with custom prompt engineering in large language models. Machine Learning with Applications, 20, 100649. find
  10. Liu, S., Agarwal, S., & May, J. (2024). Authorship style transfer with policy optimization. ArXiv Preprint ArXiv:2403.08043. ↗
  11. Lv, K., Yang, Y., Liu, T., Gao, Q., Guo, Q., & Qiu, X. (2023). Full parameter fine-tuning for large language models with limited resources. ArXiv Preprint ArXiv:2306.09782. ↗
  12. Masri, S., Raddad, Y., Khandaqji, F., Ashqar, H. I., & Elhenawy, M. (2024). Transformer Models in Education: Summarizing Science Textbooks with AraBART, MT5, AraT5, and mBART. ArXiv Preprint ArXiv:2406.07692. ↗
  13. Najjar, A. A., Ashqar, H. I., Darwish, O. A., & Hammad, E. (2025). Detecting AI-Generated Text in Educational Content: Leveraging Machine Learning and Explainable AI for Academic Integrity. ArXiv Preprint ArXiv:2501.03203. ↗
  14. Najjar, A., Ashqar, H. I., Darwish, O., & Hammad, E. (2025). Leveraging Explainable AI for LLM Text Attribution: Differentiating Human-Written and Multiple LLMs-Generated Text. ArXiv Preprint ArXiv:2501.03212. ↗
  15. Pasad, A., Chou, J.-C., & Livescu, K. (2021). Layer-wise analysis of a self-supervised speech representation model. 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 914–921. find
  16. Patel, A., Andrews, N., & Callison-Burch, C. (2022). Low-resource authorship style transfer: Can non-famous authors be imitated? ArXiv Preprint ArXiv:2212.08986. ↗ our card
  17. Radhakrishnan, S., Yang, C.-H. H., Khan, S. A., Kiani, N. A., Gomez-Cabrero, D., & Tegner, J. N. (2023). A parameter-efficient learning approach to arabic dialect identification with pre-trained general-purpose speech model. ArXiv Preprint ArXiv:2305.11244. ↗ our card
  18. Sammoudi, M., Habaybeh, A., Ashqar, H. I., & Elhenawy, M. (2024). Question-Answering (QA) Model for a Personalized Learning Assistant for Arabic Language. ArXiv Preprint ArXiv:2406.08519. ↗ our card
  19. Tami, M., Ashqar, H. I., & Elhenawy, M. (2024). Automated Question Generation for Science Tests in Arabic Language Using NLP Techniques. ArXiv Preprint ArXiv:2406.08520. ↗ our card
  20. Van Aken, B., Winter, B., Löser, A., & Gers, F. A. (2019). How does bert answer questions? a layer-wise analysis of transformer representations. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 1823– 1832. A Appendices find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/malhis-2025-jenin-arageneval-shared-task-parameter/