Elamur Research⌕
☰
home / papers / 2026 / Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer

Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer

Summary

Arabic–Russian is a poorly resourced translation pair. Its existing parallel data is mostly religious, conversational or general text, with almost no scientific terminology. Arabov fills this gap with a benchmark for translating scientific Arabic into Russian. He assembled a mixed parallel corpus of about 27,000 pairs (26,878). Scientific abstracts are combined with six general sources: religion, news, conversations, dictionaries, the Bible and Tatoeba.

Three multilingual models of increasing size were adapted with LoRA/QLoRA at ranks 8 to 64: mT5-base, NLLB-200-distilled-1.3B and Qwen2.5-7B-Instruct. They were scored with BLEU, chrF, BERTScore and COMET.

The best system was Qwen2.5-7B with QLoRA at rank 8, which clearly beat its own zero-shot output. Three-shot prompting brought no gain, and the sub-1B mT5 stayed weak even after tuning. The author concludes that both task-specific tuning and a minimum model size matter.

Models, corpus and evaluation code are released. This gives a reproducible baseline for an underserved pair and for cross-community scientific exchange.

Key points

  • The corpus has 26,878 training examples, built from scientific abstracts plus six general-domain sources. The test set has 1,500 sentences from scientific abstracts.
  • Best result: Qwen2.5-7B-Instruct with QLoRA rank 8 reached BLEU 23.15, chrF 43.89, BERTScore 0.906 and COMET 0.758. That is +4.36 BLEU and +0.051 COMET over zero-shot.
  • NLLB-200-distilled-1.3B with LoRA rank 32 reached 21.95 BLEU, a good trade-off between quality and cost.
  • mT5-base (580M) stayed unacceptable but improved steadily up to rank 64. Fine-tuning gains for the larger models ranged from +2.13 to +4.36 BLEU, while training under 1% of parameters.
  • Limitations:
    • The references are Gemma-3-4B translations, checked by experts on a random sample of 50 sentences.
    • All systems produced translations 2.1–3.1 tokens shorter than the references.

Interesting ideas

  • I1. Pick models from three distinct size regimes to locate the smallest capacity that reaches acceptable quality, instead of only testing one large model. — For Mova (learn): Run the same sweep over gparse sizes to find the cheapest parser that still meets a target LAS, and justify model size with data.
  • I2. Sweep adapter rank together with model size. In this paper small models kept gaining from higher ranks, while ~1B models leveled off around rank 32. — For Mova (tree): When adapting gparse to new treebanks or UD dialects, tune adapter capacity relative to base-model size rather than using a fixed default.
  • I3. Use another LLM's output as the reference set, audit a small random sample with experts, and trust mainly the relative comparisons. — For Mova (learn): Treat LLM-teacher silver data the same way: spot-check a fixed random sample per tier and report it as a validation signal, not ground truth.
  • I4. Report a prompting-only baseline (zero-shot and few-shot) next to fine-tuning, so the benefit of training is measured rather than assumed. — For Mova (learn): Add "LLM prompted without training" as a standing control in evaluations, alongside the negative controls.
  • I5. Track the length gap between system output and reference as a separate diagnostic, because it can quietly depress overlap metrics. — For Mova (back): Measure the length drift of the text → graph → text compressor and generation output to catch systematic omissions of content.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Machine translation

Cited by

2 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2606.30943 · arXiv 2606.30943

@misc{arabov2026bridging,
  title = {{Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer}},
  author = {Arabov, Mullosharaf K.},
  year = {2026},
  doi = {10.48550/arxiv.2606.30943},
  eprint = {2606.30943},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.30943}
}

References

18 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-Rank Adaptation of Large Language Models. In: International Conference on Learning Representations (ICLR) (2022). https://openreview.net/forum?id=nZeVKeeFYf9 ↗
  2. Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: QLoRA: Efficient Fine-tuning of Quantized LLMs. In: Advances in Neural Information Processing Systems, vol. 36 (2024). find
  3. Rei, R., Stewart, C., Farinha, A.C., Lavie, A.: COMET: A Neural Framework for MT Evaluation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685–2702. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.emnlp-main.213 ↗ our card
  4. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a Method for Automatic Evaluation of Machine Translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318. Philadelphia, USA (2002). https://doi.org/10.3115/1073083.1073135 ↗ our card
  5. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: Evaluating Text Generation with BERT. In: International Conference on Learning Representations (ICLR) (2020). https://openreview.net/forum?id=SkeHuCVFDr ↗
  6. Popovic, M.: chrF: Character n-gram F-score for Automatic MT Evaluation. In: Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392– 395. Lisbon, Portugal (2015). https://aclanthology.org/W15-3049/ ↗ our card
  7. Alzubaidi, A., Alsuwaidi, S., Boussaha, B.E.A., Al Qadi, L., Alkaabi, O., Alyafeai, M., Alobeidli, H., Hacid, H.: Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps. arXiv:2510.13430 (2025). https://arxiv.org/abs/2510.13430 ↗ our card
  8. Hadj Ameur, M.S., Guessoum, A.: A Survey on Arabic Machine Translation: Progress, Challenges, and Future Directions. Computer Science Review 38, 100307 (2020). https://doi.org/10.1016/j.cosrev.2020.100307 ↗
  9. Al-Matham, R.N., Darwish, K., Al-Rasheed, R., Alshammari, W., Alhoshan, M., Elsayed, T.: BALSAM: A Platform for Benchmarking Arabic Large Language Models. In: Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), pp. 258–277. Suzhou, China (2025). https://aclanthology.org/2025.arabicnlp-1.19/ ↗
  10. ArabicNLPWorld: Arabic-Russian Parallel Corpus. Hugging Face (2026). https://huggingface.co/datasets/ArabicNLPWorld/arabic-russian-parallel-corpus ↗
  11. ArabicNLPWorld: Arabic-Russian Scientific Translations. Hugging Face (2026). https://huggingface.co/datasets/ArabicNLPWorld/arabic-russian-scientific-translations ↗
  12. Alrashed, S., Orabona, F.: AraMix: Recycling, Refiltering, and Deduplicating to Deliver the Largest Arabic Pretraining Corpus. arXiv:2512.18834 (2025). https://arxiv.org/abs/2512.18834 ↗
  13. Arabov, M.K., Khaybullina, S.S.: Adapting Large Language Models to a Low-Resource Agglutinative Language: A Comparative Study of LoRA and QLoRA for Bashkir. arXiv:2605.04948 (2026). https://arxiv.org/abs/2605.04948 ↗
  14. Song, Y., Li, L., Lothritz, C., Ezzini, S., Sleem, L., Bissyandé, T.F., Klein, J.: Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation? In: Proceedings of the Ninth Workshop on Technologies for Machine LLM Benchmark for Arabic–Russian Translation Translation of Low Resource Languages (LoResMT 2026), pp. 1–26. Rabat, Morocco (2026). https://aclanthology.org/2026.loresmt-1.0/ ↗ our card
  15. Al-Khalifa, H., Darwish, K., Mubarak, H., Ali, M., Elsayed, T.: Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation. In: Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) (2024). https://aclanthology.org/2024.osact-1.0/ ↗
  16. Arabov, M.K.: TajPersLexon: A Tajik–Persian Lexical Resource and Hybrid Model for Cross-Script Low-Resource NLP. In: Proceedings of the First Workshop on NLP and LLMs for the Iranian Language Family, pp. 29–37. Association for Computational Linguistics, Rabat, Morocco (2026). https://doi.org/10.18653/v1/2026.silkroadnlp-1.4 ↗
  17. Kurbonovich, A.M.: Character-Level Transformer for Tajik–Persian Transliteration with a Parallel Lexical Corpus. In: Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script, pp. 75–83. Association for Computational Linguistics, Rabat, Morocco (2026). https://doi.org/10.18653/v1/2026.abjadnlp-1.10 ↗
  18. Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., Raffel, C.: mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 483–498. Association for Computational Linguistics, Online (2021). https://doi.org/10.18653/v1/2021.naacl-main.41 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/arabov-2026-bridging-scientific-heritage-arabic-russian/