Elamur Research⌕
☰
home / papers / 2026 / Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models

Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models

Summary

Translating Ancient Greek (AG) into Modern Greek (MG) has been held back by the lack of large parallel data aligned at the sentence level. Most existing sources pair texts only by excerpt or paragraph, and they contain editorial noise. The authors built the AG-MG Parallel Corpus by scraping literary, historical and biblical sources and cleaning them. They split the texts into sentences with Stanza and aligned them using VecAlign with LaBSE, after first adapting LaBSE on manually aligned pairs. Gemini 2.5 Flash then served as an automatic checker that fixes merge, split and mismatch errors. Biblical texts were paired using their verse numbers. On this data they ran the first broad benchmark for the language pair, comparing full fine-tuning, LoRA and QLoRA on NLLB, M2M100 and the Greek LLM Llama-Krikri-8B. Fine-tuning helped every model substantially, and the fully tuned Krikri achieved the best scores. The corpus and models give Greek NLP and Digital Humanities a much-needed baseline resource.

Key points

  • The corpus contains 132,481 sentence pairs, with about 2.31M AG tokens and 3.06M MG tokens. The MG side runs roughly 32% longer, which reflects the shift from synthetic to analytic grammar. Each pair carries metadata for author, translator, dialect, genre and era.
  • The data splits are as follows:
    • Train: 128,231 pairs, mostly Attic (57.3% of the whole corpus) and Hellenistic Koine (29.1%).
    • Dev and Test: 2,000 pairs each.
    • Stress set: 250 pairs from Ionic, Doric and Homeric/Epic texts.
  • LaBSE was adapted on 1,000 hand-aligned pairs. A human check of 150 pairs after the LLM refinement step estimated about 95% alignment accuracy.
  • Duplicates were removed based on the MG sentences. Alternative translations of the same AG sentence were kept in the training split only, as multiple references.
  • Llama-Krikri-8B with full fine-tuning reached the top BLEU score of 13.16. Fine-tuning improved scores by as much as +10.3 BLEU. M2M100-1.2B with QLoRA showed the largest relative improvement over its base model, whose untuned output was often hallucinated.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4cdk64dgm2w9

@inproceedings{mavromatis2026ancient,
  title = {{Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models}},
  author = {Mavromatis, Spyridon and Sofianopoulos, Sokratis and Prokopidis, Prokopis and Giagkou, Maria},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4cdk64dgm2w9},
  url = {https://doi.org/10.63317/4cdk64dgm2w9}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. ↗ our card
  2. Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. ↗
  3. Caroline Craig, Kartik Goyal, Gregory Crane, Farnoosh Shamsian, and David A Smith. 2023. Testing the limits of neural sentence alignment models on classical greek and latin texts and translations. Proceedings http://ceur-ws.org ISSN, 1613:0073. ↗
  4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088– 10115. ↗
  5. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
  6. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852. ↗ our card
  7. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3. find
  8. Marzena Karpinska and Mohit Iyyer. 2023. Large language models effectively leverage document-level context for literary translation, but critical errors persist. arXiv preprint arXiv:2304.03245. ↗ our card
  9. Huda Khayrallah, Brian Thompson, Matt Post, and Philipp Koehn. 2020. Simulated multiple reference training improves low-resource machine translation. arXiv preprint arXiv:2004.14524. ↗
  10. Ourania Kolovou. 2024. Machine translation from ancient greek to english: Experiments with opennmt. find
  11. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499. ↗
  12. Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F Wong, Siyou Liu, and Longyue Wang. 2023. A paradigm shift: The future of machine translation lies with large language models. arXiv preprint arXiv:2305.01181. ↗
  13. Chiara Palladino, Farnoosh Shamsian, Tariq Yousef, David J Wright, Anise d’Orange Ferreira, and Michel Ferreira Dos Reis. 2023. Translation alignment for ancient greek: Annotation guidelines and gold standards. Journal of Open Humanities Data, 9:22. find
  14. Katerina Papantoniou and Yannis Tzitzikas. 2024. Nlp for the greek language: A longer survey. arXiv preprint arXiv:2408.10962. ↗
  15. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. ↗ our card
  16. Maja Popović. 2015. chrf: character n-gram fscore for automatic mt evaluation. In Proceedings of the tenth workshop on statistical machine translation, pages 392–395. find
  17. Maja Popović. 2017. chrf++: words helping character n-grams. In Proceedings of the second conference on machine translation, pages 612– 618. find our card
  18. Maciej Rapacz and Aleksander Smywiński-Pohl. 2025. Low-resource interlinear translation: Morphology-enhanced neural models for ancient greek. In Proceedings of the First Workshop on Language Models for Low-Resource Languages, pages 145–165. find
  19. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. arXiv preprint arXiv:2009.09025. ↗ our card
  20. Dimitris Roussis, Leon Voukoutis, Georgios Paraskevopoulos, Sokratis Sofianopoulos, Prokopis Prokopidis, Vassilis Papavasileiou, Athanasios Katsamanis, Stelios Piperidis, and Vassilis Katsouros. 2025. Krikri: Advancing open large language models for greek. arXiv preprint arXiv:2505.13772. ↗
  21. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231. find our card
  22. Thea Sommerschield, Yannis Assael, John Pavlopoulos, Vanessa Stefanak, Andrew Senior, Chris Dyer, John Bodel, Jonathan Prag, Ion Androutsopoulos, and Nando De Freitas. 2023. Machine learning for ancient languages: A survey. Computational Linguistics, 49(3):703– 747. ↗
  23. Brian Thompson and Philipp Koehn. 2019. Vecalign: Improved sentence alignment in linear time and space. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 1342–1348. find
  24. L Voukoutis, D Roussis, G Paraskevopoulos, S Sofianopoulos, P Prokopidis, V Papavasileiou, A Katsamanis, S Piperidis, and V Katsouros. Meltemi: The first open large language model for greek. arxiv 2024. arXiv preprint arXiv:2407.20743. ↗
  25. Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023. Document-level machine translation with large language models. arXiv preprint arXiv:2304.02210. ↗ our card
  26. Tariq Yousef, Chiara Palladino, and Farnoosh Shamsian. 2023. Classical philology in the time of ai: Exploring the potential of parallel corpora in ancient language. In Proceedings of the Ancient Language Processing Workshop, pages 179–192. find
  27. Tariq Yousef, Chiara Palladino, Farnoosh Shamsian, Anise d’Orange Ferreira, and Michel Ferreira dos Reis. 2022a. An automatic model and gold standard for translation alignment of Ancient Greek. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5894–5905, Marseille, France. European Language Resources Association. find
  28. Tariq Yousef, Chiara Palladino, David J Wright, and Monica Berti. 2022b. Automatic translation alignment for ancient greek and latin. In Proceedings of the Second Workshop on Language Technologies for Historical and Ancient Languages, pages 101–107. ↗
  29. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. ↗
  30. Renjie Zheng, Mingbo Ma, and Liang Huang. 2018. Multi-reference training with pseudo-references for neural translation and text generation. arXiv preprint arXiv:1808.09564. 11. ↗
  31. Language Resource References Vatri, Alessandro and McGillivray, Barbara. 2018. The Diorisis Ancient Greek Corpus: Linguistics and Literature. Brill. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/mavromatis-2026-ancient-greek-modern-greek-machine/