Elamur Research⌕
☰
home / papers / 2026 / From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages

From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages

Paper · 2026 · Matthias Schöffel and Esteban Garces Arias

Summary

Medieval Romance texts are difficult to tag for parts of speech. Their spelling is unstable, their morphology is rich, and little annotated data exists, so tools built for modern standard languages often fail on them. The authors test this task on three varieties: Medieval Occitan (NAF6195, a 14th-century manuscript of the Vida de Sant Honorat), Medieval Catalan (the 13th-century chronicle Llibre dels Fets) and Medieval French (the anatomy section of Gui de Chauliac's 15th-century Grande Chirurgie). All tagsets were mapped to Universal Dependencies conventions. The off-the-shelf taggers UDPipe and COLaF are compared with two open LLMs, Gemma3-12B and Phi4-14B. The LLMs are used through zero-shot and few-shot prompting with several decoding strategies, monolingual fine-tuning, and bilingual or trilingual cross-lingual training. LLM-based setups outperformed the traditional tools in every configuration. Fine-tuning and multilingual training gave the largest gains, and a carefully chosen language pair sometimes beat training on all three languages. The study matters because it offers evidence-based guidance for digital humanities annotation pipelines. It shows that language relatedness and corpus properties both shape how well transfer works. Code, models and processed data are publicly released.

Key points

  • Datasets differ widely in size and genre:
    • NAF: Occitan, literary, 45,457 tokens.
    • CAT: Catalan, chronicle, 59,359 tokens.
    • Chauliac: French, medical, 2,443 tokens.
    • For the transfer experiments, each corpus is split 80/20 into training and test data.
  • Five settings were evaluated:
    • traditional taggers;
    • prompting with a fixed few-shot block mixing all three varieties, tested under beam search and temperature, top-k and nucleus sampling;
    • monolingual fine-tuning;
    • bilingual cross-lingual transfer (CLTF);
    • trilingual CLTF.
  • Per-class F1 on NAF: the best LLM method improves most over UDPipe on PROPN (25.85 → 92.47, +66.62) and NUM (+62.97). SCONJ rises from 52.94 to 94.62 under Gemma3 CLTF.
  • Transfer depends on the language pair: Catalan data helps Occitan, and CAT+FR training benefits the small Chauliac corpus. Close genealogical relationship is not enough on its own to make transfer work; the amount of data, the type of text and how consistently it was annotated play a role as well.
  • Limitations:
    • The fine-tuned 12–14B models, trained with LoRA on a single H100, are far costlier than UDPipe or COLaF.
    • Smaller LLMs, XLM-R-style encoders and learning-curve experiments were not tested.
    • The code is available at github.com/msch38/medieval-romance-pos.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations🎓Bavarian Academy of Sciences and HumanitiesMunich · Germany2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksPart-of-speech tagging · 126 works

Cite

Cite as Matthias Schöffel and Esteban Garces Arias. 2026. From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages. Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.nlp4dh-1.27

@inproceedings{schoffel2026traditional,
  title = {{From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages}},
  author = {Matthias Schöffel and Esteban Garces Arias},
  booktitle = {Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities},
  year = {2026},
  doi = {10.18653/v1/2026.nlp4dh-1.27},
  url = {https://doi.org/10.18653/v1/2026.nlp4dh-1.27}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Ahmed Awadallah, Abhishek Awasthi, Ankur Bapna, Aseem Bhargava, Sarah Benchekroun, Aaron Bingeman, Shuo Chen, Weizhu Cheng, and 1 others. 2024. Phi-4 technical report. Preprint, arXiv:2412.08905.
  2. David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. 1985. A learning algorithm for boltzmann machines. Cognitive science, 9(1):147–169. find
  3. Marcel Bollmann, Florian Petran, and Stefanie Dipper. 2019. Large-scale historical text normalization with neural networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3904–3914. Association… find
  4. Miriam Cabré. 2014. Trob-eu - troubadours and the european identity. https://www.trob-eu.net/. Accessed: 2025-06-30. ↗
  5. Jean-Baptiste Camps, Simon Gabay, Paul Fièvre, Thibault Clérice, and Florian Cafiero. 2021. Corpus and models for lemmatisation and pos-tagging of classical french theatre. Journal of Data Mining and Digital Humanities. find
  6. Maud Ehrmann, Matteo Romanello, Alex Flückiger, and Simon Clematide. 2020. Named entity recognition and classification in historical documents: a survey. ACM Computing Surveys, 53(4):1–47. find
  7. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
  8. Esteban Garces Arias, Meimingwei Li, Christian Heumann, and Matthias Assenmacher. 2025. Decoding decoded: Understanding hyperparameter effects in open-ended text generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9992–10020, Abu Dhabi, UAE. Associatio…
  9. Esteban Garces Arias, Vallari Pai, Matthias Schöffel, Christian Heumann, and Matthias Aßenmacher. 2023. Automatic transcription of handwritten old Occitan language. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15416–15439, Singapore. Association fo… find
  10. Gemma-Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. Preprint, arXiv:2403.08295. find
  11. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751.
  12. K Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilingual bert: An empirical study. In Proceedings of the 8th International Conference on Learning Representations. find
  13. Mike Kestemont, Enrique Manjavacas, and Folgert Karsdorp. 2016. Lemmatization for variation-rich languages using deep learning. In Digital Humanities 2016: Conference Abstracts, pages 193–195. find
  14. Philipp Koch, Gilary Vera Nuñez, Esteban Garces Arias, Christian Heumann, Matthias Schöffel, Alexander Häberlin, and Matthias Assenmacher. 2023. A tailored handwritten-text-recognition system for medieval Latin. In Proceedings of the Ancient Language Processing Workshop, pages 103–110, Varna, Bulgar… find
  15. Enrique Manjavacas, Ákos Kádár, and Mike Kestemont. 2019. Improving lemmatization of non-standard languages with joint learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1493–1503. Associa… find
  16. Lukas Müller, Rico Sennrich, and Martin Volk. 2021. First experiments in neural translation of historical texts. In Proceedings of the Workshop on Language Technologies for Historical and Ancient Languages, pages 23–32. Association for Computational Linguistics. find
  17. John Pavlopoulos, Vasiliki Kougia, Esteban Garces Arias, Paraskevi Platanou, Stepan Shabalin, Konstantina Liagkou, Emmanouil Papadatos, Holger Essler, Jean-Baptiste Camps, and Franz Fischer. 2024. Challenging error correction in recognised byzantine Greek. In Proceedings of the 1st Workshop on Machi… find
  18. Michael Piotrowski. 2012. Natural Language Processing for Historical Texts, volume 5 of Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. find
  19. Afra Pujol i Campeny and Marieke Meelen. 2021. Annotated corpora of historical catalan (hiscat) - llibre dels fets. find
  20. Anjali Sarawgi, Esteban Garces Arias, and Christof Zotter. 2026. Digitizing nepal’s written heritage: A comprehensive htr pipeline for old nepali manuscripts. Preprint, arXiv:2512.17111.
  21. Silke Scheible, Sarah Schulz, and Michael Wojatzki. 2011. Token-based chunking of historical german. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 84–89. Association for Computational Linguistics. find
  22. Matthias Schöffel, Esteban Garces Arias, Marinus Wiedner, Paula Ruppert, Meimingwei Li, Christian Heumann, and Matthias Aßenmacher. 2025b. Unveiling factors for enhanced pos tagging: A study of low-resource medieval romance languages. Under review. ArXiv preprint, submitted. find
  23. Matthias Schöffel, Marinus Wiedner, Esteban Garces Arias, Paula Ruppert, Christian Heumann, and Matthias Aßenmacher. 2025a. Modern models, medieval texts: A pos tagging study of old occitan. Preprint, arXiv:2503.07827. find
  24. Uwe Springmann and Anke Lüdeling. 2016. Automatic quality evaluation and (semi-)automatic improvement of mixed models for ocr on historical documents. In Proceedings of the 12th Conference on Natural Language Processing (KONVENS 2014), pages 179–185. find
  25. Milan Straka, Jan Hajič, and Jana Straková. 2016. Udpipe: Trainable pipeline for processing conll-u files performing tokenization, morphological analysis, pos tagging and parsing. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4290–4297, P…
  26. Sabine Tittel. 2004. Die "anathomie" in der "grande chirurgie" des gui de chauliac : wort- und sachgeschichtliche untersuchungen und edition. find
  27. Marinus Wiedner. 2023. Old Occitan handwriting. (modell-nr. 52822, CER=3,51%), PyLaia-Modell for handwritten Occitan from the 13th and 14th century. find
  28. Marinus Wiedner. 2025. Cometa : Corpus de l’occitan médiéval comparatif et annoté: Provence et languedoc. find
  29. Gian Wiher, Clara Meister, and Ryan Cotterell. 2022. On decoding strategies for neural text generators. Transactions of the Association for Computational Linguistics, 10:997–1012.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/schoffel-2026-traditional-taggers-llms-comparative-study/