Elamur Research⌕
☰
home / papers / 2026 / To Overfit or Not to Overfit? An Evaluation of HTR Workflow on 17Th-18Th Century French Corpus

To Overfit or Not to Overfit? An Evaluation of HTR Workflow on 17Th-18Th Century French Corpus

Summary

The study examines how well Handwritten Text Recognition (HTR) handles early modern French manuscripts, using the Comédie-Française assembly registers (roughly 7,000 pages from the 18th century) as a test case. These documents are hard for HTR. They mix many scribal hands, follow unstable spelling conventions, and repeat a lot of administrative vocabulary. The author first tested three general pre-trained models from the CREMMA lab in eScriptorium. Error rates were too high to save effort compared with manual transcription, and the broadest model did worst. The author then fine-tuned base models with Kraken on a custom ground truth of a few hundred pages. A version deliberately "overfitted" to the corpus, enriched with additional 18th-century hands, gave much lower error rates. The resulting pipeline also includes semi-automatic post-correction and NER-assisted TEI-XML encoding of persons, places, dates and plays. The work matters because it shows that small, corpus-specific models can be more practical for humanities projects than large general ones.

Key points

  • Models tested: ManuMcfrench (1,148 page pairs), ManuMcfondue (4,604 pages) and McCATmUS (180 manuscripts in 7 languages) were evaluated by CER and WER on pages from 1684, 1729, 1763 and 1784. All performed poorly, and McFondue was the weakest.
  • Ground truth: The training set has 431 pages (415,808 tokens): 331 pages from the registers plus 100 pages of MSS18 material from CREMMA-lab.
  • Base model choice: With identical training data, fine-tuning McFrench beat fine-tuning McFondue by almost 5 CER points.
  • Training setup: Training ran through the ketos command line with learning rate 0.0001 and --resize both, which gave better results than fine-tuning in the eScriptorium interface.
  • Best model: The hybrid MSS18-RCF model cut CER on a 1706 page from 20.63 to 4.83 (WER from 66.20 to 21.37). On a page from about 1725–1729, CER fell from 39.40 to 18.21, so variable hands remain difficult.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/23hc8mveght2

@inproceedings{tiger2026overfit,
  title = {{To Overfit or Not to Overfit? An Evaluation of HTR Workflow on 17Th-18Th Century French Corpus}},
  author = {Tiger, Marine},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/23hc8mveght2},
  url = {https://doi.org/10.63317/23hc8mveght2}
}

References

33 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wissam AlKendi, Franck Gechter, Laurent Heyberger, and Christophe Guyeux. 2024. Advancements and Challenges in Handwritten Text Recognition: A Comprehensive Survey. Journal of Imaging, 10(1):18. Number: 1. ↗
  2. Emanuela Boros, Maud Ehrmann, Matteo Romanello, Sven Najem-Meyer, and Frédéric Kaplan. 2024. Post-correction of Historical Text Transcripts with Large Language Models: An Exploratory Study. In Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2024), pages 133–159. Association for Computational Linguistics. find
  3. Alix Chagué, Thibault Clérice, and Laurent Romary. 2021. HTR-United : Mutualisons la vérité de terrain ! In DHNord2021 - Publier, partager, réutiliser les données de la recherche : les data papers et leurs enjeux, Lille, France. MESHS. find
  4. Alix Chagué, Thibault Clérice, Jade Norindr, Maxime Humeau, Baudoin Davoury, Elsa Van Kote, Anaïs Mazoue, Margaux Faure, and Soline Doat. 2023. Manu McFrench, from zero to hero: impact of using a generic handwriting recognition model for smaller datasets. In Digital Humanities 2023: Collaboration as Opportunity, Graz, Austria. Alliance of Digital Humanities Organizations and University of Graz. find
  5. Thibault Clérice, Ariane Pinche, Malamatenia Vlachou-Efstathiou, Alix Chagué, Jean-Baptiste Camps, Matthias Gille Levenson, Olivier Brisville-Fertin, Federico Boschetti, Franz Fischer, Michael Gervers, Agnès Boutreux, Avery Manton, Simon Gabay, Patricia O’Connor, Wouter Haverals, Mike Kestemont, Caroline Vandyck, and Benjamin Kiessling. 2024. CATMuS Medieval: A Multilingual Large-Scale Cross-Century Dataset in Latin Script for Handwritten Text Recognition and Beyond. In Elisa H. Barney Smith, Marcus Liwicki, and Liangrui Peng, editors, Document Analysis and Recognition - ICDAR 2024, volume 14806, pages 174–194. Springer Nature Switzerland, Cham. Series Title: Lecture Notes in Computer Science. find
  6. Sebastian Colutto, Philip Kahle, Hackl Guenter, and Guenter Muehlberger. 2019. Transkribus. a platform for automated text recognition and searching of historical documents. In 2019 15th International Conference on eScience (eScience), pages 463–466. find
  7. Janez Demšar and Blaž Zupan. 2021. Hands-on training about overfitting. PLOS computational Place: biology/PLoS computational biology. United States. ↗
  8. Simon Gabay, Ariane Pinche, Peter Nahon, Alix Chagué, Pauline Jacsont, Élodie Paupe, Jean-Claude Rebetez, Maxime Humeau, Christine Payot, Thibault Maillard, Yvan Jauregui, Elina Leblanc, and Loraine Chappuis. 2024. Vers un modèle diachronique pour les mains modernes françaises. find
  9. Carlos Garrido-Munoz and Jorge Calvo-Zaragoza. 2025. On the Generalization of Handwritten Text Recognition Models. find
  10. Carlos Garrido-Munoz, Antonio Rios-Vila, and Jorge Calvo-Zaragoza. 2025. Handwritten text recognition: A survey. ↗
  11. Adeline Granet, Emmanuel Morin, Harold Mouchère, Solen Quiniou, and Christian Viard-Gaudin. 2018. Transfer learning for handwriting recognition on historical documents. In Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods - ICPRAM, pages 432–439. INSTICC, SciTePress. find
  12. Alex Graves, Marcus Liwicki, Santiago Fernández, Roman Bertolami, Horst Bunke, and Jürgen Schmidhuber. 2009. A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5):855–868. ↗
  13. Tobias Hodel, David Schoch, Christa Schneider, and Jake Purcell. 2021. General Models for Handwritten Text Recognition: Feasibility and State-of-the Art. German Kurrent as an Example. Journal of Open Humanities Data, 7(0). find
  14. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy · Industrial-strength Natural Language Processing in Python. find
  15. Mark Humphries, Lianne C. Leddy, Quinn Downton, Meredith Legace, John McConnell, Isabella Murray, and Elizabeth Spence. 2024. Unlocking the Archives: Using Large Language Models to Transcribe Handwritten Historical Documents. ArXiv:2411.03340 [cs]. ↗
  16. Vinh-Nam Huynh, Ahmed Hamdi, and Antoine Doucet. 2020. When to Use OCR Post-correction for Named Entity Recognition? In Digital Libraries at Times of Massive Societal Transition, pages 33–42, Cham. Springer International Publishing. ↗
  17. Pauline Jacsont and Elina Leblanc. 2023. L’ATR en pratique : lumière sur les techniques de transcription automatique à Genève. find
  18. Benjamin Kiessling. 2019. Kraken - a Universal Text Recognizer for the Humanities. Artwork Size: 953044 Pages: 953044. find
  19. Benjamin Kiessling, Robin Tissot, Peter Stokes, and Daniel Stökl Ben Ezra. 2019. escriptorium: An open source platform for historical document analysis. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), volume 2, pages 19–19. find
  20. Caroline Koudoro-Parfait, Ljudmila Petkovic, and Glenn Roe. 2024. Analyse multilingue de l’impact de la correction automatique de la ROC sur la reconnaissance d’entités nommées spatiales dans des corpus littéraires. Traitement Automatique des Langues, 64(2):43–67. find
  21. Elvys Linhares Pontes, Ahmed Hamdi, Nicolas Sidère, and Antoine Doucet. 2019. Impact of OCR Quality on Named Entity Linking. In International Conference on Asia-Pacific Digital Libraries 2019, Kuala Lumpur, Malaysia. ↗
  22. U.-V. Marti and H. Bunke. 2002. The IAM-database: an English sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition, 5(1):39–46. ↗
  23. Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7203–7219, Online. Association for Computational Linguistics. [link]. find
  24. Thi Tuyet Hai Nguyen, Adam Jatowt, Mickael Coustaty, and Antoine Doucet. 2022. Survey of Post-OCR Processing Approaches. ACM Computing Surveys, 54(6):1–37. ↗
  25. Caroline Parfait. 2025. Des IA au service de l’espace littéraire du XIXe siècle : évaluation et analyse des outils de reconnaissance d’entités nommées spatiales. Ph.D. thesis, Sorbonne University. ↗
  26. Ljudmila Petkovic, Motasem Alrahabi, and Glenn Roe. 2022. Impact de la correction automatique de l’OCR/HTR sur la reconnaissance d’entités nommées dans un corpus bruité. Journal of Information Sciences, 21(2):42–57. ↗
  27. Ariane Pinche. 2023. Generic HTR Models for Medieval Manuscripts. The CREMMALab Project. Journal of Data Mining & Digital Humanities, Historical Documents and automatic text recognition. find
  28. Ariane Pinche and Peter Stokes. 2024. Historical documents and automatic text recognition: Introduction. Journal of Data Mining & Digital Humanities, Historical Documents and automatic text recognition. ↗
  29. Agathe Sanjuan and Martial Poirson. 2018. Comédie Française. Une histoire du théâtre, seuil, beaux livres edition. Paris. find
  30. Hugo Scheithauer. 2021. Un exemple d’exploitation des données produites grâce à la reconnaissance d’écriture manuscrite : la reconnaissance d’entités nommées. ↗
  31. R. Smith. 2007. An Overview of the Tesseract OCR Engine. In Ninth International Conference on Document Analysis and Recognition (ICDAR 2007) Vol 2, pages 629–633, Curitiba, Parana, Brazil. IEEE. find
  32. Morgan Spinec. 2013. ”Dans l’arrière scène des seuls comédiens du roy”. Thèse d’établissement, Ecole nationale des chartes. find
  33. Phillip Benjamin Ströbel, Simon Clematide, Martin Volk, and Tobias Hodel. 2022. Transformer-based htr for historical documents. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tiger-2026-overfit-or-not-overfit-evaluation/