Elamur Research⌕
☰
home / papers / 2026 / Contextual Probing for Low-Resource Named Entity Recognition in Latin

Contextual Probing for Low-Resource Named Entity Recognition in Latin

Summary

Latin NER is hard because annotated data is scarce and the language has rich morphology and free word order. The authors, from KU Leuven, propose a training-free "contextual probing" signal. Each word in a sentence is swapped, one at a time, with well-known example entities (probes) of a given type. A masked language model then scores how naturally each substitute fits, using Pseudo-Log-Likelihood (PLL). The averaged, normalized scores show how strongly a position resembles each entity type.

The scores can be used in two ways:

  • Zero-shot tagger: the scores alone label the entities.
  • Decoding aid: the scores are added to a fine-tuned model's logits, but only when that model is unsure.

Experiments use LaBERTa and LatinBERT on the LT4HALA 2026 Latin data. Training data combines four Latin NER corpora with HIPE data machine-translated into Latin. The system placed second in the coarse-grained task and third in the fine-grained task, where it ran with no training at all. This suggests that knowledge from pretraining can partly make up for missing NER annotations.

Key points

  • Compatibility score: the mean PLL of the m probes of type k placed at position i. When combined with a fine-tuned model, it is weighted by α and applied only if the model's softmax confidence is below a threshold τ.
  • Training data: five datasets.
    • Torres Aguilar (2022): medieval manuscripts.
    • Beersmans et al. (2023) and Erdmann et al. (2016): classical texts.
    • An internal STUDIUM.AI corpus of 16th–18th-century student notes.
    • A synthetic Latin set built from HIPE (French/English/German). It was translated with the Google Translate API, and labels were aligned using LaBERT embedding similarity plus Levenshtein distance.
  • Coarse-grained task, strict F1: PLL+LaBERTa reached 0.714, against 0.644 for fine-tuned LaBERTa alone. The fuzzy F1 scores were 0.771 and 0.714. LatinBERT improved from 0.618 to 0.665. Zero-shot PLL+LaBERTa scored 0.359.
  • Fine-grained task (no training data): zero-shot PLL+LaBERTa reached a strict F1 of 0.129 and a fuzzy F1 of 0.140, ranking third. LaBERTa slightly outperformed LatinBERT.
  • Submission: the best system was entered under the name "KULeuven".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing · History & humanities

Topics: Named entity recognition

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4xfc9y3we5j9

@inproceedings{trusca2026contextual,
  title = {{Contextual Probing for Low-Resource Named Entity Recognition in Latin}},
  author = {Trusca, Maria Mihaela and Depauw, Mark and Soen, Violet and Daele, Ine de and Verbruggen, Kevin and Cruys, Tim Van de},
  booktitle = {Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4xfc9y3we5j9},
  url = {https://doi.org/10.63317/4xfc9y3we5j9}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Bamman and Patrick J. Burns. 2020. Latin bert: A contextual language model for classical philology. ArXiv, abs/2009.10053. ↗
  2. Marijke Beersmans, Evelien de Graaf, Tim Van de Cruys, and Margherita Fantoli. 2023. Training and evaluation of named entity recognition models for classical Latin. In Proceedings of the Ancient Language Processing Workshop, pages 1–12. INCOMA Ltd., Shoumen, Bulgaria. find
  3. Patrick J. Burns. 2023. Latincy: Synthetic trained pipelines for latin nlp. ArXiv, abs/2305.04365. ↗
  4. Pierre Chastang, Sergio Torres Aguilar, and Xavier Tannier. 2021. A named entity recognition model for medieval latin charters. Digit. Humanit. Q., 15. find
  5. Alessio Cocchieri, Marcos Martínez Galindo, Giacomo Frisoni, Gianluca Moro, Claudio Sartori, and Giuseppe Tagliavini. 2025. Zeroner: Fueling zero-shot named entity recognition via entity type descriptions. In Annual Meeting of the Association for Computational Linguistics. find
  6. Maud Ehrmann, Matteo Romanello, Antoine Doucet, and Simon Clematide. 2022. Introducing the hipe 2022 shared task: Named entity recognition and linking in multilingual historical documents. In Advances in Information Retrieval, pages 347–354. Springer International Publishing. find
  7. Alexander Erdmann, Christopher Brown, Brian Joseph, Mark Janse, Petra Ajaka, Micha Elsner, and Marie-Catherine de Marneffe. 2016. Challenges and solutions for Latin named entity recognition. In Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH), pages 85–93. find
  8. The COL-ING 2016 Organizing Committee. find
  9. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891. Association for Computational Linguistics. find our card
  10. Guochao Jiang, Zepeng Ding, Yuchen Shi, and Deqing Yang. 2024. P-icl: Point in-context learning for named entity recognition with large language models. ArXiv, abs/2405.04960. ↗
  11. Or Shachar, Uri Katz, Yoav Goldberg, and Oren Glickman. 2025. Ner retriever: Zero-shot named entity retrieval with type-aware embeddings. In Conference on Empirical Methods in Natural Language Processing. find
  12. Sergio Torres Aguilar. 2022. Multilingual named entity recognition for medieval charters using stacked embeddings and bert-based models. In Proceedings of the Second Workshop on Language Technologies for Historical and Ancient Languages, pages 119–128. European Language Resources Association. find
  13. Mozhi Zhang, Hang Yan, Yaqian Zhou, and Xipeng Qiu. 2023. Promptner: A prompting method for few-shot named entity recognition via k nearest neighbor search. ArXiv, abs/2305.12217. ↗
  14. Yueqing Liang, Liangwei Yang, Chen Wang, Xiongxiao Xu, Philip S. Yu, and Kai Shu. 2024. Taxonomy-guided zero-shot recommendations with llms. ArXiv, abs/2406.14043. ↗
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pre-training approach. ArXiv, abs/1907.11692. ↗
  16. Vit Novotny, Kristina Luger, Michal Štefánik, Tereza Vrabcova, and Ales Horak. 2023. People and places of historical Europe: Bootstrapping annotation pipeline and a new corpus of named entities in late medieval texts. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14104–14113. Association for Computational Linguistics. find
  17. Frederick Riemenschneider and Anette Frank. 2023. Exploring large language models for classical philology. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23). Association for Computational Linguistics, Toronto, Canada. find
  18. Phillip Rust, Jonas Pfeiffer, Ivan Vulic, Sebastian Ruder, and Iryna Gurevych. 2020. How good is your tokenizer? on the monolingual performance of multilingual language models. ArXiv, abs/2012.15613. ↗ our card
  19. Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. 2020. Masked language model scoring. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 2699–2712. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/trusca-2026-contextual-probing-low-resource-named/