Elamur Research⌕
☰
home / papers / 2026 / It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models

It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models

Paper · 2026 · Cristian Santini, Marieke van Erp and Mehwish Alam

Summary

The paper tackles entity linking on historical documents, where OCR noise, language variation, multilinguality and many entities absent from knowledge bases (NIL entities) degrade standard systems. Earlier specialised solutions depend on annotated training data or hand-crafted rules, and zero-shot LLM approaches have trailed them. The authors propose MHEL-LLaMo, an unsupervised pipeline that requires no fine-tuning. BELA, a multilingual XLM-R bi-encoder, retrieves Wikidata candidates through a Faiss index, and an SQLite lookup table adds labels, descriptions, dates and entity types. The retriever's similarity score serves as a confidence signal. When a candidate passes a threshold, the mention counts as easy and BELA's top answer is kept. Otherwise, an instruction-tuned LLM handles it with chained prompts: it first decides whether the entity is NIL, then picks among the candidates. Routing only hard cases to the LLM cuts inference cost and limits hallucinations on simple mentions. Across four benchmarks in six European languages from the 19th–20th centuries, the system outperforms most specialised state-of-the-art models. This makes it a scalable option for low-resource historical entity linking.

Key points

  • The architecture has three parts: BELA candidate retrieval over Wikidata entity embeddings built from Wikipedia in 97 languages, an SQLite metadata lookup table, and an instruction-tuned LLM. Code, data and models are available at github.com/sntcristian/MHEL-LLAMO.
  • The evaluation covers English, Finnish, French, German, Italian and Swedish texts. The LLMs discussed include Gemma-3-27B-it and Poro-2-8B-it.
  • Nordic languages remain weak, with a best F1 of 0.509 for Finnish and 0.521 for Swedish. Swedish NIL-prediction recall is only 0.184, because the LLM tends to link entities rather than mark them NIL.
  • On the classical commentaries dataset AJMC, F1 ranges from 0.496 to 0.635, and most errors involve literary works and cultural objects. NIL-prediction F1 there is only 0.129–0.167, since just 2.41% of its entities are NIL.
  • Planned future work includes training the retriever on historical data and making the threshold supervised, for example with Gradient Boosted Trees. The authors also suggest LoRA-based fine-tuning of smaller LLMs and integration into relation extraction pipelines.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Télécom ParisPalaiseau · France11 works · 826 citations🔬KNAW Humanities ClusterAmsterdam · Netherlands2 works · 1 citation

Also on the byline: University of Macerata.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 worksNamed entity recognition · 200 works

Cite

Cite as Cristian Santini, Marieke van Erp and Mehwish Alam. 2026. It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.184

@inproceedings{santini2026it,
  title = {{It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models}},
  author = {Cristian Santini, Marieke {van Erp} and Mehwish Alam},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.184},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.184}
}

References

31 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lora Aroyo and Chris Welty. 2015. Truth is a lie: Crowd truth and the seven myths of human annotation. AI Magazine, 36(1):15–24.
  2. Gianmaria Balducci, Elisabetta Fersini, and Messina Enza. 2025. Beyond raw text: Knowledge-augmented italian relation extraction with large language models. In Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), pages 45–54. find
  3. Emanuela Boros, Elvys Linhares Pontes, Luis Adrián Cabrera-Diego, Ahmed Hamdi, José G Moreno, Nicolas Sidère, and Antoine Doucet. 2020. Robust named entity recognition and linking on historical multilingual documents. In Conference and Labs of the Evaluation Forum (CLEF 2020), volume 2696, pages 1–1… find
  4. Marta Boscariol, Luana Bulla, Lia Draetta, Beatrice Fi-umanò, Emanuele Lenzi, and Leonardo Piano. 2024. Evaluation of LLMs on Long-tail Entity Linking in Historical Documents. In Posters and Demos, Workshops, and Tutorials of EKAW 2024, volume 3967. CEUR. find
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Yiyi Chen, Harald Sack, and Mehwish Alam. 2022. Analyzing social media for measuring public attitudes toward controversies and their driving factors: a case study of migration. Social network analysis and mining, 12(1):135. find
  7. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  8. Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, and Fabio Petroni. 2022. Multilingual autoregressive entity linking. Transactions of the Association for Computational Linguistics, 10:274–290. find
  9. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library. arXiv preprint arXiv:2401.08281.
  10. Maud Ehrmann, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, and Antoine Doucet. 2023. find
  11. Named entity recognition and classification in historical documents: A survey. ACM Computing Surveys, 56(2):1–47. find
  12. Maud Ehrmann, Matteo Romanello, Stefan Bircher, and Simon Clematide. 2020. Introducing the clef 2020 hipe shared task: Named entity recognition and linking on historical newspapers. In European Conference on Information Retrieval, pages 524–532. Springer. find
  13. Maud Ehrmann, Matteo Romanello, Sven Najem-Meyer, Antoine Doucet, and Simon Clematide. 2022. Overview of hipe-2022: named entity recognition and linking in multilingual historical documents. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 423–446. Spr… find
  14. Carlos-Emiliano González-Gallardo, Emanuela Boros, Nancy Girdhar, Ahmed Hamdi, Jose G. Moreno, and Antoine Doucet. 2023. Yes but.. Can ChatGPT Identify Entities in Historical Documents? In 2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pages 184–189. ISSN: 2575-8152. find
  15. Carlos-Emiliano González-Gallardo, Hanh Thi Hong Tran, Ahmed Hamdi, and Antoine Doucet. 2024. Leveraging Open Large Language Models for Historical Named Entity Recognition. In Linking Theory and Practice of Digital Libraries, pages 379–395, Cham. Springer Nature Switzerland. find
  16. Arianna Graciotti, Nicolas Lazzari, Valentina Presutti, and Rocco Tripodi. 2025a. Musical heritage historical entity linking. Artificial Intelligence Review, 58(5):140. find
  17. Arianna Graciotti, Leonardo Piano, Nicolas Lazzari, Enrico Daga, Rocco Tripodi, Valentina Presutti, and Livio Pompianu. 2025b. KE-MHISTO: Towards a multilingual historical knowledge extraction benchmark for addressing the long-tail problem. In Findings of the Association for Computational Linguistic… find
  18. Paul Groth and Stefan Dietze. 2024. Understanding the impact of entity linking on the topology of entity co-occurrence networks for social media analysis. In Knowledge Engineering and Knowledge Management: 24th International Conference, EKAW 2024, Amsterdam, The Netherlands, November 26– 28, 2024, P… find
  19. Ahmed Hamdi, Elvys Linhares Pontes, Emanuela Boros, Thi Tuyet Hai Nguyen, Günter Hackl, Jose G Moreno, and Antoine Doucet. 2021. A multilingual dataset for named entity recognition, entity linking and stance detection in historical newspapers. In Proceedings of the 44th International ACM SIGIR Confe… find
  20. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. In Proceedings of the Tenth International Conference on Learning Representations (ICLR).
  21. Nikolaos Kolitsas, Octavian-Eugen Ganea, and Thomas Hofmann. 2018. End-to-end neural entity linking. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 519–529, Brussels, Belgium. Association for Computational Linguistics. find
  22. Kai Labusch and Clemens Neudecker. 2020. Named entity disambiguation and linking historic newspaper ocr with bert. CLEF (Working Notes), 2696. find
  23. Peerat Limkonchotiwat, Weiwei Cheng, Christos Christodoulopoulos, Amir Saffari, and Jens Lehmann. 2023. mReFinED: An efficient end-to-end multilingual entity linking system. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15080–15089, Singapore. Association for Comput… find
  24. Elvys Linhares Pontes, Luis Adrián Cabrera-Diego, Jose G Moreno, Emanuela Boros, Ahmed Hamdi, Antoine Doucet, Nicolas Sidere, and Mickaël Coustaty. 2022. Melhissa: a multilingual entity linking architecture for historical press articles. International journal on digital libraries, 23(2):133–160. find
  25. Yubo Ma, Yixin Cao, Yong Hong, and Aixin Sun. 2023. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10572–10601, Singapore. Association for Computational Linguisti…
  26. Mikhail Plekhanov, Nora Kassner, Kashyap Popat, Louis Martin, Simone Merello, Borislav Kozlovskii, Frédéric A Dreyer, and Nicola Cancedda. 2023. Multilingual end to end entity linking. arXiv preprint arXiv:2306.08896. — [arXiv:2306.08896](https://arxiv.org/abs/2306.08896) find
  27. Matteo Romanello and Sven Najem-Meyer. 2024. A named entity-annotated corpus of 19th century classical commentaries. Journal of Open Humanities Data, 10(1). find
  28. Adam Aron Rynkiewicz, Raul Palma, and Paulina Poniatowska-Rynkiewicz. 2025. Introducing FELA -Flexible Entity Linking Approach. Selected Papers of HybridAIMS and CAI Workshops at CAiSE 2025, pages 4–11. Place: Vienna, Austria Publisher: Zenodo. find
  29. Cristian Santini. 2024. Combining language models for knowledge extraction from italian tei editions. Frontiers in Computer Science, 6:1472512. find
  30. Cristian Santini, Nele Garay, Etienne Posthumus, and Harald Sack. 2024a. The art of relations: Analysing vasari’s biographies with network theory and semantic web. In Book of Abstracts-DHd2024. Zenodo. find
  31. Cristian Santini, Laura Melosi, and Emanuele Fron-toni. 2024b. Named Entity Recognition in Historical Italian: The Case of Giacomo Leopardi’s Zibaldone. In Posters and Demos, Workshops, and Tutorials of EKAW 2024, volume 3967, Amsterdam, Netherlands. CEUR. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/santini-2026-it-s-all-about-confidence/