Elamur Research⌕
☰
home / papers / 2026 / Matching Meaning at Scale: Evaluating Semantic Search for 18th-Century Intellectual History through the Case of Locke

Matching Meaning at Scale: Evaluating Semantic Search for 18th-Century Intellectual History through the Case of Locke

Paper · 2026 · Yu Wu, Ananth Mahadevan, Filip Ginter et al.

Summary

Research on how ideas spread through 18th-century print has mostly relied on lexical text reuse detection. That approach finds verbatim quotations but misses paraphrase and implicit engagement. The authors test whether a minimal off-the-shelf dense-retrieval semantic search pipeline can close this gap. Their case study is the reception of John Locke's An Essay Concerning Human Understanding (1689) in Eighteenth Century Collections Online (ECCO).

Heavily reused Locke passages served as queries. Experts labelled the retrieved hits with a heuristic reception taxonomy, working in an iterative human-in-the-loop process: pilot sampling came first, and deeper annotation then focused on promising results.

Semantic search found about ten times more implicit reception than string matching. This included reworked versions of Locke in scientific teaching texts and simplified educational manuals. Linguistic diagnostics, however, revealed "lexical gatekeeping": rankings still favour passages that share vocabulary with the query, even though the model copes well with OCR noise and syntactic variation.

The released expert-annotated dataset gives the field a reference point for judging retrieval on noisy historical corpora.

Key points

  • Taxonomy. Hits fall into Lexical Matches, Paraphrase Matches, Meaning Matches, Topical Matches, No Match and Don't Know. Paraphrases and Meaning Matches together count as significant reception.
  • Language analysis. Lingua was applied to 1,000 pilot samples.
    • French makes up 9.5% of Meaning Matches, against a 3.3% ECCO token baseline (nearly threefold).
    • Latin (3.4% of the corpus) shows up only as No Match.
  • Cross-lingual noise. Non-English texts make up 14.0% of top-ranked false positives, compared with 10.0% of true positives. This points to topic-driven noise at the head of the rankings.
  • English diagnostics (spaCy). The authors measured vocabulary similarity (Jaccard over lemmas), OOV rate and POS divergence.
    • True positives (N=40) have the highest vocabulary similarity, at 0.030.
    • False negatives (N=10) have the lowest overlap (0.009), the lowest hit OOV and the lowest POS divergence. In other words, the missed matches were the cleanest and most structurally similar passages.
  • Data release. The annotated data is available at github.com/COMHIS/locke-sim-data. The authors plan to extend it into a benchmark for dense retrieval in historical and low-resource domains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of HelsinkiHelsinki · Finland50 works · 5,318 citations🎓University of TurkuTurku · Finland17 works · 4,890 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 worksLiterature and Literary Theory · 519 works

Cite

Cite as Yu Wu, Ananth Mahadevan, Filip Ginter, Michael Mathioudakis and Mikko Tolonen. 2026. Matching Meaning at Scale: Evaluating Semantic Search for 18th-Century Intellectual History through the Case of Locke. Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.nlp4dh-1.20

@inproceedings{wu2026matching,
  title = {{Matching Meaning at Scale: Evaluating Semantic Search for 18th-Century Intellectual History through the Case of Locke}},
  author = {Yu Wu, Ananth Mahadevan, Filip Ginter, Michael Mathioudakis and Mikko Tolonen},
  booktitle = {Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities},
  year = {2026},
  doi = {10.18653/v1/2026.nlp4dh-1.20},
  url = {https://doi.org/10.18653/v1/2026.nlp4dh-1.20}
}

References

55 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Motasem Alrahabi and Tom Wainstain. 2025. Versus: An automatic text comparison tool for the digital humanities. In Proceedings of the First on Natural Language Processing and Language Models for Digital Humanities, pages 32–37, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria. find
  2. David Armitage and Jo Guldi. 2014. The History Manifesto. Cambridge University Press. find
  3. Emmanuelle Bermès. 2017. Text, data and link-mining in digital libraries: Looking for the heritage gold. In IFLA Satellite Meeting - Digital Humanities – Opportunities and Risks: Connecting Libraries and Research. find
  4. David M. Blei. 2012. Probabilistic topic models. Commun. ACM, 55(4):77–84. find
  5. Katherine Bode. 2018. A World of Fiction: Digital Collections and the Future of Literary History. University of Michigan Press, Ann Arbor. find
  6. Annelen Brunner, Stefan Engelberg, Fotis Jannidis, Ngoc Duyen Tanja Tu, and Lukas Weimer. 2020. Corpus REDEWIEDERGABE. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 803–812, Marseille, France. European Language Resources Association. find
  7. Simon Burrows and Mark Curran. 2012. The French Book Trade in Enlightenment Europe Project and the STN Database. Journal of Digital Humanities, 1(3). find
  8. Daniel Carey. 2006. Locke, Shaftesbury, and Hutcheson: Contesting Diversity in the Enlightenment and Beyond. Ideas in Context. Cambridge University Press, Cambridge. find
  9. Jeffrey R. Collins. 2020. In the Shadow of Leviathan: John Locke and the Politics of Conscience. Cambridge University Press. Google-Books-ID: 1tLKD-wAAQBAJ. find
  10. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2025. The FAISS Library. IEEE Transactions on Big Data, pages 1–17.
  11. Marten Düring, Matteo Romanello, Maud Ehrmann, Kaspar Beelen, Daniele Guido, Brecht Deseure, Estelle Bunout, Jana Keck, and Petros Apostolopoulos. 2023. Impresso Text Reuse at Scale. An interface for the exploration of text reuse data in semantically enriched historical newspapers. Frontiers in Big… find
  12. Research in a Digital Age: Reflections from the Mapping the Republic of Letters Project. The American Historical Review, 122(2):400–424. find
  13. Susan Fitzmaurice, Justyna A. Robinson, Marc Alexander, Iona C. Hine, Seth Mehl, and Fraser Dallachy. 2017. Linguistic DNA: Investigating Conceptual Change in Early Modern English Discourse. Studia Neophilologica, 89(sup1):21–38. find
  14. Brevin Franklin, Emily Silcock, Abhishek Arora, Tom Bryan, and Melissa Dell. 2024. News Deja Vu: Connecting Past and Present with Semantic Search. In Proceedings of the Sixth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS 2024), pages 99–112, Mexico City, Mexico. A… find
  15. Mario Giulianelli, Marco Del Tredici, and Raquel Fernández. 2020. Analysing lexical semantic change with contextualised word representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3960– 3973, Online. Association for Computational Linguistic…
  16. Andrew Goldstone and Ted Underwood. 2014. The Quiet Transformations of Literary Studies: What Thirteen Thousand Scholars Could Tell Us. New Literary History, 45(3):359–384. find
  17. Tyler Hanck. 2019. Locke’s Confusion About the Confused Idea of Substance. Thesis, University of Illinois Chicago. find
  18. James A. Harris. 2023. Of the origin of government: The afterlives of Locke and Filmer in an eighteenth-century British debate. Intellectual History Review, 33(1):33–55. find
  19. Mark J. Hill, Ville Vaara, Tanja Säily, Leo Lahti, and Mikko Tolonen. 2019. Reconstructing Intellectual Networks: From the ESTC’s bibliographic metadata to historical material. In Proceedings of the Digital Humanities in the Nordic Countries, Copenhagen, Denmark. find
  20. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength natural language processing in python. find
  21. Jenna Kanerva, Hanna Kitti, Li-Hsin Chang, Teemu Vahtola, Mathias Creutz, and Filip Ginter. 2025. Semantic search as extractive paraphrase span detection. Language Resources and Evaluation, 59(1):257–276. find
  22. Dan Edelstein. 2016. Intellectual History and Digital Humanities. Modern Intellectual History, 13(1):237– 246. find
  23. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769– 6781, Onl…
  24. Dan Edelstein, Paula Findlen, Giovanna Ceserani, Caroline Winterer, and Nicole Coleman. 2017. find
  25. Historical Pierre-Carl Langlais. 2021. Fictions littéraires de Gallica / Literary fictions of Gallica. find
  26. Daniel Layman. 2021. Locke’s Republican and Liberal Legacy. In The Lockean Mind. Routledge. Num Pages: 10. find
  27. Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’ Autume, Tomas Kocisky, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. 2021. Mind the Gap: Assessing Temporal Generalization in Neural Language… find
  28. H. P. Luhn. 1960. Key word-in-context index for technical literature (kwic index). American Documentation, 11(4):288–295. find
  29. Sean MacAvaney, Sergey Feldman, Nazli Goharian, Doug Downey, and Arman Cohan. 2022. ABNIRML: Analyzing the Behavior of Neural IR Models. Transactions of the Association for Computational Linguistics, 10:224–239. find
  30. Ananth Mahadevan, Michael Mathioudakis, Eetu Mäkelä, and Mikko Tolonen. 2025. Text reuse in large historical corpora: Insights from the optimization of a data science system. International Journal of Data Science and Analytics, 20(5):4631–4643. find
  31. Barbara McGillivray, Federico Nanni, and Kaspar Beelen. 2024. 10. Why Does Digital History Need Diachronic Semantic Search? In Computational Humanities. University of Minnesota Press. find
  32. Andrianos Michail, Juri Opitz, Yining Wang, Robin Meister, Rico Sennrich, and Simon Clematide. 2025. Cheap Character Noise for OCR-Robust Multilingual Embeddings. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11705–11716, Vienna, Austria. Association for Computational… find
  33. Grace Muzny, Michael Fang, Angel Chang, and Dan Jurafsky. 2017. A Two-stage Sieve Approach for Quote Attribution. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 460–470, Valencia, Spain. Association for Com… find
  34. Eetu Mäkelä, James Misson, Devani Singh, and Mikko Tolonen. 2025. Opening the Black Box of EEBO. Digital Scholarship in the Humanities, page fqaf086. find
  35. R. Porter. 2001. Enlightenment: Britain and the Creation of the Modern World. Penguin Books Limited. find
  36. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  37. Nils Reimers and Iryna Gurevych. 2020. Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4512–4525, Online. Association for Computational Linguistics.
  38. Hammad Rizwan, Domenic Rosati, Ga Wu, and Hassan Sajjad. 2025. Resolving Lexical Bias in Model Editing. In Forty-Second International Conference on Machine Learning. find
  39. Glenn Roe. 2024. Text reuse as cultural practice: Intertextuality in the 18th-century digital archive. Digital Enlightenment Studies, 2(1). find
  40. David Rosson, Eetu Mäkelä, Ville Vaara, Ananth Mahadevan, Yann Ryan, and Mikko Tolonen. 2023. Reception Reader: Exploring Text Reuse in Early Modern British Publications. Journal of Open Humanities Data, 9:5. ArXiv:2302.04084 [cs]. — [arXiv:2302.04084](https://arxiv.org/abs/2302.04084) find
  41. Julian Schelb, Michael Wittweiler, Marie Revellio, Barbara Feichtinger, and Andreas Spitz. 2026. Loci Similes: A Benchmark for Extracting Intertextualities in Latin Literature. Preprint, arXiv:2601.07533. find
  42. Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. Simple Entity-Centric Questions Challenge Dense Retrievers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6138–6148, Online and Punta Cana, Dominican Republic. Association for…
  43. David A. Smith, Ryan Cordel, Elizabeth Maddock Dillon, Nick Stramp, and John Wilkerson. 2014. Detecting and modeling local text reuse. In IEEE/ACM Joint Conference on Digital Libraries, pages 183– 192. find
  44. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and permuted pre-training for language understanding. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, pages 16857– 16867, Red Hook, NY, USA. Curran Associates Inc.
  45. Peter M. Stahl. 2021. Lingua: The most accurate natural language detection library for Python. https://github.com/pemistahl/lingua-py. Python bindings for the Lingua language detection library. ↗
  46. Timothy Stanton. 2018. John Locke and the Fable of Liberalism. The Historical Journal, 61(3):597–622. find
  47. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). find
  48. Iiro Tiihonen and Kira Hinderks. 2025. Genre Classification Workflow for the English Short Title Catalogue (ESTC). Transformations: A DARIAH Journal, Workflows(Metadata-based workflows):14754. find
  49. Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020-2025. Label Studio: Data labeling software. Open source software available from https://github.com/HumanSignal/label-studio. ↗
  50. Mikko Tolonen, Eetu Mäkelä, and Leo Lahti. 2022. The Anatomy of Eighteenth Century Collections Online (ECCO). Eighteenth-Century Studies, 56(1):95–123. Publisher: Johns Hopkins University Press. find
  51. Mikko Tolonen and Yann Ciarán Ryan. 2026. Computational Methods in Intellectual History, pages 239– 260. Proceedings of the British Academy. Liverpool University Press, United Kingdom. find
  52. Mikko Tolonen and Mark G. Spencer. 2025. The Reception of David Hume’s Essays in Eighteenth-Century Britain. In Max Skjönsberg and Felix Waldmann, editors, Hume’s Essays, Cambridge Critical Guides, pages 15–35. Cambridge University Press, Cambridge. find
  53. Aleksi Vesanto, Filip Ginter, Hannu Salmi, Asko Nivala, and Tapio Salakoski. 2017. A System for Identifying and Exploring Text Repetition in Large Historical Document Corpora. In Proceedings of the 21st Nordic Conference on Computational Linguistics, pages 330–333, Gothenburg, Sweden. Association fo… find
  54. Christopher N. Warren, Daniel Shore, Jessica Otis, Lawrence Wang, Mike Finegold, and Cosma Shalizi. 2016. Six Degrees of Francis Bacon: A Statistical Method for Reconstructing Large Historical Social Networks. DHQ: Digital Humanities Quarterly, 10(3):1. find
  55. Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. 2025. OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1744… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-matching-meaning-scale-evaluating-semantic/