Elamur Research⌕
☰
home / papers / 2026 / Smelling the Past: Investigating Historical Models for Olfactory Event Extraction

Smelling the Past: Investigating Historical Models for Olfactory Event Extraction

Paper · 2026 · Teresa Paccosi and Marijn Koolen

Summary

The paper asks whether encoders pretrained on historical text extract olfactory events from older documents better than models trained on present-day language. Earlier work on this task used only contemporary BERT models. The authors use the multilingual Odeuropa olfactory benchmark (Menini et al., 2022), which is annotated with a Frame Semantics scheme of Lexical Units (Smell_Word) and Frame Elements. They cover five languages: English, French, Italian, German and Dutch. For each language they fine-tune one historical and one contemporary monolingual BERT in a multitask setup built with MaChAmp, and they re-run the contemporary baselines under identical conditions.

Historical pretraining improves F1 for nearly every label in English and Dutch and gives smaller gains in German. In Italian and French, the contemporary models stay ahead. The authors go beyond raw scores by comparing strict and lenient span matching, measuring how predictions overlap with gold spans, inspecting errors by hand and comparing tokenisers. This analysis tests earlier explanations for weak performance on some Frame Elements. The study gives practical guidance on when in-domain historical pretraining pays off for diachronic semantic extraction, and the authors release their fine-tuned models.

Key points

  • Historical models used: MacBERTh (English), BERToldo (Italian), D'AlemBERT (French), GysBERT (Dutch) and Historical German BERT.
  • Contemporary baselines: bert-base-cased, bert-base-italian-cased, FlauBERT, BERTje and bert-base-german-cased.
  • Training setup:
    • MaChAmp multitask learning with 10 BIO sequence-labelling tasks, one for the Lexical Unit and one for each Frame Element.
    • A CRF decoder for each task.
    • Learning rate 1e-4, batch size 32, 30 epochs, and equal weight for all tasks.
  • Data: Slovenian, part of the original six-language benchmark, was left out because no monolingual historical model exists for it and the authors lacked expertise for error analysis. The nine Frame Elements include Smell_Source, Odour_Carrier, Quality, Perceiver, Evoked_Odorant and Effect.
  • Evaluation: span-level strict and lenient precision, recall and F1, plus overlap and tokeniser analyses. Code, data and fine-tuned models are available on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬KNAW Humanities ClusterAmsterdam · Netherlands2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 167 works

Cite

Cite as Teresa Paccosi and Marijn Koolen. 2026. Smelling the Past: Investigating Historical Models for Olfactory Event Extraction. Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4otmsnawg3kr

@inproceedings{paccosi2026smelling,
  title = {{Smelling the Past: Investigating Historical Models for Olfactory Event Extraction}},
  author = {Teresa Paccosi and Marijn Koolen},
  booktitle = {Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4otmsnawg3kr},
  url = {https://doi.org/10.63317/4otmsnawg3kr}
}

References

38 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Branden Chan, Stefan Schweter, and Timo Möller. 2020. German’s next language model. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6788–6796, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  2. Wietse De Vries, Andreas van Cranenburgh, Arianna Bisazza, Tommaso Caselli, Gertjan van Noord, and Malvina Nissim. 2019. Bertje: A dutch bert model. arXiv preprint arXiv:1912.09582.
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  4. Maud Ehrmann, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, and Antoine Doucet. 2023. Named entity recognition and classification in historical documents: A survey. ACM Computing Surveys, 56(2):1–47. find
  5. Charles J Fillmore et al. 2006. Frame semantics. Cognitive linguistics: Basic readings, 34:373– 400. find
  6. Ahmed Alajrami and Nikolaos Aletras. 2022. How does the pre-training objective affect what large language models learn about linguistic properties? arXiv preprint arXiv:2203.10415. find
  7. Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, et al. 2024. Tokenizer choice for llm training: Negligible or crucial? In Findings of the Association for Computational Linguistics: NAACL 2024, page…
  8. Alessio Palmero Aprosio, Stefano Menini, and Sara Tonelli. 2022. Bertoldo, the historical bert for italian. In Proceedings of the second workshop on language technologies for historical and ancient languages, pages 68–72. find
  9. Ryan Brate, Paul Groth, and Marieke van Erp. 2020. Towards olfactory information extraction from text: A case study on detecting smell experiences in novels. In Proceedings of the The 4th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Litera… find
  10. Rich Caruana. 1997. Multitask learning. Machine learning, 28(1):41–75. find
  11. Simon Gabay, Pedro Ortiz Suarez, Alexandre Bartz, Alix Chagué, Rachel Bawden, Philippe Gambette, and Benoît Sagot. 2022. From FreEM to d’AlemBERT: a large corpus and a language model for early Modern French. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3367–33… find
  12. Clovis Gladstone, Zhao Fang, and Spencer Dean Stewart. 2025. Ground truth generation for multilingual historical nlp using llms. arXiv preprint arXiv:2511.14688. — [arXiv:2511.14688](https://arxiv.org/abs/2511.14688) find
  13. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964. find
  14. Leah Hamilton, Caitlin Neill, and Jacob Lahne. 2023. Flavor language in expert reviews versus consumer preferences: an application to expensive american whiskeys. Food Quality and Preference. find
  15. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146. find
  16. David Howes. 2006. Charting the sensorial revolution. find
  17. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning 2001 (ICML 2001), pages 282–289. find
  18. Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoit Crabbé, Laurent Besacier, and Didier Schwab. 2020. FlauBERT: Unsupervised language model pre-training for French. In Proceedings of the Twelfth Language Resources and Evaluation Conferen…
  19. E. Lefever, I. Hendrickx, I. Croijmans, A. Van den Bosch, and A. Majid. 2018. Discovering the language of wine reviews: A text mining account. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Associ… find
  20. Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Xin Zhao, and Ji-Rong Wen. 2022. ElitePLM: An empirical study on general language ability evaluation of pretrained language models. In Proceedings of the 2022 Conference of the North American Chapter of the Assoc… find
  21. Enrique Manjavacas and Lauren Fonteyn. 2021. Macberth: Development and evaluation of a historically pre-trained language model for english (1450-1950). In Proceedings of the Workshop on Natural Language Processing for Digital Humanities, pages 23–36.
  22. Enrique Manjavacas Arevalo and Lauren Fonteyn. 2022. Non-parametric word sense disambiguation for historical languages. In Proceedings of the 2nd International Workshop on Natural Language Processing for Digital Humanities, pages 123–134, Taipei, Taiwan. Association for Computational Linguistics. of… find
  23. Stefano Menini, Teresa Paccosi, Sara Tonelli, Marieke Van Erp, Inger Leemans, Pasquale Lisena, Raphael Troncy, William Tullett, Ali Hür-riyetoğlu, Ger Dijkstra, et al. 2022. A multilingual benchmark to capture olfactory situations over time. In Proceedings of the 3rd workshop on computational appro… find
  24. Barbara Plank. 2016. What to do about non-standard (or non-canonical) language in nlp. arXiv preprint arXiv:1608.07836. — [arXiv:1608.07836](https://arxiv.org/abs/1608.07836) find
  25. Josef Ruppenhofer, Michael Ellsworth, Miriam RL Petruck, Christopher R Johnson, and Jan Scheffczyk. 2016. Framenet ii: Extended theory and practice. find
  26. Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? on the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Join…
  27. Stefan Schweter. 2020. Italian bert and electra models. Zenodo. find
  28. Stefan Schweter, Luisa März, Katharina Schmid, and Erion Çano. 2022. hmbert: Historical multilingual language models for named entity recognition. arXiv preprint arXiv:2205.15575. — [arXiv:2205.15575](https://arxiv.org/abs/2205.15575) find
  29. Dan Sperber. 1975. Rethinking symbolism. 11. CUP Archive. find
  30. Riste Stojanov, Gorjan Popovski, Gjorgjina Cenikj, Barbara Koroušić Seljak, and Tome Eftimov. 2021. A fine-tuned bidirectional encoder representations from transformers model for food named-entity recognition: Algorithm development and validation. Journal of medical Internet research, 23(8):e28229. find
  31. Steven McGregor and Barbara McGillivray. 2018. A distributional semantic methodology for enhanced search in historical records: A case study on smell. In 14th Conference on Natural Language Processing. find
  32. Simeng Sun, Brian Dillon, and Mohit Iyyer. 2022. How much do modifications to transformer language models affect their ability to learn linguistic knowledge? In Proceedings of the Third Workshop on Insights from Negative Results in NLP, pages 46–53, Dublin, Ireland. Association for Computational Lin… find
  33. Stefano Menini. 2024. Semantic frame extraction in multilingual olfactory events. In Proceedings Serra Sinem Tekiroğlu, Gözde Özbal, and Carlo Strapparava. 2014. Sensicon: An automatically constructed sensorial lexicon. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language… find
  34. Sara Tonelli and Stefano Menini. 2021. Framenet-like annotation of olfactory information in texts. In Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 11–20. find
  35. William Tullett, Inger Leemans, Hsuan Hsu, Stephanie Weismann, Cecilia Bembibre, Melanie A Kiechle, Duane Jethro, Anna Chen, Xuelei Huang, Jorge Otero-Pailos, et al. 2022. Smell, history, and heritage. The American Historical Review, 127(1):261–309. find
  36. Rob Van Der Goot, Ahmet Üstün, Alan Ramponi, Ibrahim Sharaf, and Barbara Plank. 2021. Massive choice, ample tasks (machamp): A toolkit for multi-task learning in nlp. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations…
  37. Stella Verkijk, Piek Vossen, and Pia Sommerauer. 2025. Language models lack temporal generalization and bigger is not better. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20629–20637. find
  38. Bodo Winter, Marcus Perlman, and Asifa Majid. 2018. Vision dominates in perceptual language: English sensory vocabulary is optimized for usage. Cognition, 179:213–220. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/paccosi-2026-smelling-past-investigating-historical-models/