Elamur Research⌕
☰
home / papers / 2025 / The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling

The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling

Paper · 2025 · Sophie Spliethoff, Sanne Hoeken, Silke Schwandt et al.

Summary

The paper addresses how to make a humanities concept, invective language in the sense of Schwerhoff et al. (2017), usable for machine learning on Early Modern English (EModE) sources from Tudor England (1485–1603). The authors describe a full pipeline. They take printed works from EEBO, clean them, split them into sentences and filter by language. They then draw a sample balanced by year and author confession and annotate it iteratively with domain experts. The result is the InviTE corpus, close to 2000 sentences. Each sentence is labelled for invectivity, with finer layers such as target and literal versus metaphorical type, plus annotator confidence. Invective sentences were rare in the random sample, so the authors added keyword-retrieved sentences. On binary invective detection, fine-tuned BERT-style encoders beat much larger instruction-tuned LLMs prompted zero-shot. MacBERTh, pre-trained on historical English, performs best. The study shows that careful annotation and domain-matched pre-training still matter, even though zero-shot LLMs remain a reasonable fallback when no labels exist.

Key points

  • InviTE contains 1975 EModE sentences from EEBO prints published in England between 1485 and 1603, covering the Protestant Reformation period.
  • 1443 sentences were sampled evenly across publication year and speaker group (Catholic vs. non-Catholic). Only 182 of these turned out to be invective, so 532 more were added through keyword search (keywords listed in Appendix A).
  • Pre-processing removed headers, footers and page numbers, assigned sentence IDs and used langdetect to drop non-English sentences such as Latin. Author gender and confession were added by hand.
  • Two trained domain experts annotated the data. The scheme covers invective vs. non-invective, target, and type (literal or metaphorical), and records annotator confidence.
  • Fine-tuned encoders, MacBERTh best of all, outperform zero-shot LLMs. The LLMs are still competitive without any task training.
  • Stated limitations:
    • The keyword enrichment skews class proportions, so the corpus should not be used to measure invective prevalence.
    • The IAA measures ignore dependencies between annotation categories.
    • Only zero-shot prompting was tested.
    • Only the binary label was modelled.

Interesting ideas

  • I1. Each label carries the annotator's confidence, so interpretive uncertainty becomes part of the data rather than noise hidden inside a single gold label.
  • I2. Rare phenomena are boosted with targeted keyword retrieval, and the authors openly note that this breaks the natural distribution and rules out frequency claims.
  • I3. Error analysis showed two failure modes for surface lexical cues. Devotional self-abasement uses harsh words aimed at the speaker, and "not + negative term" constructions mislead the models.
  • I4. Some derogatory metaphors only make sense against period knowledge, such as the wolf standing for Catholics in Reformation polemic.
  • I5. Instead of normalizing variable historical spelling, the authors keep it to avoid losing meaningful variation, and they filter out mixed-language sentences with a language detector before annotation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data could keep a per-item confidence derived from how strongly the witness parsers agree, and training or evaluation could weight or filter by it.
  • I2 (learn): Rare constructions could be over-sampled for training while a separate, untouched random sample is kept for any frequency estimates or evaluation figures.
  • I3 (frames): Speech-act and irony modules should decide whether something derogates from the frame's target and the negation scope in the UD tree, not from the presence of negative words.
  • I4 (world): The compiled knowledge layer could hold era- or community-specific figurative mappings, so metaphor readings come from explicit, inspectable facts rather than guesswork.
  • I5 (tokens): The tokenizer could keep the original surface form alongside a normalized lemma and flag foreign-language spans, so later stages never lose the source evidence.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLiterature and Literary Theory · 519 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Sophie Spliethoff, Sanne Hoeken, Silke Schwandt, Sina Zarrieß and Ozge Alaçam. 2025. The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling. arXiv:2509.22345.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2509.22345 · arXiv 2509.22345

@misc{spliethoff2025invite,
  title = {{The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling}},
  author = {Sophie Spliethoff, Sanne Hoeken, Silke Schwandt, Sina Zarriess and Ozge Alaccam},
  year = {2025},
  doi = {10.48550/arxiv.2509.22345},
  eprint = {2509.22345},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.22345}
}

References

32 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ali Al-Laith, Kirstine Nielsen Degn, Alexander Conroy, Bolette Sandford Pedersen, Jens Bjerring-Hansen, and Daniel Hershcovich. 2023. Sentiment classification of historical Danish and Norwegian literary texts. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 324– 334, Tórshavn, Faroe Islands. University of Tartu Library. find
  2. Christin Beck and Marisa Köllner. 2023. GHisBERT – training BERT from scratch for lexical semantic investigations across historical German language stages. In Proceedings of the 4th Workshop on Computational Approaches to Historical Language Change, pages 33–45, Singapore. Association for Computational Linguistics. find
  3. Antoinina Bevan Zlatar. 2011. Reformation Fictions. Polemical Protestant Dialogues in Elizabethan England. Oxford University Press, Oxford. find
  4. Marcel Bollmann. 2019. A large-scale comparison of historical text normalization systems. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3885–3898, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  5. Pierluigi Cassotti and Nina Tahmasebi. 2025. Sense-specific historical word usage generation. Transactions of the Association for Computational Linguistics, 13:690–708. ↗
  6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. ↗
  7. John Craig. 2016. Reformation, Politics and Polemics. The Growth of Protestantism in East Anglian Market Towns, 1500-1610. Ashgate, Aldershot. ↗
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  9. Paula Fortuna and Sérgio Nunes. 2018. A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Surveys, 51(4):1–30. ↗
  10. Stephen Gramley and Vivian Gramley. 2024. The History of English. An Introduction, 3 edition. Routledge, London / New York. find
  11. Ian Green. 2000. Print and Protestantism in Early Modern England. Oxford. ↗
  12. Jakob Hauser, Daniel Kondor, Jenny Reddish, Majid Benam, Enrico Cioni, Federica Villa, James S. Bennett, Daniel Hoyer, Pieter Francois, Peter Turchin, and R. Maria del Rio-Chanona. 2024. Large language models'expert-level global history knowledge benchmark (hist-llm). In Advances in Neural Information Processing Systems, volume 37, pages 32336–32369. Curran Associates, Inc. find
  13. Sanne Hoeken, Sophie Spliethoff, Silke Schwandt, Sina Zarrieß, and Özge Alacam. 2023. Towards Detecting Lexical Change of Hate Speech in Historical Sources. Proceedings of the 4th Workshop on Computational Approaches to Historical Language Change, pages 100–111. find
  14. Ding Huang, Jiajin Xu, Yingming Song, and Ruchen Yu. 2025. Assessing the potential of using large language models for pragmatic annotation of historical texts: A case of epistemic stance in early modern english. submitted to the Journal of Historical Pragmatics. ↗
  15. Erin Katherine Kelly. 2015. Chasing the Fox and the Wolf. Hunting in the Religious Polemic of William Turner. Reformation, 20(2):113–129. find
  16. Alexander Kästner and Wiebke Voigt. 2020. Jedermann? Überlegungen zur Potenzialität und Entgren-zung von Öffentlichkeit in der Reformation. In Sabine Müller-Mall and Jan-Philipp Kruse, editors, Digitale Transformationen der Öffentlichkeit, pages 123–161. Velbrück Wissenschaft, Weilerswist. find
  17. Enrique Manjavacas and Lauren Fonteyn. 2022. Adapting vs. Pre-training Language Models for Historical Languages. Journal of Data Mining & Digital Humanities, pages 1–19. ↗
  18. Enrique Manjavacas Arevalo and Lauren Fonteyn. 2022. Non-parametric word sense disambiguation for historical languages. In Proceedings of the 2nd International Workshop on Natural Language Processing for Digital Humanities, pages 123–134, Taipei, Taiwan. Association for Computational Linguistics. find
  19. Enrique Manjavacas Arévalo and Lauren Fonteyn. 2021. MacBERTh: Development and evaluation of a historically pre-trained language model for English (1450-1950). In Proceedings of the Workshop on Natural Language Processing for Digital Humanities (NLP4DH), pages 23–36. Association for Computational Linguistics.
  20. Barbara Plank. 2022. The “problem” of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. ↗
  21. Wenjun Qiu and Yang Xu. 2022. Histbert: A pre-trained language model for diachronic lexical semantic analysis. CoRR, abs/2202.03612. ↗
  22. James Raven and Joran Proot. 2023. Renaissance and Reformation. In James Raven, editor, The Oxford History of the Book, pages 128–159. Oxford University Press, Oxford. ↗
  23. Nils Reiter. 2020. Anleitung zur Erstellung von Annotationsrichtlinien. In Nils Reiter, Axel Pichler, and Jonas Kuhn, editors, Reflektierte algorithmische Textanalyse. Interdisziplinäre(s) Arbeiten in der CRETA-Werkstatt, pages 193–201. De Gruyter, Berlin; Boston. ↗
  24. Nils Reiter, Marcus Willand, and Evelyn Gius. 2020. Die Erstellung von Annotationsrichtlinien als Community-Aufgabe für die Digitalen Geisteswissenschaften. In Julia Nantke and Frederik Schlup-kothen, editors, Annotations in Scholarly Editions and Research. Functions, Differentiation, Systematization, pages 325–350. De Gruyter, Berlin / Boston. ↗
  25. Paul Röttger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. 2022. Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 175–190.
  26. Gerd Schwerhoff. 2020. Invektivität und Geschichtswis-senschaft. Konstellationen der Herabsetzung in historischer Perspektive - ein Forschungskonzept. Historische Zeitschrift, 311:1–36. find
  27. Gerd Schwerhoff, Dagmar Ellerbrock, Lars Koch, Sabine Müller-Mall, Marina Münkler, Joachim Scharloth, and Dominik Schrage. 2017. Invektivität. Perspektiven eines neuen Forschungsprogramms in den Kultur- und Sozialwissenschaften. Kulturwis-senschaftliche Zeitschrift, 2(1):2–24. find
  28. Rachele Sprugnoli, Sara Tonelli, Alessandro Marchetti, and Giovanni Moretti. 2016. Towards sentiment analysis for historical texts. Digital Scholarship in the Humanities, 31(4):762–772. ↗
  29. Almut Suerbaum. 2015. Language of Violence. Language as Violence in Vernacular Sermons. In Almut Suerbaum, George Southcombe, and Benjamin Thompson, editors, Polemic. Language as Violence in Medieval and Early Modern Discourse, pages 125– 148. Routledge, London / New York. find
  30. K. Todorov and G. Colavizza. 2022. An assessment of the impact of ocr noise on language models. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence (ICAART 2022) - Volume 2, pages 674–683. SciTePress. find
  31. Daniel van Strien, Kaspar Beelen, Mariona Coll Ardanuy, Kasra Hosseini, Barbara McGillivray, and Giovanni Colavizza. 2020. Assessing the impact of ocr quality on downstream nlp tasks. In Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 1: ARTIDIGH, pages 484–496. INSTICC, SciTePress. find
  32. Ran Zhang, Jihed Ouni, and Steffen Eger. 2024. Cross-lingual cross-temporal summarization: Dataset, models, evaluation. Computational Linguistics, 50(3):1001–1047. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/spliethoff-2025-invite-corpus-annotating-invectives-tudor/