📄 Weakly Supervised Named Entity Recognition for Historical Texts

Summary

Named entity recognition on historical documents usually depends on experts annotating full texts by hand, which is slow and costly. The authors ask whether weakly supervised NER, specifically distantly supervised (DS-NER) methods designed for modern text, can replace that annotation. They use the medieval French charter dataset of Torres Aguilar (2022), which draws on three sources (CBMA, CDBE, HOME) and labels persons and locations. For each source, they sample partial entity dictionaries from the training split, covering 10% to 100% of the entities, and create noisy training labels through string matching. The validation and test sets keep their original gold labels. They benchmark seven DS-NER techniques against dictionary matching and a supervised domain-specific model. They then pair the most robust technique, CuPUL, with XLM-RoBERTa base and with Stacked Embeddings, a BiLSTM-CRF model built for medieval texts. Even with only a tenth of the dictionary entries, CuPUL with Stacked Embeddings keeps more than 80% of its fully supervised F1. This suggests that a small entity list can stand in for full manual annotation of low-resource historical collections.

Key points

• Data: The dataset is the Torres Aguilar (2022) collection of medieval charters, with PERS and LOC entities.
  • Training splits contain about 38.7k tokens (CBMA, Arbois cartulary), 235.6k tokens (CDBE, Diplomata Belgica) and 114.6k tokens (HOME-Alcar).
  • The number of distinct training entities ranges from 652/347 PERS/LOC in CBMA to 4118/2662 in CDBE.
• Methods compared: The study covers positive-unlabeled learning (Conf-MPU, CuPUL), self-training approaches (BOND, SCDL, CENSOR) and other techniques (SANTA, MProto).
• Findings by method: Most techniques do well with large dictionaries but degrade sharply as dictionaries shrink, and the self-training methods are particularly unstable. CuPUL is the most noise-tolerant, mainly because it keeps recall high.
• Main result: CuPUL combined with Stacked Embeddings stays above 80% of supervised F1 at every dictionary size. The authors suggest that the BiLSTM-CRF's lower capacity may reduce overfitting.
• Limitations and code: The study covers only manually transcribed French charters, two entity types and randomly sampled internal dictionaries, and it still requires a fully annotated validation set. Code is available at github.com/msorbi/hwsner.