Elamur Research⌕
☰
home / papers / 2020 / Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

Summary

The paper addresses few-shot named entity recognition, where a system must tag new entity types from only one or a handful of labeled examples per class. The authors argue that earlier prototypical-network approaches perform poorly for two reasons. First, they classify tokens independently and ignore label dependencies. Second, they collapse the heterogeneous O class into one noisy prototype.

Their system, StructShot, avoids meta-training altogether. It trains an ordinary supervised NER model (BiLSTM or fine-tuned BERT) on a source domain and reuses its token representations. At test time, each token takes the tag of its closest support-set token. A Viterbi decoder then adds structure, using tag-transition statistics counted in the source data and mapped onto the target tag set.

The authors also propose a more realistic evaluation protocol. Support sets are drawn greedily from benchmark development sets, and models are scored on the original test sets rather than on balanced, sampled episodes.

Across in-domain and cross-domain tests in one-shot and five-shot settings, StructShot beats meta-learning baselines by 6–16 absolute F1 points. This makes it a cheap, strong baseline that fits easily alongside standard NER systems.

Key points

  • Two components. NNShot is a token-level nearest-neighbor classifier that uses squared Euclidean distance over L2-normalized features. StructShot adds Viterbi decoding with abstract transition probabilities estimated from source-domain counts, split evenly across the related target transitions.
  • Encoders. Token features come from one of two encoders: a BiLSTM-based tagger following Lample et al. (2016), or a cased BERT-base model fine-tuned using a standard cross-entropy objective. Representations are taken just before the final linear layer.
  • Evaluation and data. Datasets include OntoNotes 5, CoNLL, I2B2 2014 and WNUT. Support sets are built by greedy sampling that starts from the rarest classes, which keeps class counts balanced.
  • Gains over prior work. Earlier prototypical-network systems averaged roughly 10–30% F1. StructShot improves on such systems by 6–16 absolute F1 points.
  • Remaining errors. The system struggles with ambiguous classes: numeric I2B2 types (AGE, MEDICAL-RECORD, PHONE, IDNUM), PATIENT versus DOCTOR, and WNUT group versus corporation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition

Cited by

89 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 85 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.516

@inproceedings{yang2020simple,
  title = {{Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning}},
  author = {Yang, Yi and Katiyar, Arzoo},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.516},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.516}
}

References

11 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, and Ting Liu. 2020. Few-shot slot tagging with collapsed dependency transfer and label-enhanced task-adaptive projection network. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  2. Yujia Bao, Menghua Wu, Shiyu Chang, and Regina Barzilay. 2020. Few-shot text classification with distributional signatures. In International Conference on Learning Representations. find
  3. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  4. Colin Cherry, Greg Durrett, George Foster, Reza Haffari, Shahram Khadivi, Nanyun Peng, Xiang Ren, and Swabha Swayamdipta. 2019. Proceedings of the 2nd workshop on deep learning approaches for low-resource nlp (deeplo 2019). In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resou… find
  5. Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017. Results of the wnut2017 shared task on novel and emerging entity recognition. In Proceedings of the 3rd Workshop on Noisy User-generated Text. find our card
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the International Conference… find
  7. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  8. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research. find
  9. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP). find
  10. Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. In Neural Information Processing Systems (NIPS). find
  11. Amber Stubbs and Özlem Uzuner. 2015. Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/uthealth corpus. Journal of biomedical informatics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2020-simple-effective-few-shot-named/