Higher-Order Coreference Resolution with Coarse-to-Fine Inference
Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N18-2108.pdf
Summary
First-order coreference systems score only pairs of mentions. Each link is decided on its own, so a cluster can be made of links that each look plausible but don't fit together. In the paper's example, an ambiguous "you" connects a singular "I" with a plural "all of you". The authors extend the end-to-end span-ranking model of Lee et al. (2017) with a differentiable approximation of higher-order inference. The predicted antecedent distribution acts as attention and produces an expected antecedent vector. A learned gate merges this vector into each span's representation, and the spans are then rescored. Repeating this step is expensive, so they add a cheap bilinear "coarse" factor. It ranks every candidate antecedent and passes only the best few to the costly "fine" scorer, and the whole model is trained with a single objective. On the English CoNLL-2012 (OntoNotes) data the system reaches 73.0 average F1, a new state of the art at the time, while scoring far fewer antecedents per span. The work shows that a model can enforce consistent clusters and still be efficient.
Key points
- Baseline: Lee et al. (2017) with ELMo embeddings and tuned hyperparameters, scoring 72.3 average F1 (MUC, B³, CEAFφ4) on the test set. Adding coarse-to-fine pruning gives 72.6, and adding second-order inference gives 73.0.
- The coarse-to-fine model sends only 50 antecedents per span to the fine scorer, compared with the 250 nearest used by the distance heuristic. On the dev set, cutting K from 250 to 50 costs under 0.2 F1 with coarse-to-fine pruning but almost 5 F1 with the heuristic.
- Using the bilinear score on its own instead of the full pairwise scorer loses more than 3 F1. It works as a pre-filter, not as the only scorer.
- Second-order inference adds 0.8 F1 on the dev set, and a third iteration adds just 0.1. The gain comes mainly from higher precision, while coarse-to-fine pruning mainly raises recall.
- The code is public at github.com/kentonl/e2e-coref.
Cited by
- An Annotated Dataset of Coreference in English Literature
- Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution
- Coreference Resolution through a seq2seq Transition-Based System
- Do UD Trees Match Mention Spans in Coreference Annotations?
- Word-Level Coreference Resolution
- Multi-Sentence Argument Linking
- The KnowRef Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution
- Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling
- Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction
- BERT for Coreference Resolution: Baselines and Analysis
- SpanBERT: Improving Pre-training by Representing and Predicting Spans
- AMALGUM – A Free, Balanced, Multilayer English Web Corpus
- Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction
- Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
- Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection
- LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution
- Unifying Large Language Models and Knowledge Graphs: A Roadmap
- Scoring Coreference Chains with Split-Antecedent Anaphors
- A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
- Mitigating Gender Bias in Natural Language Processing: Literature Review
- What do you learn from context? Probing for sentence structure in contextualized word representations
- Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks
- On Generalization in Coreference Resolution
- Entity, Relation, and Event Extraction with Contextualized Span Representations
- CorefQA: Coreference Resolution as Query-based Span Prediction
- Incremental Neural Coreference Resolution in Constant Memory
- Revealing the Myth of Higher-Order Inference in Coreference Resolution
- Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution
- The CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue
- PCR4ALL: A Comprehensive Evaluation Benchmark for Pronoun Coreference Resolution in English
- Anatomy of OntoGUM—Adapting GUM to the OntoNotes Scheme to Evaluate Robustness of SOTA Coreference Algorithms
Cite
@inproceedings{lee2018higher,
title = {{Higher-Order Coreference Resolution with Coarse-to-Fine Inference}},
author = {Lee, Kenton and He, Luheng and Zettlemoyer, Luke},
booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
year = {2018},
doi = {10.18653/v1/n18-2108},
eprint = {1804.05392},
archivePrefix = {arXiv},
url = {https://doi.org/10.18653/v1/n18-2108}
}