Elamur Research
☰
home / papers / 2017 / End-to-end Neural Coreference Resolution

End-to-end Neural Coreference Resolution

What it is about

The first coreference model trained end to end without a parser or a mention detector. Every span up to 10 words is a candidate; each span gets a mention score and each pair an antecedent score, and a span chooses among all earlier spans plus a dummy antecedent fixed at score 0 (no link, or not a mention). Span vectors combine BiLSTM states at the boundaries with an attention-weighted «soft head» that learns something close to syntactic heads; pair features are same-speaker, genre, binned distance and span width. Spans are pruned greedily by mention score (0.4 per word, up to 250 antecedents) and the model maximises the marginal likelihood of all gold antecedents. On CoNLL-2012 (OntoNotes) it reaches 67.2 average F1 single and 68.8 as a five-model ensemble, +1.5/+3.1 over the previous best; distance and width features are worth 3.8 F1, and gold mentions would add over 17.5 F1, while a rule-based mention detector costs 1 F1.

Ideas

For Mova

languages/en/35-reference-and-coreference.md §8–9 (algorithms; Mova's coref crate). Code: coref/src/eval.rs (oracle mentions), coref/src/mention.rs, coref/src/sieve.rs, a new ranker next to it. Measures: CoNLL F1 on GUM dev with gold and SLM trees, with and without oracle mentions.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D17-1018.pdf

Summary

Earlier coreference resolution systems relied on syntactic parsers for head-word features and on hand-written rules to propose mentions. As a result, parsing errors carried through the pipeline, and the rules transferred poorly to new languages and domains. Lee et al. propose the first fully end-to-end neural model for the task. It treats every text span up to a set width as a possible mention and, for each span, learns a distribution over earlier spans plus a dummy option meaning "no antecedent." Each span is represented by bidirectional LSTM states at its edges together with an attention-weighted mix of its inner words, which acts as a learned head finder. The pair score splits into mention scores for each span and an antecedent score. This split lets the model discard unlikely spans before doing the costly pairwise work. Training maximizes the marginal likelihood of any correct antecedent in the gold clusters, with no parser or other external resources. On OntoNotes the model beats previous systems by 1.5 F1, or by 3.1 F1 as a 5-model ensemble. It showed that dropping pipeline components can actually raise accuracy.

Key points

  • Scoring is factored as s(i,j) = s_m(i) + s_m(j) + s_a(i,j), with the dummy antecedent fixed at 0. Scoring all span pairs naively costs O(T⁴), so the model keeps only 0.4 spans per word, which still covers about 92.7% of gold mentions.
  • Words are represented by fixed pretrained embeddings plus a character-level CNN. A separate BiLSTM runs over each sentence, because cross-sentence context did not help. Pair features encode speaker, genre and span distance.
  • The learned attention is never trained on syntactic heads, yet its top-weighted word matches the syntactic head in 68–93% of cases. Among predicted spans of 2–5 words, 75–90% are syntactic constituents.
  • Results: +1.5 F1 for a single model and +3.1 F1 for a 5-model ensemble on the OntoNotes benchmark, without a syntactic parser or a hand-engineered mention detector.
  • Error analysis shows that word embeddings raise recall but also create false links between mentions that are similar but not the same entity, such as "flight attendants" vs. "pilots."

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1018 · arXiv 1707.07045

@inproceedings{lee2017end,
  title = {{End-to-end Neural Coreference Resolution}},
  author = {Lee, Kenton and He, Luheng and Lewis, Mike and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1018},
  eprint = {1707.07045},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1018}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2015. TensorFlow: Large-scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. find