Active Learning for Anaphora Resolution

Summary

This work asks whether active learning can lower the cost of building training data for anaphora resolution. Annotating anaphoric links by hand is notoriously slow, and the author reports no earlier attempt to apply active learning to this task. The base learner is the author's probabilistic resolver for non-pronominal anaphora in biomedical text (Gasperin and Briscoe, 2008). It labels each anaphor–candidate pair as coreferent, biotype, set-member, or no relation, and it can also detect discourse-new mentions. Active learning was simulated on an annotated corpus of full-text Drosophila papers. Part of the data served for initial training, and the rest acted as an unlabelled pool. Three entropy-based uncertainty measures picked new training instances over 10 iterations, and each was compared against random selection of the same size. None of the measures beat random sampling consistently. The author attributes this mainly to the difficulty of merging the probabilities of many antecedent candidates into a single uncertainty score. The negative result is useful because it shows that methods that work well for tasks such as NER or parse selection do not carry over directly to anaphora resolution.

Key points

Corpus: five full-length Drosophila biomedical articles (about 33,300 words), released through the FlySlip project.
- Preprocessing used a gene name recogniser, the RASP parser, and biotypes derived from the Sequence Ontology (SO).
- It contains 3,048 positive cases: 1,678 coreferent, 274 biotype, 543 set-member, and 436 discourse-new.
Negative instances: there were 873,731 negatives. They were cut to 85,314 by grouping identical feature vectors and keeping one-tenth of each group, which also discarded groups with fewer than 10 members.
Model: the resolver uses 11 features and a Bayes-based decomposition inspired by Ge et al. (1998). It can be seen as a naive-Bayes variant that allows some dependence between features.
Selection measures: LE(A, a) works on pairs and selected 1,500 pairs per iteration. GE1(A) and GE2(A) work per anaphor and selected the 50 most uncertain anaphors per iteration.
Results:
- LE(A, a) chose mostly negatives and lowered overall performance, although precision rose for coreferent and set-member cases.
- GE1(A) tended to improve precision and drew about 70% proper names.
- GE2(A) tended to improve recall at the cost of precision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

8 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gasperin2009active,
  title = {{Active Learning for Anaphora Resolution}},
  author = {Gasperin, Caroline},
  booktitle = {Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing},
  year = {2009},
  doi = {10.3115/1564131.1564133},
  url = {https://doi.org/10.3115/1564131.1564133}
}

References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Jason Baldridge and Miles Osborne. 2003. Active learning for hpsg parse selection. In Walter Daelemans and Miles Osborne, editors, Proceedings of CoNLL-2003, pages 17–24. Edmonton, Canada. find
Edward J. Briscoe, John Carroll, and Rebecca Watson. 2006. The second release of the RASP system. In Proceedings of ACL-COLING 06, Sydney, Australia. find
K. Bretonnel Cohen, Lynne Fox, Philip Ogren, and Lawrence Hunter. 2005. Corpus design for biomedical natural language processsing. In Proceedings of the ACL-ISMB Workshop on Linking Biological Literature, Ontologies and Databases, Detroit. find
Karen Eilbeck and Suzanna E. Lewis. 2004. Sequence ontology annotation guide. Comparative and Functional Genomics, 5:642–647. find
Caroline Gasperin and Ted Briscoe. 2008. Statistical anaphora resolution in biomedical texts. In Proceedings of COLING 2008, Manchester, UK. find our card
Caroline Gasperin, Nikiforos Karamanis, and Ruth Seal. 2007. Annotation of anaphoric relations in biomedical full-text articles using a domain-relevant scheme. In Proceedings of DAARC 2007, pages 19–24, Lagos, Portugal. find
Niyu Ge, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora - COLING-ACL’98, Montreal, Canada. find our card
Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of ACL 2002, Philadelphia. find our card
Massimo Poesio and Renata Vieira. 1998. A corpus-based investigation of definite description use. Computational Linguistics, 24(2):183–216. find
Dan Shen, Jie Zhang, Jian Su, Guodong Zhou, and Chew-Lim Tan. 2004. Multi-criteria-based active learning for named entity recognition. In Proceedings of ACL 2004, Barcelona. find
Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. find our card
Michael Strube, Stefan Rapp, and Christoph Mller. 2002. The influence of minimum edit distance on reference resolution. In Proceedings of the EMNLP 2002, Philadelphia. find
Min Tang, Xiaoqiang Luo, and Salim Roukos. 2002. Active learning for statistical natural language parsing. In Proceedings of ACL 2002, pages 120–127, Philadelphia, Pennsylvania. Association for Computational Linguistics. find
Andreas Vlachos and Caroline Gasperin. 2006. Bootstrapping and evaluating named entity recognition in the biomedical domain. In Proceedings of BioNLP at HLT-NAACL 2006, pages 138–145, New York. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/gasperin-2009-active-learning-anaphora-resolution/

📄 Active Learning for Anaphora Resolution