Domain Adaptation with Active Learning for Coreference Resolution

Summary

Coreference systems trained on newswire perform poorly on biomedical text, and annotating coreference in a new domain is very costly. This work tackles that problem by adapting a classifier-based coreference resolver from newswire (the NPAPER training data) to MEDLINE abstracts (the GENIA-MEDCo corpus). The authors combine three components: domain adaptation (Augment, Instance Weighting, and Instance Pruning), pool-based active learning that uses uncertainty to choose which target-domain instances to label, and extra weight on the target-domain instances that have been labeled. Evaluation uses the MUC metric on 399 randomly chosen abstracts. The best configuration, Augment with target weighting, reaches almost the same F-measure as a system trained on full target annotations, while labeling only a tiny fraction of the instances. According to the authors, no earlier study had merged active learning with domain adaptation for the coreference task. It shows that a resolver for a specialized domain can be built cheaply by reusing resources from a resource-rich domain.

Key points

Training only on newswire gives 39.8% MUC F on the GENIA test set. Augment+Weighted reaches 49.0% after 100 active-learning iterations, with recall of 37.3% and precision of 71.5%.
A system fully supervised on 80 GENIA texts scores 49.2%. The proposed method needs only about 1,000 annotated instances to come close, roughly 0.63% (1/160) of the full training instances.
Augment adapts faster and performs better than IW and IP, and IP generally beats IW. In the earliest iterations, domain adaptation can actually trail the source-only baseline because there is too little target data.
Error analysis shows that newswire-trained models wrongly link compared biomedical entities, such as two different "CR" mentions, and adaptation fixes such errors. Results under B-CUBED were similar to those under MUC.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-1104

@inproceedings{zhao2014domain,
  title = {{Domain Adaptation with Active Learning for Coreference Resolution}},
  author = {Zhao, Shanheng and Ng, Hwee Tou},
  booktitle = {Proceedings of the 5th International Workshop on Health Text Mining and Information Analysis (Louhi)},
  year = {2014},
  doi = {10.3115/v1/w14-1104},
  url = {https://doi.org/10.3115/v1/w14-1104}
}

References

30 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

José Castaño, Jason Zhang, and James Pustejovsky. 2002. Anaphora resolution in biomedical literature. In Proceedings of the International Symposium on Reference Resolution. find
Yee Seng Chan and Hwee Tou Ng. 2007. Domain adaptation with active learning for word sense disambiguation. In Proceedings of the ACL2007. find
Nancy Chinchor. 2011. Statistical significance of MUC-6 results. In Proceedings of the Sixth Message Understanding Conference. find
K. Bretonnel Cohen, Arrick Lanfranchi, William Corvey, William A. Baumgartner Jr., Christophe Roeder, Philip V. Ogren, Martha Palmer, and Lawrence Hunter. 2010. Annotation of all coreference in biomedical text: Guideline selection and adaptation. In BioTxtM 2010. find
Daniel Dahlmeier and Hwee Tou Ng. 2010. Domain adaptation for semantic role labeling in the biomedical domain. Bioinformatics, 26(8):1098–1104. find
Hal Daume III. 2007. Frustratingly easy domain adaptation. In Proceedings of the ACL2007. find
Caroline Gasperin and Ted Briscoe. 2008. Statistical anaphora resolution in biomedical texts. In Proceedings of the COLING2008. find our card
Caroline Gasperin, Nikiforos Karamanis, and Ruth Seal. 2007. Annotation of anaphoric relations in biomedical full-text articles using a domain-relevant scheme. In Proceedings of the DAARC2007. find
Caroline Gasperin. 2009. Active learning for anaphora resolution. In Proceedings of the NAACL-HLT2009 Workshop on Active Learning for Natural Language Processing. find our card
Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in NLP. In Proceedings of the ACL2007. find
Jin-Dong Kim, Sampo Pyysalo, Tomoko Ohta, Robert Bossy, Ngan Nguyen, and Jun’ichi Tsujii. 2011a. Overview of BioNLP shared task 2011. In Proceedings of BioNLP Shared Task 2011 Workshop. find
Youngjun Kim, Ellen Riloff, and Nathan Gilbert. 2011b. The taming of Reconcile as a biomedical coreference resolver. In Proceedings of BioNLP Shared Task 2011 Workshop. find our card
Florian Laws, Florian Heimerl, and Hinrich Schütze. 2012. Active learning for coreference resolution. In Proceedings of the NAACL2012. find our card
David D. Lewis and William A. Gale. 1994. A sequential algorithm for training text classifiers. In Proceedings of the SIGIR1994. find
Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
Timothy A. Miller, Dmitriy Dligach, and Guergana K. Savova. 2012. Active learning for coreference resolution. In Proceedings of the BioNLP2012. find our card
Makoto Miwa, Paul Thompson, and Sophia Ananiadou. 2012. Boosting automatic event extraction from the literature using domain adaptation and coreference resolution. Bioinformatics, 28(13):1759–1765. find
MUC-6. 1995. Coreference task definition (v2.3, 8 Sep 95). In Proceedings of the Sixth Message Understanding Conference (MUC-6). find
MUC-7. 1998. Coreference task definition (v3.0, 13 Jul 97). In Proceedings of the Seventh Message Understanding Conference (MUC-7). find
Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the ACL2002. find our card
NIST. 2002. The ACE 2002 evaluation plan. ftp://jaguar.ncsl.nist.gov/ace/doc/ ACE-EvalPlan-2002-v06.pdf. find
Piyush Rai, Avishek Saha, Hal Daume, and Suresh Venkatasubramanian. 2010. Domain adaptation meets active learning. In Proceedings of the NAACL-HLT2010 Workshop on Active Learning for Natural Language Processing. find
Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. find our card
Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in noun phrase coreference resolution: Making sense of the state-of-the-art. In Proceedings of the ACL-IJCNLP2009. find our card
Min Tang, Xiaoqiang Luo, and Salim Roukos. 2002. Active learning for statistical natural language parsing. In Proceedings of the ACL2002. find
Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the MUC-6. find our card
Xiaofeng Yang, Guodong Zhou, Jian Su, and Chew Lim Tan. 2004. Improving noun phrase coreference resolution by matching strings. In Proceedings of the IJCNLP2004. find
Jian Bo Yang, Qi Mao, Qiao Liang Xiang, Ivor W. Tsang, Kian Ming A. Chai, and Hai Leong Chieu. 2012. Domain adaptation for coreference resolution: An adaptive ensemble approach. In Proceedings of the EMNLP2012. find our card
Shanheng Zhao and Hwee Tou Ng. 2010. Maximum metric score training for coreference resolution. In Proceedings of the COLING2010. find
Zhi Zhong, Hwee Tou Ng, and Yee Seng Chan. 2008. Word sense disambiguation using OntoNotes: An empirical study. In Proceedings of the EMNLP2008. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/zhao-2014-domain-adaptation-active-learning-coreference/

📄 Domain Adaptation with Active Learning for Coreference Resolution