Elamur Research⌕
☰
home / papers / 2012 / Domain-specific vs. Uniform Modeling for Coreference Resolution

Domain-specific vs. Uniform Modeling for Coreference Resolution

Summary

Coreference corpora such as ACE-02, ARRAU and OntoNotes combine several domains, but each domain is fairly small. This raises a practical question: should a resolver be trained separately for each domain or on all the data at once? Uryupina and Poesio first introduce a simple measure of domain homogeneity. Every document is described by ten mention-distribution ratios, including the share of pronouns, names, singletons and the length of the longest chain. Homogeneity C dom is then the average distance of documents to their domain centroid.

They then use the BART toolkit with J48 decision trees and system mentions to compare domain-specific and union training. Three feature sets are tested: the Soon et al. (2001) baseline, a rich set of 42 linguistic features, and a reduced, pre-selected half of that set.

Domain choice barely affects the shallow baseline. The full rich set overfits, and it does so more on small domains. With feature selection, domain-specific models usually beat union models. This suggests that splitting by domain helps knowledge-rich resolvers when overfitting is controlled.

Key points

  • Corpora: ARRAU (about 156K training tokens) covers news, medical, fiction and dialogue. ACE-02 (about 221K) and OntoNotes/CoNLL (about 1M) contain only news-type material. Because their annotation guidelines differ, cross-corpus training was ruled out.
  • Domain structure: The indicators separate ARRAU's domains cleanly (pear–rst centroid distance 0.65). ACE-02 subdomains overlap heavily. Some OntoNotes domains are near-identical (mz–wb 0.09). C dom values cannot be compared across corpora.
  • Selected features (MUC F, domain vs. union): ACE-02 scored 73.97 vs. 71.12 and OntoNotes 57.74 vs. 57.05, both significant. ARRAU stayed weaker (54.84 vs. 55.29).
  • All 42 features: Results dropped below the baseline for ARRAU and OntoNotes (e.g., OntoNotes about 49.5 vs. 54.2–54.5). Domain-specific training on small ACE domains made the drop worse.
  • Small domains: These should be merged with similar ones or trained on the union. For example, ARRAU vpc closely resembles rst.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{uryupina2012domain,
  title = {{Domain-specific vs. Uniform Modeling for Coreference Resolution}},
  author = {Uryupina, Olga and Poesio, Massimo},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/5paidqcig542},
  url = {https://doi.org/10.63317/5paidqcig542}
}

References

15 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hal III Daume. 2007. Frustratingly easy domain adaptation. In Proc. find
  2. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassell, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program–tasks, data, and evaluation. In Proc. LREC-04. find our card
  3. Veronique Hoste. 2005. Optimization Issues in Machine Learning of Coreference Resolution. Ph.D. thesis, Antwerp University. find
  4. David McClosky, Eugene Charniak, and Mark Johnson. 2010. Automatic domain adaptation for parsing. In Proc. find
  5. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proc. ACL-02, pages 104–111. find our card
  6. Barbara Plank and Gertjan van Noord. 2011. Effective measures of domain similarity for parsing. In Proc. find
  7. Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the arrau corpus. In Proc. LREC-08. find our card
  8. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proc. find our card
  9. CoNLL 2011, Portland, Oregon, June. find
  10. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistic, 27(4):521–544. find our card
  11. Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in noun phrase coreference resolution: making sense of the state-of-the-art. In Proc. find our card
  12. Olga Uryupina, Sriparna Saha, Asif Ekbal, and Massimo Poesio. 2011. Multi-metric optimization for coreference: The UniTN / IITP / Essex submission to the 2011 CONLL shared task. In Proc. CoNLL-2011. find our card
  13. Olga Uryupina. 2006. Coreference resolution with and without linguistic knowledge. In Proc. LREC-06. find our card
  14. Kees van Deemter and Rodger Kibble. 2001. On coreferring: Coreference in muc and related annotation schemes. Computational Linguistics. find our card
  15. Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. BART: a modular toolkit for coreference resolution. In Proc. ACL-08, pages 9–12. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/uryupina-2012-domain-specific-vs-uniform-modeling/