Elamur Research⌕
☰
home / papers / 2008 / BART: A Modular Toolkit for Coreference Resolution

BART: A Modular Toolkit for Coreference Resolution

Summary

Building a coreference resolver that works end-to-end, from raw text to entity chains, takes substantial engineering. As a result, many researchers either rely on gold mentions or skip coreference altogether. BART is an open-source, highly modular toolkit created at the Johns Hopkins Summer Workshop 2007 to address this. It builds on Ponzetto and Strube (2006) and borrows ideas from GUITAR and other systems.

The toolkit keeps each stage separate:

  • Preprocessing: pipelines based on chunkers, parsers, or an ACE mention tagger, with output stored in MMAX2 standoff format.
  • Feature extraction: pairwise feature extractors selected through an XML configuration file.
  • Learning: wrappers for several machine-learning back ends.
  • Training/testing logic: a pluggable encoder/decoder component that controls how pairs are sampled and how mentions are clustered.

Simply re-creating the baseline from Soon et al. (2001) is already enough to outperform comparable systems on MUC data. Adding syntactic tree-kernel features and Wikipedia-derived knowledge brings ACE-2 performance close to Ng (2007). BART therefore serves both as a ready-to-use resolver and as a shared platform for coreference research.

Key points

  • With Soon et al.'s features and the built-in maximum entropy learner, BART reaches 65.8% F on MUC6 and 62.9% F on MUC7. It beats JavaRAP on pronouns and Uryupina's (2006) Soon reimplementation.
  • On ACE-2, the extended feature set achieves F = 0.630 (BNews), 0.658 (NPaper) and 0.627 (NWire). Ng (2007) reports 0.647, 0.646 and 0.633, but trains on the full ACE training corpus. Pronoun recall with the extended set is 73.4%.
  • Three preprocessing pipelines are available:
    • a chunking pipeline (Stanford POS tagger, YamCha, Stanford NER);
    • a parsing pipeline (Charniak–Johnson reranking parser, or the Berkeley parser for a pure-Java setup);
    • the Carafe pipeline with MITRE's ACE mention tagger.
  • Supported learners are WEKA, SVMLight/TK (tree-valued features) and a MaxEnt classifier optimized with L-BFGS. Results can be inspected in MMAX2 for error analysis.
  • BART is freely available from the University of Tübingen. Groups outside the original team already use it for research.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution · Part-of-speech tagging

Cited by

61 papers of our library cite it (of the 140673 whose bibliographies we read); 28 of them have a page here, 33 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{versley2008bart,
  title = {{BART: A Modular Toolkit for Coreference Resolution}},
  author = {Versley, Yannick and Ponzetto, Simone Paolo and Poesio, Massimo and Eidelman, Vladimir and Jern, Alan and Smith, Jason and Yang, Xiaofeng and Moschitti, Alessandro},
  booktitle = {Proceedings of the ACL-08: HLT Demo Session},
  year = {2008},
  doi = {10.63317/28a2xqhvmavx},
  url = {https://doi.org/10.63317/28a2xqhvmavx}
}

References

24 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Charniak, E. and Johnson, M. (2005). Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proc. find our card
  2. Culotta, A., Wick, M., and McCallum, A. (2007). First-order probabilistic models for coreference resolution. In Proc. HLT/NAACL 2007. find our card
  3. Denis, P. and Baldridge, J. (2007). A ranking approach to pronoun resolution. In Proc. find
  4. Finkel, J. R., Grenager, T., and Manning, C. (2005). Incorporating non-local information into information extraction systems by Gibbs sampling. In Proc. find
  5. Joachims, T. (1999). Making large-scale SVM learning practical. In Schölkopf, B., Burges, C., and Smola, A., editors, Advances in Kernel Methods - Support Vector Learning. find
  6. NPaper Prec F 0.526 0.586 0.677 0.658 0.797 0.646 NWire Prec 0.474 0.652 0.775 Recl 0.663 0.641 0.544 Recl 0.608 0.604 0.535 F 0.533 0.627 0.633 find
  7. Kudoh, T. and Matsumoto, Y. (2000). Use of Support Vector Machines for chunk identification. In Proc. find
  8. Liu, D. C. and Nocedal, J. (1989). On the limited memory method for large scale optimization. Mathematical Programming B, 45(3):503–528. find
  9. McCarthy, J. F. and Lehnert, W. G. (1995). Using decision trees for coreference resolution. In Proc. find
  10. Morton, T. S. (2000). Coreference for NLP applications. In Proc. find
  11. Moschitti, A. (2006). Making tree kernels practical for natural language learning. In Proc. find
  12. Müller, C. and Strube, M. (2006). Multi-level annotation of linguistic data with MMAX2. In Braun, S., Kohn, K., and Mukherjee, J., editors, Corpus Technology and Language Pedagogy: New Resources, New Tools, New Methods. Peter Lang, Frankfurt a.M., Germany. find
  13. Ng, V. (2007). Shallow semantics for coreference resolution. In Proc. find
  14. Petrov, S., Barett, L., Thibaux, R., and Klein, D. (2006). Learning accurate, compact, and interpretable tree annotation. In COLING-ACL 2006. find our card
  15. Ponzetto, S. P. and Strube, M. (2006). Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In Proc. HLT/NAACL 2006. find our card
  16. Qiu, L., Kan, M.-Y., and Chua, T.-S. (2004). A public reference implementation of the RAP anaphora resolution algorithm. In Proc. find our card
  17. Soon, W. M., Ng, H. T., and Lim, D. C. Y. (2001). A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. find our card
  18. Steinberger, J., Poesio, M., Kabadjov, M., and Jezek, K. (2007). Two uses of anaphora resolution in summarization. Information Processing and Management, 43:1663–1680. Special issue on Summarization. find
  19. Toutanova, K., Klein, D., Manning, C. D., and Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proc. find
  20. Uryupina, O. (2006). Coreference resolution with and without linguistic knowledge. In Proc. find our card
  21. Versley, Y. (2006). A constraint-based approach to noun phrase coreference resolution in German newspaper text. In Konferenz zur Verarbeitung Natürlicher Sprache (KONVENS 2006). find
  22. Wellner, B. and Vilain, M. (2006). Leveraging machine readable dictionaries in discriminative sequence models. In Proc. find
  23. Witten, I. and Frank, E. (2005). Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. find
  24. Yang, X., Su, J., and Tan, C. L. (2006). Kernel-based pronoun resolution with structured syntactic knowledge. In Proc. CoLing/ACL-2006. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/versley-2008-bart-modular-toolkit-coreference-resolution/