Elamur Research⌕
☰
home / papers / 2011 / Building a Coreference-Annotated Corpus from the Domain of Biochemistry

Building a Coreference-Annotated Corpus from the Domain of Biochemistry

Summary

Biomedical coreference resolution has been held back by a shortage of annotated training data. Most existing resources, such as MEDSTRACT and the MEDCo abstracts, contain only abstracts. The one downloadable full-text corpus, FlySlip, covers only gene-related entities. To fill this gap, the authors built the HANAPIN corpus from 20 open-access full-text papers in marine natural products chemistry, a biochemistry subfield where many kinds of entities appear together. They wrote an annotation scheme covering four nominal coreference types: sortal, pronominal, numerical and abbreviation. The last two are frequent in chemistry writing and had not been annotated in earlier biomedical corpora. Three chemistry graduates labelled every coreferring mention of chemical compounds, drug targets, organisms, diseases and drug effects over five months. Numerical references turned out to be the most common type, and chemical compounds dominate the chains. Inter-annotator agreement was satisfactory, reaching 84% Krippendorff's alpha with the MASI distance. The planned public release of the corpus and guidelines gives researchers a multi-type, full-text resource for building and testing biomedical coreference systems.

Key points

  • Source and size: Articles were sampled at random from the journal Marine Drugs (PubMed Central Open Access subset), published 2003–2009. Experimental sections were left unannotated. The corpus totals 1,027 sentences and 27,358 words.
  • Chains: There are 395 non-singleton coreference chains.
    • Chemical compounds: 70.89%
    • Drug targets: 12.66% (mostly proteins, cell lines and pathogens)
    • Organisms: 9.87%
    • Drug effects: 3.29%
    • Diseases: 3.29%
  • Links: Of 760 coreference links, 46% are numerical, 33% sortal, 11% pronominal and 10% abbreviation. Within the sortal type, definite and indefinite noun phrases are the most frequent subtypes.
  • Reliability: Krippendorff's alpha was 75% using the original set-based distance and 84% using MASI. Both exceed the 67% level usually cited as the threshold for unreliability.
  • Comparison and scope: The paper compares HANAPIN with MEDSTRACT, MEDCo, FlySlip, CRAFT and DrugNerAr. Annotation was done in XConc with XML output. Verbal and indirect coreference are not covered and are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution · Treebanks

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{batistanavarro2011building,
  title = {{Building a Coreference-Annotated Corpus from the Domain of Biochemistry}},
  author = {Batista-Navarro, Riza Theresa and Ananiadou, Sophia},
  booktitle = {Proceedings of BioNLP 2011 Workshop},
  year = {2011},
  url = {https://aclanthology.org/W11-0210/}
}

References

16 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ron Artstein and Massimo Poesio. 2004. Inter-Coder Agreement for Computational Linguistics. Computational Linguistics, 34(4):555-596. find
  2. José Castaño, Jason Zhang and James Pustejovsky. 2002. Anaphora resolution in biomedical literature. Proceedings of the International Symposium on Reference Resolution for NLP. find
  3. K. Bretonnel Cohen, Philip V. Ogren, Lynne Fox and Lawrence E. Hunter. 2005. Empirical data on corpus design and usage in biomedical natural language processing. AMIA Annual Symposium Proceedings, pages 156-160. find
  4. K. Bretonnel Cohen, Helen L. Johnson, Karin Verspoor, Christophe Roeder, Lawrence E. Hunter. 2010. The structural and content aspects of abstracts versus bodies of full text journal articles are different. BMC Bioinformatics, 11(1):492. find
  5. K. Bretonnel Cohen, Arrick Lanfranchi, William Corvey, William A. Baumgartner Jr., Christophe Roeder, Philip V. Ogren, Martha Palmer and Lawrence E. Hunter. 2010. Annotation of all coreference in biomedical text: Guideline selection and adaptation. Proceedings of the Second Workshop on Building and… find
  6. Caroline Gasperin, Nikiforos Karamanis and Ruth Seal. 2007. Annotation of anaphoric relations in biomedical full-text articles using a domain-relevant scheme. Proceedings of the 6th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2007). find
  7. Ralph Grishman and Beth Sundheim. 1995. Design of the MUC-6 Evaluation. MUC ’95: Proceedings of the 6th Message Understanding Conference, pages 1-11. find
  8. Lynette Hirschman. 1997. MUC-7 Coreference Task Definition. Message Understanding Conference 7 Proceedings. find our card
  9. Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall, 2nd edition. find
  10. Klaus H. Krippendorff. 1980. Content Analysis: An Introduction to Its Methodology. Beverly Hills, CA: Sage Publications. find
  11. Tara McIntosh and James R.Curran. 2009. Challenges for automatically extracting molecular interactions from full-text articles. BMC Bioinformatics, 10(1):311. find
  12. Ruslan Mitkov, Richard Evans, Constantin Orasan, Catalina Barbu, Lisa Jones and Violeta Sotirova. 2005. Coreference and anaphora: developing annotating tools, annotated resources and annotation strategies. Proceedings of the Discourse Anaphora and Anaphora Resolution Colloquium (DAARC 2000), pages 4… find
  13. Rebecca J. Passoneau. 2004. Computing reliability for coreference annotation. Proceedings of the International Conference on Language Resouces (LREC). find
  14. M. Schumie, M. Weeber, B. Schijvenaars, E. van Mulligen, C. van der Eijk, R. Jelier, B. Mons and J. Kors. 2004. Distribution of information in biomedical abstracts and full-text publications. Bioinformatics, 20(16):2597-2604. find
  15. Isabel Segura-Bedmar, Mario Crespo, César de Pablo-Sánchez and Paloma Martı́nez. 2009. Resolving anaphoras for the extraction of drug- drug interactions in pharmacological documents. BMC Bioinformatics, 11(Suppl 2):S1. find
  16. Parantu K. Shah, Carolina Perez-Iratxeta, Peer Bork and Miguel A. Andrade. 2003. Information extraction from full text scientific articles: Where are the keywords? BMC Bioinformatics, 4(1): 20. find

↑ back to top