Elamur Research⌕
☰
home / papers / 2019 / ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

Summary

Most off-the-shelf NLP models degrade badly on biomedical and clinical text. Existing domain tools either focus narrowly on entity recognition and linking, such as MetaMap, or rely on older, non-neural methods, such as the GENIA tagger. The authors present scispaCy, a Python library built on spaCy. It retrains spaCy's tagger, parser and NER on biomedical data and adds tokenization rules tailored to scientific writing.

The syntactic models are trained on the GENIA 1.0 treebank, which the authors converted to UD, mixed with OntoNotes 5.0 for robustness. The main NER model is trained on MedMentions. Further NER models are benchmarked on nine biomedical corpora. The authors also release the converted GENIA corpus, aligned to the raw PubMed abstracts and their metadata.

The results come close to the state of the art. POS tagging is on par with specialized systems, and dependency parsing trails the Biaffine parser by about 2–3% while running roughly 9x faster. The library matters because it gives researchers fast, easy-to-install biomedical pipelines that plug directly into the Python ecosystem.

Key points

  • There are two core packages. en_core_sci_sm has a 58,338-word vocabulary and no word vectors. en_core_sci_md has a 101,678-word vocabulary and 98,131 vectors.
  • On GENIA, POS accuracy is 98.38 for the sm package and 98.51 for md, against 98.89 for the best BiLSTM-CRF. LAS is 87.67 and 88.79, against 91.23 for Stanford-Biaffine-v2.
  • Four entity-typed NER packages (en_ner_{bc5cdr|craft|jnlpba|bionlp13cg}_md) are released. NER is also evaluated on AnatEM, BC2GM, BC4CHEMD, Linnaeus and NCBI-Disease.
  • For entity linking, the candidate generator gains 5% absolute recall over Murty et al. (2018) while proposing 46% fewer candidates per mention on average.
  • The GENIA 1.0 corpus is released in Universal Dependencies v1.0, aligned to the untokenized abstracts and enriched with PubMed metadata such as MeSH terms, authors and journals.

Interesting ideas

  • I1. Mixing general-domain data into training. Adding a share of OntoNotes text to the biomedical training data improved results on general text without hurting biomedical scores. A similar mix could make specialized Ukrainian models less brittle on everyday text.
  • I2. Synthetic test for sentence splitting. The authors feed in sentences that are known to be single sentences and count how often the model wrongly splits them, focusing on sentences with citations. This creates a cheap, targeted evaluation without new annotation, and it could be reused for abbreviations or legal references.
  • I3. Learning boundaries instead of writing rules. Training the dependency parser on in-domain data removed the need for a hand-written sentence segmenter. Teams could try this before investing in fragile rule sets.
  • I4. Releasing corpora aligned to raw text. Linking annotations back to the original untokenized text, with metadata attached, allows tokenization and segmentation to be evaluated end to end. The metadata also offers natural supervision signals for new tasks.
  • I5. Reporting speed alongside accuracy. Wall-clock benchmarks against other pipelines make clear when a model that is slightly less accurate but much faster is the better practical choice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition · Dependency parsing · Part-of-speech tagging

Cited by

186 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 179 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-5034

@inproceedings{neumann2019scispacy,
  title = {{ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing}},
  author = {Neumann, Mark and King, Daniel and Beltagy, Iz and Ammar, Waleed},
  booktitle = {Proceedings of the 18th BioNLP Workshop and Shared Task},
  year = {2019},
  doi = {10.18653/v1/w19-5034},
  url = {https://doi.org/10.18653/v1/w19-5034}
}

References

43 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm… find our card
  2. Alan R. Aronson. 2001. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proceedings. AMIA Symposium, pages 17–21. find
  3. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
  4. Michael Bada, Miriam Eckert, Donald Evans, Kristin Garcia, Krista Shipley, Dmitry Sitnikov, William A. Baumgartner, K. Bretonnel Cohen, Karin M. Verspoor, Judith A. Blake, and Lawrence Hunter. 2011. Concept annotation in the CRAFT corpus. In BMC Bioinformatics. find
  5. Eduard H. Hovy, Mitchell P. Marcus, Martha Palmer, Lance A. Ramshaw, and Ralph M. Weischedel. 2006. OntoNotes: The 90% solution. In HLT-NAACL. find
  6. Vidur Joshi, Matthew Peters, and Mark Hopkins. 2018. Extending a parser to distant domains using a few dozen partially annotated examples. In ACL. find our card
  7. Olivier Bodenreider. 2004. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic acids research, 32 Database issue:D267–70. find
  8. Jin-Dong Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003. GENIA corpus - a semantically annotated corpus for bio-textmining. Bioinformatics, 19 Suppl 1:i180–2. find
  9. Lutz Bornmann and Rüdiger Mutz. 2014. Growth rates of modern science: A bibliometric analysis. CoRR, abs/1402.4578. — [arXiv:1402.4578](https://arxiv.org/abs/1402.4578) find
  10. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional lstm feature representations. Transactions of the Association for Computational Linguistics, 4:313–327. find our card
  11. Ekaterina Buyko, Joachim Wermter, Michael Poprat, and Udo Hahn. Automatically adapting an NLP core engine to the biology domain. In Proceedings of the ISMB 2006 Joint Linking Literature, Information and Knowledge for Biology and the 9th Bio-Ontologies Meeting. find
  12. Martin Krallinger, Florian Leitner, Obdulia Rabal, Miguel Vazquez, Julen Oyarzábal, and Alfonso Valencia. 2015. CHEMDNER: The drugs and chemical names extraction challenge. In J. Cheminformatics. find
  13. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural scaffolds for citation intent classification in scientific publications. CoRR, abs/1904.01608. find our card
  14. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In HLT-NAACL. find
  15. Nigel Collier and Jin-Dong Kim. 2004. Introduction to the bio-entity recognition task at JNLPBA. In NLP-BA/BioNLP. find our card
  16. Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sci-aky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016. BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database : the journal of biological databases… find
  17. Gamal K. O. Crichton, Sampo Pyysalo, Billy Chiu, and Anna Korhonen. 2017. A neural network multi-task learning approach to biomedical named entity recognition. In BMC Bioinformatics. find
  18. Dina Demner-Fushman, Willie J. Rogers, and Alan R. Aronson. 2017. MetaMap Lite: an evaluation of a new Java implementation of MetaMap. Journal of the American Medical Informatics Association : JAMIA, 24 4:841–844. find
  19. David McClosky and Eugene Charniak. 2008. Self-training for biomedical parsing. In ACL. find
  20. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Reranking and self-training for parser adaptation. In ACL. find our card
  21. Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. 2014. NCBI disease corpus: A resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47:1–10. find
  22. Shikhar Murty, Patrick Verga, Luke Vilnis, Irena Radovanovic, and Andrew McCallum. 2018. Hierarchical losses and new resources for fine-grained entity typing and linking. In ACL. find
  23. Timothy Dozat and Christopher D. Manning. 2016. Deep biaffine attention for neural dependency parsing. CoRR, abs/1611.01734. find our card
  24. Dat Quoc Nguyen and Karin Verspoor. 2018. From POS tagging to dependency parsing for biomedical event extraction. arXiv preprint arXiv:1808.03731. — [arXiv:1808.03731](https://arxiv.org/abs/1808.03731) find
  25. Martin Gerner, Goran Nenadic, and Casey M. Bergman. 2009. LINNAEUS: A species name identification system for biomedical literature. In BMC Bioinformatics. find
  26. Hoifung Poon, Chris Quirk, Charlie DeZiel, and David Heckerman. 2014. Literome: PubMed-scale genomic knowledge base in the cloud. Bioinformatics, 30 19:2840–2. find
  27. Yoav Goldberg and Joakim Nivre. 2012. A dynamic oracle for arc-eager dependency parsing. In Coling 2012. find
  28. Yoshimasa Tsuruoka and Jun’ichi Tsujii. 2005. Bidirectional inference with the easiest-first strategy for tagging sequence data. In HLT/EMNLP. find
  29. Hoifung Poon, Kristina Toutanova, and Chris Quirk. 2015. Distant supervision for cancer pathway extraction from text. Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing, pages 120–31. find
  30. Marco Antonio Valenzuela-Escarcega, Ozgun Babur, Gus Hahn-Powell, Dane Bell, Thomas Hicks, Enrique Noriega-Atala, Xia Wang, Mihai Surdeanu, Emek Demir, and Clayton T. Morrison. 2018. Large-scale automated machine reading discovers new cancer-driving mechanisms. In Database. find
  31. Sampo Pyysalo and Sophia Ananiadou. 2014. Anatomical entity mention recognition at literature scale. In Bioinformatics. find
  32. Sampo Pyysalo, Tomoko Ohta, Rafal Rak, Andrew Rowley, Hong-Woo Chun, Sung-Jae Jung, Sung-Pil Choi, Jun’ichi Tsujii, and Sophia Ananiadou. 2015. Overview of the cancer genetics and pathway curation tasks of BioNLP shared task 2013. In BMC Bioinformatics. find
  33. Xuan Wang, Yu Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis P. Langlotz, and Jiawei Han. 2018. Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics. find
  34. Laura Rimell and Stephen Clark. 2008. Adapting a lexicalized-grammar parser to contrasting domains. In EMNLP. find
  35. Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Jiao Li, Thomas C. Wiegers, and Zhiyong Lu. 2016. Assessing the state of the art in biomedical relation extraction: overview of the BioCreative V chemical-disease relation (CDR) task. Database : the journal of biolog… find
  36. Devendra Singh Sachan, Pengtao Xie, Mrinmaya Sachan, and Eric P. Xing. 2017. Effective use of bidirectional language modeling for transfer learning in biomedical named entity recognition. In MLHC. find
  37. Ralph Weischedel, Eduard Hovy, Martha Palmer, Mitch Marcus, Robert Belvin adn Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A large training corpus for enhanced processing. In Joseph Olive, Caitlin Christianson, and John McCary, editors, Handbook of Natural Language Processing and… find
  38. Guergana K. Savova, James J. Masanz, Philip V. Ogren, Jiaping Zheng, Sunghwan Sohn, Karin Kipper Schuler, and Christopher G. Chute. 2010. Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. Journal of the American Medical Informa… find
  39. Yuan Zhang and David I Weiss. 2016. Stack-propagation: Improved representation learning for syntax. CoRR, abs/1603.06598. find
  40. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced english universal dependencies: An improved representation for natural language understanding tasks. In LREC. find our card
  41. Ariel S. Schwartz and Marti A. Hearst. 2002. A simple algorithm for identifying abbreviation definitions in biomedical text. Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing, pages 451–62. find
  42. Larry Smith, Lorraine K. Tanabe, Rie Johnson nee Ando, Cheng-Ju Kuo, I-Fang Chung, Chun-Nan Hsu, Y Lin, Roman Klinger, Christoph M. Friedrich, Kuzman Ganchev, Manabu Torii, Hongfang Liu, Barry Haddow, Craig A. Struble, Richard J. Povinelli, Andreas Vlachos, William A. Baumgartner, Lawrence E. Hunter… find
  43. Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun’ichi Tsujii. 2005. Developing a robust part-of-speech tagger for biomedical text. In Panhellenic Conference on Informatics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/neumann-2019-scispacy-fast-robust-models-biomedical/