Elamur Research
☰
home / papers / 2019 / S2ORC: The Semantic Scholar Open Research Corpus

S2ORC: The Semantic Scholar Open Research Corpus

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/1911.02782

Summary

Researchers who mine scholarly text have mostly used corpora with clear limits. Some are small (AAN), some cover only one field (PubMed Central OA and the arXiv-based sets), and some, like RefSeer, give only short citation snippets instead of usable full text. The Allen Institute for AI team built S2ORC from the Semantic Scholar literature graph to fill this gap. Their pipeline parses about 30.5M PDFs with ScienceParse and GROBID and also uses arXiv LaTeX sources. For each paper cluster it chooses canonical metadata by preferring trusted publisher records and falling back to majority voting. It then drops clusters with too little content and links each bibliography entry to the matching paper in the corpus. The result covers 81.1M English-language papers across dozens of disciplines. Of these, 8.1M open-access papers come with structured full text in which section headings, inline citations and mentions of figures and tables are tagged and linked. The authors also pretrain a BERT model on the corpus and train paper embeddings from citation contexts. S2ORC was the biggest public source of machine-readable academic text when it came out.

Key points

  • Scale: 81.1M papers, 380.5M resolved citation links, 8.1M full-text parses from open-access PDFs and 1.5M LaTeX parses from arXiv.
  • The corpus has about three times as many full-text papers as PubMed Central OA (2.6M), the next-largest corpus with citation links. It also covers far more fields than AAN (about 25k computational linguistics papers) or the arXiv-based corpus of Saier and Färber (2019).
  • PDFs are processed with ScienceParse v3.0.0, which handles titles and authors, and GROBID v0.5.5, which handles body text, citations and bibliographies. Regex post-processing removes false citation matches and expands bracket ranges such as [3]-[5].
  • LaTeX sources give almost error-free citation, caption and equation spans, plus structured tables and formulas. However, LaTeX metadata turned out less reliable than PDF metadata, so it is not used for clustering.
  • The same pipeline was used to build CORD-19, which drew more than 75K downloads in its first month.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

292 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 278 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1911.02782 · arXiv 1911.02782

@misc{lo2019s2orc,
  title = {{S2ORC: The Semantic Scholar Open Research Corpus}},
  author = {Lo, Kyle and Wang, Lucy Lu and Neumann, Mark and Kinney, Rodney and Weld, Dan S.},
  year = {2019},
  doi = {10.48550/arxiv.1911.02782},
  eprint = {1911.02782},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1911.02782}
}

References

57 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Riaz Ahmad and Muhammad Tanvir Afzal. 2018. Cad: an algorithm for citation-anchors detection in research papers. Scientometrics, 117:1405–1423. find
  2. Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, 19 The Kaggle CORD-19 and TREC-COVID competitions. See Wang et al. (2020) for details. find
  3. Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, and Oren Etzioni. 2018. Construction of the literature graph in semantic scholar. In Proceedings of the 2018 Conference of the North American Chapter of the A… find
  4. Kimitaka Asatani, Junichiro Mori, Masanao Ochi, and Ichiro Sakata. 2018. Detecting trends in academic research from a citation network using network representation learning. In PloS one. find
  5. Awais Athar and Simone Teufel. 2012. Context-enhanced citation sentiment detection. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 597–601, Montréal, Canada. Association for Computational Lingu… find
  6. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Sci-BERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620,… find
  7. Matthew Berger, Katherine McDonough, and Lee M Seversky. 2016. cite2vec: Citation-driven document exploration via word embeddings. IEEE transactions on visualization and computer graphics, 23(1):691–700. find
  8. Chandra Bhagavatula, Sergey Feldman, Russell Power, and Waleed Ammar. 2018. Content-based citation recommendation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 238–251,… find
  9. Steven Bird, Robert Dale, Bonnie Dorr, Bryan Gibson, Mark Joseph, Min-Yen Kan, Dongwon Lee, Brett Powley, Dragomir Radev, and Yee Fan Tan. 2008. The ACL anthology reference corpus: A reference dataset for bibliographic research in computational linguistics. In Proceedings of the Sixth International… find
  10. Cornelia Caragea, Florin Adrian Bulgarov, Andreea Godea, and Sujatha Das Gollapalli. 2014. Citation-enhanced keyphrase extraction from research papers: A supervised approach. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1435– 1446, Doha, Qa… find
  11. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural scaffolds for citation intent classification in scientific publications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  12. Arman Cohan and Nazli Goharian. 2015. Scientific article summarization using citation-context and article’s discourse structure. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 390–400, Lisbon, Portugal. Association for Computational Linguistics. find
  13. Nigel Collier and Jin-Dong Kim. 2004. Introduction to the bio-entity recognition task at JNLPBA. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP), pages 73– 78, Geneva, Switzerland. COLING. find our card
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  15. Ying Ding, Guo Zhang, Tamy Chambers, Min Song, Xiaolong Wang, and Cheng xiang Zhai. 2014. Content-based citation analysis: The next generation of citation analysis. JASIST, 65:1820–1833. find
  16. Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014. Ncbi disease corpus: a resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47:1–10. find
  17. Daniel Duma and Ewan Klein. 2014. Citation resolution: A method for evaluating context-based citation recommendation systems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 358–363, Baltimore, Maryland. Association for Compu… find
  18. Masaki Eto. 2019. Extended co-citation search: Graph-based document retrieval on a co-citation network containing citation context information. Inf. Process. Manage., 56. find
  19. C. L. Giles, K. D. Bollacker, and S. Lawrence. 1998. Citeseer: an automatic citation indexing system. In Proceedings of the ACM International Conference on Digital Libraries, pages 89–98. ACM. Proceedings of the 1998 3rd ACM Conference on Digital Libraries ; Conference date: 23-06-1998 Through 26-06… find
  20. Khalid Haruna, Maizatul Akmar Ismail, Abdullahi Baffa Bichi, Victor I. Chang, Sutrisna Wibawa, and Tutut Herawan. 2018. A citation-based recommender system for scholarly paper recommendation. In ICCSA. find
  21. Qi He, Jian Pei, Daniel Kifer, Prasenjit Mitra, and Lee Giles. 2010. Context-aware citation recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW 10, page 421430, New York, NY, USA. Association for Computing Machinery. find
  22. Wenyi Huang, Zhaohui Wu, Chen Liang, Prasenjit Mitra, and C. Lee Giles. 2015. A neural probabilistic model for context based citation recommendation. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI15, page 24042410. AAAI Press. find
  23. Chanwoo Jeong, Sion Jang, Hyuna Shin, Eunjeong Park, and Sungchul Choi. 2019. A context-aware citation recommendation model with bert and graph convolutional networks. arXiv. find
  24. David Jurgens, Srijan Kumar, Raine Hoover, Dan Mc-Farland, and Dan Jurafsky. 2018. Measuring the evolution of a scientific field through citation frames. Transactions of the Association for Computational Linguistics, 6:391–406. find
  25. Anshul Kanakia, Zhihong Shen, Darrin Eide, and Kuansan Wang. 2019. A scalable hybrid research paper recommender system for microsoft academic. In The World Wide Web Conference, WWW 19, page 28932899, New York, NY, USA. Association for Computing Machinery. find
  26. J-D Kim, Tomoko Ohta, Yuka Tateisi, and Junichi Tsujii. 2003. Genia corpusa semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl 1):i180–i182. find
  27. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In International Conference on Learning Representations. find
  28. Martin Krallinger, Obdulia Rabal, Saber Ahmad Akhondi, Martı́n Pérez Pérez, Jésús López Santa-marı́a, Gael Pérez Rodrı́guez, Georgios Tsatsaronis, Ander Intxaurrondo, José Antonio Baso López, Umesh Nandal, Erin M. van Buel, A. Poorna Chandrasekhar, Marleen Rodenburg, Astrid Lægreid, Marius A… find
  29. Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sci-aky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016. Biocreative V CDR task corpus: a resource for chemical disease relation extraction. find
  30. Haifeng Liu, Xiangjie Kong, Xiaomei Bai, Wei Wang, Teshome Megersa Bekele, and Feng Xia. 2015. Context-based collaborative filtering for citation recommendation. IEEE Access, 3:1695–1703. find
  31. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a. Linguistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tec… find our card
  32. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A robustly optimized BERT pretraining approach. arXiv. find
  33. Patrice Lopez. 2009. Grobid: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In Proceedings of the 13th European Conference on Research and Advanced Technology for Digital Libraries, ECDL09, page 473474, Berlin, Heidelberg. Springer-Verlag. find
  34. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219–3232, Brussels, Belg… find our card
  35. Laurens van der Maaten and Geoffrey E. Hinton. 2008. Visualizing data using t-sne. In Journal of Machine Learning Research. find
  36. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems -Volume 2, NIPS13, page 31113119, Red Hook, NY, USA… find
  37. Sandra Mitrović and Henning Müller. 2015. Summarizing citation contexts of scientific publications. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 154–165, Cham. Springer International Publishing. find
  38. Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. ScispaCy: Fast and robust models for biomedical natural language processing. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 319–327, Florence, Italy. Association for Computational Linguistics. find
  39. Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain Marshall, Ani Nenkova, and Byron Wallace. 2018. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In Proceedings of the 56th Annual Meeting of the Associatio… find
  40. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo… find
  41. Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018b. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509, Brussels, Belgium. Association for Computational… find our card
  42. Vahed Qazvinian and Dragomir R. Radev. 2008. Scientific paper summarization using citation summary networks. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 689–696, Manchester, UK. find
  43. Coling 2008 Organizing Committee. find
  44. Dragomir R. Radev, Pradeep Muthukrishnan, and Vahed Qazvinian. 2009. The acl anthology network corpus. In Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries, NLPIR4DL 09, page 5461, USA. Association for Computational Linguistics. find
  45. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. N/A. find
  46. Tarek Saier and Michael Färber. 2019. Bibliometric-enhanced arxiv: A data set for paper-based and citation-based tasks. In Proceedings of the 8th International Workshop on Bibliometric-enhanced Information Retrieval (BIR 2019) co-located with the 41st European Conference on Information Retrieval (E… find
  47. Tarek Saier and Michael Färber. 2020. unarxive: a large scholarly data set with publications’ full-text, annotated in-text citations, and links to metadata. Scientometrics. find
  48. Zhihong Shen, Hao Ma, and Kuansan Wang. 2018. A web-scale system for scientific knowledge exploration. In Proceedings of ACL 2018, System Demonstrations, pages 87–92, Melbourne, Australia. Association for Computational Linguistics. find
  49. Henry Small. 1973. Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4):265–269. find
  50. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 08, page 990998, New York, NY, USA. Association for Comp… find
  51. Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. Automatic classification of citation function. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 103–110, Sydney, Australia. Association for Computational Linguistics. find
  52. Dominika Tkaczyk, Andrew Collins, Paraic Sheridan, and Joeran Beel. 2018. Machine learning vs. rules and out-of-the-box vs. retrained: An evaluation of open-source bibliographic reference and citation parsers. In Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries, JCDL 18, pag… find
  53. Caleb M. Trujillo and Tammy M. Long. 2018. Document co-citation analysis to enhance transdisci-plinary research. Science Advances, 4(1). find
  54. Marco Valenzuela, Vu Ha, and Oren Etzioni. 2015. Identifying meaningful citations. AAAI. find
  55. Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Darrin Eide, Kathryn Funk, Rodney Kinney, Ziyang Liu, William Merrill, Paul Mooney, Dewey Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex D. Wade, Kuansan Wang, Chris Wilhelm, Boya Xie, Douglas R… find
  56. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… find
  57. Xiao Yu, Quanquan Gu, Mianwei Zhou, and Jiawei Han. 2012. Citation prediction in heterogeneous bibliographic networks. In SDM. find