Elamur Research⌕
☰
home / papers / 2024 / Structured information extraction from scientific text with large language models

Structured information extraction from scientific text with large language models

Paper · 2024 · John M. Dagdelen, Alexander Dunn, Sang‐Hoon Lee et al.

Summary

Most experimental knowledge about solid-state materials is buried in the prose of research articles. Earlier materials NLP work mostly tagged isolated entities and did not capture how those entities relate. Classic relation extraction also handles poorly the facts that combine many linked parts, such as a composition paired with a morphology and an application. The authors fine-tune a pretrained large language model (GPT-3 or the open Llama-2) on a few hundred expert-annotated passages. The tuned model reads a sentence or abstract and outputs entities and their links in a single step, using a user-chosen format such as plain English sentences or lists of JSON objects. Three materials chemistry tasks serve as tests: host–dopant linking, cataloguing metal-organic frameworks, and general extraction of composition, phase, morphology and application. Evaluation uses strict triplet-level F1 plus an expert-judged manual score for fuzzier fields. The method works well at both sentence and document level and needs no multi-model pipeline or separate NER stage. This gives researchers with little NLP background a practical way to build large structured databases from the literature.

Key points

  • Users define an output schema (e.g., JSON with fixed keys) and annotate roughly 100–500 passages. Intermediate models can pre-fill suggested entities, which makes building training sets much faster.
  • Models: GPT-3 (closed, tuned through an API) and Llama-2 (open, with released LoRA weights). Task models include a doping model, a MOF model and a General-JSON model.
  • Scoring is deliberately strict. Formula-type fields (host, dopant, formula, mof_formula) must match exactly. For example, "Bi2Te3 nanoparticles" and "Bi2Se3 nanoparticles" count as wrong even though their Jaro-Winkler similarity is 0.977 and their character BLEU-4 is 0.858.
  • A "manual score" accepts reasonable variants, such as "silicon" written as "Si". It only counts an entity as correct when it is grouped under the right root material.
  • Code, annotated datasets and splits are available at github.com/LBNLP/NERRE and on Zenodo. Llama-2 code is at github.com/lbnlp/nerre-llama, and the weights are on Figshare.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, BerkeleyBerkeley · United States175 works · 227,440 citations🔬Lawrence Berkeley National LaboratoryBerkeley · United States3 works · 39,160 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksMaterials Science · 26 worksMaterials Chemistry · 26 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 167 works

💡 Concepts 2Key ideas and methods explained on our concept pages.Information extraction · 50 worksMorphology · 50 works

Cite

Cite as John M. Dagdelen, Alexander Dunn, Sang‐Hoon Lee, Nicholas Walker, Andrew Rosen, Gerbrand Ceder, Kristin Aslaug Persson and Anubhav Jain. 2024. Structured information extraction from scientific text with large language models. Nature Communications.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1038/s41467-024-45563-x

@inproceedings{dagdelen2024structured,
  title = {{Structured information extraction from scientific text with large language models}},
  author = {John M. Dagdelen, Alexander Dunn, Sang‐Hoon Lee, Nicholas Walker, Andrew Rosen, Gerbrand Ceder, Kristin Aslaug Persson and Anubhav Jain},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-45563-x},
  url = {https://doi.org/10.1038/s41467-024-45563-x}
}

References

3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 1. 2. 3. 4. 5. 6. 7. Saal, J. E., Oliynyk, A. O. & Meredig, B. Machine learning in materials discovery: confirmed predictions and their underlying approaches. Annu. Rev. Mater. Res. 50, 49–69 (2020). Choudhary, K. et al. Recent advances and applications of deep learning methods in materials science.… find
  2. Sierepeklis, O. & Cole, J. M. A thermoelectric materials database auto-generated from the scientific literature using chemdataextractor. Sci. data 9, 648 (2022). 9. Beard, E. J. & Cole, J. M. Perovskite- and dye-sensitized solar-cell device databases auto-generated using chemdataextractor. Sci. Data… find
  3. Database 2016 (2016). 24. Bravo, Á., Piñero, J., Queralt-Rosinach, N., Rautschka, M. & Furlong, L. I. Extraction of relations between genes and diseases from text and large-scale data analysis: Implications for translational research. BMC Bioinformatics 16, 1–17 (2015). 25. Yang, X. et al. Pcmsp: A… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dagdelen-2024-structured-information-extraction-scientific-text/