Elamur Research⌕
☰
home / papers / 2026 / SciTrue: Evidence-Grounded Claim Verification in Science

SciTrue: Evidence-Grounded Claim Verification in Science

Paper · 2026 · Neset Tan, Minghao Li and Mark Gahegan

Summary

SciTrue is a demonstration system for checking scientific claims. It targets a known weakness of general-purpose and web-search LLMs: they often produce citations that are invented, misattributed, or only loosely connected to the statement they are meant to support.

The pipeline works in several stages:

  • An agent first rewrites the user's claim into a clear scientific statement.
  • The system pulls candidate papers through the Semantic Scholar API.
  • It extracts the most relevant sentence from each paper and labels it with several attributes: support or refutation, underlying assumptions and conditions, rhetorical role, and strength of the claim–paper relationship.
  • It writes a concise summary in which every statement is tied to a specific source.
  • It gives a graded verdict.
  • It splits the claim into subclaims, each with aligned evidence, shown in an interactive web interface.

Annotators compared SciTrue with GPT-4o-search-preview and Perplexity Sonar Pro. SciTrue was rated higher on all criteria, including summary traceability, attribution accuracy, context and assumption awareness, and source credibility. The authors argue this shows that scientific fact-checking tools need strict source filtering and auditable, fine-grained attribution. A demo is available at www.scitrue.org.

Key points

  • Retrieval and evidence labels: Retrieval uses the Semantic Scholar API, which covers more than 214 million papers. Evidence sentences are categorised with Wei's (2023) rubric: claims of superiority, new proposals, and new findings or cause–effect relationships. The claim–article link is rated strong, medium or weak.
  • Verdict scale: Verdicts take four levels: fully, mostly, partially, or not supported.
  • Evaluation setup: The human evaluation used 60 claims spread evenly over four domains (health/nutrition, environment/climate, social science, technology/policy). Five articles were retrieved per claim, giving about 300 attributions per system and over 900 full-text sources overall.
  • Annotation: Two annotators, each with at least a science bachelor's degree, followed a rubric and reached 90% agreement. Disagreements were settled by discussion.
  • Baseline weaknesses and SciTrue's limitation: Both baselines drew fewer than half of their sources from scientific articles, leaning on blogs and news sites, and they hallucinated citations. SciTrue's own coverage is limited to what Semantic Scholar indexes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AucklandAuckland · New Zealand15 works · 888 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksInformation Systems · 1,730 worksSociology and Political Science · 626 works

Cite

Cite as Neset Tan, Minghao Li and Mark Gahegan. 2026. SciTrue: Evidence-Grounded Claim Verification in Science. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-demo.27

@inproceedings{tan2026scitrue,
  title = {{SciTrue: Evidence-Grounded Claim Verification in Science}},
  author = {Neset Tan, Minghao Li and Mark Gahegan},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-demo.27},
  url = {https://doi.org/10.18653/v1/2026.eacl-demo.27}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Isabelle Augenstein, Sebastian Riedel, Pontus Stenetorp, Leon Derczynski, Georgios Spithourakis, Alex Dutton, and Yang Ji. 2019. Multifc: A real-world multi-domain dataset for evidence-based fact checking of claims. In NAACL-HLT.
  2. Marcel Binz, Stephan Alaniz, Adina Roskies, Balazs Aczel, Carl T Bergstrom, Colin Allen, Daniel Schad, Dirk Wulff, Jevin D West, Qiong Zhang, Richard M Shiffrin, Samuel J Gershman, Vencislav Popov, Emily M Bender, Marco Marelli, Matthew M Botvinick, Zeynep Akata, and Eric Schulz. 2025. How should th… find
  3. Julia G Bottesini, Christie Aschwanden, Mijke Rhemtulla, and Simine Vazire. 2023. How do science journalists evaluate psychology research? Adv. Methods Pract. Psychol. Sci., 6(3). find
  4. Neset Tan, Trung Nguyen, Josh Bensemann, Alex Peng, Qiming Bao, Yang Chen, Mark Gahegan, and Michael Witbrock. 2023. Multi2Claim: Generating scientific claims from multi-choice questions for scientific fact-checking. In Proceedings of the 17th Conference of the European Chapter of the Association fo… find
  5. Neşet Özkan Tan, Niket Tandon, David Wadden, Oyvind Tafjord, Mark Gahegan, and Michael Witbrock. 2024. Faithful reasoning over scientific claims. Proceedings of the AAAI Symposium Series, 3(1):263– 272. find
  6. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Majid Mittal. 2018. Fever: a large-scale dataset for fact extraction and verification. In NAACL-HLT, pages 809–819.
  7. Lihu Chen, Shuojie Fu, Gabriel Freedman, Cemre Zor, Guy Martin, James Kinross, Uddhav Vaghela, Ovidiu Serban, and Francesca Toni. 2025. Pub-guard-LLM: Detecting fraudulent biomedical articles with reliable explanations. arXiv [cs.CL]. find
  8. Deepa Tilwani, Revathy Venkataramanan, and Amit P Sheth. 2024. Neurosymbolic AI approach to attribution in large language models. arXiv [cs.CL]. find
  9. Consensus. https://consensus.app/. ↗
  10. Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, and Lucy Lu Wang. 2021. Ms^2: Multi-document summarization of medical studies. In EMNLP. find
  11. Elicit. https://elicit.org/. ↗
  12. Google DeepMind. 2024. Gemini 1.5 technical report. Accessed June 2025. find
  13. Antonios Kanavouras and Frank Coutelieris. 2020. Similarity among physical phenomena recognized on the basis of the classification of existing knowledge. Journal of Mathematical Sciences and Modelling, 3(2):47–54. find
  14. Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, and Hao Peng. 2024. Source-aware training enables knowledge attribution in language models. arXiv [cs.CL].
  15. Thomas S. Kuhn. 1962. The Structure of Scientific Revolutions. University of Chicago Press. find
  16. Meta. 2025. Gemini 2.5 report. find
  17. Brent Mittelstadt, Sandra Wachter, and Chris Russell. 2023. To protect science, we must use LLMs as zero-shot translators. Nat. Hum. Behav., 7(11):1830– 1832. find
  18. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  19. Chantal Shaib, Millicent Li, Sebastian Joseph, Iain James Marshall, Junyi Jessy Li, and Byron Wallace. 2023. Summarizing, simplifying, and synthesizing medical evidence using gpt-3 (with varying success). ArXiv, abs/2305.06299. find
  20. David Wadden, Shanchan Lin, Kyle Lo Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7546.
  21. Byron C. Wallace, Sayantani Saha, Frank Soboczenski, and Iain James Marshall. 2021. Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization. In AMIA Summits on Translational Science Proceedings. find
  22. Xin Wei. 2023. ClaimDistiller: Scientific claim extraction with supervised contrastive learning. pages 65–77. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2026-scitrue-evidence-grounded-claim-verification/