Elamur Research⌕
☰
home / papers / 2026 / From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence

From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence

Summary

Professional fact-checks contain detailed evidence and reasoning, but this material is buried in free-form prose, so automated verification systems rarely reuse it. The authors build PrimeFacts, a resource and extraction pipeline based on 13,106 text-claim fact-checks from PolitiFact, collected up to September 2025. Each record includes the claim, the verdict and the cited sources. In-article hyperlinks serve as anchors that locate the sentences carrying evidence. The pipeline has three variants: keeping anchored sentences verbatim, having LLMs rewrite them into self-contained premises (Mode B), and extracting premises openly without anchors, with source attribution (Mode C). The authors also propose a faithfulness score that combines forward entailment with an asymmetric lexical-overlap penalty, and they run a small human study. Decontextualized premises are easier to retrieve across articles and improve zero-shot verdict prediction. These gains hold across six instruction-tuned LLMs and both 2-class and 5-class labels. The work shows that existing journalistic evidence can be reused for automated fact-checking.

Key points

  • The corpus contains 13,106 PolitiFact articles by 661 journalists and 49,718 in-text hyperlinks used as evidence anchors.
  • Rewriting anchored sentences into stand-alone premises raises Mean Reciprocal Rank by up to 30% (relative) over verbatim sentences.
  • Using the extracted evidence for verdict prediction improves Macro-F1 by roughly 10–20 points over the no-evidence baseline.
  • The resource is released on Hugging Face as xplainlp/prime-facts. It contains only metadata and annotations; the original article texts are excluded for copyright reasons.
  • The authors list several limitations:
    • evaluation covers only English-language PolitiFact;
    • evidence without a hyperlink can be missed;
    • multi-hop reasoning between premises is not modeled;
    • some risk of LLM hallucination remains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/453datkp6z9s

@inproceedings{sahitaj2026articles,
  title = {{From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence}},
  author = {Sahitaj, Premtim and Kolanowski, Jawan and Sahitaj, Ariana and Solopova, Veronika and Upravitelev, Max and Röder, Daniel and Maab, Iffat and Yamagishi, Junichi and Möller, Sebastian and Schmitt, Vera},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/453datkp6z9s},
  url = {https://doi.org/10.63317/453datkp6z9s}
}

References

48 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. 2018. Where is Your Evidence: Improving Fact-checking by Justification Modeling. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 85– 90, Brussels, Belgium. Association for Computational Linguistics. find
  2. Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information. find
  3. Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Langua… find our card
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. find
  5. Regina Cazzamatta. 2025a. Building a cross-border european information network: Hyperlink connections among fact-checking organizations. Media and Communication, 13. find
  6. Regina Cazzamatta. 2025b. Decoding Correction Strategies: How Fact-Checkers Uncover Falsehoods Across Countries. Journalism Studies, 26(7):777–799. find
  7. Regina Cazzamatta. 2025c. Redefining objectivity: Exploring types of evidence by fact-checkers in four European countries. European Journal of Communication, 40(2):144–164. find
  8. Tong Chen, Hongwei Wang, Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, Dong Yu, and Hongming Zhang. 2024. Dense X retrieval: What retrieval granularity should we use? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15159–15177. Association for Comput… find
  9. Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, and Michael Collins. 2021. Decontextualization: Making Sentences Stand-Alone. Transactions of the Association for Computational Linguistics, 9:447–461. find
  10. Xingyu Deng, Xi Wang, and Mark Stevenson. 2025. The next phase of scientific fact-checking: advanced evidence retrieval from complex structured academic papers. In Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR), pages 4… find
  11. Max Glockner, Yufang Hou, and Iryna Gurevych. 2022. Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5916–5936, Abu Dhabi, United Arab Emirates. Association for Computat… find
  12. Lucas Graves. 2016. Deciding What’s True: The Rise of Political Fact-Checking in American Journalism. Columbia University Press, New York. find
  13. Anisha Gunjal and Greg Durrett. 2024. Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3751–3768, Miami, Florida, USA. Association for Computational Linguistics. find
  14. Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. A Survey on Automated Fact-Checking. Transactions of the Association for Computational Linguistics, 10:178–206. find our card
  15. Edda Humprecht. 2020. How Do They Debunk “Fake News”? A Cross-National Comparison of Transparency in Fact Checks. Digital Journalism, 8(3):310–327. find
  16. Shan Jiang, Simon Baumgartner, Abe Ittycheriah, and Cong Yu. 2020a. Factoring Fact-Checks: Structured Information Extraction from Fact-Checking Articles. In Proceedings of The Web Conference 2020, WWW ’20, pages 1592–1603, New York, NY, USA. Association for Computing Machinery. find
  17. Shan Jiang, Simon Baumgartner, Abe Ittycheriah, and Cong Yu. 2020b. Factoring fact-checks: Structured information extraction from fact-checking articles. In Proceedings of The Web Conference 2020, pages 1592–1603. find
  18. Shailza Jolly, Pepa Atanasova, and Isabelle Augenstein. 2022. Generating Fluent Fact Checking Explanations with Unsupervised Post-Editing. Information, 13(10):500. find
  19. Lasha Kavtaradze. 2024. Challenges of automating fact-checking: A technographic case study. Emerging Media, 2(2):236–258. find
  20. Kashif Khan, Ruizhe Wang, and Pascal Poupart. 2022. WatClaimCheck: A new Dataset for Claim Entailment and Inference. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1293–1304, Dublin, Ireland. Association for Computational Lin… find
  21. Martin Klein, Herbert Van de Sompel, Robert Sanderson, Harihar Shankar, Lyudmila Balakireva, Ke Zhou, and Richard Tobin. 2014. Scholarly context not found: one in five articles suffers from reference rot. PloS one, 9(12):e115253. find
  22. Belinda Z. Li, Emmy Liu, Alexis Ross, Abbas Zeitoun, Graham Neubig, and Jacob Andreas. 2025. Language modeling with editable external knowledge. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3070–3090. Association for Computational Linguistics. find
  23. Xing Han Lù. 2024. BM25S: Orders of magnitude faster lexical search via eager sparse scoring. find
  24. Huanhuan Ma, Weizhi Xu, Yifan Wei, Liuji Chen, Liang Wang, Qiang Liu, Shu Wu, and Liang Wang. 2024. EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification. find
  25. Iffat Maab, Edison Marrese-Taylor, Sebastian Padó, and Yutaka Matsuo. 2024. Media bias detection across families of language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Paper… find
  26. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On Faithfulness and Factuality in Abstractive Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. find
  27. Meta. 2025. Llama 4 scout (17b, 16 experts) — instruct version. https://huggingface.co/meta-llama/ Llama-4-Scout-17B-16E-Instruct. Accessed: 2025-10-13. ↗
  28. Meta AI. 2024. Llama 3.3 70B Instruct. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct. Accessed: 2025-10-13. ↗
  29. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress Test Evaluation for Natural Language Inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computation… find
  30. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeño, Rubén Míguez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Watheq Mansour, Bayan Hamdan, Zien Sheikh Ali, Nikolay Babulkov, Alex Nikolov, Gautam Kishore Shahi, Julia Maria Struß, Thomas Mandl, Mucahid Kutlu, a… find
  31. Wojciech Ostrowski, Arnav Arora, Pepa Atanasova, and Isabelle Augenstein. 2021. Multi-Hop Fact Checking of Political Claims. find
  32. Rrubaa Panchendrarajan and Arkaitz Zubiaga. 2024. Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research. Natural Language Processing Journal, 7:100066. find
  33. Premtim Sahitaj, Iffat Maab, Junichi Yamagishi, Jawan Kolanowski, Sebastian Möller, and Vera Schmitt. 2025. Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs. find
  34. Chris Samarinas, Wynne Hsu, and Mong-Li Lee. 2021. Improving evidence retrieval for automated explainable fact-checking. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, pages 84–91. find
  35. Artsiom Sauchuk, James Thorne, Alon Halevy, Nicola Tonellotto, and Fabrizio Silvestri. 2022. On the role of relevance in natural language processing tasks. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1785–1789. find
  36. Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. 2023. Averitec: A dataset for real-world claim verification with evidence from the web. Advances in Neural Information Processing Systems, 36:65128–65167. find
  37. Tal Schuster, Adam Fisch, and Regina Barzilay. 2021. Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 624–643, Online. Assoc… find our card
  38. Christian Stab and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619–659. find our card
  39. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: A Large-scale Dataset for Fact Extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo… find our card
  40. Xiaoou Wang, Elena Cabrio, and Serena Villata. 2025a. Safe: Structured argumentation for fact-checking with explanations. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25, pages 11114– 11118. International Joint Conferences on Artificial Intelli… find
  41. Xiaoou Wang, Elena Cabrio, and Serena Villata. 2025b. When automated fact-checking meets argumentation: Unveiling fake news through argumentative evidence. Argument and Computation, 16. find
  42. Greta Warren, Irina Shklovski, and Isabelle Augenstein. 2025. Show me the work: Fact-checkers’ requirements for explainable automated fact-checking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–21. find
  43. Albert Weichselbraun. 2021. Inscriptis - a python-based HTML to text conversion library optimized for knowledge extraction from the web. Journal of Open Source Software, 6(66):3557. find
  44. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… find
  45. Rui Xing, Timothy Baldwin, and Jey Han Lau. 2024. Evaluating transparency of machine generated fact checking explanations. arXiv e-prints, pages arXiv–2406. find
  46. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. find our card
  47. Fengzhu Zeng and Wei Gao. 2024. JustiLM: Few-shot Justification Generation for Explainable Fact-Checking of Real-world Claims. Transactions of the Association for Computational Linguistics, 12:334–354. find
  48. Ke Zhou, Claire Grover, Martin Klein, and Richard Tobin. 2015. No more 404s: predicting referenced link rot in scholarly articles for pro-active archiving. In Proceedings of the 15th ACM/IEEE-CS joint conference on digital libraries, pages 233–236. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sahitaj-2026-articles-premises-building-primefacts-extraction/