Elamur Research⌕
☰
home / papers / 2026 / Can LLMs Automate Fact-Checking Article Writing?

Can LLMs Automate Fact-Checking Article Writing?

Paper · 2026 · Dhruv Sahnan, David Corney, Irene Larraz et al.

Summary

Research on automated fact-checking usually ends at spotting claims, retrieving evidence, assigning a verdict and, at most, producing a short justification. Professional fact-checkers go further. They publish full articles that explain where a claim came from, its context and possible readings, counter-arguments, and how the evidence leads to the verdict. No existing pipeline covers this time-consuming step. The authors propose adding article generation as a new pipeline stage. They interviewed four experts from leading fact-checking organizations to define what a good article needs. The experts also explained why they distrust AI for this job: invented arguments and URLs, and a weak grasp of context. Based on this, the authors built QRAFT, a multi-agent LLM framework. Given a claim, its verdict and evidence documents, it extracts evidence nuggets, plans an outline and writes a draft. It then revises the draft through a simulated editorial question-and-answer exchange between agents. QRAFT beats several generic text-generation baselines on automatic metrics, but professional fact-checkers rate it well below expert-written articles. Human oversight therefore remains essential.

Key points

  • Task setup: the input is a claim, its veracity label and the evidence set, all supplied by a fact-checker. The output is a full article, compared against an expert-written reference. Evidence retrieval and verification are outside the scope.
  • Expert desiderata: the experts (anonymized as P1–P4) named six qualities a good article needs:
    • accurate clarification of the claim;
    • the claim's origin and context;
    • transparent, source-verifiable writing;
    • a clear structure (claim, then arguments, then conclusion);
    • an explanation of why the claim matters.
  • Architecture: QRAFT uses three LLM agents (GPT-4o and GPT-4o-mini) working in two stages. The first stage handles nuggets, outline and draft. The second stage is an agent-to-agent editorial review that produces edits.
  • Evaluation: QRAFT was assessed with automatic metrics, LLM-as-a-judge and expert human evaluation. It led on all automatic measures but still fell short of human articles. Specific scores are not included in the provided excerpt.
  • Code: available at github.com/mbzuai-nlp/qraft.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technische Universität DarmstadtDarmstadt · Germany109 works · 35,920 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations🎓Indian Institute of Technology DelhiNew Delhi · India22 works · 1,036 citations🤝Full FactLondon · United Kingdom2 works · 248 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Dhruv Sahnan, David Corney, Irene Larraz, Giovanni Zagni, Ruben Miguez, Zhuohan Xie, Iryna Gurevych, Elizabeth Churchill, Tanmoy Chakraborty and Preslav Nakov. 2026. Can LLMs Automate Fact-Checking Article Writing?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.644

@article{sahnan2026can,
  title = {{Can LLMs Automate Fact-Checking Article Writing?}},
  author = {Dhruv Sahnan, David Corney, Irene Larraz, Giovanni Zagni, Ruben Miguez, Zhuohan Xie, Iryna Gurevych, Elizabeth Churchill, Tanmoy Chakraborty and Preslav Nakov},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.644},
  url = {https://doi.org/10.1162/tacl.a.644}
}

References

61 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352–7364, Online. Association for Computational Linguistics. https://doi.org/…
  2. Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Langua…
  3. Cheng-Han Chiang and Hung-yi Lee. 2024. Merging facts, crafting fallacies: Evaluating the contradictory nature of aggregated factual claims in long-form generations. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2734–2751, Bangkok, Thailand. Association for Computatio… find
  4. Anubrata Das, Houjiang Liu, Venelin Kovatchev, and Matthew Lease. 2023. The state of human-centered NLP technology for fact-checking. Information Processing & Management, 60(2): 103219. https://doi.org/10.1016/j .ipm.2022.103219 — [doi:10.1016/j](https://doi.org/10.1016/j) find
  5. Subhabrata Dutta, Timo Kaufmann, Goran Glavaš, Ivan Habernal, Kristian Kersting, Frauke Kreuter, Mira Mezini, Iryna Gurevych, Eyke Hüllermeier, and Hinrich Schütze. 2025. Problem solving through Human-AI preference-based cooperation. Computational Linguistics, pages 1–35. https://doi.org /10.1162… ↗
  6. Lucas Graves. 2017. Anatomy of a fact check: Objective practice and the contested epistemology of fact checking. Communication, Culture and Critique, 10(3):518–537. https://doi.org/10.1111/cccr.12163 — [doi:10.1111/cccr.12163](https://doi.org/10.1111/cccr.12163) ↗
  7. Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206. https://doi.org/10.1162/tacl a 00454 ↗
  8. Naeemul Hassan, Chengkai Li, and Mark Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM 2015, Melbourne, VIC, Australia,October19–23,2015, pages 1835–1838. ACM. https://… — [doi:10.1145/2806416](https://doi.org/10.1145/2806416) find
  9. Zhe Hu, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Hua Wu, and Lifu Huang. 2022. PLANET: Dynamic content planning in autoregressive transformers for long-form text generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2288… find
  10. Prerna Juneja and Tanushree Mitra. 2022. Human and technological infrastructures of fact-checking. Proceedings of the ACM on Human Computer Interaction, 6(CSCW2). https://doi.org/10.1145/3555143 — [doi:10.1145/3555143](https://doi.org/10.1145/3555143) ↗
  11. Dayoon Ko, Jinyoung Kim, Hahyeon Choi, and Gunhee Kim. 2024. GrowOVER: How can LLMs adapt to growing real-world knowledge? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3282–3308, Bangkok, Thailand. Association for Computati… find
  12. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025b. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems…
  13. Kim Alice Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(1).
  14. Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. Prometheus 2: An open source language model specialized in evaluating other language models. In Proceedings of the 2024 Conference on Empirical Metho…
  15. Hui Huang, Xingyuan Bu, Hongli Zhou, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. 2025a. An empirical study of LLM-as-a-judge for LLM evaluation: Fine-tuned judge model is not a general substitute for GPT-4. In Findings of the Association for Computational Linguistics: ACL 2025, pages… find
  16. Fantine Huot, Reinald Jennimaria Palomaki, Jakobovits, Elizabeth Clark, and Mirella Lapata. 2025. Agents’ Room: Narrative generation through multi-step collaboration. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24–28, 2025. OpenReview.net. find
  17. Lev Konstantinovskiy, Oliver Price, Mevan Babakar, and Arkaitz Zubiaga. 2021. Toward automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection. Digital Threats, 2(2). https://doi.org/10 .1145/3412869 find
  18. Neema Kotonya and Francesca Toni. 2020. Explainable automated fact-checking: A survey. In Proceedings of the 28th International Amplayo, Shoshana Conference on Computational Linguistics, pages 5430–5443, Barcelona, Spain (Online). International Committee on Computational Linguistics. https://doi.org… ↗
  19. Singapore. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.741 ↗
  20. Preslav Nakov, David Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alberto Barrón-Cedeño, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. 2021a. Automated fact-checking for assisting human fact-checkers. In Proceedings of the Thirtieth International Joint Conference on Artificial In… — [doi:10.24963/ijcai.2021/619](https://doi.org/10.24963/ijcai.2021/619) find
  21. Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163–177. https://doi.org/10 .1162/tacl_a_00453 find
  22. Yi Liang, You Wu, Honglei Zhuang, Li Chen, Jiaming Shen, Yiling Jia, Zhen Qin, Sumit Sanghai, Xuanhui Wang, Carl Yang, and Michael Bendersky. 2024. Integrating planning into single-turn long-form text generation. ArXiv preprint, abs/2410.06203. — [arXiv:2410.06203](https://arxiv.org/abs/2410.06203) find
  23. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeño, Rubén Mı́guez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Nikolay Babulkov, Alex Nikolov, Gautam Kishore Shahi, Julia Maria Struß, and Thomas Mandl. 2021b. The CLEF-2021 CheckThat! Lab on detecting check-… — [doi:10.1007/978-3-030-72240-1_75](https://doi.org/10.1007/978-3-030-72240-1_75) find
  24. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  25. Houjiang Liu, Anubrata Das, Alexander Boltz, Didi Zhou, Daisy Pinaroc, Matthew Lease, and Min Kyung Lee. 2024. Human-centered NLP fact-checking: Co-designing with fact-checkers using Matchmaking for AI. Proceedings of the ACM on Human Computer Interaction, 8(CSCW2). https://doi.org/10.1145/3686962 ↗
  26. Team OpenAI. 2023. GPT-4 technical report.
  27. Yuxiang Liu and Kevin Chen-Chuan Chang. 2025. Writing like the best: Exemplar-based expository text generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25739–25764, Vienna, Austria. Association for Computational Lingui… find
  28. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
  29. Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. DeClarE: Debunking fake news and false claims using evidence-aware deep learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 22–32, Brussels, Belgium. Association for Co…
  30. D. Gordon Rohman. 1965. Pre-writing the stage of discovery in the writing process. College Composition and Communication, 16(2):106–112. https://doi.org/10.2307/354885 — [doi:10.2307/354885](https://doi.org/10.2307/354885) ↗
  31. Shaden Shaar, Nikolay Babulkov, Giovanni Da San Martino, and Preslav Nakov. 2020. That is a known lie: Detecting previously fact-checked claims. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3607–3618, Online. Association for Computational Linguist…
  32. USA. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-emnlp.552 ↗
  33. Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A. Metoyer. 2025. Limitations of the LLM-as-a-judge approach for evaluating LLM outputs in expert knowledge tasks.In Proceedings of the 30th International Conference on Intelligent User Interfaces, IUI… find
  34. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. 2024. Assisting in writing Wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  35. Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pages 18–22, Baltimore, MD, USA. Association for Computational Linguistics. https://doi.org/10 .3115…
  36. Zejiang Shen, Tal August, Pao Siangliulue, Kyle Lo, Jonathan Bragg, Jeff Hammerbacher, Doug Downey, and Joseph Chee Chang. 2023. Beyond summarization: Designing AI support for real-world expository writing tasks. ArXiv, abs/2304.02623. — [arXiv:2304.02623](https://arxiv.org/abs/2304.02623) find
  37. Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, Daiyuan Li, Yu Hu, and Mingkui Tan. 2025. Generating long-form story using dynamic hierarchical outlining with memory-enhancement. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Ling… find
  38. Shuqian Sheng, Yi Xu, Luoyi Fu, Jiaxin Ding, Lei Zhou, Xinbing Wang, and Chenghu Zhou. 2024. Is reference necessary in the evaluation of NLG systems? When and where? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T… find
  39. Amir Soleimani, Christof Monz, and Marcel Worring. 2020. BERT for evidence retrieval and claim verification. In Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Proceedings, Part II, pages 359–366, Berlin, Heidelberg. Springe… — [doi:10.1007/978-3-030-45442-5_45](https://doi.org/10.1007/978-3-030-45442-5_45) find
  40. Qiyao Wang, Shiwen Ni, Huaren Liu, Shule Lu, Guhong Chen, Xi Feng, Chi Wei, Qiang Qu, Hamid Alinejad-Rokny, Yuan Lin, and Min Yang. 2024a. AutoPatent: A multi-agent framework for automatic patent generation. ArXiv preprint, abs/2412.09796. — [arXiv:2412.09796](https://arxiv.org/abs/2412.09796) find
  41. William Yang Wang. 2017. ‘‘Liar, liar pants on fire’’: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426, Vancouver, Canada. Association for Computational Linguistics. ht…
  42. Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. VeriScore: Evaluating the factuality of verifiable claims in long-form text generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9447–9474, Miami, Florida, Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang…
  43. Zhuohan Xie, Rui Xing, Yuxia Wang, Jiahui Geng, Hasan Iqbal, Dhruv Sahnan, Iryna Gurevych, and Preslav Nakov. 2025. FIRE: Fact-checking with iterative retrieval and verification. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2901–2914, Albuquerque, New Mexico. Assoc… find
  44. Greta Warren, Irina Shklovski, and Isabelle Augenstein. 2025. Show me the work: Fact-checkers’ requirements for explainable automated fact-checking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. New York, NY, USA. Association for Computing Machinery. https://doi.or… — [doi:10.1145/3706598](https://doi.org/10.1145/3706598) ↗
  45. Rui Xing, Shraey Bhatia, Timothy Baldwin, and Jey Han Lau. 2022. Automatic explanation generation for climate science claims. In Proceedings of the 20th Annual Workshop of the Australasian Language Technology Association, pages 122–129, Adelaide, Australia. Australasian Language Technology Associati… find
  46. Edward Wates and Robert Campbell. 2007. Author’s version vs. publisher’s version: An analysis of the copy-editing function. Learned Publishing, 20(2):121–129. https://doi.org/10.1087/174148507X185090 — [doi:10.1087/174148507x185090](https://doi.org/10.1087/174148507x185090) ↗
  47. Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. 2023. DOC: Improving long story coherence with detailed outline control. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3378–3465, Toronto, Canada. Association for Comput…
  48. Constance A. Weaver and Walter Kintsch. 1991. Expository text. In Handbook of Reading Research, volume 2, pages 230–245. Lawrence Erlbaum Associates, Mahwah, NJ. find
  49. Dustin Wright and Isabelle Augenstein. 2020. Claim check-worthiness detection as positive unlabelled learning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 476–488, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.findings -emnlp… ↗
  50. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates. Association for Computatio… find
  51. Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. 2022. PRIMERA: Pyramid-based masked sentence pre-training for multi-document summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5245–5263, Dublin, Ireland.…
  52. Lili Yao, Nanyun Peng, Ralph M. Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-Write: Towards better automatic storytelling. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference,… find
  53. Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V. Chawla, and Xiangliang Zhang. 2020. Justice or prejudice? Quantifying biases in LLM-as-a-judge. In The Thirteenth International Conference on Learning Representations,… find
  54. Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2019. GEAR: Graph-based evidence aggregating and reasoning for fact verification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 892–901, Florence, Italy. Associ…
  55. Fengzhu Zeng and Wei Gao. 2024. JustiLM: Few-shot justification generation for explainable fact-checking of real-world claims. Transactions of the Association for Computational Linguistics, 12:334–354. https://doi.org/10.1162/tacl a 00649 ↗
  56. Anni Zou, Zhuosheng Zhang, and Hai Zhao. 2023. Decker: Double check with heterogeneous knowledge for commonsense fact verification. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11891–11904, Toronto, Canada. Association for Computational Linguistics. https://doi.org/… find
  57. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. find
  58. Guido Zuccon, Bevan Koopman, and Razia Shaik. 2023. ChatGPT hallucinates when attributing answers. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, SIGIR-AP ’23, pages 46–51, New York, NY, USA. Associatio… — [doi:10.1145/3624918.3625329](https://doi.org/10.1145/3624918.3625329) find
  59. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot arena. In Advances Section I.
  60. Content General Question Round Q. As a fact-checker, what do you expect from a fact-check article? Q. Can you comment on any challenges for AI that you may foresee in the process of writing a fact-check article? II. Sample Data Presentation Claim: ‘‘In December 2023, conspiracy theorist \<person 1\>’s… find
  61. Focused Question Round 1 Q. What do you expect from an article written for the given data? Q. What do you think an AI-based system might miss if it generated a fact-check article from the given data? IV. Presenting Generated Article Article: – Redacted – V. Focused Question Round 2 Q. What are your… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sahnan-2026-can-llms-automate-fact-checking/