Elamur Research⌕
☰
home / papers / 2025 / FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation

FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation

Paper · 2025 · Farima Fatahi Bayat, Lechen Zhang, Sheza Munir et al.

Summary

Existing long-form factuality benchmarks are fixed snapshots, which exposes them to data contamination. They also cover narrow scenarios, such as biographies or LM-written questions, rather than what real users ask. The authors address this by mining in-the-wild prompts from LMSYS-Chat-1M. They cluster the prompts with BERTopic into 382 tasks and keep only prompts judged verifiable and useful. To evaluate responses, they built VERIFY, a pipeline that does the following:

  • splits an answer into decontextualized content units
  • keeps only the units that can be checked
  • searches the Web for evidence
  • labels each checked unit Supported, Unsupported, or Undecidable

A hallucination score that penalizes both wrong and unverifiable units ranks the prompts. The prompts that most often trigger errors form FactBench: 1,000 prompts over 150 topics, split into Easy, Moderate and Hard tiers. The benchmark is meant to be refreshed periodically. VERIFY agrees with human judgments more closely than competing evaluators. Benchmarking several proprietary and open-weight LMs shows that accuracy falls as prompt difficulty rises and that larger models are not automatically more factual. This makes FactBench a contamination-resistant, realistic testbed for hallucination research.

Key points

  • Data pipeline: Cleaning left 294,333 distinct prompts. HDBSCAN set aside 48.5% of them as niche outliers. The final FactBench contains 1K prompts across 150 topics in three difficulty tiers.
  • Agreement with humans: VERIFY's Pearson correlation with human labels reaches 0.97 on units that are factual and 0.73 on units that are not. On average, it gains 15.7% balanced accuracy over baseline methods.
  • Human annotations released: Two annotators each judged 4,467 content units. In a separate check of 570 Undecidable units, 57% turned out to be factual, which led the authors to set the score weight α = 0.5.
  • Model scale: Llama3.1-405B-Instruct performs about the same as, or worse than, Llama3.1-70B-Instruct.
  • Refusals: Gemini1.5-Pro refuses almost 10% of Hard prompts, and roughly a quarter of its refusals are unjustified. Most of these unjustified refusals stem from misinformation worries, while others come from inflated ethical or legal risk.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Farima Fatahi Bayat, Lechen Zhang, Sheza Munir and Lu Wang. 2025. FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.1587

@inproceedings{bayat2025factbench,
  title = {{FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation}},
  author = {Farima Fatahi Bayat, Lechen Zhang, Sheza Munir and Lu Wang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1587},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1587}
}

References

39 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama3 https://ai.meta.com/blog/meta-llama-3/. model. ↗
  2. Anthropic. 2024. Claude 3.5 sonnet. https://www.anthropic.com/news/claude-3-5-sonnet. ↗
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  4. Liang Chen, Yang Deng, Yatao Bian, Zeyu Qin, Bingzhe Wu, Tat-Seng Chua, and Kam-Fai Wong. 2023a. Beyond factuality: A comprehensive evaluation of large language models as knowledge generators. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6325– 6341… find
  5. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023b. Felm: Benchmarking factuality evaluation of large language models. Preprint, arXiv:2310.00741. find
  6. Cohere. 2024. Command r+. https://cohere.com/blog/command-r-plus-microsoft-azure. ↗
  7. Gemini, Rohan Anil, Sebastian Borgeaud, and Jean-Baptiste et al. 2024. Gemini: A family of highly capable multimodal models. Preprint, arXiv:2312.11805.
  8. Jonathan Goldsmith. 2014. Wikipedia: A pythonic wrapper for the wikipedia api. https://github.com/goldsmith/Wikipedia. ↗
  9. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. Preprint, arXiv:2203.05794.
  10. Anisha Gunjal and Greg Durrett. 2024. Molecular facts: Desiderata for decontextualization in llm fact verification. Preprint, arXiv:2406.20079. find
  11. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  12. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. Preprint, arXiv:2311.05232.
  13. Lev Konstantinovskiy, Oliver Price, Mevan Babakar, and Arkaitz Zubiaga. 2020. Towards automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection. Preprint, arXiv:1809.08193. — [arXiv:1809.08193](https://arxiv.org/abs/1809.08193) find
  14. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large-scale hallucination evaluation benchmark for large language models. Preprint, arXiv:2305.11747.
  15. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. Preprint, arXiv:2109.07958.
  16. Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. 2023. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4791–4797, Singapore… find
  17. Inbal Magar and Roy Schwartz. 2022. Data contamination: From memorization to exploitation. Preprint, arXiv:2203.08242. find
  18. Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. 2024. ExpertQA: Expert-curated questions and attributed answers. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  19. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… find
  20. Claudia Malzer and Marcus Baum. 2020. A hybrid approach to hierarchical density-based cluster selection. In 2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI). IEEE. find
  21. Federico Ruggeri, Francesco Antici, Andrea Galassi, Katerina Korre, Arianna Muti, and Alberto Barrón-Cedeño. 2023. On the definition of prescriptive annotation guidelines for language-agnostic subjectivity detection. find
  22. Leland McInnes, John Healy, and James Melville. 2020. Umap: Uniform manifold approximation and projection for dimension reduction. Preprint, arXiv:1802.03426.
  23. Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. Veriscore: Evaluating the factuality of verifiable claims in long-form text generation. Preprint, arXiv:2406.19276.
  24. Meta. 2024. Introducing llama 3.1: Our most capable models to date. https://ai.meta.com/blog/ meta-llama-3-1/. Accessed: 2024-09-10. ↗
  25. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. find
  26. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
  27. Yuxia Wang, Revanth Gangi Reddy, Zain Muhammad Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, Isabelle Augenstein, Iryna Gurevych, and Preslav Nakov. 2024. Factcheck-bench: Fine-grained evaluation benchmark for automat… find
  28. MistralAI. 2024. Mistral large https://mistral.ai/news/mistral-large-2407. ↗
  29. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2024a. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22,…
  30. Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, and Yoav Shoham. 2024. Generating benchmarks for factuality evaluation of language models. In Proceedings of the 18th Conference of the European Chapter of the Association for…
  31. Preslav Nakov, Alberto Barrón-Cedeño, Giovanni Da San Martino, Firoj Alam, Rubén Míguez, Tommaso Caselli, Mucahid Kutlu, Wajdi Zaghouani, Chengkai Li, Shaden Shaar, Hamdy Mubarak, Alex Nikolov, and Yavuz Selim Kartal. 2022. Overview of the clef-2022 checkthat! lab task 1 on identifying relevant clai… find
  32. Jingwei Ni, Minjing Shi, Dominik Stammbach, Mrinmaya Sachan, Elliott Ash, and Markus Leippold. 2024. AFaCTA: Assisting the annotation of factual claim detection with reliable LLM annotators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa… find
  33. OpenAI. 2024a. https://openai.com/index/hello-gpt-4o/. 2024-05-13. ↗
  34. Gpt-4o. Version: OpenAI. 2024b. New embedding models and api updates. https://openai.com/index/ new-embedding-models-and-api-updates/. ↗
  35. Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024b. Long-form factuality in large language models. Preprint, arXiv:2403.18802.
  36. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
  37. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations.
  38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2024. Lmsys-chat-1m: A large-scale real-world llm conversation dataset. Preprint, arXiv:2309.11998.
  39. OpenAI. 2024c. New models and developer products announced at devday. https://openai.com/blog/ new-models-and-developer-products-announced-at-devday. Version: turbo-2024-04-09. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bayat-2025-factbench-dynamic-benchmark-wild-language/