Elamur Research⌕
☰
home / papers / 2025 / TripleFact: Defending Data Contamination in the Evaluation of LLM-driven Fake News Detection

TripleFact: Defending Data Contamination in the Evaluation of LLM-driven Fake News Detection

Paper · 2025 · Cheng Xu and Nan Yan

Summary

Large language models can see evaluation benchmarks during pre-training. This benchmark data contamination (BDC) can make fake news detection look better than it really is, because models may recall debunked claims instead of reasoning about them. Static datasets such as LIAR assume test items are unseen, and LLMs break that assumption. The authors propose TripleFact, an evaluation framework built to resist contamination. It has three parts:

  • HAPT tests models against misinformation written by humans to be adversarial.
  • RTW-AV uses a real-time web agent to collect fresh claims and checks the labels later.
  • ECVE replaces real entities with synthetic ones to remove memorized entity associations.

The scores are combined into a TripleFact Score (TFS). The authors evaluated 17 LLMs, including GPT, LLaMA and DeepSeek models. They also fine-tuned models with deliberately injected contamination at increasing levels. Conventional metrics rose by up to 23%, while TFS stayed nearly constant. TripleFact therefore offers a way to separate real detection skill from memorization when benchmarking LLMs on misinformation.

Key points

  • The authors injected four types of contamination: semantic, information, data and label. Label-level contamination raised the HAPT output from 0.40 to 0.52 and the RTW-AV output from 0.62 to 0.85.
  • ECVE offsets these inflated scores. Its BRR value fell to −0.18 under full label-level contamination, which exposes reliance on memorized entities.
  • TFS ranged from 0.49 to 0.57 across all contamination settings, which the authors report as an absolute error of at most 4%.
  • The experiments are built on the LIAR2 dataset (released on Hugging Face as chengxuphd/liar2). Data from X and PolitiFact was also used but cannot be redistributed. The code is available at github.com/chengxuphd/triplefact.
  • The authors note several limitations. The framework was tested mainly on English, and low-resource languages remain untested. HAPT depends on crowd-sourced adversarial examples, synthetic entities may change a claim's meaning, and RTW-AV assumes that fact-checkers eventually reach consensus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

Cite

Cite as Cheng Xu and Nan Yan. 2025. TripleFact: Defending Data Contamination in the Evaluation of LLM-driven Fake News Detection. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.431

@inproceedings{xu2025triplefact,
  title = {{TripleFact: Defending Data Contamination in the Evaluation of LLM-driven Fake News Detection}},
  author = {Cheng Xu and Nan Yan},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.431},
  url = {https://doi.org/10.18653/v1/2025.acl-long.431}
}

References

68 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Co… find
  2. Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2):211–36. find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  4. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations. find
  5. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21)…
  6. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating llms by human preference. Preprint, arXiv:2403.04132.
  7. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20:273–297.
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, and et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.
  9. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, and et al. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. find
  10. Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, and Arman Cohan. 2024. Unveiling the spectrum of data contamination in language model: A survey from detection to remediation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16078–16092, Bang… find
  11. Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. 2023. Investigating data contamination in modern benchmarks for large language models. Preprint, arXiv:2311.09783.
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. 2024. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. In Findings of th… find
  13. Team GLM, :, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, and et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. Preprint, arXiv:2406.12793.
  14. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  15. Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206.
  16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In International Conference on Learning Representations.
  17. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  18. Benjamin D. Horne, Jeppe Nørregaard, and Sibel Adali. 2019. Robust fake news detection over time and attack. ACM Trans. Intell. Syst. Technol., 11(1). find
  19. Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher Choquette Choo, and Nicholas Carlini. 2023. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Lan…
  20. Minhao Jiang, Ken Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. 2024. Does data contamination make a difference? insights from intentionally contamination pre-training data for language models. In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Founda… find
  21. Kashif Khan, Ruizhe Wang, and Pascal Poupart. 2022. WatClaimCheck: A new dataset for claim entailment and inference. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1293–1304, Dublin, Ireland. Association for Computational Lin… find
  22. Kalev Leetaru and Philip A Schrodt. 2013. Gdelt: Global data on events, location, and tone, 1979–2012. In ISA annual convention, volume 2, pages 1–49. Citeseer. find
  23. Mehrnaz Moslemi and Amal Zouaq. 2024. TagDebias: Entity and concept tagging for social bias mitigation in pretrained language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1553–1567, Mexico City, Mexico. Association for Computational Linguistics. find
  24. Stephan Lewandowsky. 2025. Free speech, fact checking, and the right to accurate information. Science, 387(6734). find
  25. Changmao Li and Jeffrey Flanigan. 2024. Task contamination: Language models may not be few-shot anymore. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):18471–18480. find
  26. Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024a. Culturellm: Incorporating cultural differences into large language models. In Advances in Neural Information Processing Systems, volume 37, pages 84799–84838. Curran Associates, Inc.
  27. Sihang Li, Jin Huang, Jiaxi Zhuang, Yaorui Shi, Xiaochen Cai, Mingjun Xu, Xiang Wang, Linfeng Zhang, Guolin Ke, and Hengxing Cai. 2025. Scilitllm: How to adapt llms for scientific literature understanding. In The Thirteenth International Conference on Learning Representations. find
  28. Xiang Li, Yunshi Lan, and Chao Yang. 2024b. Treeeval: Benchmark-free evaluation of large language models through tree planning. Preprint, arXiv:2402.13125. — [arXiv:2402.13125](https://arxiv.org/abs/2402.13125) find
  29. Yucheng Li, Frank Guerin, and Chenghua Lin. 2024c. Latesteval: Addressing data contamination in language model evaluation through dynamic and time-sensitive test construction. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):18600– 18607. find
  30. Yiheng Liu, Hao He, Tianle Han, Xu Zhang, Mengyuan Liu, Jiaming Tian, Yutong Zhang, Jiaqi Wang, Xiaohui Gao, Tianyang Zhong, Yi Pan, Shaochen Xu, Zihao Wu, Zhengliang Liu, Xin Zhang, Shu Zhang, Xintao Hu, Tuo Zhang, Ning Qiang, and 2 others. 2024. Understanding llms: A comprehensive overview from tr… — [arXiv:2401.02038](https://arxiv.org/abs/2401.02038) find
  31. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. The flan collection: Designing data and methods for effective instruction tuning. In Proceedings of the 40th International Conference on Machine Learning, vol…
  32. Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2024. KnowTun-ing: Knowledge-aware fine-tuning for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, OpenAI.…
  33. Preprint, Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2024. Fine-tuning or retrieval? comparing knowledge injection in LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 237–250, Miami, Florida, USA. Association for Computational…
  34. Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. 2024. A survey on the use of large language models (llms) in fake news. Future Internet, 16(8). find
  35. Kellin Pelrine, Anne Imouza, Camille Thibault, Meilina Reksoprodjo, Caleb Gupta, Joel Christoph, Jean-François Godbout, and Reihaneh Rabbany. 2023. Towards reliable misinformation mitigation: Generalization, uncertainty, and GPT-4. In Proceedings of the 2023 Conference on Empirical Methods in Natura… find
  36. Gordon Pennycook and David G Rand. 2021. The psychology of fake news. Trends in cognitive sciences, 25(5):388–402. find
  37. Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3391–3401, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  38. Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh. 2022. Impact of pretraining term frequencies on few-shot numerical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 840–854, Abu Dhabi, United Arab Emirates. Association for Computational Li…
  39. Y.M. Rocha, G.A. de Moura, G.A. Desidério, C.H. de Oliveira, F.D. Lourenço, and L.D. de Figueiredo Nicolete. 2021. The impact of fake news on social media and its influence on health during the covid-19 pandemic: A systematic review. Journal of Public Health, pages 1–10. find
  40. Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2024. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations. find
  41. Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data, 8(3):171–188. find
  42. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. SIGKDD Explor. Newsl., 19(1):22–36. find
  43. Nataliya Stepanova and Björn Ross. 2023. Temporal generalizability in multimodal misinformation detection. In Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP, pages 76–88, Singapore. Association for Computational Linguistics. find
  44. Yifan Sun, Han Wang, Dongbai Li, Gang Wang, and Huan Zhang. 2025. The emperor’s new clothes in benchmarking? a rigorous examination of mitigation strategies for llm benchmark data contamination. Preprint, arXiv:2503.16402. — [arXiv:2503.16402](https://arxiv.org/abs/2503.16402) find
  45. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. Preprin…
  46. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, and et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. Preprint, arXiv:2307.09288.
  47. Petter Törnberg. 2023. Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. Preprint, arXiv:2304.06588.
  48. Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. 2023. A causal view of entity bias in (large) language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15173–15184, Singapore. Association for Computational Linguistics. find
  49. William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426, Vancouver, Canada. Association for Computational Linguistics.
  50. Przemyslaw M. Waszak, Wioleta Kasprzycka-Waszak, and Alicja Kubanek. 2018. The spread of medical fake news in social media – the pilot quantitative study. Health Policy and Technology, 7(2):115–118. find
  51. Bingbing Wen, Chenjun Xu, Bin HAN, Robert Wolfe, Lucy Lu Wang, and Bill Howe. 2024. From human to model overconfidence: Evaluating confidence dynamics in large language models. In NeurIPS 2024 Workshop on Behavioral Machine Learning. find
  52. Jiaying Wu, Jiafeng Guo, and Bryan Hooi. 2024. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 3367–3378, New York, NY, USA. Association for Computing… find
  53. Chunqiu Steven Xia, Yinlin Deng, and Lingming Zhang. 2024. Top leaderboard ranking = top coding proficiency, always? evoeval: Evolving coding benchmarks via llm. Preprint, arXiv:2403.19114. — [arXiv:2403.19114](https://arxiv.org/abs/2403.19114) find
  54. Cheng Xu, Shuhao Guan, Derek Greene, and M-Tahar Kechadi. 2024. Benchmark data contamination of large language models: A survey. Preprint, arXiv:2406.04244.
  55. Cheng Xu and M-Tahar Kechadi. 2023. Fuzzy deep hybrid network for fake news detection. In Proceedings of the 12th International Symposium on Information and Communication Technology, SOICT ’23, page 118–125, New York, NY, USA. Association for Computing Machinery. find
  56. Cheng Xu and M-Tahar Kechadi. 2024. An enhanced fake news detection system with fuzzy deep learning. IEEE Access, 12:88006–88021. find
  57. Cheng Xu, Jing Wang, Tianlong Zheng, Yue Cao, and Fan Ye. 2022. Prediction of prognosis and survival of patients with gastric cancer by a weighted improved random forest model: an application of machine learning in medicine. Archives of Medical Science, 18(5):1208–1220. find
  58. Cheng Xu and Nan Yan. 2023. AROT-COV23: A dataset of 500k original arabic tweets on COVID-19. In 4th Workshop on African Natural Language Processing. find
  59. Nan Yan and Cheng Xu. 2024. Decolonizing african NLP: A survey on power dynamics and data colonialism in tech development. In 5th Workshop on African Natural Language Processing. find
  60. Jiahao Ying, Yixin Cao, Bo Wang, Wei Tang, Yizhe Yang, and Shuicheng Yan. 2024. Have seen me before? automating dataset updates towards reliable and timely evaluation. Preprint, arXiv:2402.11894. — [arXiv:2402.11894](https://arxiv.org/abs/2402.11894) find
  61. Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, and Shikun Zhang. 2024. Freeeval: A modular framework for trustworthy and efficient evaluation of large language models. Preprint, arXiv:2404.06003. find
  62. Reza Zafarani, Xinyi Zhou, Kai Shu, and Huan Liu. 2019. Fake news research: Theories, detection strategies, and open problems. In Proc of the 25th ACM SIGKDD Int’l. Conf on Knowledge Discovery & Data Mining, KDD’19, page 3207–3208, New York, NY, USA. ACM. find
  63. Fengzhu Zeng and Wei Gao. 2024. JustiLM: Few-shot justification generation for explainable fact-checking of real-world claims. Transactions of the Association for Computational Linguistics, 12:334–354.
  64. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Preprint, arXiv:2306.05685.
  65. Model Name HAPT RTW-AV ECVE o3-mini-2025-01-31 - with medium - with high o1-preview-2024-09-12 o1-mini-2024-09-12 gpt-4o-2024-08-06 gpt-4o-mini-2024-07-18 gpt-4-turbo-2024-04-09 gpt-3.5-turbo-0125 0.63 1.98 4.19 8.94 0.34 0.07 \<0.01 0.26 0.01 9.91 30.81 59.98 277.82 4.25 2.98 0.35 10.32 0.54 14.12 4… find
  66. Xinyi Zhou and Reza Zafarani. 2020. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Comput. Surv., 53(5). *Table 5: OpenAI API cost of a single experiment by TripleFact framework ($).* Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, and Xi… find
  67. Model Name HAPT RTW-AV ECVE Llama-3.3-70B Llama-3.2-3B Llama-3.2-1B Llama-3.1-405B Llama-3.1-70B Llama-3.1-8B Llama-3-70B Llama-3-8B \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 ~0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 \<0.5 ~1 \<0.5 \<0.5 \<0.5 \<0.5 DeepSeek-R1 DeepSeek-V3 \<0.5 \<0.5 ~1 ~1 ~1.5 ~1.5 find
  68. Kaijie Zhu, Jindong Wang, Qinlin Zhao, Ruochen Xu, and Xing Xie. 2024b. Dyval 2: Dynamic evaluation of large language models by meta probing agents. Preprint, arXiv:2402.14865.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2025-triplefact-defending-data-contamination-evaluation/