Elamur Research⌕
☰
home / papers / 2026 / Assessing Web Search Credibility and Response Groundedness in Chat Assistants

Assessing Web Search Credibility and Response Groundedness in Chat Assistants

Paper · 2026 · Ivan Vykopal, Matúš Pikuliak, Simon Ostermann et al.

Summary

Chat assistants with built-in web search can cite external pages, but those pages may come from unreliable or propaganda outlets. That creates a risk that the assistants spread misinformation instead of correcting it. The authors propose an evaluation methodology that measures two things together: how credible the cited sources are, and whether the assistant's statements are actually backed by those citations. They selected 100 debunked claims, 20 from each of five misinformation-prone topics. Each claim was posed to GPT-4o, GPT-5, Perplexity and Qwen Chat from two user perspectives: a Fact-Checker trying to verify it and a Claim Believer looking for confirmation. Cited domains were labelled using fact-checking organisation lists and Media Bias/Fact Check. Responses were then split into atomic units and checked against the cited evidence with an LLM judge. Perplexity relied on the most credible sources, while GPT-4o cited non-credible sites more often on sensitive topics. Because groundedness is tied to source quality, the method can catch answers that faithfully follow their citations even when those citations are disinformation. Factuality-only checks miss this failure mode.

Key points

  • Dataset: 100 claims drawn from fact-checkers listed in the Duke Reporters' Lab, 20 per topic. The five topics cover health, the war between Russia and Ukraine, local matters, climate change, and politics in the U.S. The war claims are tied to Russian disinformation campaigns.
  • Claim preparation: Claims were reworded with Llama3.3 70B and then manually checked, so that assistants would not simply retrieve the original debunking pages.
  • Prompting setup: Three Fact-Checker templates and two Claim Believer templates were adapted from Kaur et al. (2024). The Claim Believer setup also included a question-form variant to test false presuppositions.
  • Metrics: The paper reports a Credibility Rate and a Non-Credibility Rate for cited sources. Groundedness follows a VERIFY-style pipeline with Llama 3.3 70B as judge, covering unit extraction, decontextualisation, evidence retrieval and a final judgment.
  • Further findings and resources: When GPT-5 automatically switched on its thinking mode, it cited more credible sources and less disinformation. All experiments were in English only. Code and data are released at github.com/kinit-sk/web-search-analysis.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬German Research Centre for Artificial IntelligenceKaiserslautern · Germany77 works · 29,479 citations🎓Brno University of TechnologyBrno · Czechia11 works · 32,326 citations🤝Kempelen Institute of Intelligent TechnologiesBratislava · Slovakia11 works · 82 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Ivan Vykopal, Matúš Pikuliak, Simon Ostermann and Marian Simko. 2026. Assessing Web Search Credibility and Response Groundedness in Chat Assistants. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.115

@inproceedings{vykopal2026assessing,
  title = {{Assessing Web Search Credibility and Response Groundedness in Chat Assistants}},
  author = {Ivan Vykopal, Matúš Pikuliak, Simon Ostermann and Marian Simko},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.115},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.115}
}

References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alan Agresti and Brent A. Coull. 1998. Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician, 52(2):119– 126. find
  2. American Sunlight Project. 2025. A pro-russia content network foreshadows the automated future of info ops. find
  3. Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James Glass, and Preslav Nakov. 2018. Predicting factuality of reporting and bias of news media sources. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’18, Brussels, Belgium.
  4. Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, and Preslav Nakov. 2020. What was written vs. who read it: News media profiling using text analysis and social media context. In Proceedings of the 58th Annual Meeting of the Association for Computational Lingu… find
  5. Aaron Chatterji, Thomas Cunningham, David J Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. 2025. How people use chatgpt. Working Paper 34255, National Bureau of Economic Research. find
  6. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. Felm: Benchmarking factuality evaluation of large language models. In Advances in Neural Information Processing Systems, volume 36, pages 44502–44523. Curran Associates, Inc. find
  7. Farima Fatahi Bayat, Lechen Zhang, Sheza Munir, and Lu Wang. 2025. FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 33090–33110, Vienna, Austr…
  8. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  9. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst., 43(2).
  10. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547. the Association for Computational Linguistics: ACL 2024, pages 14308–14331, Bangkok, Thailand. Association for Computational Linguistics.
  11. Lucas Fonseca Lage and Simon Ostermann. 2025. Openfactscore: Open-source atomic evaluation of factuality in text generation. Preprint, arXiv:2507.05965. — [arXiv:2507.05965](https://arxiv.org/abs/2507.05965) find
  12. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
  13. Zain Muhammad Mujahid, Dilshod Azizov, Maha Tufail Agro, and Preslav Nakov. 2025. Profiling news media for factuality and bias using LLMs and the fact-checking methodology of human experts. In Findings of the Association for Computational Linguistics: ACL 2025, pages 798–819, Vienna, Austria. Associ… find
  14. McKenzie Sadeghi and Isis Blachez. 2025. A well-funded moscow-based global ‘news’ network has infected western artificial intelligence tools worldwide with russian propaganda. find
  15. Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A comprehensive survey of hallucination in large language, image, video and audio foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11709–11724, Miami, Flo… find
  16. Judith Sieker, Clara Lachenmaier, and Sina Zarrieß. 2025. LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High. Preprint, arXiv:2505.22354.
  17. Ilan Strauss, Jangho Yang, Tim O’Reilly, Sruly Rosenblat, and Isobel Moure. 2025. The Attribution Crisis in LLM Search Results: Estimating Ecosystem Exploitation. find
  18. Sander van der Linden, Anthony Leiserowitz, Seth Rosenthal, and Edward Maibach. 2017. Inoculating the public against misinformation about climate change. Global Challenges, 1(2):1600008. find
  19. Daniel Kansaon, Philipe de Freitas Melo, Savvas Zannettou, and Fabricio Benevenuto. 2025. From fake news to real protests: Whatsapp’s role in brazilian political coordination. Proceedings of the International AAAI Conference on Web and Social Media, 19(1):1007–1020. find
  20. Ivan Vykopal, Matúš Pikuliak, Ivan Srba, Robert Moro, Dominik Macko, and Maria Bielikova. 2024. Disinformation capabilities of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14830–14847, Bangkok, Thaila… find
  21. Navreet Kaur, Monojit Choudhury, and Danish Pruthi. 2024. Evaluating Large Language Models for Health-related Queries with Presuppositions. In Findings of Yuxia Wang, Revanth Gangi Reddy, Zain Muhammad Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Na…
  22. Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, and Preslav Nakov. 2024b. Factuality of large language models: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19519–19529, Miami, Florida…
  23. Przemyslaw M. Waszak, Wioleta Kasprzycka-Waszak, and Alicja Kubanek. 2018. The spread of medical fake news in social media – the pilot quantitative study. Health Policy and Technology, 7(2):115–118. find
  24. Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024. Long-form factuality in large language models. In Advances in Neural Information Processing Systems, volume 37, pages 80756–80827. Curran Associates,…
  25. Kai-Cheng Yang. 2025. News source citing patterns in ai search systems. Preprint, arXiv:2507.05301. — [arXiv:2507.05301](https://arxiv.org/abs/2507.05301) find
  26. Xinyan Yu, Sewon Min, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. CREPE: Open-domain question answering with false presuppositions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10457–10480, Toronto, Canada. Association…
  27. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Curran Associates Inc., Red Hook, NY, USA.
  28. Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. mGTE: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vykopal-2026-assessing-web-search-credibility-response/