Elamur Research
☰
home / papers / 2023 / A Survey of Hallucination in Large Foundation Models

A Survey of Hallucination in Large Foundation Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2309.05922

Summary

Large foundation models across text, image, video and audio often produce output that sounds plausible but is false or invented. Earlier surveys covered only natural language generation or LLMs. This paper claims to be the first review of hallucination across all four modalities. It is purely a literature survey with no new experiments. The authors group recent work, mostly from March–September 2023, into a taxonomy: text (general, multilingual and domain-specific LLMs), image, video and audio. A summary table records, for each cited paper, whether it handles detection or mitigation, plus its task, datasets and evaluation metrics. Text-side approaches include zero-resource self-consistency checks, retrieval and knowledge-base grounding, automated feedback and iterative prompting with formal methods. The authors also argue that hallucination can be useful in creative settings. Finally, they outline future work: automated hallucination metrics, human-AI evaluation, adversarial testing and grounding in curated knowledge such as knowledge graphs. For practitioners, the survey works as a map of benchmarks and tools across modalities.

Key points

  • The taxonomy covers four modalities: Text, Image, Video and Audio. Text is split into LLMs, Multilingual LLMs and Domain-specific LLMs (medicine: Med-HALT; law).
  • The text-side tools and benchmarks covered are:
    • SelfCheckGPT for black-box detection;
    • PURR for editing out hallucinations;
    • HaluEval as a benchmark;
    • LLM-AUGMENTER and Chain of Knowledge for knowledge grounding;
    • HALOCHECK, tested on BLOOM 7B.
  • In multilingual MT, Pfeiffer et al. compare the M2M models with ChatGPT across more than 100 translation directions.
  • Non-text resources and metrics include:
    • image: CHAIR on MSCOCO, and M-HalDetect for VQA;
    • video: FactVC on ActivityNet Captions and YouCook2;
    • audio: LP-MusicCaps, scored with BLEU, METEOR and ROUGE-L.
  • The authors keep an updated reading list at github.com/vr25/hallucination-foundation-model-survey.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

121 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 117 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.05922 · arXiv 2309.05922

@misc{rawte2023survey,
  title = {{A Survey of Hallucination in Large Foundation Models}},
  author = {Rawte, Vipula and Sheth, Amit and Das, Amitava},
  year = {2023},
  doi = {10.48550/arxiv.2309.05922},
  eprint = {2309.05922},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.05922}
}

References

40 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when its lying. arXiv preprint arXiv:2304.13734. find our card
  2. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. find our card
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. find
  4. Eric T Castaldo and Edmund Y Yang. 2007. Severe sepsis attributable to community-associated methicillin-resistant staphylococcus aureus: an emerging fatal problem. The American Surgeon, 73(7):684–687. find
  5. Anthony Chen, Panupong Pasupat, Sameer Singh, Hongrae Lee, and Kelvin Guu. 2023. Purr: Efficiently editing language model hallucinations by denoising language model corruptions. find
  6. Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. 2023. Chatlaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv:2306.16092. find our card
  7. Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. 2022. Plausible may not be faithful: Probing object hallucination in vision-language pre-training. arXiv preprint arXiv:2210.07688. find
  8. SeungHeon Doh, Keunwoo Choi, Jongpil Lee, and Juhan Nam. 2023. Lp-musiccaps: Llm-based pseudo music captioning. arXiv preprint arXiv:2307.16372. — [arXiv:2307.16372](https://arxiv.org/abs/2307.16372) find
  9. Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, and Shizhu Liu. 2023. Halo: Estimation and reduction of hallucinations in open-source weak large language models. arXiv preprint arXiv:2308.11764. find
  10. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2023. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Co… find our card
  11. Jie Huang and Kevin Chen-Chuan Chang. 2023. Citation: A key to building responsible and accountable large language models. arXiv preprint arXiv:2307.02185. find
  12. Susmit Jha, Sumit Kumar Jha, Patrick Lincoln, Nathaniel D Bastian, Alvaro Velasquez, and Sandeep Neema. 2023. Dehallucinating large language models using formal methods guided iterative prompting. In 2023 IEEE International Conference on Assured Autonomy (ICAA), pages 149–152. IEEE. find
  13. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38. find our card
  14. Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, pages 706–715. find
  15. Sumith Kulal, Tim Brooks, Alex Aiken, Jiajun Wu, Jimei Yang, Jingwan Lu, Alexei A Efros, and Krishna Kumar Singh. 2023. Putting people in their place: Affordance-aware human insertion into scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17089– 1709… find
  16. Juncheng B Li, Jackson Sam Michaels, Laura Yao, Lijun Yu, Zach Wood-Doughty, and Florian Metze. 2023a. Audio-journey: Efficient visual+ llm-aided audio encodec diffusion. In Workshop on Efficient Systems for Foundation Models@ ICML2023. find
  17. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023b. Helma: A large-scale hallucination evaluation benchmark for large language models. arXiv preprint arXiv:2305.11747. find our card
  18. KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023c. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355. find our card
  19. Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Lidong Bing, Shafiq Joty, and Soujanya Poria. 2023d. Chain of knowledge: A framework for grounding large language models with structured knowledge bases. arXiv preprint arXiv:2305.13269. find
  20. Anisha Gunjal, Jihan Yin, and Erhan Bas. 2023. Detecting and preventing hallucinations in large vision language models. arXiv preprint arXiv:2308.06394. find
  21. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023e. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355. find our card
  22. Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon, and William Yang Wang. 2023. find
  23. Let’s Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1… find our card
  24. Jing Wu, Jennifer Hobbs, and Naira Hovakimyan. 2023. Hallucination improves the performance of unsupervised visual representation learning. arXiv preprint arXiv:2307.12168. — [arXiv:2307.12168](https://arxiv.org/abs/2307.12168) find
  25. Hui Liu and Xiaojun Wan. 2023. Models see hallucinations: Evaluating the factuality in video captioning. arXiv preprint arXiv:2303.02961. find
  26. Yongsheng Yu, Heng Fan, and Libo Zhang. 2023. Deficiency-aware masked transformer for video inpainting. arXiv preprint arXiv:2307.08629. — [arXiv:2307.08629](https://arxiv.org/abs/2307.08629) find
  27. Junyu Luo, Cao Xiao, and Fenglong Ma. 2023. Zero-resource hallucination prevention for large language models. arXiv preprint arXiv:2309.02654. find
  28. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023a. How language model hallucinations can snowball. find our card
  29. Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. find
  30. Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. 2023b. Mitigating language model hallucination with interactive question-knowledge alignment. find
  31. Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, and Mark Steedman. 2023. Sources of hallucination by large language models on inference tasks. arXiv preprint arXiv:2305.14552. find
  32. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023c. Siren’s song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. find our card
  33. Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2023. Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation. find
  34. Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813. find our card
  35. Jonas Pfeiffer, Francesco Piccinno, Massimo Nicosia, Xinyi Wang, Machel Reid, and Sebastian Ruder. 2023. mmt5: Modular multilingual pre-training solves source language hallucinations. find
  36. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. arXiv preprint arXiv:1809.02156. find our card
  37. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695. find
  38. Logesh Kumar Umapathi, Ankit Pal, and Malaikannan Sankarasubbu. 2023. Med-halt: Medical domain hallucination test for large language models. arXiv preprint arXiv:2307.15343. find
  39. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation. arXiv preprint arXiv:2307.03987. find our card
  40. Kyle Wiggers. 2023. Are ai models doomed to always hallucinate? find