Elamur Research⌕
☰
home / papers / 2023 / AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation

AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation

Paper · 2023 · Junyang Wang, Yuhang Wang, Guohai Xu et al.

Summary

Multi-modal Large Language Models (MLLMs) often produce answers that sound plausible but do not match the image. Existing ways to measure this "hallucination" have two weaknesses. Some are expensive because they depend on human judges, GPT-4, or trained detector models. Others are narrow: POPE, for example, covers only object existence in yes/no questions. AMBER is a benchmark that needs no LLM and checks both open-ended image description (generative) and yes/no questioning (discriminative). It covers three kinds of hallucination: existence, attribute and relation. The authors selected 1,004 clean, unambiguous images that MLLMs are unlikely to have seen in training. They annotated each image with present objects, attributes, contact relations, and plausible but absent objects. Scoring is fully automatic: nouns are extracted with toolkits such as nltk and compared against the annotations. Nine models were tested, including GPT-4V, and all of them still hallucinated. The authors also analyse contributing factors and suggest mitigations. AMBER matters because it makes hallucination checks cheap and repeatable.

Key points

  • Dataset construction
    • Images were drawn from test splits of open datasets and from copyright-free sources, giving 1,004 images after filtering.
    • Attribute annotations cover state, number and action.
    • Relation annotations mark whether two objects are in direct contact.
    • Counterfactual "hallucinatory target objects" supply false-premise questions.
  • Evaluation setup
    • The generative task uses the single prompt "Describe this image." and is scored with metrics including CHAIR and Cover.
    • The discriminative task uses templated yes/no questions and is scored with Acc and F1.
    • The two tasks are combined into an AMBER Score.
  • Models tested: LLaVA, MiniGPT-4, mPLUG-Owl, InstructBLIP, LLaVA-1.5, Qwen-VL, CogVLM, mPLUG-Owl2 and GPT-4V.
  • Factor analysis
    • Higher image resolution lowers hallucination for most models.
    • GPT-4V barely changes with resolution, with AMBER Score staying around 91.2–91.4.
    • A larger LLM does not consistently help. MiniGPT-4 13B hallucinates more in generation than the 7B version (CHAIR 14.3 vs 13.6), while LLaVA-1.5 13B hallucinates less (7.0 vs 7.8).
    • For MiniGPT-v2, adding instruction data in stage-3 reduces generative hallucination but weakens discriminative performance.
  • Limitations noted by the authors
    • Attribute and relation checks apply only to the discriminative task.
    • Noun extraction makes errors, such as treating "orange" as an object or splitting compound nouns like "water bottle".
    • The list of absent target objects is limited.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEvaluation & benchmarks · 1,341 works

Cite

Cite as Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang and Jitao Sang. 2023. AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation. arXiv:2311.07397.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2311.07397 · arXiv 2311.07397

@misc{wang2023amber,
  title = {{AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation}},
  author = {Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang and Jitao Sang},
  year = {2023},
  doi = {10.48550/arxiv.2311.07397},
  eprint = {2311.07397},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.07397}
}

References

28 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966.
  2. Jun Chen, Deyao Zhu Xiaoqian Shen Xiang Li, Zechun Liu Pengchuan Zhang, Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: Large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478.
  3. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna.lmsys. org (accessed 14 April 2023). ↗
  4. Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. 2023. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287. — [arXiv:2311.03287](https://arxiv.org/abs/2311.03287) find
  5. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. Instructblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500.
  6. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In Conference on Computer Vision and Pattern Recognition (CVPR).
  7. Anisha Gunjal, Jihan Yin, and Erhan Bas. 2023. Detecting and preventing hallucinations in large vision language models. arXiv preprint arXiv:2308.06394.
  8. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355.
  9. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer.
  10. Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023a. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), llava-1.5, and other multi-modality models. arXiv preprint…
  11. Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023b. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565.
  12. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023c. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744.
  13. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023d. Visual instruction tuning. arXiv preprint arXiv:2304.08485.
  14. OpenAI. 2023. Gpt-4 technical report. abs/2303.08774.
  15. ArXiv, Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. arXiv preprint arXiv:1809.02156.
  16. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  17. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  18. Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. 2023a. Evaluation and analysis of hallucination in large vision-language models. arXiv preprint arXiv:2308.15126. — [arXiv:2308.15126](https://arxiv.org/abs/2308.15126) find
  19. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. 2023b. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079.
  20. Yang Wu, Shilong Wang, Hao Yang, Tian Zheng, Hongbo Zhang, Yanyan Zhao, and Bing Qin. 2023. An early evaluation of gpt-4v (ision). arXiv preprint arXiv:2310.16534. — [arXiv:2310.16534](https://arxiv.org/abs/2310.16534) find
  21. Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2023a. Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794. — [arXiv:2309.06794](https://arxiv.org/abs/2309.06794) find
  22. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023b. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178.
  23. Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023c. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. arXiv preprint arXiv:2311.04257.
  24. Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045.
  25. Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, and Xiangjun Fan. 2023. Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. arXiv preprint arXiv:2310.01779. — [arXiv:2310.01779](https://arxiv.org/abs/2310.01779) find
  26. Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. 2023a. Don’t make your llm an evaluation benchmark cheater. arXiv preprint arXiv:2311.01964.
  27. Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023b. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754.
  28. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2023-amber-llm-free-multi-dimensional/