Elamur Research⌕
☰
home / papers / 2021 / On Hallucination and Predictive Uncertainty in Conditional Language Generation

On Hallucination and Predictive Uncertainty in Conditional Language Generation

Summary

Neural text generators often produce content that their input does not support, a problem known as hallucination. Earlier explanations, such as language priors, dataset misalignment and domain shift, were tied to individual tasks. Xiao and Wang propose a shared cause: predictive uncertainty, measured as the entropy of the next-token distribution. Using deep ensembles, they split this uncertainty into an aleatoric part (from the data) and an epistemic part (from the model).

They tested the idea on two tasks:

  • Image captioning: MSCOCO, with objects and verbs checked for hallucination.
  • Data-to-text generation: ToTTo.

Hallucination rates rose consistently with uncertainty, and epistemic uncertainty predicted hallucination better than aleatoric uncertainty did. Building on this, the authors introduce uncertainty-aware beam search (UABS), which adds an uncertainty penalty to the decoding score. Penalizing the epistemic component gives the best balance between fewer hallucinations and standard metric quality.

The work offers a task-independent explanation of hallucination and a simple decoding-time fix that requires no retraining. The trade-off is shorter, more cautious outputs.

Key points

  • Captioning setup: Four MSCOCO models (FC, Att2In, BUTD, Transformer) were trained on the Karpathy split with the self-critical.pytorch framework by Luo et al. Training used batch size 50, 30 epochs and the Adam optimizer. Object hallucination was detected with synonym lists, following the approach behind the CHAIR metric.
  • Correlations: Pearson correlation with hallucination was about 0.27–0.36 for epistemic uncertainty, compared with roughly 0.13–0.30 for aleatoric uncertainty, across all four models.
  • Action verbs: The pattern also holds for verbs, which were extracted with the spaCy POS tagger. Hallucination rates climb from near zero in the lowest-uncertainty bins to 22–44% in the highest bin.
  • Generic captions: In the captioning comparison, one alternative penalty raised the share of generic captions from 1.0% to 28.4%. Epistemic UABS kept generic outputs rare while cutting hallucination.
  • ToTTo results: UABS used a five-model ensemble with 100 human-rated outputs per penalty weight. Small penalties improved faithfulness at some cost to BLEU and fluency. Larger penalties dropped low-confidence statements, producing shorter texts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 73 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.eacl-main.236 · arXiv 2103.15025

@inproceedings{xiao2021hallucination,
  title = {{On Hallucination and Predictive Uncertainty in Conditional Language Generation}},
  author = {Xiao, Yijun and Wang, William Yang},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-main.236},
  eprint = {2103.15025},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.eacl-main.236}
}

References

34 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077–6086. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find our card
  3. Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424. — [arXiv:1505.05424](https://arxiv.org/abs/1505.05424) find
  4. Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. find
  5. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325. find our card
  6. Stefan Depeweg, Jose-Miguel Hernandez-Lobato, Finale Doshi-Velez, and Steffen Udluft. 2018. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International Conference on Machine Learning, pages 1184–1193. PMLR. find
  7. Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128–3137. find our card
  8. Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in neural information processing systems, pages 5574–5584. find our card
  9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  10. Karen Kukich. 1983. Design of a knowledge-based report generator. In Proceedings of the 21st annual meeting on Association for Computational Linguistics, pages 145–150. Association for Computational Linguistics. find
  11. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pages 6402–6413. find
  12. Ruotian Luo, Brian Price, Scott Cohen, and Gregory Shakhnarovich. 2018. Discriminability objective for training descriptive captions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6964–6974. find
  13. Kathleen McKeown. 1992. bridge University Press. find
  14. Text generation. Cam-Mathias Müller, Annette Rios, and Rico Sennrich. 2019. Domain robustness in neural machine translation. arXiv preprint arXiv:1911.03109. find our card
  15. Armen Der Kiureghian and Ove Ditlevsen. 2009. Aleatory or epistemic? does it matter? Structural safety, 31(2):105–112. find
  16. Radford M Neal. 1995. BAYESIAN LEARNING FOR NEURAL NETWORKS. Ph.D. thesis, University of Toronto. find
  17. Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055– 5070, Online. Association for Computational Lingu… find our card
  18. Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong Pan, and Chin-Yew Lin. 2019. A simple recipe towards reducing hallucination in neural surface realisation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2673– 2679. find
  19. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. find
  20. Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In International Conference on Machine Learning, pages 3956–3965. find our card
  21. Geoffrey E Hinton and Drew Van Camp. 1993. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory, pages 5–13. find
  22. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. find our card
  23. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. find our card
  25. Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. Totto: A controlled table-to-text generation dataset. arXiv preprint arXiv:2004.14373. find our card
  26. Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7008–7024. find our card
  27. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045. find our card
  28. Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. arXiv preprint arXiv:1910.08684. — [arXiv:1910.08684](https://arxiv.org/abs/1910.08684) find
  29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  30. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575. find our card
  31. Sam Wiseman, Stuart M Shieber, and Alexander M Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263. find our card
  32. Yijun Xiao and William Yang Wang. 2019. Quantifying uncertainties in natural language processing tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7322–7329. find
  33. Jiacheng Xu, Shrey Desai, and Greg Durrett. 2020. Understanding neural abstractive summarization models via uncertainty. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6275–6281. find
  34. Lingxue Zhu and Nikolay Laptev. 2017. Deep and confident prediction for time series at uber. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 103–110. IEEE. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xiao-2021-hallucination-predictive-uncertainty-conditional-language/