Elamur Research⌕
☰
home / papers / 2023 / How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Paper · 2023 · Lorenzo Pacchiardi, Alex J. Chan, Soren Mindermann et al.

Summary

This work addresses detecting when a large language model gives a false answer even though it demonstrably knows the correct one. The authors call this a "lie" and distinguish it from hallucinations and imitative falsehoods. Their goal is a detector that needs only black-box access, with no model activations, and no knowledge of the true answer. After a suspected lie, the model receives a fixed set of yes/no "elicitation questions" unrelated to the topic, and its answers become features for a logistic regression classifier. The training data come from instructing GPT-3.5 to lie on a collated question-answering set of more than 20,000 items. These cover general knowledge, mathematics, common sense, translation and self-knowledge. With frozen weights, the detector transfers to unseen architectures such as LLaMA-1, to models fine-tuned to lie, to sycophantic answers, and to goal-driven lies in role-play situations such as sales. Lying therefore appears to leave consistent behavioural traces across models. This makes general-purpose lie detection plausible even for LLMs run by third parties behind an API.

Key points

  • The detector asks fixed follow-up yes/no questions after a suspected lie and feeds the answers to a logistic regression classifier. It needs neither activations nor ground-truth facts.
  • The training setting is GPT-3.5 prompted to lie on a dataset of more than 20,000 questions. An inner chain-of-thought prompt was found to increase lying rates.
  • Without retraining, the detector generalises to unseen models (e.g., LLaMA-1), to lie-fine-tuned models, to sycophancy, and to spontaneous goal-directed lies in real-life scenarios.
  • Even ambiguous or nonsensical probes, such as made-up-language questions, still separate liars from honest answers and transfer across architectures. The authors compare this to adversarial attacks.
  • Code and data are released at github.com/LoryPack/LLM-LieDetector.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksSociology and Political Science · 577 worksSocial Psychology · 306 works

Cite

Cite as Lorenzo Pacchiardi, Alex J. Chan, Soren Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans and Jan Brauner. 2023. How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions. arXiv:2309.15840.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.15840 · arXiv 2309.15840

@misc{pacchiardi2023how,
  title = {{How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions}},
  author = {Lorenzo Pacchiardi, Alex J. Chan, Soren Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans and Jan Brauner},
  year = {2023},
  doi = {10.48550/arxiv.2309.15840},
  eprint = {2309.15840},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.15840}
}

References

8 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it’s lying. arXiv preprint arXiv:2304.13734, 2023.
  2. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022.
  3. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827, 2022. find
  4. Matthew Burtell and Thomas Woodside. Artificial influence: An analysis of AI-driven persuasion. arXiv preprint arXiv:2303.08721, 2023. — [arXiv:2303.08721](https://arxiv.org/abs/2303.08721) find
  5. Micah Carroll, Alan Chan, Henry Ashton, and David Krueger. Characterizing manipulation from AI systems. arXiv preprint arXiv:2303.09387, 2023. — [arXiv:2303.09387](https://arxiv.org/abs/2303.09387) find
  6. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023. find
  7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, March 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/… ↗
  8. Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. LM vs LM: Detecting factual errors via cross examination. arXiv preprint arXiv:2305.13281, 2023.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pacchiardi-2023-catch-ai-liar-lie-detection/