Elamur Research⌕
☰
home / papers / 2024 / Language Models in Dialogue: Conversational Maxims for Human-AI Interactions

Language Models in Dialogue: Conversational Maxims for Human-AI Interactions

Paper · 2024 · Erik Miehling, Manish Nagireddy, Prasanna Sattigeri et al.

Summary

The paper asks why modern chat-oriented language models misbehave in conversation: they answer when they should ask, rarely admit ignorance, pad responses, hallucinate, and drift toward sycophancy. The authors argue that most of these failures can be read as breaches of conversational principles. They review work on human dialogue from the social sciences, along with known defects of fine-tuned LLMs. From this they derive a taxonomy of six maxims for human-AI dialogue. Grice's four (quantity, quality, relevance, manner) are rephrased to be more prescriptive. Two new ones are added: benevolence, covering harmful content, and transparency, covering knowledge limits, operational constraints and intents. Each maxim is split into concrete submaxims that apply to both human and AI speakers. They then test whether several language models can correctly judge these submaxims. The models turn out to carry a built-in ranking of principles: concern about harm leaks into unrelated judgments, for example marking an on-topic reply as irrelevant. This matters for using LLMs as judges and for future alignment and annotation schemes.

Key points

  • Six maxims are proposed: quantity, quality, relevance, manner, benevolence, transparency. The last two are new additions beyond Grice (1975).
  • Quality is split into truthfulness and honesty. Truthfulness means agreement with external evidence. Honesty means agreement with what the speaker internally holds, which for an AI means its internal states or activations.
  • Models trained under the Askell et al. (2021) goals of being helpful, honest and harmless become overly moralistic when judging submaxims, conflating benevolence with dimensions like relevance.
  • The work builds on earlier Gricean analyses of conversational agents, e.g., Panfili et al. (2021) on Alexa, who proposed an extra "priority" maxim.
  • The authors note limitations: a Western-centric view of good conversation and strong context dependence. Planned uses are lightweight breakdown detectors, guides for human labeling, and constitutional directives for multi-turn alignment.

Interesting ideas

  • I1. Separating truth (matches the facts) from honesty (matches what the speaker believes) gives two distinct checks instead of one blurry "correctness" score.
  • I2. Quantity is judged against the needs revealed so far in the dialogue, not in isolation, so the right amount of detail depends on what is already shared.
  • I3. An LLM judge's own value ranking can corrupt its labels on unrelated dimensions, so evaluator bias must be measured, not assumed away.
  • I4. Transparency treats knowing one's own limits as part of good conversation, instead of answering at all costs.
  • I5. Reading conversational failures as violations of named maxims turns vague complaints into a diagnosable taxonomy.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): The world of a text could track a speaker's stated beliefs separately from established facts, so answers can tell "false" apart from "the character is lying."
  • I2 (back): Generation from the meaning graph could omit nodes already grounded in the discourse and expand only what the question leaves open.
  • I3 (learn): When using LLM teacher tiers, add negative-control items with sensitive content but harmless target labels, and test whether the teacher's labels on syntax or roles shift.
  • I4 (answer): Readers could explicitly abstain and explain which entity or fact is missing from the world when a question cannot be grounded.
  • I5 (frames): The prag crate could tag which maxim (including benevolence or transparency) an utterance flouts, and use that tag as the explainable trigger for implicature or irony.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

 Fields, topics and concepts

Cite

Cite as Erik Miehling, Manish Nagireddy, Prasanna Sattigeri, Elizabeth M. Daly, David Piorkowski and John T. Richards. 2024. Language Models in Dialogue: Conversational Maxims for Human-AI Interactions. Findings of the Association for Computational Linguistics: EMNLP 2024.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.findings-emnlp.843

@inproceedings{miehling2024language,
  title = {{Language Models in Dialogue: Conversational Maxims for Human-AI Interactions}},
  author = {Erik Miehling, Manish Nagireddy, Prasanna Sattigeri, Elizabeth M. Daly, David Piorkowski and John T. Richards},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.843},
  url = {https://doi.org/10.18653/v1/2024.findings-emnlp.843}
}

References

83 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor, Ioana Baldini, Sara E Berger, Bishwaranjan Bhattacharjee, Djallel Bouneffouf, Subhajit Chaudhury, Pin-Yu Chen, Lamogha Chiazor, et al. 2024. Detectors for safe and reliable llms: Implementations, uses, and limitations. arXiv preprint arXiv:2403.06009. ↗
  2. Blaise Agüera y Arcas. 2022. Do large language models understand us? Daedalus, 151(2):183–197. ↗
  3. Rahul Aralikatte, Shashi Narayan, Joshua Maynez, Sascha Rothe, and Ryan McDonald. 2021. Focus attention: Promoting faithfulness and diversity in summarization. arXiv preprint arXiv:2105.11921. ↗
  4. Arthur Team. 2023. Hedging answers experiment. https://www.arthur.ai/gap-articles/ hedging-answers-experiment. Accessed: 2024-01-18. ↗
  5. Zahra Ashktorab, Mohit Jain, Q Vera Liao, and Justin D Weisz. 2019. Resilient chatbots: Repair strategy preferences for conversational breakdowns. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–12. find
  6. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. 2021. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861. ↗
  7. Amos Azaria and Tom Mitchell. 2023. The internal state of an LLM knows when its lying. arXiv preprint arXiv:2304.13734. ↗
  8. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. ↗
  9. Penelope Brown and Stephen C Levinson. 1987. Politeness: Some Universals in Language Usage, volume 4. Cambridge University Press. find
  10. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827. ↗
  11. Herbert H Clark and Edward F Schaefer. 1989. Contributing to discourse. Cognitive Science, 13(2):259– 294. ↗
  12. Ajeya Cotra. 2021. Why AI alignment could be hard with modern deep learning. Cold Takes. find
  13. Nouha Dziri, Ehsan Kamalloo, Kory W Mathewson, and Osmar Zaiane. 2019. Evaluating coherence in dialogue systems using entailment. arXiv preprint arXiv:1904.03371. ↗
  14. European Union. 2021. Proposal for a regulation laying down harmonised rules on artificial intelligence (artificial intelligence act). https://eur-lex.europa.eu/legal-content/ EN/TXT/?uri=CELEX:52021PC0206. Accessed: 2023-12-06. ↗
  15. Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, and William Saunders. 2021. Truthful AI: Developing and governing AI that does not lie. arXiv preprint arXiv:2110.06674. ↗
  16. Lukas Fluri, Daniel Paleka, and Florian Tramèr. 2023. Evaluating superhuman models with consistency checks. arXiv preprint arXiv:2306.09983. ↗
  17. Robert E. Frederking. 1996. Grice’s maxims: “Do the right thing”. Center for Machine Translation, Carnegie Mellon University. find
  18. Leo Gao, John Schulman, and Jacob Hilton. 2023. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, pages 10835–10866. PMLR. ↗
  19. Google. 2024. Our next-generation model: Gemini 1.5. https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/gemini-15. ↗
  20. Herbert P Grice. 1975. Logic and conversation. In Speech Acts, pages 41–58. Brill. ↗
  21. Sandra Harris. 1995. Pragmatics and power. Journal of Pragmatics, 23(2):117–135. ↗
  22. Austin Z Henley and David Piorkowski. 2024. Supporting annotators with affordances for efficiently labeling conversational data. arXiv preprint arXiv:2403.07762. ↗
  23. Ryuichiro Higashinaka, Masahiro Araki, Hiroshi Tsukahara, and Masahiro Mizukami. 2019. Improving taxonomy of errors in chat-oriented dialogue systems. In 9th International Workshop on Spoken Dialogue System Technology, pages 331–343. Springer. ↗
  24. Ryuichiro Higashinaka, Kotaro Funakoshi, Masahiro Araki, Hiroshi Tsukahara, Yuka Kobayashi, and Masahiro Mizukami. 2015. Towards taxonomy of errors in chat-oriented dialogue systems. In Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue, pages 87–95. find
  25. Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. 2024. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv preprint arXiv:2401.05566. ↗
  26. International Organization for Standardization. 2023. ISO/IEC 42001:2023, Information Technology – Artificial Intelligence – Management Systems. Available from ISO (https://www.iso.org/standard/ 81230.html). ↗
  27. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38. ↗
  28. Alexandra Kallia. 2004. Linguistic Politeness: The Implicature Approach. Walter de Gruyter GmbH & Co. KG, Berlin, Germany. ↗
  29. Greg Kamradt. 2023. Pressure testing GPT-4-128K with long context recall. https://twitter.com/ GregKamradt/status/1722386725635580292. Accessed: 2024-01-22. ↗
  30. Atoosa Kasirzadeh and Iason Gabriel. 2023. In conversation with artificial intelligence: Aligning language models with human values. Philosophy & Technology, 36(2):1–24. find
  31. Elinor Ochs Keenan. 1976. The universality of conversational postulates. Language in Society, 5(1):67–80. ↗
  32. Mark Kingwell. 1993. Is it rational to be polite? The Journal of Philosophy, 90(8):387–404. ↗
  33. Grgur Kovač, Masataka Sawayama, Rémy Portelas, Cédric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. Large language models as superpositions of cultural perspectives. arXiv preprint arXiv:2307.07870. ↗
  34. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. 2022. Language generation models can cause harm: So what can we do about it? An actionable survey. arXiv preprint arXiv:2210.07700. ↗
  35. Jae-gil Lee and Kwan Min Lee. 2022. Polite speech strategies and their impact on drivers’ trust in autonomous vehicles. Computers in Human Behavior, 127:107015. ↗
  36. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499. ↗
  37. BA Levinstein and Daniel A Herrmann. 2023. Still no lie detector for language models: Probing empirical and conceptual roadblocks. arXiv preprint arXiv:2307.00175. ↗
  38. David Lewis. 1979. Scorekeeping in a language game. Journal of Philosophical Logic, 8:339–359. ↗
  39. Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. 2023a. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? arXiv preprint arXiv:2312.03729. ↗
  40. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023b. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172. ↗
  41. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052. ↗
  42. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. 2023. A holistic approach to undesired content detection in the real world. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 15009–15018.
  43. Christopher A Miller. 2005. Trust in adaptive automation: The role of etiquette in tuning trust via analogic and affective methods. In Proceedings of the 1st International Conference on Augmented Cognition, pages 22–27. find
  44. Robert J Moore. 2018. A natural conversation framework for conversational UX design. Studies in Conversational UX Design, pages 181–204. ↗
  45. Robert J Moore, Sungeun An, and Guang-Jie Ren. 2023. The IBM natural conversation framework: a new paradigm for conversational UX design. Human– Computer Interaction, 38(3-4):168–193. ↗
  46. Ted Moskovitz, Aaditya K Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca D Dragan, and Stephen McAleer. 2023. Confronting reward model overoptimization with constrained RLHF. arXiv preprint arXiv:2310.04373. ↗
  47. Manish Nagireddy, Lamogha Chiazor, Moninder Singh, and Ioana Baldini. 2024. SocialStigmaQA: A benchmark to uncover stigma amplification in generative language models. In Proceedings of the 2024 AAAI Conference on Artificial Intelligence. find
  48. Yunju Nam, Hyenyeong Chung, and Upyong Hong. 2023. Language artificial intelligences’ communicative performance quantified through the Gricean conversation theory. Cyberpsychology, Behavior, and Social Networking, 26(12):919–923. ↗
  49. Gabriele Prato, Jerry Huang, Prasanna Parthasarathi, Shagun Sodhani, and Sarath Chandar. 2023. EpiK-Eval: Evaluation for language models as epistemic models. arXiv preprint arXiv:2310.15372. ↗
  50. Matthew Purver, Jonathan Ginzburg, and Patrick Healey. 2003. On the means for clarification in dialogue. Current and New Directions in Discourse and Dialogue, pages 235–255. ↗
  51. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, OpenAI. 2023. New models and developer products an-even when users do not intend to! arXiv preprint nounced at DevDay. https://openai.com/blog/ arXiv:2310.03693. new-models-and-developer-products-announced-at-devday. Accessed: 2024-02-01. ↗
  52. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language under-OpenAI. 2024. Memory and new controls standing by generative pre-training. for chatgpt. https://openai.com/blog/ Ansh Radhakrishnan, Karina Nguyen, Anna Chen, memory-and-new-controls-for-chatgpt. Carol Chen, Carson Denison, Danny Hernandez, Esin Accessed: 2024-02-15. ↗
  53. Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, et al. 2023. Question decomposition im-Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, proves the faithfulness of model-generated reasoning. Carroll Wainwright, Pamela Mishkin, Chong Zhang, arXiv preprint arXiv:2307.11768. ↗
  54. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc-Rachel Reichman. 1978. Conversational coherency. tions with human feedback. Advances in Neural Cognitive Science, 2(4):283–327. Information Processing Systems, 35:27730–27744. find
  55. Laura Panfili, Steve Duman, Andrew Nave, Katherine Phelps Ridgeway, Nathan Eversole, and Ruhi Sarikaya. 2021. Human-AI interactions through a Gricean lens. Proceedings of the Linguistic Society of America, 6(1):288–302. ↗
  56. Raja Parasuraman and Christopher A Miller. 2004. Trust and etiquette in high-criticality automated systems. Communications of the ACM, 47(4):51–55. ↗
  57. Peter S Park, Simon Goldstein, Aidan O’Gara, Michael Chen, and Dan Hendrycks. 2023. AI deception: A survey of examples, risks, and potential solutions. arXiv preprint arXiv:2308.14752. ↗
  58. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. 2022. Discovering language model behaviors with model-written evaluations. arXiv preprint arXiv:2212.09251. ↗
  59. Saffronand Song Francisand Cai Trevorand Ring Romanand Aslanides Johnand Glaese Ameli-aand McAleese Natand Irving Geoffrey Perez, Ethanand Huang. 2022. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. ↗
  60. Jonas Pfister. 2010. Is there a need for a maxim of politeness? Journal of Pragmatics, 42(5):1266–1282. ↗
  61. Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki, John Richards, and David Piorkowski. 2017. Detecting egregious conversations between customers and virtual agents. arXiv preprint arXiv:1711.05780. ↗
  62. Vidya Setlur and Melanie Tory. 2022. How do you converse with an analytical chatbot? Revisiting Gricean maxims for designing analytical conversational behavior. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1– 17. find
  63. Omar Shaikh, Kristina Gligorić, Ashna Khetan, Matthias Gerstgrasser, Diyi Yang, and Dan Jurafsky. 2023. Grounding or guesswork? Large language models are presumptive grounders. arXiv preprint arXiv:2311.09144. ↗
  64. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548. ↗
  65. Randall D Spain and Poornima Madhavan. 2009. The role of automation etiquette and pedigree in trust and dependence. In Proceedings of the Human Factors and Ergonomics Society annual meeting, volume 53, pages 339–343. SAGE Publications Sage CA: Los Angeles, CA. ↗
  66. Dan Sperber and Deirdre Wilson. 1986. Relevance: Communication and Cognition, volume 142. Citeseer. find
  67. Robert C Stalnaker. 1978. Assertion. In Pragmatics, pages 315–332. Brill. find
  68. Chloe Rose Stuart-Ulin. 2018. Microsoft’s politically correct chatbot is even worse than its racist one. Retrieved August, 30:2021. find
  69. Djoko Susanto. 2018. Basuki tjahaya purnama’s conversational maxims: A political discourse analysis. In Proceedings of the Language in the Online & Offline World 6: The Fortitude. find
  70. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models. arXiv preprint arXiv:2401.11817. ↗
  71. Yuanzhao Zhai, Han Zhang, Yu Lei, Yue Yu, Kele Xu, Dawei Feng, Bo Ding, and Huaimin Wang. 2023. Uncertainty-penalized reinforcement learning from human feedback with diverse reward LoRA ensembles. arXiv preprint arXiv:2401.00243. ↗
  72. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, The Information. 2024. OpenAI develops Yulong Chen, et al. 2023. Siren’s song in the AI web search product in challenge to Google. ocean: A survey on hallucination in large language https://www.theinformation.com/articles/ models. arXiv preprint arXiv:2309.01219. openai-develops-web-search-product-in-challenge-to-google. Accessed: 2024-02-15. ↗
  73. Qingxiao Zheng, Yiliu Tang, Yiren Liu, Weizi Liu, and Yun Huang. 2022. UX research on conversational Joanna Thornborrow. 2014. Power talk: Language and human-AI interaction: A literature review of the interaction in institutional discourse. Routledge. ACM digital library. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Ran Tian, Shashi Narayan, Thibault Sellam, and Systems, pages 1–24. find
  74. Ankur P Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. Andy Zou, Long Phan, Sarah Chen, James Campbell, arXiv preprint arXiv:1910.08684. ↗
  75. Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, David R Traum and Elizabeth A Hinkelman. 1992. find
  76. Conet al. 2023a. Representation engineering: A top-versation acts in task-oriented spoken dialogue. Com-down approach to AI transparency. arXiv preprint putational Intelligence, 8(3):575–599. arXiv:2310.01405. ↗
  77. Chaojun Wang and Rico Sennrich. 2020. On exposure Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrik-bias, hallucination and domain shift in neural mason. 2023b. Universal and transferable adversarial chine translation. arXiv preprint arXiv:2005.03642. attacks on aligned language models. arXiv preprint arXiv:2307.15043. ↗
  78. Hongmin Wang. 2020. Revisiting challenges in data-to-text generation with fact grounding. arXiv preprint arXiv:2001.03830. ↗
  79. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958. ↗
  80. Justin D Weisz, Mohit Jain, Narendra Nath Joshi, James Johnson, and Ingrid Lange. 2019. BigBlueBot: Teaching strategies for successful human-agent interactions. In Proceedings of the 24th International Conference on Intelligent User Interfaces, pages 448– 459. find
  81. Anna Wierzbicka. 2003. Cross-Cultural Pragmatics: The Semantics of Human Interaction. Mouton de Gruyter. find
  82. Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. 2021. Detoxifying language models risks marginalizing minority voices. arXiv preprint arXiv:2104.06390. ↗
  83. Rongwu Xu, Brian S Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. 2023. The Earth is flat because...: Investigating LLMs’ belief towards misinformation via persuasive conversation. arXiv preprint arXiv:2312.09085. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/miehling-2024-language-models-dialogue-conversational-maxims/