Elamur Research⌕
☰
home / papers / 2023 / The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation

The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation

Paper · 2023 · Rongwu Xu, Brian S. Lin, Shujian Yang et al.

Summary

LLMs store a great deal of factual knowledge, but earlier work on how easily they accept false information mostly tested single-turn prompts. This paper asks whether a model that already answers a simple factual question correctly can be argued out of that answer during a multi-turn persuasive dialogue. The authors built Farm (Fact to Misinform). It contains easy questions from BoolQ, Natural Questions and TruthfulQA, converted into shuffled multiple-choice items with an added "don't know" option. Each question is paired with a GPT-4-generated false claim and persuasive texts that use logical, credibility and emotional appeals. The test has three stages. It first confirms the model's correct initial answer. It then presents misinformation turn by turn, with implicit belief checks along the way. Finally, it asks the original question again. Across five mainstream LLMs, correct beliefs proved fragile. ChatGPT changed its answer in 50.1% of cases and GPT-4 in 20.7%. Repetition and rhetorical appeals were especially effective. The study reveals a safety-relevant robustness gap and releases a public benchmark for mitigation research.

Key points

  • Farm data: 1,500 GPT-4-answerable questions (500 each from BoolQ, NQ, TruthfulQA). NQ misinformation comes in two variants: NQ1 uses "Not A" targets, and NQ2 uses the most plausible wrong option. After human validation removed 48 failed generations, the dataset has 1,952 entries.
  • Persuasion design: Four strategies are tested: Repetition plus Logical (LO), Credibility (CR) and Emotional (EM) appeals. Each appeal type has three distinct messages per false claim, which simulates several persuasion turns.
  • Quality of generated appeals: GPT-4-based checks found 97.3–99.5% NLI entailment with the false claim and about 99.8–99.9% alignment with the intended strategy. The authors note that this ease of generation is itself a misuse risk.
  • Per-model evaluation: Each model is tested only on the questions it initially answers correctly. Even the weakest model tested, Vicuna-v1.5-7B, answers 51% of the questions correctly.
  • Main findings: Belief change rates were 50.1% for ChatGPT and 20.7% for GPT-4. The authors also observe that sycophantic replies do not necessarily mean the model's belief has changed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States567 works · 335,189 citations🎓Tsinghua UniversityBeijing · China439 works · 94,783 citations🎓Shanghai Jiao Tong UniversityShanghai · China232 works · 20,305 citations🎓Nanyang Technological UniversitySingapore · Singapore193 works · 27,739 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksSocial Sciences · 2,009 worksSociology and Political Science · 630 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Rongwu Xu, Brian S. Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu and Han Qiu. 2023. The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation. arXiv:2312.09085.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2312.09085 · arXiv 2312.09085

@misc{xu2023earth,
  title = {{The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation}},
  author = {Rongwu Xu, Brian S. Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu and Han Qiu},
  year = {2023},
  doi = {10.48550/arxiv.2312.09085},
  eprint = {2312.09085},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2312.09085}
}

References

98 references, 51 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alaa Abd-Alrazaq, Rawan AlSaad, Dari Alhuwail, Arfan Ahmed, Padraig Mark Healy, Syed Latifi, Sarah Aziz, Rafat Damseh, Sadam Alabed Alrazak, Javaid Sheikh, et al. 2023. Large language models in medical education: Opportunities, challenges, and future directions. JMIR Medical Education, 9(1):e48291. find
  2. Simran Arora, Avanika Narayan, Mayee F Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, and Christopher Re. 2022. Ask me anything: A simple strategy for prompting language models. In The Eleventh International Conference on Learning Representations.
  3. Danny Axsom, Suzanne Yates, and Shelly Chaiken. 1987. Audience response as a heuristic cue in persuasion. Journal of personality and social psychology, 53(1):30. find
  4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  5. John T Cacioppo and Richard E Petty. 1979. Effects of message repetition and position on cognitive response, recall, and persuasion. Journal of personality and Social Psychology, 37(1):97. find
  6. Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, and Ludwig Schmidt. 2023. Are aligned neural networks adversarially aligned?
  7. Kushal Chawla, Weiyan Shi, Jingwen Zhang, Gale Lucas, Zhou Yu, and Jonathan Gratch. 2023. Social influence dialogue systems: A survey of datasets and models for social influence tasks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pag… find
  8. Canyu Chen and Kai Shu. 2023a. Can llm-generated misinformation be detected? ArXiv preprint, abs/2309.13788.
  9. Canyu Chen and Kai Shu. 2023b. Combating misinformation in the age of llms: Opportunities and challenges. ArXiv preprint, abs/2311.05656.
  10. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879, Vancouver, Canada. Association for Computational Lin…
  11. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. 2023. Extending context window of large language models via positional interpolation. ArXiv preprint, abs/2306.15595. find
  12. Sijing Chen, Lu Xiao, and Jin Mao. 2021. Persuasion strategies of misinformation-containing posts in the social media. Information Processing & Management, 58(5):102665. find
  13. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  14. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. ArXiv preprint, abs/2204.02311.
  15. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
  16. Anthony G Cohn and Jose Hernandez-Orallo. 2023. Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of llms. ArXiv preprint, abs/2304.11164. find
  17. William D Crano and Radmila Prislin. 2006. Attitudes and persuasion. Annu. Rev. Psychol., 57:345–374. find
  18. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2023. A survey for in-context learning. ArXiv preprint, abs/2301.00234.
  19. Haiping Du, Siyu Teng, Hong Chen, Jiaqi Ma, Xiao Wang, Chao Gou, Bai Li, Siji Ma, Qinghai Miao, Xiaoxiang Na, et al. 2023a. Chat with chatgpt on intelligent vehicles: An ieee tiv perspective. IEEE Transactions on Intelligent Vehicles. find
  20. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023b. Improving factuality and reasoning in language models through multiagent debate. ArXiv preprint, abs/2305.14325.
  21. Ullrich KH Ecker, Stephan Lewandowsky, John Cook, Philipp Schmid, Lisa K Fazio, Nadia Brashier, Panayiota Kendeou, Emily K Vraga, and Michelle A Amazeen. 2022. The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1):13–29. find
  22. Harry Barton Essel, Dimitrios Vlachopoulos, Ako-sua Tachie-Menson, Esi Eduafua Johnson, and Papa Kwame Baah. 2022. The impact of a virtual teaching assistant (chatbot) on students’ learning in ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1):1–19. find
  23. Jörg Friedrichs. 2014. Useful lies: The twisted rationality of denial. Philosophical Psychology, 27(2):212– 234. find
  24. Melanie Gagich, Emilie Zickel, and Terri Pantuso. 2023. Rhetorical appeals: Logos, pathos, and ethos defined. find
  25. Dhir Gala and Amgad N Makaryus. 2023. The utility of language models in cardiology: A narrative review of the benefits and concerns of chatgpt-4. International Journal of Environmental Research and Public Health, 20(15):6438. find
  26. Robert H Gass and John S Seiter. 2015. Persuasion: Social inflence and compliance gaining. Routledge. find
  27. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. ArXiv preprint, abs/2203.05794.
  28. Changwu Huang, Zeqi Zhang, Bifei Mao, and Xin Yao. 2022. An overview of artificial intelligence ethics. IEEE Transactions on Artificial Intelligence. find
  29. Kensuke Ito. 2023. Truth and falsity in communication: Assertion, denial, and interpretation. Erkenntnis, 88(2):657–674. find
  30. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  31. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031, Copenhagen, Denmark. Association for Computational Linguistics.
  32. Bohan Jiang, Zhen Tan, Ayushi Nirmal, and Huan Liu. 2023. Disinformation detection: An evolving challenge in the age of llms. ArXiv preprint, abs/2309.15847. — [arXiv:2309.15847](https://arxiv.org/abs/2309.15847) find
  33. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423–438.
  34. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. ArXiv preprint, abs/2207.05221. find
  35. Nora Kassner and Hinrich Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7811–7818, Online. Association for Computational Linguistics.
  36. Celeste Kidd and Abeba Birhane. 2023. How ai can distort human beliefs. Science, 380(6651):1222–1223. find
  37. Chokri Kooli. 2023. Chatbots in education and research: A critical examination of ethical implications and solutions. Sustainability, 15(7):5614. find
  38. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  39. Kiho Lee. 2023. Chatgpt "dan" (and other "jailbreaks"). find
  40. Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. Advances in Neural Information Processing Systems, 35:34586–34599.
  41. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the…
  42. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. 2022. Large language models with controllable working memory.
  43. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXiv e-prints, pages arXiv–2305.
  44. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
  45. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114…
  46. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv preprint, abs/1907.11692.
  47. James H Lubowitz. 2023. Chatgpt, an artificial intelligence chatbot, is impacting medical literature. Arthroscopy, 39(5):1121–1122. find
  48. Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. ArXiv preprint, abs/2303.08896. find
  49. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics.
  50. Michael McTear. 2022. Conversational ai: Dialogue systems, conversational agents, and chatbots. Springer Nature. find
  51. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022a. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, A…
  52. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022b. Rethinking the role of demonstrations: What makes in-context learning work? ArXiv preprint, abs/2202.12837.
  53. Marvin Minsky. 1988. Society of mind. Simon and Schuster. find
  54. John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020. TextAttack: A framework for adversarial attacks, data augmentation, and adversarial training in NLP. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pa…
  55. Raymond S Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology, 2(2):175–220. find
  56. Brendan Nyhan and Jason Reifler. 2010. When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2):303–330. find
  57. R OpenAI. 2023. Gpt-4 technical report. arXiv, pages 2303–08774.
  58. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  59. Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang. 2023. On the risk of misinformation pollution with large language models. ArXiv preprint, abs/2305.13661.
  60. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. find
  61. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain,… — Discovering Language Model Behaviors with Model-Written Evaluations
  62. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
  63. Pouya Pezeshkpour and Estevam Hruschka. 2023. Large language models sensitivity to the order of options in multiple-choice questions.
  64. Raunak M Pillai and Lisa K Fazio. 2021. The effects of repeating false and misleading information on belief. Wiley Interdisciplinary Reviews: Cognitive Science, 12(6):e1573. find
  65. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with" gradient descent" and beam search. ArXiv preprint, abs/2305.03495.
  66. Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying LMs with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5203–5212, Online. Association for Computatio…
  67. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  68. Christof Rapp. 2002. Aristotle’s rhetoric. The Stanford Encyclopedia of Philosophy. find
  69. Lisa Rashotte. 2007. Social influence. The Blackwell encyclopedia of sociology. find
  70. Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. Parallel context windows for large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1…
  71. Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. 2023. Investigating the factual knowledge boundary of large language models with retrieval augmentation. ArXiv preprint, abs/2307.11019.
  72. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418–5426, Online. Association for Computational Linguistics. find
  73. Gulyamov Said, Khudoberganov Azamat, Sharopov Ravshan, and Abduvaliev Bokhadir. 2023. Adapting legal systems to the development of artificial intelligence: Solving the global problem of ai in judicial processes. International Journal of Cyber Law, 1(4). find
  74. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen tau Yih. 2023. Trusting your evidence: Hallucinate less with context-aware decoding.
  75. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Onlin…
  76. Nicolaj Siggelkow. 2007. Persuasion with case studies. Academy of management journal, 50(1):20–24. find
  77. Andreas Stokke. 2013. Lying and asserting. The Journal of Philosophy, 110:33–60. find
  78. Briony Swire-Thompson, Joseph DeGutis, and David Lazer. 2020. Searching for the backfire effect: Measurement and design considerations. Journal of applied research in memory and cognition, 9(3):286– 299. find
  79. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2020. oLMpics-on what language model pre-training captures. Transactions of the Association for Computational Linguistics, 8:743–758.
  80. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. ArXiv preprint, abs/2307.09288.
  81. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation. ArXiv preprint, abs/2307.03987.
  82. Boshi Wang, Xiang Yue, and Huan Sun. 2023a. Can chatgpt defend the truth? automatic dialectical evaluation elicits llms’ deficiencies in reasoning. ArXiv preprint, abs/2305.13160. — [arXiv:2305.13160](https://arxiv.org/abs/2305.13160) find
  83. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023b. Voyager: An open-ended embodied agent with large language models. ArXiv preprint, abs/2305.16291. find
  84. Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. 2023c. Augmenting language models with long-term memory. ArXiv preprint, abs/2306.07174. find
  85. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023a. Jailbroken: How does llm safety training fail?
  86. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V. Le. 2023b. Simple synthetic data reduces sycophancy in large language models.
  87. Orion Weller, Aleem Khan, Nathaniel Weir, Dawn Lawrie, and Benjamin Van Durme. 2022. Defending against misinformation attacks in open-domain question answering. ArXiv preprint, abs/2212.10002. — [arXiv:2212.10002](https://arxiv.org/abs/2212.10002) find
  88. Magdalena Wysocka, Oskar Wysocki, Maxime Delmas, Vincent Mutel, and Andre Freitas. 2023. Large language models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery. ArXiv preprint, abs/2305.17819. — [arXiv:2305.17819](https://arxiv.org/abs/2305.17819) find
  89. Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2023. Adaptive chameleon or stubborn sloth: Unraveling the behavior of large language models in knowledge conflicts. ArXiv preprint, abs/2305.13300.
  90. Lu Xu, Leslie Sanders, Kay Li, James CL Chow, et al. 2021. Chatbot for health care and oncology applications using artificial intelligence and machine learning: systematic review. JMIR cancer, 7(4):e27850. find
  91. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. ArXiv preprint, abs/2303.18223.
  92. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pages 12697–12706. PMLR.
  93. Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang. 2023. Why does chatgpt fall short in providing truthful answers. ArXiv preprint, abs/2304.10513. — [arXiv:2304.10513](https://arxiv.org/abs/2304.10513) find
  94. Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [MASK]: Learning vs. learning to recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5017–5033, Online. Association for Co…
  95. Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2023. Context-faithful prompting for large language models. ArXiv preprint, abs/2303.11315.
  96. Hazem Zohny, John McMillan, and Mike King. 2023. Ethics of generative ai. find
  97. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendry… — Representation Engineering: A Top-Down Approach to AI Transparency
  98. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023b. Universal and transferable adversarial attacks on aligned language models.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2023-earth-flat-because-investigating-llms/