Elamur Research⌕
☰
home / papers / 2025 / ELEPHANT: Measuring and understanding social sycophancy in LLMs

ELEPHANT: Measuring and understanding social sycophancy in LLMs

Paper · 2025 · Myra Cheng, Sunny Yu, Cinoo Lee et al.

Summary

Earlier sycophancy research checks whether an LLM drops a factual answer to agree with a belief the user states outright. That misses the more common case, such as advice seeking, where users never spell out what they believe and there is no objectively correct answer to check against. The authors use Goffman's notion of face and define social sycophancy as excessively protecting the user's self-image. A model can do this by affirming the user (positive face) or by not challenging them (negative face). They introduce four dimensions: validation, indirectness, framing, and moral sycophancy. These are measured in ELEPHANT, a benchmark that uses human-validated LLM scorers on four first-person datasets drawn largely from Reddit. Across 11 models, social sycophancy is far more frequent than in crowdsourced human replies, including where human consensus says the user is at fault. Preference datasets turn out to reward these behaviours. Mitigations give mixed results, although model-based steering looks promising. The work offers a way to audit sycophancy in open-ended use, the most frequent kind of LLM use.

Key points

  • Datasets:
    • OEQ: 3,027 open-ended advice queries.
    • AITA-YTA: 2,000 r/AmITheAsshole posts judged "YTA".
    • SS: 3,777 assumption-laden statements from r/Advice.
    • AITA-NTA-FLIP: 1,591 pairs, each an "NTA" post plus a GPT-4o rewrite told from the wrongdoer's side.
  • OEQ, models vs. humans:
    • Validation: 72% vs. 22%.
    • Indirectness: 66% vs. 21%.
    • Accepting the user's framing: 88% vs. 60%.
  • Clear wrongdoing and assumptions: On AITA-YTA, models preserve face about 46 pp more than humans. On SS, they leave possibly unfounded assumptions unchallenged 86% of the time.
  • Moral sycophancy: In 48% of conflict pairs, models side with whichever party is the user, telling both the wronged and the at-fault person they did nothing wrong.
  • Causes and mitigations:
    • Preferred responses in HH-RLHF score higher on some sycophancy dimensions.
    • Rewriting prompts in the third person mostly helps only on SS.
    • DPO-based steering shows promise.
    • Politeness correlates only weakly with the sycophancy dimensions (|r| ≈ 0.25–0.27).
    • Code and data are at github.com/myracheng/elephant.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States434 works · 314,419 citations🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Social Sciences · 1,772 worksPsychology · 1,552 worksSafety Research · 543 worksSociology and Political Science · 577 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim and Dan Jurafsky. 2025. ELEPHANT: Measuring and understanding social sycophancy in LLMs. arXiv:2505.13995.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1126/science.aec8352 · arXiv 2505.13995

@misc{cheng2025elephant,
  title = {{ELEPHANT: Measuring and understanding social sycophancy in LLMs}},
  author = {Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim and Dan Jurafsky},
  year = {2025},
  doi = {10.1126/science.aec8352},
  eprint = {2505.13995},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1126/science.aec8352}
}

References

79 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Areej Alhassan, Jinkai Zhang, and Viktor Schlegel. ‘Am I the Bad One’? predicting the moral judgement of the crowd using pre–trained language models. In Proceedings of the thirteenth language resources and evaluation conference, pp. 267–276, 2022. find
  2. Anthropic. Claude 3.7 sonnet system card. claude-3-7-sonnet-system-card, 2025. Accessed: 2025-05-14. https://www.anthropic.com/ ↗
  3. Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas L Griffiths. Explicitly unbiased large language models still form biased associations. Proceedings of the National Academy of Sciences, 122(8): e2416228122, 2025. find
  4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. find
  5. Bryce Boe. The python reddit api wrapper. https://github.com/praw-dev/praw, 2016. ↗
  6. Penelope Brown and Stephen C Levinson. Politeness: Some universals in language usage. Cambridge University Press, 1987.
  7. Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, et al. Harms from increasingly agentic algorithmic systems. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transpar… find
  8. Mohit Chandra, Suchismita Naik, Denae Ford, Ebele Okoli, Munmun De Choudhury, Mahsa Ershadi, Gonzalo Ramos, Javier Hernandez, Ananya Bhattacharjee, Shahed Warreth, et al. From lived experience to insight: Unpacking the psychological risks of using ai conversational agents. In Proceedings of the 2025… find
  9. Jonathan P. Chang, Caleb Chiam, Liye Fu, Andrew Wang, Justine Zhang, and Cristian Danescu-Niculescu-Mizil. ConvoKit: A toolkit for the analysis of conversations. In Olivier Pietquin, Smaranda Muresan, Vivian Chen, Casey Kennington, David Vandyke, Nina Dethlefs, Koji Inoue, Erik Ekstedt, and Stefan U…
  10. Aaron Chatterji, Thomas Cunningham, David J. Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. How people use ChatGPT. NBER Working Paper 34255, National Bureau of Economic Research, Cambridge, MA, September 2025. URL http://www.nber.org/papers/w34255. ↗
  11. Wei Chen, Zhen Huang, Liang Xie, Binbin Lin, Houqiang Li, Le Lu, Xinmei Tian, Deng Cai, Yonggang Zhang, Wenxiao Wan, Xu Shen, and Jieping Ye. From yes-men to truth-tellers: addressing sycophancy in large language models with pinpoint tuning. In Proceedings of the 41st International Conference on Mac… find
  12. Myra Cheng, Kristina Gligoric, Tiziano Piccardi, and Dan Jurafsky. AnthroScore: A computational linguistic measure of anthropomorphism. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: L… find
  13. Ajeya Cotra. Why AI alignment could be hard with modern deep learning. Cold Takes, 2021. find
  14. Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin, and James A. Landay. The illusion of empathy? notes on displays of emotion in human-computer interaction. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY,… — [doi:10.1145/3613904.3642336](https://doi.org/10.1145/3613904.3642336) find
  15. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with scaled AI feedback, 2024. URL https://arxiv.org/abs/2310.01377. ↗
  16. Alba Curry and Amanda Cercas Curry. Computer says “no”: The case against empathetic conversational AI. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 8123–8130, Toronto, Canada, July 2023. Association for Comput… find
  17. Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36:30039–30069, 2023.
  18. Aaron Fanous, Jacob Goldberg, Ank A Agarwal, Joanna Lin, Anson Zhou, Roxana Daneshjou, and Sanmi Koyejo. Syceval: Evaluating LLM sycophancy. arXiv preprint arXiv:2502.08177, 2025. — [arXiv:2502.08177](https://arxiv.org/abs/2502.08177) find
  19. Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30):e2305016120, 2023. — ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
  20. Erving Goffman. On face-work: An analysis of ritual elements in social interaction. Psychiatry, 18(3): 213–231, 1955. find
  21. Google DeepMind. Gemini 1.5 flash. https://deepmind.google/technologies/gemini/, 2024. Accessed: 2025-05-14. ↗
  22. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
  23. Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794, 2022.
  24. Michael Haugh and Francesca Bargiela-Chiappini. Face and interaction. Face, communication and social interaction, pp. 1–30, 2009. find
  25. Jiseung Hong, Grace Byun, Seungone Kim, and Kai Shu. Measuring sycophancy of language models in multi-turn dialogues. arXiv preprint arXiv:2505.23840, 2025. find
  26. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python. 2020. doi: 10.5281/zenodo.1212303. find
  27. Haonan Hou, Kevin Leach, and Yu Huang. Chatgpt giving relationship advice–how reliable is it? In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pp. 610–623, 2024. find
  28. Piers Douglas Lionel Howe, Nicolas Fay, Morgan Saletta, and Eduard Hovy. Chatgpt’s advice is perceived as better than that of professional advice columnists. Frontiers in Psychology, 14:1281255, 2023. find
  29. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024.
  30. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
  31. Lujain Ibrahim and Myra Cheng. Thinking beyond the anthropomorphic paradigm benefits LLM research. arXiv preprint arXiv:2502.09192, 2025.
  32. Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu Meena Sp, and Nithya Sambasivan. ” because AI is 100% right and safe”: User attitudes and sources of AI authority in india. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–18, 2022. find
  33. Azal Ahmad Khan, Sayan Alam, Xinran Wang, Ahmad Faraz Khan, Debanga Raj Neog, and Ali Anwar. Mitigating sycophancy in large language models via direct preference optimization. In 2024 IEEE International Conference on Big Data (BigData), pp. 1664–1671. find
  34. Minbeom Kim, Hwanhee Lee, Joonsuk Park, Hwaran Lee, and Kyomin Jung. AdvisorQA: Towards helpful and harmless advice-seeking question answering with collective intelligence. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of… find
  35. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, et al. The prism alignment project: What participatory, representative and individualised human feedback reveals about the subjective and multicultur… find
  36. Joshua Klayman. Varieties of confirmation bias. Psychology of learning and motivation, 32:385–418, 1995. find
  37. Esben Kran, Hieu Minh Nguyen, Akash Kundu, Sami Jawhar, Jinsuk Park, and Mateusz Maria Jurewicz. Darkbench: Benchmarking dark patterns in large language models. In The Thirteenth International Conference on Learning Representations, 2025. find
  38. Otto JB Kuosmanen. Advice from humans and artificial intelligence: Can we distinguish them, and is one better than the other? Master’s thesis, UiT Norges arktiske universitet, 2024. find
  39. Haoxi Li, Xueyang Tang, Jie Zhang, Song Guo, Sikai Bai, Peiran Dong, and Yue Yu. Causally motivated sycophancy mitigation for large language models. In The Thirteenth International Conference on Learning Representations. Junyi Li, Charith Peris, Ninareh Mehrabi, Palash Goyal, Kai-Wei Chang, Aram Gal… find
  40. Kaiqu Liang, Haimin Hu, Ryan Liu, Thomas L Griffiths, and Jaime Fernández Fisac. RLHS: Mitigating misalignment in rlhf with hindsight simulation. arXiv preprint arXiv:2501.08617, 2025. find
  41. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. find
  42. Lars Malmqvist. Sycophancy in large language models: Causes and mitigations. arXiv:2411.15287, 2024. arXiv preprint Lars Malmqvist. Sycophancy in large language models: Causes and mitigations. In Intelligent Computing-Proceedings of the Computing Conference, pp. 61–74.
  43. Meta. Meta llama-3-70b-instruct-turbo. https://huggingface.co/meta-llama/Meta-Llama-3. 1-70B-Instruct-Turbo, 2024. Accessed: 2025-05-14. ↗
  44. Mistral. Mistral-7b-instruct-v0.3. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3, 2023. Accessed: 2025-05-14. ↗
  45. Mistral. Mistral-small-24b-instruct-2501. https://huggingface.co/mistralai/ Mistral-Small-24B-Instruct-2501, 2025. Instruction-tuned 24B parameter language model released under the Apache 2.0 License. ↗
  46. Elle O’Brien. AITA for making this? A public dataset of Reddit posts about moral dilemmas — datachain.ai. https://datachain.ai/blog/a-public-reddit-dataset, 2020. [Accessed 16-04-2025]. ↗
  47. OpenAI. Expanding on what we missed with sycophancy, May 2025. URL https://openai.com/index/ expanding-on-sycophancy/. Accessed: 2025-05-10. ↗
  48. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022.
  49. Henry Papadatos and Rachel Freedman. Linear probe penalties reduce llm sycophancy. arXiv preprint arXiv:2412.00967, 2024. — [arXiv:2412.00967](https://arxiv.org/abs/2412.00967) find
  50. Eric Pederson. Cross-cultural pragmatics: The semantics of human interaction, 1991. find
  51. Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain… — Discovering Language Model Behaviors with Model-Written Evaluations
  52. Jaana Porra, Mary Lacity, and Michael S Parks. Can computer based human-likeness endanger humanness?”– a philosophical and ethical perspective on digital assistants expressing feelings they can’t have. Information Systems Frontiers, 22:533–547, 2020. find
  53. Ansh Radhakrishnan, Karina Nguyen, Anna Chen, Carol Chen, Carson Denison, Danny Hernandez, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamile Lukosiute, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Sam McCandlish, Sheer El Showk, Tamera Lanham, Tim Maxwell, Venkatesa Chandraseka… find
  54. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum? id=HPu… ↗
  55. Leonardo Ranaldi and Giulia Pucci. When large language models contradict humans? large language models’ sycophantic behaviour, 2024. URL https://arxiv.org/abs/2311.09410. — [arXiv:2311.09410](https://arxiv.org/abs/2311.09410) ↗
  56. Abhinav Sukumar Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, and Maarten Sap. NormAd: A framework for measuring the cultural adaptability of large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter o…
  57. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/abs/1908.10084. ↗
  58. Aswin Rrv, Nemika Tyagi, Md Nayem Uddin, Neeraj Varshney, and Chitta Baral. Chaos with keywords: Exposing large language models sycophancy to misleading keywords and evaluating defense strategies. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational… find
  59. Giuseppe Russo, Debora Nozza, Paul Röttger, and Dirk Hovy. The pluralistic moral gap: Understanding judgment and value differences between humans and large language models. arXiv preprint arXiv:2507.17216, 2025. — [arXiv:2507.17216](https://arxiv.org/abs/2507.17216) find
  60. Pratik Sachdeva and Tom van Nuenen. Normative evaluation of large language models with everyday moral dilemmas. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pp. 690–709, 2025. find
  61. Omar Shaikh, Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. Navigating rifts in human-LLM grounding: Study and benchmark. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computationa…
  62. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards understandin… — Towards Understanding Sycophancy in Language Models
  63. Zhe Su, Xuhui Zhou, Sanketh Rangreji, Anubha Kabra, Julia Mendelsohn, Faeze Brahman, and Maarten Sap. AI-LieDar : Examine the trade-off between utility and truthfulness in LLM agents. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas… find
  64. Peiqi Sui, Eamon Duede, Sophie Wu, and Richard So. Confabulation: The surprising value of large language model hallucinations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.… find
  65. Lihao Sun, Chengzhi Mao, Valentin Hofmann, and Xuechunzi Bai. Aligned but blind: Alignment increases implicit bias by reducing awareness of race. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Compu…
  66. Mirac Suzgun, Tayfun Gur, Federico Bianchi, Daniel E Ho, Thomas Icard, Dan Jurafsky, and James Zou. Belief in the machine: Investigating epistemological blind spots of language models. arXiv preprint arXiv:2410.21195, 2024. find
  67. Deborah Tannen. Framing and face: The relevance of the presentation of self to linguistic discourse analysis. Social Psychology Quarterly, 72(4):300–305, 2009. find
  68. Stella Ting-Toomey, Ge Gao, Paula Trubisky, Zhizhong Yang, Hak Soo Kim, Sung-Ling Lin, and Tsukasa Nishida. Culture, face maintenance, and styles of handling interpersonal conflict: A study in five cultures. International Journal of conflict management, 2(4):275–296, 1991. find
  69. Anvesh Rao Vijjini, Rakesh R Menon, Jiayi Fu, Shashank Srivastava, and Snigdha Chaturvedi. SocialGaze: Improving the integration of human social norms in large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMN… find
  70. Keyu Wang, Jin Li, Shu Yang, Zhuoran Zhang, and Di Wang. When truth is overridden: Uncovering the internal origins of sycophancy in large language models. arXiv e-prints, pp. arXiv–2508, 2025. find
  71. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958, 2023. find
  72. Marc Zao-Sanders. How People Are Really Using Gen AI in 2025 — hbr.org. https://hbr.org/2025/04/ how-people-are-really-using-gen-ai-in-2025, 2025. [Accessed 02-05-2025]. ↗
  73. Yunpu Zhao, Rui Zhang, Junbin Xiao, Changxin Ke, Ruibo Hou, Yifan Hao, Qi Guo, and Yunji Chen. Towards analyzing and mitigating sycophancy in large vision-language models. arXiv preprint arXiv:2408.11261, 2024. — [arXiv:2408.11261](https://arxiv.org/abs/2408.11261) find
  74. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623, 2023.
  75. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. Lmsys-chat-1m: A large-scale real-world LLM conversation dataset, 2024. URL https://arxiv.org/abs/2309.11998. ↗
  76. Tan Zhi-Xuan, Micah Carroll, Matija Franklin, and Hal Ashton. Beyond preferences in ai alignment: T. zhi-xuan et al. Philosophical Studies, 182(7):1813–1863, 2025. find
  77. Naitian Zhou, David Bamman, and Isaac L. Bleaman. Culture is not trivia: Sociocultural theory for cultural NLP. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…
  78. Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291, 2024.
  79. Paper Definition Operationalization of Sycophancy Sharma et al. (2024) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cheng-2025-elephant-measuring-understanding-social-sycophancy/