Elamur Research⌕
☰
home / papers / 2024 / Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

Summary

Many studies measure the values and opinions of large language models (LLMs) by giving them multiple-choice questionnaires. The usual motivation is real-world impact, yet actual users rarely talk to chatbots in survey format. To find out whether such constrained evaluations tell us anything reliable, the authors examine the widely cited Political Compass Test (PCT) in depth.

They first reviewed Google Scholar, arXiv and the ACL Anthology. Most prior PCT-based studies push models to pick one predefined answer and seldom check whether results hold up when the prompt is changed. The authors then ran controlled experiments on up to 10 open and proprietary LLMs:

  • removing the forced-choice instruction;
  • varying how strongly models are pushed to choose;
  • paraphrasing prompts minimally;
  • moving to an open-ended answer setting.

Every change shifted the political positions the models expressed, so PCT-style results are unstable and do not generalise. The authors recommend three practices:

  • design evaluations around realistic usage in specific applications;
  • always add robustness checks;
  • limit conclusions to the tested setting rather than claiming a model's "general" ideology.

Key points

  • Test format: The PCT has 62 single-sentence propositions on six topics. Respondents choose from four agreement levels with no neutral option, and the answers place them on an economic left–right axis and a social libertarian–authoritarian axis.

  • Literature review: As of February 2024, the search yielded 57 unique articles, of which 12 actually apply the PCT to LLMs.

    • 10 of these 12 force a single multiple-choice answer.
    • Only three do any robustness testing beyond repeating the same prompt.
    • All 12 test some GPT-3.5 version.

  • Models tested:

    • Llama2-chat (7b, 13b, 70b);
    • Mistral 7b Instruct v0.1 and v0.2;
    • Zephyr 7b β;
    • GPT-3.5 and GPT-4 (versions 0613 and 1106).

    All were run at temperature zero.

  • Forced-choice escalation: The forced-choice instructions grow increasingly coercive, up to emotional pressure such as claiming a job or a grandmother's life is at stake. The answers models give depend on which variant is used.

  • Open resource: Code and data are released at github.com/paul-rottger/llm-values-pct.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

100 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 98 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.acl-long.816

@inproceedings{rottger2024political,
  title = {{Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models}},
  author = {Röttger, Paul and Hofmann, Valentin and Pyatkin, Valentina and Hinck, Musashi and Kirk, Hannah and Schuetze, Hinrich and Hovy, Dirk},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.816},
  url = {https://doi.org/10.18653/v1/2024.acl-long.816}
}

References

46 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Verena Rieser, and Zeerak Talat. 2023. Mirages. on anthropomorphism in dialogue systems. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4776–4790, Singapore. Association for Computational Linguist… find
  2. Giuseppe Attanasio. 2023. Simple Generation. https://github.com/MilaNLProc/simple-generation. ↗
  3. Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120. find
  4. Ethan Busby, D Flynn, James N Druckman, and P D’Angelo. 2018. Studying framing effects on political preferences. Doing news framing analysis II: Empirical and theoretical perspectives, pages 27–50. find
  5. Dennis Chong and James N Druckman. 2007. Framing theory. Annu. Rev. Polit. Sci., 10:103–126. find
  6. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388. find our card
  7. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031. find
  8. Cristina España-Bonet. 2023. Multilingual coarse political stance classification of media. the editorial line of a ChatGPT and bard newspaper. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11757–11777, Singapore. Association for Computational Linguistics. find
  9. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… find our card
  10. Sasuke Fujimoto and Takemoto Kazuhiro. 2023. Revisiting the political biases of chatgpt. Frontiers in Artificial Intelligence, 6. find
  11. Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nishanth Sastry, Jose Such, and Guillermo Suarez-Tangil. 2023. Ai in the gray: Exploring moderation policies in dialogic large language models vs. human answers in controversial topics. In Proceedings of the 32nd ACM International Conference on Information… find
  12. Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation. arXiv preprint arXiv:2301.01768. find
  13. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2020. Aligning ai with shared human values. In International Conference on Learning Representations. find our card
  14. Janus. 2022. Simulators. LessWrong online forum, 2nd September. https://www.lesswrong.com/posts/ vJFdjigzmcXMhNTsx/. ↗
  15. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  16. Graham Kalton and Howard Schuman. 1982. The effect of the question on survey responses: A review. Journal of the Royal Statistical Society Series A: Statistics in Society, 145(1):42–57. find
  17. Hannah Kirk, Andrew Bean, Bertie Vidgen, Paul Rottger, and Scott Hale. 2023. The past, present and better future of feedback learning in large language models for subjective human preferences and values. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages… find
  18. R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. find
  19. Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. 2022. Who is GPT-3? an exploration of personality, values and demographics. In Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS), pages 218–227, Abu Dhabi, UAE. Association for Computatio… find
  20. Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2023. More human than human: Measuring chatgpt political bias. Public Choice, pages 1–21. find
  21. Arvind Narayanan and Sayash Kapoor. 2023. Does chatgpt have a liberal bias? https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias. ↗
  22. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Trai… find
  23. Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. 2023. The shifted and the overlooked: A task-oriented investigation of user-GPT interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language P… find
  24. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2023. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263. models. In Proceedings of the First Workshop on Bangla Language Proc… find our card
  25. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. find
  26. David Rozado. 2023a. Danger in the machine: The perils of political and demographic biases embedded in ai systems. Manhattan Institute. find
  27. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023. Zephyr: Direct distillation of lm alignment. arXiv preprint arXiv:2310.16944. find
  28. David Rozado. 2023b. The political biases of chatgpt. Social Sciences, 12(3):148. find
  29. Merel van den Broek. 2023. Chatgpt’s left-leaning liberal bias. University of Leiden. find
  30. David Rozado. 2024. The political preferences of llms. arXiv preprint arXiv:2402.01789. — [arXiv:2402.01789](https://arxiv.org/abs/2402.01789) find
  31. Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (R… find
  32. Jérôme Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, and Markus Pauly. 2023. The self-perception and political biases of chatgpt. arXiv preprint arXiv:2304.07333. — [arXiv:2304.07333](https://arxiv.org/abs/2304.07333) find
  33. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org. find our card
  34. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. In Thirty-seventh Conference on Neural Information Processing Systems. find
  35. Howard Schuman and Stanley Presser. 1977. Question wording as an independent variable in survey analysis. Sociological Methods & Research, 6(2):151–170. find
  36. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations. find
  37. Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature, 623(7987):493–498. find our card
  38. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Dallas Card, and David Jurgens. 2023. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. arXiv preprint arXiv:2311.09718. find
  39. Surendrabikram Thapa, Ashwarya Maratha, Khan Md Hasib, Mehwish Nasim, and Usman Naseem. 2023. Assessing political inclination of Bangla language Jindong Wang, HU Xixu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Wei Ye, Haojun Huang, Xiubo Geng, et al. 2023. On the robustness of cha… find
  40. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Röttger, and Barbara Plank. 2024a. Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. arXiv preprint arXiv:2404.08382. — [arXiv:2404.08382](https://arxiv.org/abs/2404.08382) find
  41. Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024b. " my answer is c": First-token probabilities do not match text answers in instruction-tuned language models. arXiv preprint arXiv:2402.14499. find
  42. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483. find our card
  43. Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, et al. 2023. Cvalues: Measuring the values of chinese large language models from safety to responsibility. arXiv preprint arXiv:2307.09705. find
  44. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. (inthe)wildchat: 570k chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations. find
  45. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024a. Large language models are not robust multiple choice selectors. In The Twelfth International Conference on Learning Representations. find
  46. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2024b. LMSYS-chat-1m: A large-scale real-world LLM conversation dataset. In The Twelfth International Conference on Learning… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/