Elamur Research⌕
☰
home / papers / 2026 / Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models

Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models

Paper · 2026 · David Guzman Piedrahita, Irene Strauss, Rada Mihalcea et al.

Summary

Most studies of political bias in LLMs focus on demographics or on the left–right axis, which assumes a democratic setting. This paper asks a different question: do models lean toward democratic or authoritarian systems of governance? The authors build a three-part probing framework. First, an adapted F-scale questionnaire measures abstract authoritarian attitudes. Second, FavScore, a new metric built from public-opinion survey items, rates how favorably models judge current world leaders. Third, role-model prompts check whether political preferences appear in contexts that are not explicitly political. Eight prominent LLMs were tested in English and in Mandarin. In English, the models mostly reject authoritarian values and rate authoritarian leaders poorly. In Mandarin, that democratic tilt weakens, and approval of authoritarian leaders rises significantly. In both languages, models often name authoritarian figures when asked for role models of a given nationality. The work matters because it shows that geopolitical value bias depends on language and can surface in everyday, apolitical use. It also gives auditors a reusable tool for this kind of evaluation.

Key points

  • Value-centric probe: 30 F-scale items (Adorno et al., 1950) from nine categories, such as authoritarian aggression and conventionalism. Models answer on the original 6-point Likert scale, and English–Mandarin differences are tested with a sign test.
  • FavScore: 39 questions adapted from Pew, ANES and Eurobarometer, organized into five categories covering overall approval, approval of the leader's policies, emotional affect, perceived traits, and expectations about the future. Scores range from –1 to +1 for each leader in power as of April 2025, and regime types come from the V-Dem dataset.
  • Language effect: Mandarin prompts yield noticeably higher favorability toward authoritarian leaders than English prompts.
  • Role-model probing: Authoritarian figures are frequently suggested as role models. Adding a prosocial definition of "role model" reduces this tendency but does not remove it, while synonyms give similar or higher rates.
  • Resources and limitations: Code is released on GitHub (irenestrauss/Democratic-Authoritarian-Bias-LLMs). The study covers only two languages and uses LLM-judge annotation with human validation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TorontoToronto · Canada96 works · 142,007 citations🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations🎓University of ZurichZurich · Switzerland78 works · 12,260 citations🎓ETH ZurichZurich · Switzerland72 works · 5,176 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 worksInterpretability · 130 works

Cite

Cite as David Guzman Piedrahita, Irene Strauss, Rada Mihalcea and Zhijing Jin. 2026. Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.27

@inproceedings{piedrahita2026democratic,
  title = {{Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models}},
  author = {David Guzman Piedrahita, Irene Strauss, Rada Mihalcea and Zhijing Jin},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.27},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.27}
}

References

38 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. T. W. Adorno, Else Frenkel-Brunswik, Daniel J. Levinson, and R. Nevitt Sanford. 1950. The Authoritarian Personality. Harper & Brothers, New York. find
  2. Yunus Akdal. 2025. The political compass test and the death of politics. Columbia Political Review. find
  3. Bob Altemeyer. 1981. Right-Wing Authoritarianism. University of Manitoba Press. find
  4. ANES. 2021. Anes 2020 time series study full release. [dataset and documentation]. July 19, 2021 version. find
  5. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
  6. Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring political bias in large language models: What is said and how it is said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11142–11159, Bangkok, Thailand. A…
  7. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pages 610–623, New York, NY, USA. Association… find
  8. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics.
  9. Wayne Brittenden. 2001. The political compass test. find
  10. Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. Humans or LLMs as the judge? a study on judgement bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8301–8327, Miami, Florida, USA. Association for Computational Lingu…
  11. Richard Christie and Marie Jahoda. 1954. Studies in the Scope and Method of The Authoritarian Personality. Free Press. find
  12. Alan C. Elms and Stanley Milgram. 1966. Personality characteristics associated with obedience and defiance toward authoritative command. Journal of Experimental Research in Personality. find
  13. Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, and Soroush Vosoughi. 2022. Quantifying and alleviating political bias in language models. Artificial Intelligence, 304:103654. find
  14. Eurobarometer. 2024. Standard eurobarometer std102: Standard eurobarometer 102 – autumn 2024 (v1.00). find
  15. Ervin Locklear and Micheal Stratil. 1982. Authoritarianism: Validation of the balanced f scale through observer ratings. Technical report, Pembroke State University. find
  16. David Exler, Mark Schutera, Markus Reischl, and Luca Rettenberger. 2025. Large means left: Political bias in large language models increases with their number of parameters.
  17. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume…
  18. Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole-Dayan, Deb Roy, and Jad Kabbara. 2024. On the relationship between truth and political bias in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9004– 9018… find
  19. Anna Lührmann and Staffan I. Lindberg. 2019. A third wave of autocratization is here: What is new about it? Democratization, 26(7):1095–1113. find
  20. Anna Lührmann, Marcus Tannenberg, and Staffan I. Lindberg. 2018. Regimes of the world (row): Opening new avenues for the comparative study of political regimes. Politics and Governance, 6(1):60–77. find
  21. Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2024. More human than human: Measuring chatgpt political bias. Public Choice, 198(1):3–23. find
  22. Anna Neumann, Elisabeth Kirsten, Muhammad Bilal Zafar, and Jatinder Singh. 2025. Position is power: System prompts as a mechanism of bias in large language models (llms). In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’25, page 573–598, New York, NY, U… find
  23. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097– 1179.
  24. Thomas B. Pepinsky. 2025. State, society, and the politics of democratic backsliding. SSRN Electronic Journal. Available at SSRN. find
  25. Stephan Haggard and Robert Kaufman. 2021. Backsliding: Democratic Regress in the Contemporary World. Cambridge University Press. find
  26. Saghar Hosseini, Hamid Palangi, and Ahmed Hassan Awadallah. 2023. An empirical study of metrics to measure representational harms in pre-trained language models. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 121–134, Toronto, Canada. Association…
  27. Minseok Jung, Aurora Zhang, Junho Lee, and Paul Pu Liang. 2024. Quantitative insights into language model usage and trust in academia: An empirical study. find
  28. Pew. 2010. Obama’s ratings little affected by recent turmoil. find
  29. Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. 2024. Hidden persuaders: LLMs’ political leaning and their influence on voters. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4244– 4275, Miami, Florida, USA. Association for Computa… find
  30. Lee Rainie. 2025. Close encounters of the ai kind: The increasingly human-like way people are engaging with language models. find
  31. Luca Rettenberger, Markus Reischl, and Mark Schutera. 2025. Assessing political bias in large language models. Journal of Computational Social Science, 8(2):1– 17. find
  32. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. Llms-as-judges: A comprehensive survey on llm-based evaluation methods.
  33. Weixin Liang, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, and James Zou. 2025. The widespread adoption of large language model-assisted writing across society. find
  34. David Rozado. 2024. The political preferences of llms. PLOS ONE, 19(7):e0306621. find
  35. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML 2023, pages 29971–30004, Honolulu, HI, USA. PMLR.
  36. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A. Rothkopf, and Kristian Kersting. 2022. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 4(3):258–268. find
  37. Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. 2024. Self-preference bias in llm-as-a-judge.
  38. Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V. Chawla, and Xiangliang Zhang. 2025. Justice or prejudice? quantifying biases in LLM-as-a-judge. In The Thirteenth International Conference on Learning Representations.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/piedrahita-2026-democratic-or-authoritarian-probing-new/