Elamur Research⌕
☰
home / papers / 2026 / The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models

The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models

Paper · 2026 · Konrad Löhr, Shuzhou Yuan and Michael Färber

Summary

LLMs increasingly mediate how people find political information, so hidden ideological skews could quietly influence public opinion. This EACL 2026 Findings paper measures political bias in eight prominent LLMs with the two-dimensional Political Compass Test (PCT), which has an economic left–right axis and a social libertarian–authoritarian axis. The PCT's scoring rules are not public, so the authors first annotated each proposition's direction on both axes, using human annotators and several LLMs. With this transparent scoring, they ran three experiments. First, they recorded each model's baseline position. Second, they used persona prompting to surface explicit stereotypes about social groups. Third, they posed the PCT in several European languages to expose implicit, language-triggered stereotypes.

Every model landed on the left of the compass. Language-induced shifts were generally larger than persona-induced ones. For most models, the two kinds of stereotype pointed in the same direction, which suggests the models internally represent their own biases in a coherent way. The work implies that safety and bias audits run only in English can miss stronger biases that appear in other languages.

Key points

  • Proposition annotations came from undergraduate students and three models: GPT-4.1-mini, Llama-4-Scout-17B-16E-Instruct and Llama-3.3-70B-Instruct. The models disagreed on just 5 of 120 annotations.
  • Each of the 8 models was tested with 19 personas and 7 languages. All languages were European, and the personas included countries from the Anglosphere.
  • Implicit bias produced the most striking shift in Gemini-2.0-flash, which moved toward a right-leaning, authoritarian position when prompted in French.
  • Explicit and implicit stereotypes agreed for most models, but not for Gemini-2.0-flash and Gemini-2.0-flash-lite. For example, GPT-4.1-mini gave nearly identical shifts for an Italian-speaking persona (+0.092 economic, +0.644 social) and for Italian-language propositions (+0.158 economic, +0.644 social).
  • The authors list several limitations: the PCT lacks psychometric validation, the language and region coverage is narrow, and structured prompts differ from real conversations. All generated data is released on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technische Universität DresdenDresden · Germany13 works · 7,336 citations🎓Center for Scalable Data Analytics and Artificial IntelligenceDresden · Germany10 works · 77 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Konrad Löhr, Shuzhou Yuan and Michael Färber. 2026. The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.118

@inproceedings{lohr2026hidden,
  title = {{The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models}},
  author = {Konrad Löhr, Shuzhou Yuan and Michael Färber},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.118},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.118}
}

References

32 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jan Batzner, Volker Stocker, Stefan Schmid, and Gjergji Kasneci. 2024. Germanpartiesqa: Benchmarking commercial large language models for political bias and sycophancy. ArXiv, abs/2407.18008. — [arXiv:2407.18008](https://arxiv.org/abs/2407.18008) find
  2. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics.
  3. Kai Chen, Zihao He, Jun Yan, Taiwei Shi, and Kristina Lerman. 2024. How susceptible are large language models to ideological manipulation? ArXiv, abs/2402.11725. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3290 other…
  5. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  6. DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun — DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence https://github.com/k0nr4dloehr/The-Hidden-Bias-Data Li, Huazuo Gao, Shirong Ma, Wangding Zeng, Xiao Bi, Zihui Gu, Hanwei Xu, Damai Dai, Kai Dong, Liyue Zhang, Yishi Piao, and 21 others. 2024. Deepseek-co… ↗
  7. Wenchao Dong, Assem Zhunis, Dongyoung Jeong, Hyojin Chin, Jiyoung Han, and Meeyoung Cha. 2024. Persona setting pitfall: Persistent outgroup biases in large language models arising from social identity adoption. ArXiv, abs/2409.03843. — [arXiv:2409.03843](https://arxiv.org/abs/2409.03843) find
  8. Mats Faulborn, Indira Sen, Max Pellert, Andreas Spitz, and David Garcia. 2025. Only a little to the left: A theory-grounded measure of political bias in large language models. Preprint, arXiv:2503.16148. find
  9. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. In Annual Meeting of the Association for Computational Linguistics.
  10. Emilio Ferrara. 2023. Genai against humanity: Nefarious applications of generative artificial intelligence and large language models. J. Comput. Soc. Sci., 7:549–569. find
  11. Sasuke Fujimoto and Kazuhiro Takemoto. 2023. Revisiting the political biases of chatgpt. Frontiers in Artificial Intelligence, 6. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  13. Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation. SSRN Electronic Journal. find
  14. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. Ai generates covertly racist decisions about people based on their dialect. find
  15. Hyukhun Koh, Dohyung Kim, Minwoo Lee, and Kyomin Jung. 2024. Can llms recognize toxicity? a structured investigation framework and toxicity metric. In Conference on Empirical Methods in Natural Language Processing. find
  16. Emilia Agis Lerner, Florian E. Dorner, Elliott Ash, and Naman Goel. 2024. Whose preferences? differences in fairness preferences and their impact on the fairness of ai utilizing human feedback. In Annual Meeting of the Association for Computational Linguistics. find
  17. Uwe Messer. 2025. How do people react to political bias in generative artificial intelligence (ai)? Computers in Human Behavior: Artificial Humans, 3:100108. find
  18. Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2023. More human than human: measuring chatgpt political bias. Public Choice, 198:3–23. find
  19. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  20. Milo Phillips-Brown. 2023. Algorithmic neutrality. ArXiv, abs/2303.05103. — [arXiv:2303.05103](https://arxiv.org/abs/2303.05103) find
  21. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schutze, and Dirk Hovy. 2024. Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models. In Annual Meeting of the Association for Computational…
  22. David Rozado. 2024. The political preferences of llms. PLOS ONE, 19. find
  23. Jérôme Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, and Markus Pauly. 2023. The self-perception and political biases of chatgpt. ArXiv, abs/2304.07333. — [arXiv:2304.07333](https://arxiv.org/abs/2304.07333) find
  24. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? Preprint, arXiv:2303.17548.
  25. Nikhil Sharma, Q. V. Liao, and Ziang Xiao. 2024. Generative echo chamber? effect of llm-powered search systems on diverse information seeking. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. find
  26. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Y. Gal, Nicolas Papernot, and Ross Anderson. 2023. The curse of recursion: Training on generated data makes models forget. ArXiv, abs/2305.17493.
  27. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631:755 – 759. find
  28. Christopher Summerfield, Lisa Argyle, Michiel A. Bakker, Teddy Collins, Esin Durmus, Tyna Eloundou, Iason Gabriel, Deep Ganguli, Kobi Hackenburg, Gillian Hadfield, Luke Hewitt, Saffron Huang, Hélène Landemore, Nahema Marchal, Aviv Ovadya, Ariel Procaccia, Mathias Risse, Bruce Schneier, Elizabeth Seg… — [arXiv:2409.06729](https://arxiv.org/abs/2409.06729) find
  29. Raphael Tang, Xinyu Crystina Zhang, Jimmy J. Lin, and Ferhan Ture. 2023. What do llamas really think? revealing preference biases in language model representations. ArXiv, abs/2311.18812. — [arXiv:2311.18812](https://arxiv.org/abs/2311.18812) find
  30. Supriti Vijay, Aman Priyanshu, and Ashique Khudabukhsh. 2024. When neutral summaries are not that neutral: Quantifying political neutrality in llm-generated news summaries. ArXiv, abs/2410.09978. — [arXiv:2410.09978](https://arxiv.org/abs/2410.09978) find
  31. Haoyi Xiong, Jiang Bian, Yuchen Li, Xuhong Li, Mengnan Du, Shuaiqiang Wang, Dawei Yin, and Sumi Helal. 2024. When search engine services meet large language models: Visions and challenges. IEEE Transactions on Services Computing, 17:4558–4577. find
  32. Shuzhou Yuan, Ercong Nie, Mario Tawfelis, Helmut Schmid, Hinrich Schütze, and Michael Färber. 2025. Hateful person or hateful model? investigating the role of personas in hate speech detection by large language models. arXiv preprint arXiv:2506.08593. — [arXiv:2506.08593](https://arxiv.org/abs/2506.08593) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lohr-2026-hidden-bias-study-explicit-implicit/