Elamur Research⌕
☰
home / papers / 2026 / When Do Language Models Endorse Limitations on Human Rights Principles?

When Do Language Models Endorse Limitations on Human Rights Principles?

Paper · 2026 · Keenan Samway, Miu Nicole Takagi, Rada Mihalcea et al.

Summary

LLMs are increasingly used in courts, asylum processing and content moderation. In these settings, human rights often have to be weighed against other goals. This paper asks how readily such models approve government actions that restrict rights guaranteed by the Universal Declaration of Human Rights (UDHR). The authors used GPT-4.1 to build 144 English scenarios, each describing a government curbing a specific right. The scenarios vary the article involved, the severity of the restriction and the emergency context. They were translated into eight languages, giving 1,152 items, and posed to eleven LLMs. Answers were collected both on a Likert scale and as free text, and GPT-4.1 classified the free-text answers. Models approved restrictions on economic, social and cultural rights more easily than restrictions on civil and political ones. Approval was higher in Chinese and Hindi than in English or Romanian. Results also depended heavily on answer format, emergency framing and persona prompts. The authors conclude that rights-related behaviour of LLMs does not generalise across contexts. They argue for multilingual, multi-format alignment testing and human oversight.

Key points

  • Dataset: 144 base scenarios × 8 languages = 1,152 items, covering 24 UDHR articles.
    • Political/civil rights: articles 4–21.
    • Economic/social/cultural rights: articles 22–27.
    • Only the mildest (1) and harshest (3) severity levels were analysed.
    • The languages include six high-resource and two low-resource ones.
  • Rights most and least protected:
    • Limits on movement (Art. 13) and education (Art. 26) were accepted most often.
    • Freedom from torture (Art. 5), presumption of innocence (Art. 11) and recognition before the law (Art. 6) were defended most strongly.
  • Emergency framing: a natural-disaster emergency raised approval of restrictions compared with civil unrest or ordinary conditions.
  • Persona steering: persona prompts casting the model as a defender of personal liberties, or alternatively as an advocate for state power, shifted endorsement scores substantially.
  • Format sensitivity: Likert and open-ended answers frequently diverged.
  • Resources: code is released at github.com/keenansamway/llm-human-rights. GPT-4.1 served as both scenario generator and judge, which the authors acknowledge as a possible source of bias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CopenhagenCopenhagen · Denmark115 works · 15,292 citations🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations🔬Max Planck SocietyMunich · Germany16 works · 42,253 citations🔬Max Planck Institute for Intelligent SystemsStuttgart · Germany8 works · 854 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 worksPolitical Science and International Relations · 182 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksLegal AI · 134 works

Cite

Cite as Keenan Samway, Miu Nicole Takagi, Rada Mihalcea, Bernhard Schölkopf, Ilias Chalkidis, Daniel Hershcovich and Zhijing Jin. 2026. When Do Language Models Endorse Limitations on Human Rights Principles?. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.347

@inproceedings{samway2026when,
  title = {{When Do Language Models Endorse Limitations on Human Rights Principles?}},
  author = {Keenan Samway, Miu Nicole Takagi, Rada Mihalcea, Bernhard Schölkopf, Ilias Chalkidis, Daniel Hershcovich and Zhijing Jin},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.347},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.347}
}

References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marwa Abdulhai, Gregory Serapio-García, Clement Crepy, Daria Valter, John Canny, and Natasha Jaques. 2024. Moral foundations of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17737–17752, Miami, Florida, USA. Association for Computational Linguistics.
  2. Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. Investigating cultural alignment of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422, Bangkok, Thailand. Association for Computational Linguistics.
  3. UN General Assembly and 1 others. 1948. Universal declaration of human rights. UN General Assembly, 302(2):14–25. find
  4. Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring political bias in large language models: What is said and how it is said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 11142–11159. Association for Computational Linguistics.
  5. Michal Bravansky, Filip Trhlík, and Fazl Barez. 2025. Rethinking AI cultural alignment. In ICLR 2025 Workshop on Bidirectional Human-AI Alignment. ↗
  6. Ilias Chalkidis. 2025. Decoding alignment: A critical survey of llm development initiatives through value-setting and data-centric lens. Preprint, arXiv:2508.16982. ↗
  7. Ilias Chalkidis and Stephanie Brandl. 2024. Llama meets EU: Investigating the European political spectrum through the lens of LLMs. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 481–498, Mexico City, Mexico. Association for Computational Linguistics. ↗
  8. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4299–4307. ↗
  9. Council of Europe. 1950. European convention on human rights. As amended by Protocols Nos. 11, 14 and 15. find
  10. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023a. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 11737–11762. Association for Computational Linguistics.
  11. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023b. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11737–11762, Toronto, Canada. Association for Computational Linguistics.
  12. Jillian Fisher, Ruth E. Appel, Chan Young Park, Yujin Potter, Liwei Jiang, Taylor Sorensen, Shangbin Feng, Yulia Tsvetkov, Margaret E. Roberts, Jennifer Pan, Dawn Song, and Yejin Choi. 2025. Political neutrality in AI is impossible- but here is how to approximate it. CoRR, abs/2503.05728. ↗
  13. Rafiya Javed, Jackie Kay, David Yanni, Abdullah Zaini, Anushe Sheikh, Maribeth Rauh, Ramona Comanescu, Iason Gabriel, and Laura Weidinger. 2025. Do llms exhibit demographic parity in responses to queries about human rights? CoRR, abs/2502.19463. ↗
  14. Zhijing Jin, Max Kleiman-Weiner, Giorgio Piatti, Sydney Levine, Jiarui Liu, Fernando Gonzalez Adauto, Francesco Ortu, András Strausz, Mrinmaya Sachan, Rada Mihalcea, Yejin Choi, and Bernhard Schölkopf. 2025. Language model alignment in multilingual trolley problems. In The Thirteenth International Conference on Learning Representations. ↗
  15. George Kateb. 2014. Human Dignity. Harvard University Press, Cambridge, MA and London, England. find
  16. Ariba Khan, Stephen Casper, and Dylan Hadfield-Menell. 2025. Randomness, not representation: The unreliability of evaluating cultural alignment in llms. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT 2025, Athens, Greece, June 23-26, 2025, pages 2151–2165. ACM. ↗
  17. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew M. Bean, Katerina Margatina, Rafael Mosquera Gómez, Juan Ciro, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott Hale. 2024. The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024. find
  18. Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. Culturellm: Incorporating cultural differences into large language models. In Advances in Neural Information Processing Systems, volume 37, pages 84799–84838. Curran Associates, Inc.
  19. Yinhong Liu, Zhijiang Guo, Tianya Liang, Ehsan Shareghi, Ivan Vulić, and Nigel Collier. 2025. Aligning with logic: Measuring, evaluating and improving logical preference consistency in large language models. In Forty-second International Conference on Machine Learning. ↗
  20. Meta. 2025. More speech and fewer mistakes. find
  21. Rada Mihalcea, Oana Ignat, Longju Bai, Angana Borah, Luis Chiruzzo, Zhijing Jin, Claude Kwizera, Joan Nwatu, Soujanya Poria, and Thamar Solorio. 2024. Why AI is WEIRD and should not be this way: Towards AI for everyone, with everyone, by everyone. CoRR, abs/2410.16315. ↗
  22. Michal Mochtak. 2024. Chasing the authoritarian spectre: Detecting authoritarian discourse with large language models. European Journal of Political Research, n/a(n/a). ↗
  23. Grigor Nalbandyan, Rima Shahbazyan, and Evelina Bakhturina. 2025. SCORE: Systematic COnsistency and robustness evaluation for large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track), pages 470–484, Albuquerque, New Mexico. Association for Computational Linguistics. ↗
  24. OpenAI. 2023. Using gpt-4 for content moderation. find
  25. David Guzman Piedrahita, Irene Strauss, Bernhard Schölkopf, Rada Mihalcea, and Zhijing Jin. 2025. Democratic or authoritarian? probing a new dimension of political biases in large language models. Preprint, arXiv:2506.12758. ↗
  26. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. ↗
  27. Siqi Shen, Mehar Singh, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Rada Mihalcea. 2025. Revisiting llm value probing strategies: Are they robust and expressive? Preprint, arXiv:2507.13490. ↗
  28. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L. Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024. Position: A roadmap to pluralistic alignment. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net.
  29. Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, and Monojit Choudhury. 2023. Probing the moral development of large language models through defining issues test. CoRR, abs/2309.13356. ↗
  30. UK Home Office. 2024. Evaluation of ai trials in the asylum decision making process. find
  31. Aleksandra Urman and Mykola Makhortykh. 2025. The silence of the llms: Cross-lingual analysis of guardrail-related political bias and false information prevalence in chatgpt, google bard (gemini), and bing chat. Telematics Informatics, 96:102211. ↗
  32. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  33. Xinhua. 2025. China’s local judicial systems embrace ai to improve efficiency. find
  34. Neemesh Yadav, Jiarui Liu, Francesco Ortu, Roya Ensafi, Zhijing Jin, and Rada Mihalcea. 2025. Revealing hidden mechanisms of cross-country content moderation with natural language processing. CoRR, abs/2503.05280. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/samway-2026-when-do-language-models-endorse/