Elamur Research⌕
☰
home / papers / 2026 / Carefully Considering Culture: Analyzing LLM Alignment in Single- and Multi-Cultural Settings using Cultural Consensus Theory

Carefully Considering Culture: Analyzing LLM Alignment in Single- and Multi-Cultural Settings using Cultural Consensus Theory

Paper · 2026 · Krishna Pothugunta and John P. Lalor

Summary

Most survey-based evaluations of LLM cultural alignment compare answer distributions at the country level. This hides two things: how strongly people within a country actually agree, and whether a country contains several cultures. Pothugunta and Lalor address this gap with cultural consensus theory (CCT), a method from anthropology. CCT derives a consensus answer key, per-respondent competence scores and the variance explained by a shared factor, all from an agreement matrix.

They apply it to World Values Survey Wave 7 data across 10 countries and 12 domains. Countries are split into single- and multi-culture groups by ethnic fractionalization. Responses come from ten LLMs under six country-steering prompts. The models are scored both as members of the human population and as a separate stand-in population.

The analysis finds three domain-dependent failure patterns:

  • Consensus Gap: models fail to agree among themselves (e.g., HWB).
  • Heterogeneity Gap: models converge on a consensus that does not match the human one (e.g., POST).
  • Consensus Inflation: models match the human consensus but overstate its strength (e.g., POC).

The approach provides diagnostics of when LLMs flatten real human diversity into homogenized answers.

Key points

  • Data and models: WVS Wave 7 was used with 12 domains (e.g., EV, HWB, POC, POST, PCPR, POS, SC-TOM).
    • Single-culture countries: Japan, Armenia, Germany, Greece, Netherlands.
    • Multi-culture countries: Colombia, Mexico, Malaysia, Peru, United States.
    • Ten LLMs (including GPT-4o, GPT-OSS:120B, Llama3.1:70B, Qwen3:32B and Phi3:instruct) were queried with 6 prompts at temperature 0, giving 60 response rows per setting.
  • Method: Competence comes from the first eigenvector of the agreement matrix, normalized by its 99th percentile. Consensus answers are competence-weighted averages. CCT models were fit with AnthroTools 2.0.
  • New metrics: Consensus Consistency (CC) is the share of items where LLM and human consensus answers match after rounding. ∆VE is LLM minus human variance explained; a positive value signals inflated consensus.
  • Single- vs. multi-culture: After FDR correction, CC shows no significant difference between the two groups, but ∆VE does in some domains. In HWB, ∆VE moves from −0.305 in single-culture countries to a much smaller gap in multi-culture ones (q = 0.036).
  • Availability: Code and data are released at github.com/nd-ball/llm-alignment-cct.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Notre DameNotre Dame · United States59 works · 37,127 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksSocial Psychology · 330 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Krishna Pothugunta and John P. Lalor. 2026. Carefully Considering Culture: Analyzing LLM Alignment in Single- and Multi-Cultural Settings using Cultural Consensus Theory. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1323

@inproceedings{pothugunta2026carefully,
  title = {{Carefully Considering Culture: Analyzing LLM Alignment in Single- and Multi-Cultural Settings using Cultural Consensus Theory}},
  author = {Krishna Pothugunta and John P. Lalor},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1323},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1323}
}

References

30 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, and 1 others. 2024. Phi-3 technical report: A highly capable language model locally on your phone, 2024. URL https://arxiv.org/abs/2404.14219, 2(6):4. ↗
  2. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  3. Alberto Alesina, Arnaud Devleeschauwer, William Easterly, Sergio Kurlat, and Romain Wacziarg. 2003. Fractionalization. Journal of Economic growth, 8(2):155–194. find
  4. Badr Alkhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. Investigating cultural alignment of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404–12422.
  5. Royce Anders and William H. Batchelder. 2015. Cultural Consensus Theory for the Ordinal Data Case. Psychometrika, 80(1):151–181. find
  6. Jaeryang Baek, Ayana Hussain, Danny Liu, Nicholas Vincent, and Lawrence H Kim. 2025. Open webui: An open, extensible, and usable interface for ai interaction. arXiv preprint arXiv:2510.02546. — [arXiv:2510.02546](https://arxiv.org/abs/2510.02546) find
  7. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  8. Roy G d’Andrade, Richard A Shweder, and Robert A Le Vine. 1984. Cultural meaning systems. Adams, Robert McC, Ed.; And Others Behavioral and Social Science Research: A National, 197. find
  9. Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the…
  10. Kevin Durrheim, Maria Schuld, Martin Mafunda, and Sindisiwe Mazibuko. 2023. Using word embeddings to investigate cultural biases. British Journal of Social Psychology, 62(1):617–629. find
  11. Austin C Kozlowski, Matt Taddy, and James A Evans. 2019. The geometry of culture: Analyzing the meanings of class through word embeddings. American Sociological Review, 84(5):905–949. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  13. Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2025. Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art. Transactions of the Association for Computational Linguistics, 13:652–689.
  14. E.D. Gutiérrez, Ekaterina Shutova, Patricia Lichtenstein, Gerard De Melo, and Luca Gilardi. 2016. Detecting Cross-Cultural Differences Using a Multilingual Topic Model. Transactions of the Association for Computational Linguistics, 4:47–60. find
  15. Wolfgang Messner, Tatum Greene, and Josephine Matalone. 2025. From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models. Journal of Public Policy & Marketing, 44(3):370– 391. find
  16. Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, Bjorn Puranen, and 1 others. 2022. World values survey: Round seven–country-pooled datafile version 5.0. find
  17. Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. 2025. Survey of Cultural Awareness in Language Models: Text and Beyond. Computational Linguistics, pages 1–96.
  18. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, and 1 others. 2022. Challenges and strategies in cross-cultural nlp. In Proceedings of the 60th Annual Meeting of the Assoc…
  19. Benjamin Grant Purzycki and Alastair Jamieson-Lane. 2017. AnthroTools: An R Package for Cross-Cultural Ethnographic Data Analysis. Cross-Cultural Research, 51(1):51–74. find
  20. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  21. Graham M Jones, Shai Satran, and Arvind Satyanarayan. 2025. Toward cultural interpretability: A linguistic anthropological framework for describing and evaluating large language models. Big Data & Society, 12(1):20539517241303118. find
  22. Roger M Keesing. 1974. Theories of culture. Annual review of anthropology, 3:73–97. find
  23. Ariba Khan, Stephen Casper, and Dylan Hadfield-Menell. 2025. Randomness, not representation: The unreliability of evaluating cultural alignment in llms. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pages 2151– 2165. find
  24. Hannah Rose Kirk, Alexander Whitefield, Paul Rottger, Andrew M Bean, Katerina Margatina, Rafael Mosquera-Gomez, Juan Ciro, Max Bartolo, Adina Williams, He He, and 1 others. 2024. The prism alignment dataset: What participatory, representative and individualised human feedback reveals about the subje…
  25. Jiyuan Ren, Zhaocheng Du, Zhihao Wen, Qinglin Jia, Sunhao Dai, Chuhan Wu, and Zhenhua Dong. 2025. Few-shot llm synthetic data with distribution matching. In Companion Proceedings of the ACM on Web Conference 2025, pages 432–441. find
  26. A Kimball Romney, Susan C Weller, and William H Batchelder. 1986. Culture as consensus: A theory of culture and informant accuracy. American anthropologist, 88(2):313–338. find
  27. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schütze, and Dirk Hovy. 2024. Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models. In Proceedings of the 62nd Annual Meeting of the Association…
  28. Sougata Saha, Saurabh Kumar Pandey, Harshit Gupta, and Monojit Choudhury. 2025. Reading between the lines: Can LLMs identify cross-cultural communication gaps? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Languag… find
  29. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971–30004. PMLR.
  30. Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Chosh… — Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pothugunta-2026-carefully-considering-culture-analyzing-llm/