Elamur Research⌕
☰
home / papers / 2026 / Distributional Alignment for Large Language Models under Domain Shift

Distributional Alignment for Large Language Models under Domain Shift

Paper · 2026 · Viet Thanh Pham, Lizhen Qu, Zhuang Li et al.

Summary

The paper addresses distribution prediction for LLMs: estimating the share of a country's population that would choose each option of a multiple-choice survey question. Existing methods are unreliable, especially outside familiar domains and cultures. Token log-probabilities depend on option wording, repeated sampling is costly and prompt-sensitive, and verbalized distributions tend to be poorly calibrated.

The authors propose Evi-DA, which grounds predictions in World Values Survey (WVS) data. It builds an evidence bank of WVS items, each with group-level answer distributions and a discretized (low/medium/high) Welzel value signature per option. For a new question, the system:

  1. Retrieves similar, well-supported items.
  2. Has the model predict value signatures for the unseen options.
  3. Outputs a JSON distribution conditioned on that evidence.

Training uses a two-stage GRPO procedure with survey-based rewards. The rewards cover value accuracy, distribution fidelity, output format and cultural bias.

Across six open-source backbones and seven baselines, Evi-DA gives the lowest Jensen–Shannon divergence both in-domain and out-of-domain. This makes it a more robust tool for survey piloting and population simulation.

Key points

  • Value representation: Welzel values cover Secular and Emancipative dimensions, split into 8 sub-indices. Each answer option is summarized by the mean value profile of the respondents who chose it, then binned into LMH levels.
  • Retrieval: Candidate items are ranked by cosine similarity of text-encoder embeddings (top-K = 10). Items with too few valid respondents in the target group are filtered out.
  • Evaluation data: 2,403 country-conditioned questions, of which 2,203 are in-domain and 200 are out-of-domain. The ethics statement also mentions Pew Global Attitudes data.
  • Results: Average JSD is 0.16 in-domain and 0.23 out-of-domain. This is up to 44% lower than token log-probability estimation and 21% lower than the best baseline under shift. GRPO training usually adds further gains.
  • Limitations: Only WVS values are used in training, and there is no expert judgment of what error level is acceptable. Code and data are available at github.com/thanhpv2102/Evi-DA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Monash UniversityMelbourne · Australia60 works · 54,866 citations🎓RMIT UniversityMelbourne · Australia17 works · 703 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Viet Thanh Pham, Lizhen Qu, Zhuang Li and Gholamreza Haffari. 2026. Distributional Alignment for Large Language Models under Domain Shift. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1026

@inproceedings{pham2026distributional,
  title = {{Distributional Alignment for Large Language Models under Domain Shift}},
  author = {Viet Thanh Pham, Lizhen Qu, Zhuang Li and Gholamreza Haffari},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1026},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1026}
}

References

31 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2022. World values survey: All rounds – country-pooled datafile. Dataset. find
  2. Bhavik Agarwal, Ishan Joshi, and Viktoria Rojkova. 2025. Think inside the json: Reinforcement strategy for strict llm schema adherence. Preprint, arXiv:2502.14905. find
  3. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.
  4. Nicolas Bougie and Narimasa Watanabe. 2025. Citysim: Modeling urban behaviors and city dynamics with large-scale llm-driven agent simulation. Preprint, arXiv:2506.21805. find
  5. Yong Cao, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovich. 2025. Specializing large language models to simulate survey response distributions for global populations. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association f…
  6. Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, and Yejin Choi. 2025. CulturalBench: A robust, diverse and challenging benchmark for measuring LMs’ cultural knowledge through human-AI red-teami…
  7. Yun-Shiuan Chuang, Agam Goyal, Nikunj Harlalka, Siddharth Suresh, Robert Hawkins, Sijia Yang, Dhavan Shah, Junjie Hu, and Timothy Rogers. 2024. Simulating opinion dynamics with networks of LLM-based agents. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3326– 3346, M… find
  8. Jukka Corander, Ulpu Remes, and Timo Koski. 2021. On the jensen-shannon divergence and the variation distance for categorical probability distributions. Kybernetika, 57(6):879–907. find
  9. Esin DURMUS, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the…
  10. Ashutosh Dwivedi, Siddhant Shivdutt Singh, and Ashutosh Modi. 2025. EtiCor++: Towards understanding etiquettical bias in LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9355–9376, Vienna, Austria. Association for Computational Linguistics. find
  11. Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. 2024. Modular pluralism: Pluralistic alignment via multi-LLM collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4151–4171, Miami, F…
  12. Antonino Ferraro, Antonio Galli, Valerio La Gatta, Marco Postiglione, Gian Marco Orlando, Diego Russo, Giuseppe Riccio, Antonio Romano, and Vincenzo Moscato. 2024. Agent-based modelling meets generative ai in social network simulations. Preprint, arXiv:2411.16031. — [arXiv:2411.16031](https://arxiv.org/abs/2411.16031) find
  13. Aaron Grattafiori and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  14. Reem Masoud, Ziquan Liu, Martin Ferianc, Philip C. Treleaven, and Miguel Rodrigues Rodrigues. 2025. Cultural alignment in large language models: An explanatory analysis based on hofstede’s cultural dimensions. In Proceedings of the 31st International Conference on Computational Linguistics, pages 84…
  15. Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2025. Benchmarking distributional alignment of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long P…
  16. Mistral AI. 2025. Introducing mistral 3. Mistral AI blog post. find
  17. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2024. Having beer after prayer? measuring cultural bias in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16366–16393, Bangkok, Thailand. Associatio…
  18. Pew Research Center. 2025. find
  19. Spring 2024 survey data. Global Attitudes Survey (35-country survey conducted Jan 5–May 22, 2024). Accessed 2026-01-06. find
  20. Viet Thanh Pham, Zhuang Li, Lizhen Qu, and Gholamreza Haffari. 2025a. CultureInstruct: Curating multi-cultural instructions at scale. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1:… find
  21. Viet Thanh Pham, Lizhen Qu, Zhuang Li, Suraj Sharma, and Gholamreza Haffari. 2025b. SurveyPilot: an agentic framework for automated human opinion collection from social media. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 43… find
  22. Viet Thanh Pham, Shilin Qu, Farhad Moghimifar, Suraj Sharma, Yuan-Fang Li, Weiqing Wang, and Reza Haf. 2024. Multi-cultural norm base: Frame-based norm discovery in multi-cultural settings. In Proceedings of the 28th Conference on Computational Natural Language Learning, pages 24–35, Miami, FL, USA.… find
  23. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.
  24. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  25. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024. Position: a roadmap to pluralistic alignment. In Proceedings of the 41st International Conference on M…
  26. Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9). find
  27. Cheng Wang, Gyuri Szarvas, Georges Balazs, Pavel Danchenko, and Patrick Ernst. 2024. Calibrating verbalized probabilities for large language models. Preprint, arXiv:2410.06707. — [arXiv:2410.06707](https://arxiv.org/abs/2410.06707) find
  28. Minghan Wang, Viet Thanh Pham, Farhad Moghimifar, and Thuy-Trang Vu. 2025. Proverbs run in pairs: Evaluating proverb translation capability of large language model. In Findings of the Association for Computational Linguistics: ACL 2025, pages 1646–1662, Vienna, Austria. Association for Computational… find
  29. Christian Welzel. n.d. Constructing indices for secular and emancipative values. World Values Survey Association. Accessed 2025-12-29. find
  30. An Yang, Anfeng Li, Baosong Yang, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  31. Jiayi Zhang, Simon Yu, Derek Chong, Anthony Sicilia, Michael R. Tomz, Christopher D. Manning, and Weiyan Shi. 2025. Verbalized sampling: How to mitigate mode collapse and unlock llm diversity. Preprint, arXiv:2510.01171. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pham-2026-distributional-alignment-large-language-models/