Elamur Research⌕
☰
home / papers / 2026 / Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities

Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities

Paper · 2026 · Wajdi Zaghouani

Summary

LLMs are increasingly used in social science and humanities (SSH) work for annotation, qualitative coding and survey simulation. However, they are judged mainly by task-oriented NLP benchmarks, which reveal little about interpretive validity or cultural fit, and this gap grows outside English. In this position paper, Zaghouani argues that multilingual reasoning LLMs are not neutral tools but hermeneutic instruments that shape how meaning gets produced. The argument draws on hermeneutics (Gadamer, Ricoeur), philosophy of technology and STS (Latour, Ihde, Winner), and multilingual NLP. The paper reviews evidence that models lean toward Western, often US-centric, values even when operating in other languages, with Arabic as a recurring case. It then sets out an evaluation framework that defines measurable indicators of how well models fit cultural contexts, how consistently they behave across languages and how faithful their reasoning is, along with transparency requirements for interpretive tasks. A multilingual political discourse analysis scenario illustrates the framework. No new corpus or model is released. The contribution is a conceptual and methodological basis for responsibly embedding LLMs in comparative, multilingual research pipelines.

Key points

  • The framework centres on three operationalized dimensions: cultural alignment, cross-lingual stability and reasoning faithfulness, plus transparency rules for interpretive research.
  • The paper cites CAMeL (Naous et al., 2024), which shows that both multilingual and Arabic-specific models favour Western entities when prompted in Arabic. Naous and Xu (2025) link part of this bias to frequency-based tokenization and polysemy, and find smaller gaps when models are tested in English.
  • It cites Sukiennik et al. (2025), who tested 10 LLMs against 20 countries using Hofstede's Values Survey Module. The outputs drifted toward a moderate middle, and the United States was the best-aligned country.
  • It argues that existing resources fall short for interpretive SSH needs. XNLI targets inference rather than culturally situated interpretation, and MBBQ shows that bias varies by language. The Aya effort covers 101 languages but still leaves disparities.
  • It flags a methodological risk for comparative framing analysis (following Entman): cross-national differences may reflect model priors rather than real differences.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Northwestern University in QatarDoha · Qatar39 works · 148 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 worksGeneral Social Sciences · 254 worksLiterature and Literary Theory · 541 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksMultilingual & cross-lingual NLP · 882 works

Cite

Cite as Wajdi Zaghouani. 2026. Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities. Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2mrtfpgxeerx

@inproceedings{zaghouani2026toward,
  title = {{Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities}},
  author = {Wajdi Zaghouani},
  booktitle = {Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2mrtfpgxeerx},
  url = {https://doi.org/10.63317/2mrtfpgxeerx}
}

References

60 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdurahman, S., Salkhordeh Ziabari, A., Moore, A. K., Bartels, D. M., and Dehghani, M. 2025. A Primer for Evaluating Large Language Models in Social-Science Research. Advances in Methods and Practices in Psychological Science, 8(2). find
  2. Adelani, D. I., Abbott, J., Neubig, G., et al. 2021. MasakhaNER: Named entity recognition for African languages. Transactions of the Association for Computational Linguistics, 9:1116–1131.
  3. Al Emadi, M. M. and Zaghouani, W. 2024. Emotional Toll and Coping Strategies: Navigating the Effects of Annotating Hate Speech Data. In Proceedings of the Workshop on Legal and Ethical Issues in Human Language Technologies @ LREC-COLING 2024, 66–72. find
  4. Charfi, A., Bessghaier, M., Atalla, A., Akasheh, R., Al-Emadi, S., and Zaghouani, W. 2024b. Stance Detection in Arabic with a Multi-Dialectal Cross-Domain Stance Corpus. Social Network Analysis and Mining, 14(1), 161. find
  5. Chen, G. H., Chen, S., Liu, Z., Jiang, F., and Wang, B. 2024. Humans or LLMs as the judge? A study on judgement bias. In Proceedings of EMNLP 2024, 8301–8327.
  6. Chong, D. and Druckman, J. N. 2007. Framing theory. Annual Review of Political Science, 10:103– 126. find
  7. Chowdhery, A., Narang, S., Devlin, J., et al. 2023. PaLM: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.
  8. Argyle, L. P., Busby, E., Fulda, N., Gubler, J., Rytting, C., and Wingate, D. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.
  9. Cicchetti, D. V. 1994. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4):284–290. find
  10. Bender, E. M. and Friedman, B. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. find
  11. Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman, S., Schwenk, H., and Stoyanov, V. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of EMNLP 2018, 2475–2485. find
  12. Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of FAccT 2021, 610–623. find
  13. Egami, N., Hinck, M., Stewart, B., and Wei, H. 2024. Using large language model annotations for the social sciences: A general framework with applications. Working paper. find
  14. Blasi, D. E., Anastasopoulos, A., and Neubig, G. 2022. Systematic inequalities in language technology performance across the world’s languages. In Proceedings of ACL 2022.
  15. Entman, R. M. 1993. Framing: Toward clarification of a fractured paradigm. Journal of Communication, 43(4):51–58. find
  16. Bommasani, R., Hudson, D. A., Adeli, E., et al. 2021. On the opportunities and risks of foundation models. arXiv:2108.07258.
  17. Brown, T. B., Mann, B., Ryder, N., et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, 33.
  18. Carlson, N. A. and Burbano, V. 2025. The use of LLMs to annotate data in management research: Foundational guidelines and warnings. Strategic Management Journal. find
  19. Charfi, A., Bessghaier, M., Akasheh, R., Atalla, A., and Zaghouani, W. 2024a. Hate Speech Detection with ADHAR: A Multi-Dialectal Hate Speech Corpus in Arabic. Frontiers in Artificial Intelligence, 7, Article 1391472. find
  20. Gadamer, H. G. 1975. Truth and Method. Seabury Press. find
  21. Gilardi, F., Alizadeh, M., and Kubli, M. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30):e2305016120.
  22. Gu, J., Jiang, X., et al. 2024. A survey on LLM-asa-Judge. arXiv:2411.15594. find
  23. Halterman, A. and Keith, K. A. 2024. Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts. arXiv:2407.10747. — [arXiv:2407.10747](https://arxiv.org/abs/2407.10747) find
  24. Hayes, A. S. 2025. Conversing with qualitative data: Enhancing qualitative research through large language models. International Journal of Qualitative Methods. find
  25. Heseltine, M. and Clemm von Hohenberg, B. 2024. Large language models as a substitute for human experts in annotating political text. Research and Politics, 11(1):1–10. find
  26. Naous, T. and Xu, W. 2025. On the origin of cultural biases in language models: From pre-training data to linguistic phenomena. In Proceedings of NAACL 2025.
  27. Ihde, D. 1990. Technology and the Lifeworld: From Garden to Earth. Indiana University Press. find
  28. Neplenbroek, V., Bisazza, A., and Fernández, R. 2024. MBBQ: A dataset for cross-lingual comparison of stereotypes in generative LLMs. In Proceedings of COLM 2024. find
  29. Joshi, P., Santy, S., Budhiraja, A., Bali, K., and Choudhury, M. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of ACL 2020, 6282–6293.
  30. Ollion, E., Shen, R., Macanovic, A., and Chatelain, A. 2023. ChatGPT for text annotation? Mind the hype! SocArXiv. find
  31. Krippendorff, K. 2004. Content Analysis: An Introduction to Its Methodology (2nd ed.). Sage Publications. find
  32. Orife, I., Kreutzer, J., Sibanda, B., et al. 2020. Masakhane – Machine translation for Africa. arXiv:2003.11529. — [arXiv:2003.11529](https://arxiv.org/abs/2003.11529) find
  33. Kuhn, J. 2019. Computational text analysis within the Humanities: How to combine working practices from the contributing fields? Language Resources and Evaluation, 53(4):565–602. find
  34. Rathje, S., Mirea, D. M., Sucholutsky, I., et al. 2024. GPT is an effective tool for multilingual psychological text analysis. Proceedings of the National Academy of Sciences, 121(34):e2308950121. find
  35. Laabar, S. and Zaghouani, W. 2024. Multi-Dimensional Insights: Annotated Dataset of Stance, Sentiment, and Emotion in Facebook Comments on Tunisia’s July 25 Measures. In Proceedings of the Second Workshop on NLP for Political Sciences @ LREC-COLING 2024, 22–32. find
  36. Ricoeur, P. 1976. Interpretation Theory. Texas Christian University Press. find
  37. Lanham, T., Chen, A., Radhakrishnan, A., et al. 2023. Measuring faithfulness in chain-of-thought reasoning. arXiv:2307.13702.
  38. Latour, B. 1987. Science in Action. Harvard University Press. find
  39. Li, C., Chen, M., Wang, J., Sitaram, S., and Xie, X. 2024. CultureLLM: Incorporating cultural differences into large language models. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024).
  40. Liang, P., Bommasani, R., Lee, T., et al. 2023. Holistic evaluation of language models. Transactions on Machine Learning Research. find
  41. Linegar, M., Kocielnik, R., and Alvarez, R. M. 2023. Large language models and political science. Frontiers in Political Science, 5:1257092. find
  42. Mitchell, M., Wu, S., Zaldivar, A., et al. 2019. Model cards for model reporting. In Proceedings of FAccT 2019, 220–229. find
  43. Moretti, F. 2013. Distant Reading. Verso Books. find
  44. Naous, T., Ryan, M. J., Ritter, A., and Xu, W. 2024. Having beer after prayer? Measuring cultural bias in large language models. In Proceedings of ACL 2024, 16366–16393.
  45. Rockwell, G. and Sinclair, S. 2016. Hermeneutica: Computer-Assisted Interpretation in the Humanities. MIT Press. find
  46. Rystrøm, J., Kirk, H. R., and Hale, S. 2025. Multilingual != Multicultural: Evaluating gaps between multilingual capabilities and cultural alignment in LLMs. arXiv:2502.16534. find
  47. Singh, S., et al. 2024. Aya Dataset: An open-access collection for multilingual instruction tuning. arXiv:2402.06619.
  48. Sukiennik, N., et al. 2025. An evaluation of cultural value alignment in LLM. arXiv:2504.08863. — [arXiv:2504.08863](https://arxiv.org/abs/2504.08863) find
  49. Tao, Y., Viberg, O., Baker, R. S., and Kizilcec, R. F. 2024. Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9).
  50. Tornberg, P. 2023. ChatGPT-4 outperforms experts and crowd workers in annotating political Twitter messages with zero-shot learning. arXiv:2304.06588.
  51. Touvron, H., Lavril, T., Izacard, G., et al. 2023. LLaMA: Open and efficient foundation language models. arXiv:2302.13971.
  52. Turpin, M., Michael, J., Perez, E., and Bowman, S. R. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems.
  53. Üstün, A., Aryabumi, V., Yong, Z. X., et al. 2024. Aya model: An instruction finetuned open-access multilingual language model. arXiv:2402.07827.
  54. Vallejo Vera, S., et al. 2025. LLMs as annotators: The effect of party cues on labelling decisions by large language models. find
  55. Humanities and Social Sciences Communications. van Zundert, J. 2016. Screwmeneutics and hermenumericals: The computationality of hermeneutics. In A New Companion to Digital Humanities, Wiley-Blackwell. find
  56. Wei, J., Wang, X., Schuurmans, D., et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  57. Wilkerson, J. and Casas, A. 2017. Large-scale computerized text analysis in political science: Opportunities and challenges. Annual Review of Political Science, 20:529–544. find
  58. Winner, L. 1980. Do artifacts have politics? Daedalus, 109(1):121–136. find
  59. Zheng, L., Chiang, W. L., Sheng, Y., et al. 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems.
  60. Ziems, C., Held, W., Shaikh, O., et al. 2024. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2026-responsible-epistemically-grounded-multilingual-llms/