Elamur Research⌕
☰
home / papers / 2026 / Do LLMs Transfer Political Framing across Languages? A Cross-Lingual Analysis of LLM-Generated Discourse

Do LLMs Transfer Political Framing across Languages? A Cross-Lingual Analysis of LLM-Generated Discourse

Paper · 2026 · Nooredeen Awwad, Ebtihal Ismail Enfes and Kolawole John Adebayo

Summary

The paper asks whether a multilingual LLM frames the same political topic differently depending on the language it is prompted in. The authors use migration as a test case and compare English and Arabic. Ten argumentative prompts were written in English and rendered into Arabic by native speakers, who aimed for equivalent meaning rather than literal wording. Two models, GPT-4o and the Arabic-centric Jais-13B, each produced 200 outputs per language, giving 800 texts in total, all under identical decoding settings. Each output was assigned a dominant frame using hand-built keyword lexicons for each language, normalised by text length. The authors then ran a chi-square test and a qualitative reading of the outputs. Frame distribution depended strongly on language. Security framing led in both languages. English leaned more toward institutional and electoral themes, while Arabic concentrated on security with humanitarian and communal undertones. The authors call this effect cross-lingual framing drift. They argue that bias and alignment checks done only in English cannot be assumed to hold in other languages.

Key points

  • Design: 2 models (GPT-4o, Jais-13B) × 2 languages (English, Arabic) × 200 generations, for N = 800. Decoding was fixed: temperature 0.9, max 120 tokens, presence and frequency penalty 0.3.
  • Frame scheme: Security/Legal, Humanitarian/Communal, Political/Institutional, plus Other for outputs with no lexicon hits, mixed signals or weak evidence. Ties and zero-match cases were resolved by manual review.
  • Lexicon reliability was checked by manually annotating a random sample of 50 outputs per language.
  • Language and frame were clearly associated: χ² = 43.32, p = 2.11 × 10⁻⁹. In the quantitative tally, Arabic outputs had no Political/Institutional-dominant cases, while English showed a broader spread.
  • Stated limitations:
    • The method is lexicon-only, with no embedding or classifier comparison. This matters especially for Arabic, where framing often sits in discourse structure.
    • The study covers one topic and only two models.
    • The authors stress that the differences concern framing emphasis, not partisan polarity.
  • Data, code and settings are released on GitHub.

Interesting ideas

  • I1. Keep every generation setting fixed and vary only the input language, using meaning-preserving rather than literal prompt translations, so any distribution shift can be blamed on language alone.
  • I2. Sample many outputs at high temperature to expose a model's latent preferences as a distribution instead of trusting one deterministic answer.
  • I3. Give the coder an explicit Other bucket for no-signal or mixed cases, and route ties to human review, rather than forcing a label.
  • I4. Keep the interpretive lens (security vs. humanitarian vs. institutional) separate from ideological stance, so that a difference in emphasis is not misread as bias.
  • I5. Validate an automatic coder against a small manually annotated random sample per condition before trusting large-scale counts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Mova could run text → graph → text and translation through the meta-world, then check that frame and role distributions stay the same, treating any drift as a measurable defect.
  • I2 (learn): sample LLM teacher tiers repeatedly on the same item to estimate label spread and systematic teacher bias before accepting their output as silver data.
  • I3 (frames): prag could emit an explicit undecided frame or speech-act label when evidence is weak or conflicting, keeping uncertainty visible and explainable instead of guessing.
  • I4 (world): store perspective/emphasis and evaluative stance as distinct attributes of a text's claims, so answers can say how something is framed without implying whose side the text takes.
  • I5 (learn): add per-domain manual spot-checks of silver parses and frames as a cheap control alongside the agreement of witness parsers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of Ireland, MaynoothMaynooth · Ireland5 works · 10 citations🎓University of KentCanterbury · United Kingdom2 works · 1 citation

Also on the byline: Eastern Mediterranean University.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSemitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 worksDiscourse analysis · 466 works

Cite

Cite as Nooredeen Awwad, Ebtihal Ismail Enfes and Kolawole John Adebayo. 2026. Do LLMs Transfer Political Framing across Languages? A Cross-Lingual Analysis of LLM-Generated Discourse. Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3azkyw9gr5hf

@inproceedings{awwad2026do,
  title = {{Do LLMs Transfer Political Framing across Languages? A Cross-Lingual Analysis of LLM-Generated Discourse}},
  author = {Nooredeen Awwad, Ebtihal Ismail Enfes and Kolawole John Adebayo},
  booktitle = {Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026)},
  year = {2026},
  doi = {10.63317/3azkyw9gr5hf},
  url = {https://doi.org/10.63317/3azkyw9gr5hf}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Goshi Aoki. 2024. Large language models in politics and democracy: A comprehensive survey. ↗
  2. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. ↗
  3. Aeron Davis. 2010. Political communication and social theory. Routledge. ↗
  4. Frank Esser and Barbara Pfetsch. 2004. Comparing political communication: Theories, cases, and challenges. Cambridge University Press. find
  5. Jillian Fisher, Shangbin Feng, Robert Aron, Thomas Richardson, Yejin Choi, Daniel W Fisher, Jennifer Pan, Yulia Tsvetkov, and Katharina Reinecke. 2025. Biased llms can influence political decision-making. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6559–6607.
  6. Lance Calvin Lim Gamboa, Yue Feng, and Mark G. Lee. 2025. Social bias in multilingual language models: A survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27857–27880, Suzhou, China. Association for Computational Linguistics. find
  7. William Guey, Pierrick Bougault, Vitor D. de Moura, Wei Zhang, and Jose O. Gomes. 2025. Mapping geopolitical bias in 11 large language models: A bilingual, dual-framing analysis of u.s.-china tensions. arXiv preprint arXiv:2503.XXXXX. Submitted 31 Mar 2025. find
  8. Chadi Helwe, Oana Balalau, and Davide Ceolin. 2025. Navigating the political compass: Evaluating multilingual LLMs across languages and nationalities. In Findings of the Association for Computational Linguistics: ACL 2025, pages 17179–17204, Vienna, Austria. Association for Computational Linguistics. find
  9. Michał Krzyżanowski, Anna Triandafyllidou, and Ruth Wodak. 2018. The mediatization and the politicization of the “refugee crisis” in europe. find
  10. Lincan Li, Jiaqi Li, Catherine Chen, Fred Gui, Hongjia Yang, Chenxiao Yu, Zhengguang Wang, Jianing Cai, Junlong Aaron Zhou, Bolin Shen, et al. 2024. Political-llm: Large language models in political science. arXiv preprint arXiv:2412.06864. ↗
  11. Damjan Mandelc. 2025. Religion, migration, and the far-right: How european populism frames religious pluralism. Religions, 16(9):1192. ↗
  12. Afrozah Nadeem, Mark Dras, and Usman Naseem. 2026. Framing political bias in multilingual llms across pakistani languages. arXiv preprint arXiv:2506.00068. Version 3, January 30, 2026. ↗
  13. OpenAI. 2023. Gpt-4 technical report.
  14. Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. 2025. A survey of multilingual large language models. Patterns, 6(1):101118. ↗
  15. Lukas Rettenberger, Markus Reischl, and Matthias Schutera. 2025. Assessing political bias in large language models. Journal of Computational Social Science, 8:42. ↗
  16. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In International conference on machine learning, pages 29971–30004. PMLR. ↗
  17. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, and Eric Xing. 2023a. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. ↗
  18. S. Sengupta, B. AlKhamissi, S. Alawadhi, et al. 2023b. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv preprint arXiv:2308.16149. ↗
  19. Oleg Smirnov. 2026. The language you ask in: Language-conditioned ideological divergence in llm analysis of contested political documents. ↗
  20. Stefan Stieglitz and Linh Dang-Xuan. 2013. Social media and political communication: a social media analytics framework. Social network analysis and mining, 3(4):1277–1291. ↗
  21. Helene Thiollet. 2011. Migration as diplomacy: Labor migrants, refugees, and arab regional politics in the oil-rich countries. International Labor and Working-Class History, 79(1):103–121. ↗
  22. Junho Yoo and Youhyun Shin. 2025. Fair or framed? political bias in news articles generated by LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16904–16930, Suzhou, China. Association for Computational Linguistics. ↗
  23. D. Zhou and Y. Zhang. 2024. Political biases and inconsistencies in bilingual gpt models—the cases of the u.s. and china. Scientific Reports, 14:25048. 7. Appendix 7.1. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/awwad-2026-do-llms-transfer-political-framing/