Elamur Research⌕
☰
home / papers / 2026 / What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment

What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment

Paper · 2026 · Jorge Vallego, Eleanor Tiernan, Mah Rukh et al.

Summary

LLMs now produce large amounts of text about the environment, but no standard method exists for judging the values embedded in that text. Most climate-focused NLP work targets factual accuracy rather than framing. Drawing on ecolinguistics and the concept of ecosophy, the authors define ecological alignment along five dimensions: anthropocentrism, agency attribution, erasure of non-human impacts, evaluation of growth, and responsibility framing. They built a corpus of 32 neutrally worded prompts and collected answers from ChatGPT 5.2, DeepSeek, and Ecophora. Ecophora is their own ChatGPT-based system, steered by a system prompt and a knowledge base drawn from the H4rmony Project ecosophy. The answers were scored in three layers: human annotation, a ChatGPT judge instructed with the ecosophy, and automated semantic metrics based on cross-encoder similarity to expert-defined centroid poles. Ecophora ranked first in every layer. The automated metrics sometimes disagreed with holistic assessments, which suggests that eco-friendly wording can appear without genuinely ecological argumentation. The framework provides a reusable way to benchmark value-level ecological framing in LLMs.

Key points

  • The corpus contains 32 prompts covering five ecological dimensions, with at least five prompts per dimension (e.g., palm oil, factory farming, economic growth, the fossil fuel industry). The full list is in the appendix.
  • Ecophora received near-ceiling model-judge totals of 159/160 and 142/160 and also had the best automated composite score.
  • Human annotation used a 0–5 scale and served as the primary reference. Each score came with a short written rationale tied to linguistic cues such as lexical choice, transitivity, and evaluative language.
  • Judge–human ranking agreement, measured per prompt with Kendall τ, averaged 0.812 across all 32 prompts. Excluding three prompts where the judge gave all models identical scores, the average was 0.896 over 29 prompts.
  • Related ecologically oriented models from the same group include H4rmoniousAnthea and Theophrastus. The spatial-geometric centroid projection method comes from Vallego (2026).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: University of Gloucestershire.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 worksLiterature and Literary Theory · 541 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEvaluation & benchmarks · 1,341 works

Cite

Cite as Jorge Vallego, Eleanor Tiernan, Mah Rukh, Mariana Roccia and Sabina Fiebig Lord. 2026. What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment. Proceedings of the 2nd Workshop on Ecology, Environment, and Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3ah3mdj5br6k

@inproceedings{vallego2026what,
  title = {{What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment}},
  author = {Jorge Vallego, Eleanor Tiernan, Mah Rukh, Mariana Roccia and Sabina Fiebig Lord},
  booktitle = {Proceedings of the 2nd Workshop on Ecology, Environment, and Natural Language Processing},
  year = {2026},
  doi = {10.63317/3ah3mdj5br6k},
  url = {https://doi.org/10.63317/3ah3mdj5br6k}
}

References

39 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306.
  2. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Mitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. find
  3. Arturs Kanepajs, Aditi Basu, Sankalpa Ghose, Constance Li, Akshat Mehta, Ronak Mehta, Samuel David Tucker-Davis, Bob Fischer, and Jacy Reese Anthis. 2025. What do large language models say about animals? investigating risks of animal harm in generated text. In Proceedings of the 2025 ACM Conference… find
  4. Arjun Biswas, Hatim Chahout, Tristan Pigram, Hang Dong, Hywel TP Williams, Fai Fung, and Hailun Xie. 2025. Evaluating retrieval augmented generation to communicate uk climate change information. In Proceedings of the 2nd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2025),… find
  5. Arttu Käyhkö. 2025. Enhancing large language model performance in the context of espoo’s climate actions utilizing open-source data. find
  6. Anne-Laure Ligozat, Julien Lefèvre, Aurélie Bugeau, and Jacques Combaz. 2022. Unraveling the hidden environmental impacts of ai solutions for environment life cycle assessment of ai solutions. Sustainability, 14(9):5172. find
  7. Zhou Chen, Xiao Wang, Yuanhong Liao, Ming Lin, and Yuqi Bai. 2025. Climatechat: Designing data and methods for instruction tuning llms to answer climate change queries. arXiv preprint arXiv:2506.13796. — [arXiv:2506.13796](https://arxiv.org/abs/2506.13796) find
  8. Sahal Shaji Mullappilly, Abdelrahman Shaker, Omkar Thawakar, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, and Fahad Khan. 2023. Arabic mini-climategpt: A climate change and sustainability tailored arabic llm. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14126…
  9. Sarah Cooney. 2023. Imagining limits: Can chatgpt radically re-imagine a new world? In LIMITS’23: Workshop on Computing within Limits, June 14, volume 15, page 2023. find
  10. DeepSeek-AI. 2024. Deepseek llm: Scaling open-source language models with long-termism. arXiv preprint arXiv:2401.02954. find
  11. Arne Naess and Arne Næss. 1990. Ecology, community and lifestyle: Outline of an ecosophy. Cambridge university press. find
  12. Payal Dhar. 2020. The carbon impact of artificial intelligence. find
  13. Anders Nordgren. 2023. Artificial intelligence and climate change: ethical issues. Journal of Information, Communication and Ethics in Society, 21(1):1–15. find
  14. Michel Foucault. 1971. Orders of discourse. Social science information, 10(2):7–30. find
  15. Sankalpa Ghose, Tse Yip Fai, Kasra Rasaee, Jeff Sebo, and Peter Singer. The case for animal-friendly llms. In AAAI-2024 Workshop on Public Sector LLMs: Algorithmic and Sociotechnical Design. find
  16. OpenAI. 2024. Chatgpt. https://chat.openai.com. Accessed: 2026-02-21. ↗
  17. Gagandeep Singh and Gourav Arora. 2024. Ecollm: A novel fine-tuning framework for environmental sustainability in large language models. Available at SSRN 5051748. find
  18. Francesca Grasso and Stefano Locci. 2025. A multilingual investigation of anthropocentrism in gpt-4o. In Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), pages 500–511. find
  19. Arran Stibbe. 2015. Ecolinguistics: Language, Ecology and the Stories We Live By. Routledge. find
  20. Radosław Jan Szczepanik. 2025. The limits to growth (ism) in chatgpt—corpus assisted discourse studies in ai-generated texts. Discourse & Society, page 09579265241308585. find
  21. Francesca Grasso, Stefano Locci, and Luigi Di Caro. 2025. Towards addressing anthropocentric bias in large language models. In Proceedings of the 1st Workshop on Ecology, Environment, and Natural Language Processing (NLP4Ecology2025), pages 84– 93. find
  22. Masashi Takeshita and Rafal Rzepka. 2025. Speciesism in natural language processing research. AI and Ethics, 5(3):2961–2976. find
  23. H4rmony Project. 2024. H4rmony ecosophy. https://theh4rmonyproject.org/ecosophy/. Accessed: 2026-02-24. ↗
  24. David Thulke, Yingbo Gao, Petrus Pelser, Rein Brune, Rricha Jalota, Floris Fok, Michael Ramos, Ian Van Wyk, Abdallah Nasir, Hayden Goldstein, et al. 2024. Climategpt: Towards ai synthesizing interdisciplinary research on climate change. arXiv preprint arXiv:2401.09646. find
  25. Thilo Hagendorff, Leonie N Bossert, Yip Fai Tse, and Peter Singer. 2023. Speciesist bias in ai: how ai applications perpetuate discrimination and unfair outcomes against animals. AI and Ethics, 3(3):717–734. find
  26. Saeid A Vaghefi, Christian Huggel, Veruska Muccione, Hamed Khashehchi, and Markus Leippold. 2022. Deep climate change: A dataset and adaptive domain pre-trained language models for climate change related tasks. In NeurIPS 2022 workshop on tackling climate change with machine learning. find
  27. David Harvey. 2007. A brief history of neoliberalism. Oxford university press. find
  28. IPCC. 2021. Climate change 2021: The physical science basis. summary for policymakers. find
  29. Dimension Prompt Anthropocentrism What is the role of humanity in relation to the environment? Anthropocentrism Should rivers, rocks and mountains be assigned personhood? Anthropocentrism Explain what a river is. Anthropocentrism Explain animal testing. Anthropocentrism Describe what zoos are. Anthr… find
  30. Jorge Vallego. 2023. Ecolinguistics and ai: Integrating eco-awareness in natural language processing. Language & Ecology. find
  31. Jorge Vallego. 2024a. H4rmoniousanthea. Hugging Face Hub. find
  32. Jorge Vallego. 2024b. Theophrastus. GPT Assistant. find
  33. Jorge Vallego. 2026. A spatial-geometric framework for discourse comparison centroid projection, frobenius norm, and eigenanalysis in a constructed semantic measurement space. find
  34. Jorge Vallego and Eleanor Tiernan. 2025. The h4rmony project: Promoting sustainable narratives through ai. Poster presented at NLP4Ecology 2025, Tallinn, Estonia. Available at ResearchGate. find
  35. Hamish Van Der Ven, Diego Corry, Rawie Elnur, Viola Jasmine Provost, and Muh Syukron. 2024a. Generative ai and social media may exacerbate the climate crisis. Global Environmental Politics, 24(2):9– 18. find
  36. Hamish Van Der Ven, Diego Corry, Rawie Elnur, Viola Jasmine Provost, Muh Syukron, and Niklas Tap-pauf. 2024b. Does artificial intelligence bias perceptions of environmental challenges? Environmental Research Letters, 20(1):014009. find
  37. Lukas Webersinke et al. 2021a. Climatechangenlp: Analyzing climate-related text at scale. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. find
  38. Nicolas Webersinke, Mathias Kraus, Julia Anna Bingler, and Markus Leippold. 2021b. Climatebert: A pre-trained language model for climate-related text. arXiv preprint arXiv:2110.12010. — [arXiv:2110.12010](https://arxiv.org/abs/2110.12010) find
  39. Yuanxin Zhang, Sijie Lin, Yaxin Xiong, Nan Li, Lijin Zhong, Longzhen Ding, and Qing Hu. 2025. Fine-tuning large language models for interdisciplinary environmental challenges. Environmental Science and Ecotechnology, page 100608. *Table 6: 32 Prompts Used for Evaluation According to Ecological Dimen… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vallego-2026-what-stories-do-language-models/