Elamur Research⌕
☰
home / papers / 2026 / Introducing a Green Leaderboard for Sustainable Risk Prediction in Streaming NLP Shared Tasks.

Introducing a Green Leaderboard for Sustainable Risk Prediction in Streaming NLP Shared Tasks.

Paper · 2026 · Alba María Mármol-Romero, Adrián Moreno Muñoz and Arturo Montejo-Raez

Summary

NLP shared tasks usually rank submissions by one accuracy metric and ignore how much computation and energy each system needs. In streaming early risk detection, a model reads a user's messages round by round, so these costs accumulate. They also limit deployment on phones or wearables. The authors propose the Green Early Detection Score (GED), which combines classification quality, detection speed, and environmental cost. It comes in two variants: one based on CO2 emissions, and one based on energy per prediction for on-device use. They apply GED retrospectively to systems from three MentalRiskES editions (2023–2025). MentalRiskES is a Spanish social-media mental health task whose participants logged their footprint with CodeCarbon. The result is what the authors call the first Green Leaderboard for the early detection setting. Re-ranking changes positions considerably: lighter classical ML and hybrid systems move into the top places, most clearly under the energy variant. The paper argues that evaluation criteria shape which models get built. It also shows that efficiency can be scored with existing shared-task infrastructure.

Key points

  • Predictive effectiveness is defined as P = Macro-F1 · (1 − ERDE30). Systems therefore score well only if they are both accurate and early.
  • Emissions and energy per prediction are converted to efficiency scores in [0, 1] with min–max normalization. This is done separately for each edition, because datasets and submission pools differ between years.
  • The unified dataset covers more than 170 systems. The 2023–2024 editions use the MentalRiskES dataset, and 2025 uses the PRECOM dataset.
  • A Pareto analysis of ERDE30 against mean CO2 per prediction suggests that earlier detection does not require higher emissions.
  • Top-10 lists based only on Macro-F1 are dominated by compute-heavy transformer systems, while GED_CO2 and especially GED_Energy favor lightweight and hybrid models. Planned extensions include inference latency, memory footprint, and normalization that accounts for different hardware.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Universidad de JaénJaén · Spain8 works · 1,671 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 worksEngineering · 133 worksElectrical and Electronic Engineering · 18 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Alba María Mármol-Romero, Adrián Moreno Muñoz and Arturo Montejo-Raez. 2026. Introducing a Green Leaderboard for Sustainable Risk Prediction in Streaming NLP Shared Tasks. Proceedings of the 2nd Workshop on Ecology, Environment, and Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3cbext6ixjh2

@inproceedings{marmolromero2026introducing,
  title = {{Introducing a Green Leaderboard for Sustainable Risk Prediction in Streaming NLP Shared Tasks.}},
  author = {Alba María Mármol-Romero, Adrián Moreno Muñoz and Arturo Montejo-Raez},
  booktitle = {Proceedings of the 2nd Workshop on Ecology, Environment, and Natural Language Processing},
  year = {2026},
  doi = {10.63317/3cbext6ixjh2},
  url = {https://doi.org/10.63317/3cbext6ixjh2}
}

References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nesrine Bannour, Sahar Ghannay, Aurélie Névéol, and Anne-Laure Ligozat. 2021. Evaluating the carbon footprint of nlp methods: a survey and analysis of existing tools. In Proceedings of the second workshop on simple and efficient natural language processing, pages 11–21. find
  2. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. find
  3. Verónica Bolón-Canedo, Laura Morán-Fernández, Brais Cancela, and Amparo Alonso-Betanzos. 2024. A review of green artificial intelligence: Towards a more sustainable future. Neurocomputing, 599:128096. find
  4. Benoit Courty, Victor Schmidt, Sasha Luccioni, Goyal-Kamal, MarionCoutarel, Boris Feld, Jérémy Lecourt, LiamConnell, Amine Saboni, Inimaz, supatomic, Mathilde Léval, Luis Blanche, Alexis Cruveiller, ouminasara, Franklin Zhao, Aditya Joshi, Alexis Bogroff, Hugues de Lavoreille, Niko Laskaris, Edoardo… find
  5. Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248):1– 43. find
  6. Daniel Hershcovich, Nicolas Webersinke, Mathias Kraus, Julia Bingler, and Markus Leippold. 2022. Towards climate awareness in nlp research. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2480–2494. find
  7. Lynn H Kaack, Priya L Donti, Emma Strubell, George Kamiya, Felix Creutzig, and David Rolnick. 2022. Aligning artificial intelligence with climate change mitigation. Nature Climate Change, 12(6):518–527. find
  8. Loïc Lannelongue, Jason Grealey, and Michael Inouye. 2021. Green algorithms: quantifying the carbon footprint of computation. Advanced science, 8(12):2100707. find
  9. David E Losada and Fabio Crestani. 2016. A test collection for research on depression and language use. In International conference of the cross-language evaluation forum for European languages, pages 28–39. Springer. find
  10. Alba María Mármol-Romero, Adrián Moreno-Muñoz, Flor Miriam Plaza-del Arco, María Dolores Molina-González, Maria Teresa Martín-Valdivia, Luis Alfonso Ureña-López, and Arturo Montejo-Raéz. 2023. Overview of mentalriskes at iberlef 2023: Early detection of mental disorders risk in spanish. Procesamient… find
  11. Aimee Van Wynsberghe. 2021. Sustainable ai: Ai for sustainability and the sustainability of ai. AI and Ethics, 1(3):213–218. 10. find
  12. Language Resource References Álvarez-Ojeda, Pablo and Cantero-Romero, María Victoria and Semikozova, Anastasia and Montejo-Ráez, Arturo. 2025. The precom-sm corpus: Gambling in spanish social media. find
  13. Mármol Romero, Alba María and Moreno-Muñoz, Adrián and Plaza-Del-Arco, Flor Miriam and Molina-González, M. Dolores and Montejo-Ráez, Arturo. 2024. MentalRiskES: A New Corpus for Early Detection of Mental Disorders in Spanish. ELRA and ICCL. find
  14. Alba María Mármol-Romero, Adrián Moreno-Muñoz, Flor Miriam Plaza-Del-Arco, M Dolores Molina-González, and Arturo Montejo-Ráez. 2024. Environmental impact measurement in the mentalriskes evaluation campaign. In Proceedings of the Second International Workshop Towards Digital Language Equality (TDLE):… find
  15. Victor Schmidt, Kamal Goyal, Aditya Joshi, Boris Feld, Liam Conell, Nikolas Laskaris, Doug Blank, Jonathan Wilson, Sorelle Friedler, and Sasha Luccioni. 2021. Codecarbon: estimate and track carbon emissions from machine learning computing. Cited on, 20. find
  16. Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020. Green ai. Communications of the ACM, 63(12):54–63.
  17. Emma Strubell, Ananya Ganesh, and Andrew Mc-Callum. 2019. Energy and policy considerations for deep learning in nlp. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 3645–3650.
  18. María Estrella Vallecillo-Rodríguez, María Victoria Cantero-Romero, Isabel Cabrera-de Castro, Luis Alfonso Ureña-López, Arturo Montejo-Ráez, and María Teresa Martín-Valdivia. 2024. Overview of refutes at iberlef 2024: Automatic generation of counter speech in spanish. Procesamiento del Lenguaje Natu… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/romero-2026-introducing-green-leaderboard-sustainable-risk/