Elamur Research⌕
☰
home / papers / 2026 / Geolocating News about Extreme Climate Events: A Comparative Analysis of Off-the-Shelf Tools for Toponym Identification in German

Geolocating News about Extreme Climate Events: A Comparative Analysis of Off-the-Shelf Tools for Toponym Identification in German

Paper · 2026 · Brielen Madureira, Mariana Madruga de Brito and Andreas Niekler

Summary

Finding the location of a reported climate disaster is a routine need in climate impact research. Pipelines usually begin by extracting place names with an off-the-shelf NER tool whose output goes unchecked. The authors measure how much that unexamined choice matters for German news.

They ran Flair, Spacy and Stanza over 983 German newspaper articles (2000–2024) covering seven hazard types, each labelled by humans with the country of the event. Every detected location was looked up in Geonames and Nominatim, and ambiguous names were settled with a spatial-closeness heuristic. Each article then received a country from three intentionally simple rules: majority vote, closest-to-centroid, and keyword proximity.

The tools were compared in two ways. First they were checked against one another, since no gold toponyms exist. Then they were judged extrinsically against the human country labels and the corpus-wide ranking of country frequencies.

Flair came out ahead, while Spacy and Stanza swapped places across experiments. Picking a single tool blindly can alter over a tenth of predictions, lower accuracy by about 5 points, and distort which countries look prominent in German media.

Key points

  • Corpus: 983 relevant German articles from the wiso-net aggregator. They come from a human-annotated sample of 3,150, of which 1,008 were relevant and 25 were excluded. The seven hazard categories are floods, storms, droughts, wildfires, landslides, and both heat and cold waves. Article length ranges from 178 to 11,512 characters, averaging 2,468.
  • Tools and models: Flair de-ner-large, Spacy de_core_news_lg and Stanza de. All use a four-label scheme, and only LOC is used here. The authors note that a single German location label merges several concepts: geopolitical entities, nationalities, facilities and places.
  • Geocoding: up to 20 matches per toponym were cached. After filtering and deduplication by name and country, 11,300 entries remained for Nominatim and 10,392 for Geonames. Toponyms were neither lemmatised nor disambiguated with context, so German declension can cause lookup misses.
  • Metrics:
    • IoU over exact character spans for agreement between tools;
    • exact and partial country match against the gold labels;
    • Kendall τ and Spearman correlation for country-frequency rankings.
  • Code is published at codeberg.org/briemadu/german-ner-geoparsing.

Interesting ideas

  • I1. When no gold annotation exists for an intermediate layer, judge each component variant by how often it flips the final decision; this turns "which tagger is better" into a measurable downstream question.
  • I2. Downstream rules are kept deliberately simple so that differences coming from upstream are not hidden behind clever modelling.
  • I3. Ambiguous place names are resolved by choosing the reading closest to the places that are already unambiguous in the same document, a coherence-based disambiguation.
  • I4. Location evidence is restricted to sentences containing an event trigger word, and the authors suggest filtering toponyms by part of speech and syntactic role (for example, excluding "German" as a dialect adjective).
  • I5. Agreement between tools on exact spans serves as a first, gold-free quality check before any extrinsic test.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Swap tagger or parser variants and count how many answers on school tests or fable QA change, giving a gold-free signal for which upstream errors actually matter.
  • I2 (learn): Ablation experiments on the parser could pair it with a minimal frame extractor, so that changes in output can be credited to the tree rather than to compensating logic.
  • I3 (world): When linking entities to Wikidata, prefer the candidate most consistent with entities already firmly anchored in the text's world.
  • I4 (frames): Attach locations to event frames through dependency paths to the event verb or noun, rather than counting document-wide mentions, which avoids being misled by background places.
  • I5 (learn): Span-level IoU between witness parsers' entities or constituents could decide which silver examples are trustworthy enough to train on.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Leipzig UniversityLeipzig · Germany26 works · 8,622 citations🔬Helmholtz Centre for Environmental ResearchLeipzig · Germany3 works · 1 citation

Also on the byline: Leipzig University of Applied Sciences.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Brielen Madureira, Mariana Madruga de Brito and Andreas Niekler. 2026. Geolocating News about Extreme Climate Events: A Comparative Analysis of Off-the-Shelf Tools for Toponym Identification in German. arXiv:2605.03414.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2605.03414 · arXiv 2605.03414

@misc{madureira2026geolocating,
  title = {{Geolocating News about Extreme Climate Events: A Comparative Analysis of Off-the-Shelf Tools for Toponym Identification in German}},
  author = {Brielen Madureira, Mariana Madruga {de Brito} and Andreas Niekler},
  year = {2026},
  doi = {10.48550/arxiv.2605.03414},
  eprint = {2605.03414},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2605.03414}
}

References

39 references, 2 of them with a card of ours. ↗ leads to the work.

  1. M. Gritta, M. T. Pilehvar, N. Collier, A pragmatic guide to geoparsing evaluation: Toponyms, Named Entity Recognition and pragmatics, Language Resources and Evaluation 54 (2020) 683–712. URL: http://link.springer.com/10.1007/s10579-019-09475-3. doi: 10.1007/s10579-019-09475-3 . ↗
  2. X. Hu, Z. Zhou, H. Li, Y. Hu, F. Gu, J. Kersten, H. Fan, F. Klan, Location Reference Recognition from Texts: A Survey and Comparison, ACM Computing Surveys 56 (2024) 1–37. URL: https://dl.acm.org/doi/10.1145/3625819. doi: 10.1145/3625819 . ↗
  3. D. Otto, M. Pfeiffer, M. M. de Brito, M. Gross, Fixed Amidst Change: 20 Years of Media Coverage on Carbon Capture and Storage in Germany, Sustainability 14 (2022). URL: https://www.mdpi.com/2071-1050/14/12/7342. doi: 10.3390/su14127342 . ↗
  4. J. Sodoge, C. Kuhlicke, M. M. d. Brito, Automatized spatio-temporal detection of drought impacts from newspaper articles using natural language processing and machine learning, Weather and Climate Extremes 41 (2023) 100574. URL: https://www.sciencedirect.com/science/article/pii/ S2212094723000270. doi: https://doi.org/10.1016/j.wace.2023.100574 . ↗
  5. J. H. Lochner, A. Stechemesser, L. Wenz, Climate summits and protests have a strong impact on climate change media coverage in Germany, Communications Earth & Environment 5 (2024) 279. URL: https://doi.org/10.1038/s43247-024-01434-3. doi: 10.1038/s43247-024-01434-3 . ↗
  6. P. H. L. Alencar, J. Sodoge, E. Nora Paton, M. Madruga De Brito, Flash droughts and their impacts—using newspaper articles to assess the perceived consequences of rapidly emerging droughts, Environmental Research Letters 19 (2024) 074048. URL: https://iopscience.iop.org/ article/10.1088/1748-9326/ad58fa. doi: 10.1088/1748-9326/ad58fa . ↗
  7. I. Kong, R. S. Purves, Analyzing Geographic Bias of Newspaper Articles Reporting Global Climate Disasters, Annals of the American Association of Geographers (2025) 1–19. URL: https://www.tandfonline.com/doi/full/10.1080/24694452.2025.2564220. doi: 10.1080/24694452. 2025.2564220 . ↗
  8. E. Amitay, N. Har’El, R. Sivan, A. Soffer, Web-a-where: geotagging web content, in: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, ACM, Sheffield United Kingdom, 2004, pp. 273–280. URL: https://dl.acm.org/doi/10.1145/ 1008992.1009040. doi: 10.1145/1008992.1009040 . ↗
  9. G. Andogah, G. Bouma, J. Nerbonne, Every document has a geographical scope, Data & Knowledge Engineering 81-82 (2012) 1–20. URL: https://linkinghub.elsevier.com/retrieve/pii/ S0169023X12000687. doi: 10.1016/j.datak.2012.07.002 . ↗
  10. B. R. Monteiro, C. A. Davis, F. Fonseca, A survey on the geographic scope of textual documents, Computers & Geosciences 96 (2016) 23–34. URL: https://linkinghub.elsevier.com/retrieve/pii/ S0098300416301972. doi: 10.1016/j.cageo.2016.07.017 . ↗
  11. F. Melo, B. Martins, Automated Geocoding of Textual Documents: A Survey of Current Approaches, Transactions in GIS 21 (2017) 3–38. URL: https://onlinelibrary.wiley.com/doi/10.1111/tgis.12212. doi: 10.1111/tgis.12212 . ↗
  12. W. Zong, D. Wu, A. Sun, E.-P. Lim, D. H.-L. Goh, On assigning place names to geography related web pages, in: Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, ACM, Denver CO USA, 2005, pp. 354–362. URL: https://dl.acm.org/doi/10.1145/1065385.1065464. doi: 10.1145/1065385.1065464 . ↗
  13. S. J. Lee, H. Liu, M. D. Ward, Lost in Space: Geolocation in Event Data, Political Science Research and Methods 7 (2019) 871–888. URL: https://www.cambridge.org/core/product/identifier/ S2049847018000237/type/journal_article. doi: 10.1017/psrm.2018.23 . ↗
  14. D. Benikova, C. Biemann, M. Reznicek, NoSta-D named entity annotation for German: Guidelines and dataset, in: N. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, S. Piperidis (Eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), European Language Resources Association (ELRA), Reykjavik, Iceland, 2014, pp. 2524–2531. URL: https://aclanthology.org/L14-1251/. ↗
  15. M. Riedl, S. Padó, A named entity recognition shootout for German, in: I. Gurevych, Y. Miyao (Eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 120–125. URL: https://aclanthology.org/P18-2020/. doi: 10.18653/v1/P18-2020 . ↗
  16. K. Labusch, C. Neudecker, D. Zellhöfer, Bert for named entity recognition in contemporary and historic german, in: Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers, German Society for Computational Linguistics & Language Technology, Erlangen, Germany, 2019, pp. 1–9. URL: https://konvens.org/proceedings/2019/papers/ KONVENS2019_paper_4.pdf. ↗
  17. E. Leitner, G. Rehm, J. Moreno-Schneider, A dataset of German legal documents for named entity recognition, in: N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, S. Piperidis (Eds.), Proceedings of the Twelfth Language Resources and Evaluation Conference, European Language Resources Association, Marseille, France, 2020, pp. 4478–4485. URL: https://aclanthology.org/2020.lrec-1.551/. ↗
  18. K. Ortmann, A. Roussel, S. Dipper, Evaluating Off-the-Shelf NLP Tools for German, in: Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers, German Society for Computational Linguistics & Language Technology, Erlangen, Germany, 2019, pp. 212– 222. URL: https://sfb1102.uni-saarland.de/sfbunisb/uploads/2020/10/KONVENS2019_paper_55.pdf. ↗
  19. S. Scheible, R. J. Whitt, M. Durrell, P. Bennett, Evaluating an ‘off-the-shelf’ POS-tagger on early Modern German text, in: K. Zervanou, P. Lendvai (Eds.), Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, Association for Computational Linguistics, Portland, OR, USA, 2011, pp. 19–23. URL: https://aclanthology.org/W11-1503/. ↗
  20. R. Laarmann-Quante, L. Prepens, T. Zesch, Evaluating automatic spelling correction tools on German primary school children’s misspellings, in: D. Alfter, E. Volodina, T. François, P. Desmet, F. Cornillie, A. Jönsson, E. Rennes (Eds.), Proceedings of the 11th Workshop on NLP for Computer Assisted Language Learning, LiU Electronic Press, Louvain-la-Neuve, Belgium, 2022, pp. 95–107. URL: https://aclanthology.org/2022.nlp4call-1.10/. ↗
  21. M. Gritta, M. T. Pilehvar, N. Limsopatham, N. Collier, What’s missing in geographical parsing?, Language Resources and Evaluation 52 (2018) 603–623. URL: http://link.springer.com/10.1007/ s10579-017-9385-8. doi: 10.1007/s10579-017-9385-8 . ↗
  22. J. Wang, Y. Hu, Enhancing spatial and textual analysis with EUPEG: An extensible and unified platform for evaluating geoparsers, Transactions in GIS 23 (2019) 1393–1419. URL: https://onlinelibrary.wiley.com/doi/10.1111/tgis.12579. doi: 10.1111/tgis.12579 . ↗
  23. Z. Liu, K. Janowicz, L. Cai, R. Zhu, G. Mai, M. Shi, Geoparsing: Solved or Biased? An Evaluation of Geographic Biases in Geoparsing, AGILE: GIScience Series 3 (2022) 1–13. URL: https://agile-giss.copernicus.org/articles/3/9/2022/. doi: 10.5194/agile-giss-3-9-2022 . ↗
  24. N. Doms, T. Schlachter, L. Hahn-Woernle, A Geo-Parser for German Documents with Environmental Context, in: V. Wohlgemuth, H. Kandil, A. Ramzy (Eds.), Advances and New Trends in Environmental Informatics, Springer Nature Switzerland, Cham, 2025, pp. 21–33. URL: https://link.springer.com/10.1007/978-3-031-85284-8_2. doi: 10.1007/978-3-031-85284-8_2 , series Title: Progress in IS. ↗
  25. M. Won, P. Murrieta-Flores, B. Martins, Ensemble Named Entity Recognition (NER): Evaluating NER Tools in the Identification of Place Names in Historical Corpora, Frontiers in Digital Humanities 5 (2018) 2. URL: http://journal.frontiersin.org/article/10.3389/fdigh.2018.00002/full. doi: 10.3389/fdigh.2018.00002 . ↗
  26. L. Kriesch, S. Losacker, A geolocated dataset of German news articles, Scientific Data 12 (2025) 1128. URL: https://www.nature.com/articles/s41597-025-05422-w. doi: 10.1038/ s41597-025-05422-w . ↗
  27. J. L. Leidner, G. Sinclair, B. Webber, Grounding spatial named entities for information extraction and question answering, in: Proceedings of the HLT-NAACL 2003 Workshop on Analysis of Geographic References, 2003, pp. 31–38. URL: https://aclanthology.org/W03-0105/. ↗
  28. M. Badieh Habib Morgan, M. van Keulen, Named entity extraction and disambiguation: the missing link, ESAIR ’13, Association for Computing Machinery, New York, NY, USA, 2013, p. 37–40. URL: https://doi.org/10.1145/2513204.2513217. doi: 10.1145/2513204.2513217 . ↗
  29. A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, R. Vollgraf, FLAIR: An easy-to-use framework for state-of-the-art NLP, in: W. Ammar, A. Louis, N. Mostafazadeh (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 54–59. URL: https://aclanthology.org/N19-4010/. doi: 10.18653/v1/N19-4010 . ↗
  30. M. Honnibal, I. Montani, S. Van Landeghem, A. Boyd, spaCy: Industrial-strength Natural Language Processing in Python (2020). URL: https://spacy.io/. doi: 10.5281/zenodo.1212303 . ↗
  31. P. Qi, Y. Zhang, Y. Zhang, J. Bolton, C. D. Manning, Stanza: A python natural language processing toolkit for many human languages, in: A. Celikyilmaz, T.-H. Wen (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Association for Computational Linguistics, Online, 2020, pp. 101–108. URL: https://aclanthology.org/2020.acl-demos.14/. doi: 10.18653/v1/2020.acl-demos.14 . ↗
  32. D. A. Smith, G. Crane, Disambiguating Geographic Names in a Historical Digital Library, in: G. Goos, J. Hartmanis, J. Van Leeuwen, P. Constantopoulos, I. T. Sølvberg (Eds.), Research and Advanced Technology for Digital Libraries, volume 2163, Springer Berlin Heidelberg, Berlin, Heidelberg, 2001, pp. 127–136. URL: http://link.springer.com/10.1007/3-540-44796-2_12. doi: 10. 1007/3-540-44796-2_12 , series Title: Lecture Notes in Computer Science. ↗
  33. R. C. Pasley, P. D. Clough, M. Sanderson, Geo-tagging for imprecise regions of different sizes, in: Proceedings of the 4th ACM workshop on Geographical information retrieval, ACM, Lisbon Portugal, 2007, pp. 77–82. URL: https://dl.acm.org/doi/10.1145/1316948.1316969. doi: 10.1145/ 1316948.1316969 . ↗
  34. M. A. Radke, N. Gautam, A. Tambi, U. A. Deshpande, Z. Syed, Geotagging Text Data on the Web—A Geometrical Approach, IEEE Access 6 (2018) 30086–30099. URL: https://ieeexplore.ieee.org/document/8371593/. doi: 10.1109/ACCESS.2018.2843814 . ↗
  35. C. Spearman, The Proof and Measurement of Association between Two Things, The American Journal of Psychology 15 (1904) 72. URL: https://www.jstor.org/stable/1412159?origin=crossref. doi: 10.2307/1412159 . ↗
  36. M. G. Kendall, A new measure of rank correlation, Biometrika 30 (1938) 81–93. URL: https://doi.org/10.1093/biomet/30.1-2.81. ↗
  37. N. Li, S. Zahra, M. Brito, C. Flynn, O. Görnerup, K. Worou, M. Kurfali, C. Meng, W. Thiery, J. Zscheischler, G. Messori, J. Nivre, Using LLMs to build a database of climate extreme impacts, in: D. Stammbach, J. Ni, T. Schimanski, K. Dutia, A. Singh, J. Bingler, C. Christiaen, N. Kushwaha, V. Muccione, S. A. Vaghefi, M. Leippold (Eds.), Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024), Association for Computational Linguistics, Bangkok, Thailand, 2024, pp. 93–110. URL: https://aclanthology.org/2024.climatenlp-1. 7/. doi: 10.18653/v1/2024.climatenlp-1.7 . ↗
  38. M. Madruga de Brito, J. Sodoge, H. Kreibich, C. Kuhlicke, Comprehensive assessment of flood socioeconomic impacts through text-mining, Water Resources Research 61 (2025) e2024WR037813. URL: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2024WR037813. doi: https://doi. org/10.1029/2024WR037813 . ↗
  39. T. M. N. Carvalho, A. Niekler, C. Kuhlicke, J. Zscheischler, M. M. de Brito, Global synthesis of peer-reviewed articles reveals blind spots in climate impacts research (2025). URL: http://dx.doi.org/10.21203/rs.3.rs-6095740/v1. doi: 10.21203/rs.3.rs-6095740/v1 . ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/madureira-2026-geolocating-news-about-extreme-climate/