Elamur Research⌕
☰
home / papers / 2026 / PolyglotQL: A Pipeline for Multilingual Text-to-SPARQL Dataset Generation

PolyglotQL: A Pipeline for Multilingual Text-to-SPARQL Dataset Generation

Summary

Work on turning natural-language questions into SPARQL relies on scattered benchmarks. They differ in file format, target knowledge graph and language, and few systems publish reusable multilingual tooling. The authors built PolyglotQL, an open-source ETL toolkit that collects existing QA-over-KG resources (the QALD series, LC-QuAD, RuBQ and others) and converts them into one common schema. It then cleans the records, uses an LLM classifier to drop entries that are not questions, and machine-translates the questions. Finally, it adds a JSON context field that links mentions in each question to Wikidata QIDs and PIDs. Accompanying scripts score models by executing both gold and predicted queries on live endpoints. As a demonstration, the authors release an English–German split of about 318k annotated examples. They fine-tuned and tested two multilingual open LLM families with and without this ID context. Fine-tuning alone gave weak results in both languages, while explicit entity and relation mappings produced consistent, significant improvements. The resource offers a reproducible way to build multilingual text-to-SPARQL data, and its modular design allows extension to new languages and graphs.

Key points

  • The pipeline has six stages: download/extraction, filtering, question-vs-statement classification (Gemma 27B, 8-bit), translation (NousResearch Hermes-3, 4-bit), a second post-translation filtering pass, and entity/relation mapping (Hermes-3 for English, Llama-3.3 variants for German).
  • The full aggregated pool covers about 895,954 examples across 12 languages and 15 KGs, mainly Wikidata and DBpedia. The paper's experiments use only the English–German split.
  • About 200,000 German rows were machine-translated, and 157,246 remained after heuristic filtering (repetitions, length mismatches, wrong detected language) plus native-speaker spot checks of 200 samples.
  • Context annotations total 318,161 (160,915 English and 157,246 German), plus 784 for the QALD-10 test set.
  • The evaluation compares setup v1 (question and KG name only) with v2 (ID mappings provided in both training and inference). Code and logs are available at github.com/julioc-p/polyglotql.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5ow3k3fbz296

@inproceedings{perez2026polyglotql,
  title = {{PolyglotQL: A Pipeline for Multilingual Text-to-SPARQL Dataset Generation}},
  author = {Perez, Julio and Barth, Fabio and Rehm, Georg},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5ow3k3fbz296},
  url = {https://doi.org/10.63317/5ow3k3fbz296}
}

References

25 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eleftherios et al. Avramidis. 2024. Occiglot at WMT24: European open-source large language models evaluated on translation. In Proceedings of the Ninth Conference on Machine Translation, pages 292–298, Miami, Florida, USA. Association for Computational Linguistics. find
  2. Nikita Baramiia, Alina Rogulina, Sergey Petrakov, Valerii Kornilov, and Anton Razzhigaev. 2022. Ranking approach to monolingual question answering over knowledge graphs. In NLIWoD@ ESWC, pages 32–37. find
  3. Manuel Borroto, Francesco Ricca, Bernardo Cuteri, and Vito Barbara. 2023. Sparql-qa enters the qald challenge. In QALD-10 – The 10th challenge on question answering over linked data: Shifting from DBpedia to Wikidata as a KG for KGQA. https://www.nfdi4datascience.de https://hivemind-project.eu ↗
  4. Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024a. Leveraging small language models for text2sparql tasks to improve the resilience of ai assistance. arXiv preprint arXiv:2405.17076. find
  5. Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024b. Leveraging small language models for text2sparql tasks to improve the resilience of ai assistance. find
  6. Ruixiang Cui, Rahul Aralikatte, Heather Lent, and Daniel Hershcovich. 2021. Multilingual Compositional Wikidata Questions. arXiv e-prints, page arXiv:2108.03509. find
  7. Dennis Diefenbach, Kamal Singh, and Pierre Maret. 2017. Wdaqua-core0: A question answering component for the research community. In Semantic Web Challenges, pages 84–89, Cham. Springer International Publishing. find
  8. Mohnish Dubey, Debayan Banerjee, Abdelrahman Abdelkawi, and Jens Lehmann. 2019. Lc-quad 2.0: A large dataset for complex question answering over wikidata and dbpedia. In Proceedings of the 18th International Semantic Web Conference (ISWC). Springer. find
  9. Team Gemma, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  10. Aaron et al. Grattafiori. 2024. The llama 3 herd of models. find
  11. Yu Gu, Sue Kase, Michelle Vanni, Brian Sadler, Percy Liang, Xifeng Yan, and Yu Su. 2021. Beyond iid: three levels of generalization for question answering on knowledge bases. In Proceedings of the Web Conference 2021, pages 3477– 3488. ACM. find
  12. Albert Q. et al. Jiang. 2023. Mistral 7b. find
  13. Lucie-Aimée Kaffee, Kemele M. Endris, Elena Simperl, and Maria-Esther Vidal. 2019. Ranking knowledge graphs by capturing knowledge about languages and labels. In Proceedings of the 10th International Conference on Knowledge Capture, K-CAP 2019, Marina Del Rey, CA, USA, November 19-21, 2019. ACM. find
  14. Vladislav Korablinov and Pavel Braslavski. 2020. RuBQ: A Russian Dataset for Question Answering over Wikidata. arXiv e-prints, page arXiv:2005.10659. find
  15. Aleksandr Perevalov, Dennis Diefenbach, Ricardo Usbeck, and Andreas Both. 2022. Qald-9-plus: A multilingual dataset for question answering over dbpedia and wikidata translated by native speakers. find
  16. Ivan Rybin, Vladislav Korablinov, Pavel Efimov, and Pavel Braslavski. 2021. Ru{bq} 2.0: An innovated russian question answering dataset. In Eighteenth Extended Semantic Web Conference - Resources Track. find
  17. Kanchan Shivashankar, Khaoula Benmaarouf, and Nadine Steinmetz. 2022. From graph to graph: Amr to sparql. In Proceedings of the 7th Natural Language Interfaces for the Web of Data (NLI-WoD) co-located with the 19th European Semantic Web Conference (ESWC 2022), Hersonissos, Greece, 29th May. find
  18. Yu Su, Huan Sun, Brian Sadler, Mudhakar Srivatsa, Izzeddin Gür, Zenghui Yan, and Xifeng Yan. 2016. On generating characteristic-rich question sets for QA evaluation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 562–572, Austin, Texas. Association f… find
  19. Alon Talcmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In NAACL. find
  20. Ryan Teknium, Jeffrey Quesnelle, and Chen Guang. 2024. Hermes 3 technical report. find
  21. Priyansh Trivedi, Gaurav Maheshwari, Mohnish Dubey, and Jens Lehmann. 2017. Lc-quad: A corpus for complex question answering over knowledge graphs. In International Semantic Web Conference, pages 210–218. Springer. find
  22. Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo, Felix Conrads, Michael Röder, and Giulio Napolitano. 2018. 8th challenge on question answering over linked data (qald-8) (invited paper). In Semdeep-/NLIWoD@ISWC. find
  23. Ricardo Usbeck, Michael Röder, Michael Hoffmann, Felix Conrads, Jonathan Huthmann, Axel-Cyrille Ngonga-Ngomo, Christian Demmler, and Christina Unger. 2019. Benchmarking question answering systems. Semantic Web, 10(2):293– 304. find
  24. Ricardo Usbeck, Xi Yan, Aleksandr Perevalov, Longquan Jiang, Julius Schulz, Angelie Kraft, Cedric Möller, Junbo Huang, Jan Reineke, and Axel-Cyrille et al. Ngonga Ngomo. 2024. Qald-10–the 10th challenge on question answering over linked data: Shifting from dbpedia to wikidata as a kg for kgqa. Seman… find
  25. Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 201–206. Association… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/perez-2026-polyglotql-pipeline-multilingual-text-sparql/