Elamur Research⌕
☰
home / papers / 2026 / Text-to-SQL Task-oriented Dialogue Ontology Construction

Text-to-SQL Task-oriented Dialogue Ontology Construction

Paper · 2026 · Renato Vukovic, Carel van Niekerk, Michael Heck et al.

Summary

Task-oriented dialogue (TOD) systems rely on an explicit ontology of domains, slots, values, user intents and system actions to structure their external database and keep behaviour transparent. Building these ontologies usually needs manual labels or supervised models that treat term extraction and relation extraction as separate stages. The authors present TeQoDO, a training-free approach in which an LLM builds the ontology from scratch by writing SQL against an SQLite database. Tables act as domains, columns as slots, and stored entries as values. Dialogues are processed one at a time. For each one, the model inspects the current DB content and runs a dialogue state tracking step to separate new information from what is already stored. It then issues updates guided by a notion of dialogue success. Retrieved results are enriched with similar values or example column entries. On MultiWOZ and SGD, TeQoDO beats fine-tuned baselines, and its ontology supports dialogue state tracking about as well as the gold ontology. It also scales to Wikipedia and arXiv ontologies, pointing towards human-readable knowledge extracted from LLMs.

Key points

  • The method maps TOD ontology levels onto SQL structures in SQLite. Domains become tables, slots become columns and values become entries, and intents and actions are kept in separate tables. It uses SELECT, PRAGMA and update queries.
  • It is evaluated on MultiWOZ and Schema-Guided Dialogue (SGD) with hierarchy-aware similarity metrics adapted from Lo et al. (2024). It surpasses two supervised baselines: GenDSI from Finch et al. (2024) and DORE from Vukovic et al. (2024).
  • Ablations show that the modular TOD concepts (the DST step and dialogue success) are key. These components help larger models most, while GPT-OSS-20B (3.6B active parameters) does not benefit from them.
  • The approach generalises to open models: LLaMa-4-scout (109B total, 17B active), Qwen-3-235B (22B active), GPT-OSS-20B and LLaMa-3.1-8B.
  • A prompt revised by GPT-4o using its own explanation of the task improves MultiWOZ results by roughly 10% over the manually written prompt. Results remain sensitive to prompt wording.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Heinrich Heine University DüsseldorfDüsseldorf · Germany33 works · 1,487 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 4Research subjects and languages the work is about.Dialogue systems · 624 worksKnowledge graphs & ontologies · 192 worksSemantic parsing & meaning representation · 185 worksCode & programming languages · 129 works

Cite

Cite as Renato Vukovic, Carel van Niekerk, Michael Heck, Benjamin Ruppik, Hsien-chin Lin, Shutong Feng, Nurul Lubis and Milica Gašić. 2026. Text-to-SQL Task-oriented Dialogue Ontology Construction. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.689

@article{vukovic2026text,
  title = {{Text-to-SQL Task-oriented Dialogue Ontology Construction}},
  author = {Renato Vukovic, Carel {van Niekerk}, Michael Heck, Benjamin Ruppik, Hsien-chin Lin, Shutong Feng, Nurul Lubis and Milica Gašić},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.689},
  url = {https://doi.org/10.1162/tacl.a.689}
}

References

63 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. We thank the reviewers and the action editor for their insightful comments and suggested revisions, which significantly improved the quality of this paper. This work was funded by the European Research Council (ERC) under the Horizon 2020 research and innovation program (Grant No. STG2018 804636) an… find
  2. Stuti Agrawal, Pranav Pillai, Nishi Uppuluri, Revanth Gangi Reddy, Sha Li, Gokhan Tur, Dilek Hakkani-Tur, and Heng Ji. 2024. Dialog Flow Induction for Constrainable LLM-Based Chatbots. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 66–77, Kyo… find
  3. Dean Allemang and Juan Sequeda. 2024. Increasing the Accuracy of LLM Question-Answering Systems with Ontologies. In The Semantic Web – ISWC 2024: 23rd International Semantic Web Conference, Baltimore, MD, USA, November 11–15, 2024, Proceedings, Part III, page 324–339, Berlin, Heidelberg. Springer-Ve… find
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — Language Models are Few-Shot Learners
  5. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language…
  6. Sergio Burdisso, Srikanth Madikeri, and Petr Motlicek. 2024. Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5421–5440, Miami, Florida,… find
  7. Daniel C. Dennett. 1987. The Intentional Stance. MIT Press, Cambridge, MA. find
  8. Donald D. Chamberlin and Raymond F. Boyce. 1974. SEQUEL: A Structured English Query Language. In Proceedings of the 1974 ACM SIGFIDET (Now SIGMOD) Workshop on Data Description, Access and Control, SIGFIDET ’74, page 249–264, New York, NY, USA. Association for Computing Machinery. find
  9. Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2024. Evaluating Large Language Models in Class-Level Code Generation. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, New Y… find
  10. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2023. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. Transactions on Machine Learning Research. find
  11. Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur. 2020. MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. In Proceedings of the Twelfth Language… find
  12. Prafulla Kumar Choubey, Xiangyu Peng, Shilpa Bhagavath, Caiming Xiong, Shiva Kumar Pentyala, and Chien-Sheng Wu. 2025. Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents. In Findings of the Association for Computational Linguistics: ACL 2… find
  13. Shutong Feng, Hsien-chin Lin, Christian Geishauser, Nurul Lubis, Carel van Niekerk, Michael Heck, Benjamin Matthias Ruppik, Renato Vukovic, and Milica Gasic. 2024. Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation. In Proceedings of the 25th Annual Meet… find
  14. Ondřej Cífka and Antoine Liutkus. 2023. Black-box Language Model Explanation by Context Length Probing. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1067–1079, Toronto, Canada. Association for Computational Linguistics. find
  15. Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, and Arman Cohan. 2024. Unveiling the Spectrum of Data Contamination in Language Model: A Survey from Detection to Remediation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16078–16092, Bang… find
  16. James D. Finch and Jinho D. Choi. 2024. Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12527–12544, Miami, Florida, USA. Association for Computational Linguistics. find
  17. James D. Finch, Boxin Zhao, and Jinho D. Choi. 2024. Transforming Slot Schema Induction with Generative Dialogue State Inference. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 317–324, Kyoto, Japan. Association for Computational Linguistics. find
  18. Naihao Deng, Yulong Chen, and Yue Zhang. 2022. Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2166–2187, Gyeongju, Republic of Korea. International Committee on Computational Linguis… find
  19. Katerina T Frantzi and Sophia Ananiadou. 1999. The C-value/NC-value Domain-independent Method for Multi-word Term Extraction. Journal of Natural Language Processing, 6(3):145– 179. find
  20. Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv preprint: arXiv:2310.06825. find
  21. Thomas R. Gruber. 1995. Toward Principles for the Design of Ontologies used for Knowledge Sharing? International Journal of Human-Computer Studies, 43(5):907–928. find
  22. Juyeon Kim, Jeongeun Lee, Yoonho Chang, Chanyeol Choi, Junseong Kim, and Jy-yong Sohn. 2024. Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models. find
  23. Michael Heck, Nurul Lubis, Carel van Niekerk, Shutong Feng, Christian Geishauser, Hsien-Chin Lin, and Milica Gašić. 2022. Robust Dialogue State Tracking with Weak Supervision and Sparse Data. Transactions of the Association for Computational Linguistics, 10:1175– 1192. find
  24. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs get lost in Multi-turn Conversation. arXiv preprint arXiv:2505.06120.
  25. Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C.C. Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2024. Can LLM Already Serve as A Database Interface? A Big Bench for Large-Scale Database…
  26. Vojtěch Hudeček and Ondrej Dusek. 2023. Are Large Language Models All You Need for Task-Oriented Dialogue? In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 216– 228, Prague, Czechia. Association for Computational Linguistics. find
  27. Vojtěch Hudeček, Ondřej Dušek, and Zhou Yu. 2021. Discovering Dialogue Slots with Weak Supervision. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages… find
  28. Andy Lo, Albert Q Jiang, Wenda Li, and Mateja Jamnik. 2024. End-to-End Ontology Learning with Large Language Models. In Advances in Neural Information Processing Systems, volume 38. Curran Associates, Inc. find
  29. James MacQueen. 1967. Some Methods for Classification and Analysis of Multivariate Observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA. find
  30. Pere-Lluís Huguet Cabot and Roberto Navigli. 2021. REBEL: Relation Extraction By End-to-end Language generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2370–2381, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  31. MetaAI. 2024. Meta Llama 3.1. Blog post on Meta’s AI site. Accessed 2025-10-30. find
  32. MetaAI. 2025. The Llama 4 Herd: The Beginning of a New Era of Natively Multimodal AI. Blog post on Meta’s AI site. Accessed 2025-10-30. find
  33. Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. 1991. Adaptive Mixtures of Local Experts. Neural Computation, 3(1):79–87. find
  34. George A. Miller. 1994. WordNet: A Lexical Database for English. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
  35. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut David Milward and Martin Beveridge. 20… find
  36. Hiroshi Nakagawa and Tatsunori Mori. 2002. A Simple but Powerful Automatic Term Extraction Method. In COLING-02: COMPUTERM 2002: Second International Workshop on Computational Terminology. find
  37. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  38. OpenAI. 2024. GPT-4o mini: advancing cost-efficient intelligence. Accessed 2025-01-13. find
  39. OpenAI. 2025. Introducing GPT-OSS. Blog post on OpenAI website. Accessed 2025-10-30. find
  40. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Tra… — Training language models to follow instructions with human feedback
  41. Stephen Roller, Douwe Kiela, and Maximilian Nickel. 2018. Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 358–363, Melbourne, Australia. Associa… find
  42. Peter J. Rousseeuw. 1987. Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis. Journal of Computational and Applied Mathematics, 20:53–65. find
  43. Axel Polleres. 2014. SPARQL. In Encyclopedia of Social Network Analysis and Mining, pages 1960–1966, New York, NY. Springer New York. find
  44. Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11709–11724, Miami, Flo… find
  45. Qwen-Team. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388.
  46. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a unified Text-to-text Transformer. J. Mach. Learn. Res., 21(1).
  47. Vinay Samuel, Yue Zhou, and Henry Peng Zou. 2025. Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5058–5070, Abu Dhabi, UAE. Association…
  48. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards Scalable Multi-domain Conversational Agents: The Schema-guided Dialogue Dataset. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8689–8696.
  49. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. find
  50. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In Interna-Amirhossein Razavi, Mina Soltangheis, Negar Arabzadeh, Sara Salamat, Morteza Zihayat, and Ebrahim Bagheri. 2025. Benchmarking Prompt Sensitivity in Large Language Models. In Jiaxi Yang, Binyuan Hui, Min Yang,… find
  51. Armand Stricker and Patrick Paroubek. 2024. A Few-shot Approach to Task-oriented Dialogue Enhanced with Chitchat. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 590–602, Kyoto, Japan. Association for Computational Linguistics. find
  52. Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013. POMDP-based Statistical Spoken Dialog Systems: A Review. Proceedings of the IEEE, 101(5):1160–1179. find
  53. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving Open Language Models at a Practical Size. arXiv preprint arXiv:2408.00118. find
  54. Dian Yu, Mingqiu Wang, Yuan Cao, Izhak Shafran, Laurent Shafey, and Hagen Soltau. 2022. Unsupervised Slot Schema Induction for Task-oriented Dialog. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page… find
  55. Renato Vukovic, David Arps, Carel van Niekerk, Benjamin Matthias Ruppik, Hsien-chin Lin, Michael Heck, and Milica Gasic. 2024. Dialogue Ontology Relation Extraction via Constrained Chain-of-Thought Decoding. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dia… find
  56. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018…
  57. Renato Vukovic, Michael Heck, Benjamin Ruppik, Carel van Niekerk, Marcus Zibrowius, and Milica Gasic. 2022. Dialogue Term Extraction using Transfer Learning and Topological Data Analysis. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 564–581… find
  58. Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Zhouhao Sun, Shi Jun, Ting Liu, and Bing Qin. 2024. Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9386–9406, Bangkok, Thailand. Asso… find
  59. Xuezhi Wang and Denny Zhou. 2024. Chain-of-Thought Reasoning without Prompting. arXiv preprint arXiv:2402.10200. find
  60. Joachim Wermter and Udo Hahn. 2006. You Can’t Beat Frequency (Unless You Use Linguistic Knowledge) – A Qualitative Evaluation of Association Measures for Collocation and Term Extraction. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the A… find
  61. Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, and Pengcheng He. 2024. Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective. In The Twelfth International Conference on Learning Representations (ICLR). find
  62. Xuanhe Zhou, Zhaoyan Sun, and Guoliang Li. 2024. DB-GPT: Large Language Model Meets Database. Data Science and Engineering, 9(1):102–111. find
  63. Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gašić, and Minlie Huang. 2023. ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format. In Proceedi… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vukovic-2026-text-sql-task-oriented-dialogue/