Elamur Research⌕
☰
home / papers / 2023 / RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL

RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL

Summary

Seq2seq pre-trained models such as T5 and BART are a strong approach to Text-to-SQL. However, a single decoder must handle two jobs at once: picking the correct tables and columns (schema linking) and producing the SQL keyword structure (skeleton parsing). Queries with many schema items or complex operators make this harder. A pilot experiment on Spider's dev set supports the idea of separating the two. A T5-Base model predicting only skeletons reached about 80% exact match, while full SQL generation with T5-3B stays near 70%.

RESDSQL separates the two jobs in two steps:

  • Ranking-enhanced encoding: a separate RoBERTa-based cross-encoder scores every table and column against the question. Only the highest-ranked items are passed, in ranked order, to the seq2seq encoder.
  • Skeleton-aware decoding: the decoder first writes the SQL skeleton and then the complete query, so the skeleton steers the rest of generation.

The framework is tested on Spider and three robustness variants. The authors report state-of-the-art results on Spider and strong robustness, which shows that lightening the schema-linking load can beat heavier graph-based or grammar-based designs.

Key points

  • Cross-encoder design: RoBERTa with a BiLSTM pooling module, plus a column-enhanced layer that feeds column information into table embeddings. Schema items use semantic names rather than original names. Classification AUC is 0.9973 for tables and 0.9957 for columns.
  • Robustness results (RESDSQL-3B + NatSQL, EM/EX %):
    • Spider-DK: 53.3 / 66.0
    • Spider-Syn: 69.1 / 76.9
    • Spider-Realistic: 77.4 / 81.9
  • Ablation on RESDSQL-Base (77.9 EX):
    • Removing schema ranking lowers EX to 70.1.
    • Removing skeleton parsing lowers EX to 77.1.
  • Decoding check: candidate queries are tested for executability with an off-the-shelf SQL executor, following Suhr et al. (2020).
  • Code: github.com/RUCKBReasoning/RESDSQL (AAAI 2023, Renmin University of China).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics Code & programming languages Semantic parsing & meaning representation

 Institutions

🎓Renmin University of ChinaBeijing · China26 works · 13,818 citations

Cite

Cite as Haoyang Li, Jing Zhang, Cuiping Li and Hong Chen. 2023. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL. arXiv:2302.05965.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2302.05965 · arXiv 2302.05965

@misc{li2023resdsql,
  title = {{RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL}},
  author = {Haoyang Li, Jing Zhang, Cuiping Li and Hong Chen},
  year = {2023},
  doi = {10.48550/arxiv.2302.05965},
  eprint = {2302.05965},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.05965}
}

References

48 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bogin, B.; Gardner, M.; and Berant, J. 2019. Global Reasoning over Database Structures for Text-to-SQL Parsing. In EMNLP-IJCNLP 2019, 3657–3662. find
  2. Cai, R.; Xu, B.; Zhang, Z.; Yang, X.; Li, Z.; and Liang, Z. 2018. An Encoder-Decoder Framework Translating Natural Language to Database Queries. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, 3977–3983. find
  3. Cai, R.; Yuan, J.; Xu, B.; and Hao, Z. 2021. SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, 7664–7676. find
  4. Cao, R.; Chen, L.; Chen, Z.; Zhao, Y.; Zhu, S.; and Yu, K. 2021. LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations. In ACL/IJC-NLP 2021, 2541–2555. find
  5. Chen, D.; Lin, Y.; Li, W.; Li, P.; Zhou, J.; and Sun, X. 2020. Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial I… find
  6. Deng, X.; Awadallah, A. H.; Meek, C.; Polozov, O.; Sun, H.; and Richardson, M. 2021. Structure-Grounded Pretraining for Text-to-SQL. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Onli… our card
  7. Devlin, J.; Chang, M.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019,… our card
  8. Gan, Y.; Chen, X.; Huang, Q.; Purver, M.; Woodward, J. R.; Xie, J.; and Huang, P. 2021a. Towards Robustness of Text-to-SQL Models against Synonym Substitution. In ACL/IJC-NLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, 2505–2515. find
  9. Gan, Y.; Chen, X.; and Purver, M. 2021. Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization. In EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, 8926–8931. find
  10. Gan, Y.; Chen, X.; Xie, J.; Purver, M.; Woodward, J. R.; Drake, J. H.; and Zhang, Q. 2021b. Natural SQL: Making SQL Easier to Infer from Natural Language Specifications. In Findings of EMNLP 2021, 2030–2042. find
  11. Gao, C.; Li, B.; Zhang, W.; Lam, W.; Li, B.; Huang, F.; Si, L.; and Li, Y. 2022. Towards Generalizable and Robust Text-to-SQL Parsing. In Findings of EMNLP 2022. find
  12. Giordani, A.; and Moschitti, A. 2012. Automatic Generation and Reranking of SQL-derived Answers to NL Questions. In Proceedings of the Second International Conference on Trustworthy Eternal Systems via Evolving Software, Data and Knowledge, 59–76. find
  13. Guo, J.; Zhan, Z.; Gao, Y.; Xiao, Y.; Lou, J.; Liu, T.; and Zhang, D. 2019. Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019… find
  14. Hochreiter, S.; and Schmidhuber, J. 1997. Long Short-Term Memory. Neural Comput., 1735–1780. find
  15. Hui, B.; Geng, R.; Wang, L.; Qin, B.; Li, Y.; Li, B.; Sun, J.; and Li, Y. 2022. S 2 SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers. In Findings of ACL 2022, 1254–1262. find
  16. Iyer, S.; Konstas, I.; Cheung, A.; Krishnamurthy, J.; and Zettlemoyer, L. 2017. Learning a Neural Semantic Parser from User Feedback. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, 963–973. find
  17. Krishnamurthy, J.; Dasigi, P.; and Gardner, M. 2017. Neural Semantic Parsing with Type Constraints for Semi-Structured Tables. In EMNLP 2017, 1516–1526. find
  18. Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In ACL 2020, 7871–7880. Lin, T.; Goyal, P.; Girshick, R. B.; He, K.; and… our card
  19. Lin, X. V.; Socher, R.; and Xiong, C. 2020. Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing. In Findings of EMNLP 2020, 4870–4888. our card
  20. Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692. find
  21. Loshchilov, I.; and Hutter, F. 2019. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019. find
  22. Qi, J.; Tang, J.; He, Z.; Wan, X.; Cheng, Y.; Zhou, C.; Wang, X.; Zhang, Q.; and Lin, Z. 2022. RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL. In EMNLP 2022. find
  23. Qin, B.; Wang, L.; Hui, B.; Li, B.; Wei, X.; Li, B.; Huang, F.; Si, L.; Yang, M.; and Li, Y. 2022. SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17,… find
  24. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res., 140:1–140:67. our card
  25. Rubin, O.; and Berant, J. 2021. SmBoP: Semi-autoregressive Bottom-up Semantic Parsing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, 311–324. find
  26. Schlichtkrull, M. S.; Kipf, T. N.; Bloem, P.; van den Berg, R.; Titov, I.; and Welling, M. 2018. Modeling Relational Data with Graph Convolutional Networks. In The Semantic Web - 15th International Conference, ESWC 2018, 593–607. find
  27. Scholak, T.; Schucher, N.; and Bahdanau, D. 2021. find
  28. PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. In EMNLP 2021, 9895–9901. find
  29. Shaw, P.; Chang, M.; Pasupat, P.; and Toutanova, K. 2021. find
  30. Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? In ACL/IJCNLP 2021, 922–938. find
  31. Shaw, P.; Uszkoreit, J.; and Vaswani, A. 2018. Self-Attention with Relative Position Representations. In NAACL-HLT, 464–468. our card
  32. Shazeer, N.; and Stern, M. 2018. Adafactor: Adaptive Learning Rates with Sublinear Memory Cost. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, 4603–4611. find
  33. Shi, P.; Ng, P.; Wang, Z.; Zhu, H.; Li, A. H.; Wang, J.; dos Santos, C. N.; and Xiang, B. 2021. Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, 13806–13814. find
  34. Suhr, A.; Chang, M.; Shaw, P.; and Lee, K. 2020. Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, 8372–8388. find
  35. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 5998–6008. our card
  36. Wang, B.; Shin, R.; Liu, X.; Polozov, O.; and Richardson, M. 2020a. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, 7567–7578. our card
  37. Wang, C.; Tatwawadi, K.; Brockschmidt, M.; Huang, P.-S.; Mao, Y.; Polozov, O.; and Singh, R. 2018. Robust Text-to-SQL Generation with Execution-Guided Decoding. arXiv preprint arXiv:1807.03100. find
  38. Wang, K.; Shen, W.; Yang, Y.; Quan, X.; and Wang, R. 2020b. Relational Graph Attention Network for Aspect-based Sentiment Analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, 3229–3238. our card
  39. Xu, P.; Kumar, D.; Yang, W.; Zi, W.; Tang, K.; Huang, C.; Cheung, J. C. K.; Prince, S. J. D.; and Cao, Y. 2021. Optimizing Deeper Transformers on Small Datasets. In ACL/I-JCNLP 2021, 2089–2102. find
  40. Yin, P.; and Neubig, G. 2017. A Syntactic Neural Model for General-Purpose Code Generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, 440–450. our card
  41. Yu, T.; Li, Z.; Zhang, Z.; Zhang, R.; and Radev, D. R. 2018a. TypeSQL: Knowledge-Based Type-Aware Neural Text-to-SQL Generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, 588–594. our card
  42. Yu, T.; Wu, C.; Lin, X. V.; Wang, B.; Tan, Y. C.; Yang, X.; Radev, D. R.; Socher, R.; and Xiong, C. 2021. GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing. In 9th International Conference on Learning Representations, ICLR 2021. find
  43. Yu, T.; Yasunaga, M.; Yang, K.; Zhang, R.; Wang, D.; Li, Z.; and Radev, D. R. 2018b. SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task. arXiv preprint arXiv:1810.05237. — [arXiv:1810.05237](https://arxiv.org/abs/1810.05237) find
  44. Yu, T.; Zhang, R.; Yang, K.; Yasunaga, M.; Wang, D.; Li, Z.; Ma, J.; Li, I.; Yao, Q.; Roman, S.; Zhang, Z.; and Radev, D. R. 2018c. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018 Conference on Empirical Metho… our card
  45. Zelle, J. M.; and Mooney, R. J. 1996. Learning to Parse Database Queries Using Inductive Logic Programming. In Proceedings of the Thirteenth National Conference on Artificial Intelligence and Eighth Innovative Applications of Artificial Intelligence Conference, AAAI 96, IAAI 96, 1050– 1055. our card
  46. Zhang, Y.; Warstadt, A.; Li, X.; and Bowman, S. R. 2021. When Do You Need Billions of Words of Pretraining Data? In ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, 1112–1125. find
  47. Zhong, R.; Yu, T.; and Klein, D. 2020. Semantic Evaluation for Text-to-SQL with Distilled Test Suites. In EMNLP 2020, 396–411. find
  48. Zhong, V.; Xiong, C.; and Socher, R. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv preprint arXiv:1709.00103. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-resdsql-decoupling-schema-linking-skeleton/