Elamur Research⌕
☰
home / papers / 2025 / Weaver: Interweaving SQL and LLM for Table Reasoning

Weaver: Interweaving SQL and LLM for Table Reasoning

Paper · 2025 · Rohit Khoja, Devanshu Gupta, Yanjie Fu et al.

Summary

Answering questions over tables is hard when cells or surrounding paragraphs hold free text (or images). SQL cannot interpret such content semantically, while LLMs struggle with long inputs and with counting, arithmetic and temporal logic. Earlier hybrids either embed LLM calls inside SQL, which breaks down on multi-step logic, or follow fixed pipelines with few reasoning steps. Weaver addresses this with a modular, plan-driven pipeline that works in four stages:

  • It cleans and filters the table and describes the relevant columns.
  • An LLM writes a step-by-step plan, and a second LLM checks it.
  • The plan is executed, alternating SQL steps (filtering, aggregation, formatting) with LLM steps that add semantically inferred columns. Each step saves an intermediate table.
  • A final LLM call extracts the answer.

Optimizations reduce cost: SQL operations are moved earlier, consecutive SQL steps are merged, and LLM calls are chunked and parallelized. On hybrid subsets of several Table QA benchmarks, Weaver beats prior methods with fewer API calls and errors, especially on multi-hop queries. Its stored intermediate tables make the reasoning transparent and easy to debug.

Key points

  • Evaluation data: WikiTQ, TabFact and FinQA, plus 3,000 queries each from the multimodal FinQA MM and OTT-QA MM. Because no dedicated hybrid benchmark exists, the authors curated hybrid subsets from these datasets.
  • Plan verification: removing the second-LLM check costs about 1% accuracy. Common failure types are SQL errors (unsupported MySQL functions, hallucinated columns) and planning errors.
  • Compatibility: the pipeline works with different database engines (e.g., MySQL, SQLite) and can be extended to VLM steps for image content.
  • Limitations: it handles English tables only and does not support hierarchical or multi-table queries.
  • Code: available at coral-lab-asu.github.io/weaver.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PennsylvaniaPhiladelphia · United States221 works · 63,234 citations🎓Arizona State UniversityTempe · United States50 works · 2,032 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksComputer Networks and Communications · 64 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Rohit Khoja, Devanshu Gupta, Yanjie Fu, Dan Roth and Vivek Gupta. 2025. Weaver: Interweaving SQL and LLM for Table Reasoning. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1436

@inproceedings{khoja2025weaver,
  title = {{Weaver: Interweaving SQL and LLM for Table Reasoning}},
  author = {Rohit Khoja, Devanshu Gupta, Yanjie Fu, Dan Roth and Vivek Gupta},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1436},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1436}
}

References

33 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nikhil Abhyankar, Vivek Gupta, Dan Roth, and Chandan K. Reddy. 2025. H-STAR: LLM-driven hybrid SQL-text adaptive reasoning on tables. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1:… find
  2. Asim Biswal, Liana Patel, Siddarth Jha, Amog Kam-setty, Shu Liu, Joseph E. Gonzalez, Carlos Guestrin, and Matei Zaharia. 2024. Text2sql is not enough: Unifying ai and databases with tag. CoRR, abs/2408.14717. — [arXiv:2408.14717](https://arxiv.org/abs/2408.14717) find
  3. Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W. Cohen. 2021a. Open question answering over tables and text. In International Conference on Learning Representations. find
  4. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2023. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research. find
  5. Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020. Tabfact: A large-scale dataset for table-based fact verification. In International Conference on Learning Representations.
  6. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021b. FinQA: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natura…
  7. Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. HiTab: A hierarchical table dataset for question answering and natural language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistic… find
  8. Google DeepMind. 2024. Gemini 2.0. find
  9. Parker Glenn, Parag Dakle, Liang Wang, and Preethi Raghavan. 2024. BlendSQL: A scalable dialect for unifying hybrid question answering in relational algebra. In Findings of the Association for Computational Linguistics: ACL 2024, pages 453–466, Bangkok, Thailand. Association for Computational Lingui… find
  10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  11. Vivek Gupta, Maitrey Mehta, Pegah Nokhiz, and Vivek Srikumar. 2020. INFOTABS: Inference on tables as semi-structured data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2309–2324, Online. Association for Computational Linguistics.
  12. Jinyang Li, Binyuan Hui, GE QU, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM already serve as a database interface? a BIg bench for large-scale database groun…
  13. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024a. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  14. Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina Semnani, Chen Yu, and Monica Lam. 2024b. SUQL: Conversational search over structured and unstructured data with large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4535–4555, Mexico City, Mexico. Ass… find
  15. Shu Liu, Asim Biswal, Audrey Cheng, Xiangxi Mo, Shiyi Cao, Joseph E Gonzalez, Ion Stoica, and Matei Zaharia. 2024c. Optimizing llm queries in relational workloads. CoRR. find
  16. Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. 2023. Binding language models in symbolic languages. In The Eleventh International Conference on Learning Representations.
  17. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. find
  18. Hanjun Dai, Bethany Wang, Xingchen Wan, Bo Dai, Sherry Yang, Azade Nova, Pengcheng Yin, Mangpo Phothilimthana, Charles Sutton, and Dale Schuurmans. 2024. Uqe: A query engine for unstructured Suyash Mathur, Jainit Bafna, Kunal Kartik, Harshita Khandelwal, Manish Shrivastava, Vivek Gupta, Mohit Bansal… find
  19. Akash Mittal, Anshul Bheemreddy, and Huili Tao. 2024. Semantic sql–combining and optimizing semantic predicates in sql. arXiv preprint arXiv:2404.03880. — [arXiv:2404.03880](https://arxiv.org/abs/2404.03880) find
  20. Md Mahadi Hasan Nahid and Davood Rafiei. 2024a. NormTab: Improving symbolic reasoning in LLMs through tabular data normalization. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3569–3585, Miami, Florida, USA. Association for Computational Linguistics. find
  21. Md Mahadi Hasan Nahid and Davood Rafiei. 2024b. Tabsqlify: Enhancing reasoning capabilities of llms through table decomposition. In NAACL-HLT. find
  22. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, and et al. 2024. Gpt-4o system card.
  23. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1470–…
  24. Liana Patel, Siddharth Jha, Melissa Pan, Harshit Gupta, Parth Asawa, Carlos Guestrin, and Matei Zaharia. 2024. Semantic operators: A declarative model for rich, ai-based data processing. arXiv preprint arXiv:2407.11418. — [arXiv:2407.11418](https://arxiv.org/abs/2407.11418) find
  25. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore. Association for Computat…
  26. Zirui Wu and Yansong Feng. 2024. ProTrix: Building models for planning and reasoning over tables with sentence context. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4378–4406, Miami, Florida, USA. Association for Computational Linguistics. find
  27. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018…
  28. Yunjia Zhang, Jordan Henkel, Avrilia Floratou, Joyce Cahoon, Shaleen Deep, and Jignesh M Patel. 2024. Reactable: enhancing react for table question answering. Proceedings of the VLDB Endowment, 17(8):1981–1994. find
  29. Fengbin Zhu, Ziyang Liu, Fuli Feng, Chao Wang, Moxin Li, and Tat Seng Chua. 2024. Tat-llm: A specialized language model for discrete reasoning over financial tabular and textual data. In Proceedings of the 5th ACM International Conference on AI in Finance, pages 310–318. find
  30. Yucheng Shi, Peng Shu, Zhengliang Liu, Zihao Wu, Quanzheng Li, Tianming Liu, Ninghao Liu, and Xiang Li. 2024. Mgh radiology llama: A llama 3 70b model for radiology. arXiv preprint arXiv:2408.11848. — [arXiv:2408.11848](https://arxiv.org/abs/2408.11848) find
  31. Zhao Tan, Xiping Liu, Qing Shu, Xi Li, Changxuan Wan, Dexi Liu, Qizhi Wan, and Guoqiong Liao. 2024. Enhancing text-to-SQL capabilities of large language models through tailored promptings. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and… find
  32. Prasham Yatinkumar Titiya, Jainil Trivedi, Chitta Baral, and Vivek Gupta. 2025. Mmtbench: A unified benchmark for complex multimodal table reasoning. arXiv preprint arXiv:2505.21771. — [arXiv:2505.21771](https://arxiv.org/abs/2505.21771) find
  33. Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. 2024. Chain-of-table: Evolving tables in the reasoning chain for table understanding. In ICLR. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khoja-2025-weaver-interweaving-sql-llm-table/