Elamur Research⌕
☰
home / papers / 2026 / Language Effects in Text-to-SQL Across English and Portuguese

Language Effects in Text-to-SQL Across English and Portuguese

Paper · 2026 · Lucas Nobre, Suele Sousa, Savio Teles et al.

Summary

The paper asks how far LLM-based Text-to-SQL systems degrade when questions are posed in Portuguese instead of English, and whether reasoning features help close that gap. The authors built BIRD_PT, a Brazilian Portuguese version of the BIRD benchmark. They machine-translated the questions and external knowledge with gemini-2.5-pro but left database schemas and values in English. They then tested nine proprietary and open-weight models (Gemini, Claude, DeepSeek and Qwen families) under four controlled setups that switch internal reasoning and a guided SQL-generation plan on or off. Execution accuracy was the metric.

Every model scored lower on Portuguese inputs, though the size of the drop differed widely between families. Turning on reasoning by itself did not reliably help and often hurt, especially in Portuguese. Pairing reasoning with a plan gave the most consistent improvements, but these were smaller than in English. The study shows that multilingual Text-to-SQL is still unsolved and that language understanding, reasoning and task-specific guidance need to be designed together.

Key points

  • BIRD_PT resource: Only questions and domain knowledge were translated, with schema and values kept in English. A manual check of 100 random question–knowledge pairs found just one clear translation error.
  • Plan selection: 22 candidate SQL-generation plans, produced by Claude-4-Sonnet, Gemini-2.5-Flash and GPT-4.1, were screened with Gemini-2.5-Flash. The single best plan, of the kind that includes query examples, was then used for all experiments.
  • Reasoning alone: Average execution accuracy shifted from 59.1% to 58.2% on BIRD and from 56.0% to 53.0% on BIRD_PT when reasoning was enabled. deepseek-v3.1 lost 3.3pp in Portuguese.
  • Model families: Claude and Gemini models were the most accurate and showed the smallest cross-lingual gaps. Qwen3-Next and deepseek-v3.1 led the open-weight group but lost about 2.6pp and 4.7pp on average in Portuguese.
  • English thinking: Reasoning models produced their thinking tokens only in English, even for Portuguese inputs. The authors list human validation of BIRD_PT and fully localized schemas as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 4Research subjects and languages the work is about.English · 2,263 worksReasoning · 1,426 worksSemantic parsing & meaning representation · 185 worksCode & programming languages · 129 works

Cite

Cite as Lucas Nobre, Suele Sousa, Savio Teles and Anderson Soares. 2026. Language Effects in Text-to-SQL Across English and Portuguese. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{nobre2026language,
  title = {{Language Effects in Text-to-SQL Across English and Portuguese}},
  author = {Lucas Nobre, Suele Sousa, Savio Teles and Anderson Soares},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.27/}
}

References

43 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Katrin Affolter, Kurt Stockinger, and Abraham Bernstein. 2019. A comparative survey of recent natural language interfaces for databases. The VLDB Journal, 28(5):793–819. find
  2. Alnur Ali, Ashutosh Baheti, Jonathan Chang, Ta-Chung Chi, Brandon Cui, Andrew Drozdov, Jonathan Frankle, Abhay Gupta, Pallavi Koppol, Sean Kulinski, Jonathan Li, Dipendra Misra, Krista Opsahl-Ong, Jose Javier Gonzalez Ortiz, Matei Zaharia, and Yue Zhang. 2025. A State-of-the-Art SQL Reasoning Model… — [arXiv:2509.21459](https://arxiv.org/abs/2509.21459) find
  3. Anthropic. 2025a. Claude haiku 4.5 [large language model – system card]. https://www.anthropic.com/claude-haiku-4-5-system-card. Accessed: 2025-12-01. ↗
  4. Anthropic. 2025b. Claude sonnet 4.5: System card. https://www.anthropic.com/ claude-sonnet-4-5-system-card. Accessed: 2025-11-08. ↗
  5. BerriAI. 2024. Litellm: Unified python sdk and proxy server for 100+ llms. Accessed: 2025-12-05. find
  6. Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 others. 2025. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv preprint.…
  7. William Alberto Cruz-Castañeda and Marcellus Amadeus. 2025. Large Languages Models in Brazilian Portuguese: A Chronological Survey. Journal of the Brazilian Computer Society, 31(1):1168–1187. find
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LL…
  9. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025. Deepseek-v3 technical report. Preprint, arXiv:2… find
  10. Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, lu Chen, Jinshu Lin, and Dongfang Lou. 2023. C3: Zero-shot Text-to-SQL with ChatGPT. Preprint, arXiv:2307.07306. — [arXiv:2307.07306](https://arxiv.org/abs/2307.07306) find
  11. Han Fu, Feifei Li, Chang Liu, Jian Tan, Bin Wu, and Jianling Sun. 2023. CatSQL: Towards real world natural language to SQL applications. Proceedings of the VLDB Endowment, 16(6):1534–1547. find
  12. Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2019. Towards complex text-to-SQL in cross-domain database with intermediate representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4524–4535, Florence,… find
  13. Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2025. Next-generation database interfaces: A survey of llm-based text-to-sql. IEEE Transactions on Knowledge and Data Engineering, 37(12):7328–7345. find
  14. Zixian Huang, Wenhao Zhu, Gong Cheng, Lei Li, and Fei Yuan. 2024. MindMerger: Efficiently Boosting LLM Reasoning in non-English Languages. Advances in Neural Information Processing Systems, 37:34161–34187. find
  15. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  16. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marcelo Archanjo José and Fabio Gagliardi Cozman. 2021. mrat-sql+gap: A portuguese text-to-sql transformer. In Intelligent Systems, pages 511–525,… find
  17. Marcos Menon José, Marcelo Archanjo José, Denis Deratani Mauá, and Fábio Gagliardi Cozman. 2022. Integrating question answering and text-to-sql in portuguese. In Computational Processing of the Portuguese Language, pages 278–287, Cham. Springer International Publishing. find
  18. George Katsogiannis-Meimarakis and Georgia Koutrika. 2023. A survey on deep learning approaches for text-to-sql. The VLDB Journal, 32(4):905–936. find
  19. Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2025. MCS-SQL: Leveraging multiple prompts and multiple-choice selection for text-to-SQL generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 337–353, Abu Dhabi, UAE. Association for Computation…
  20. Simone Papicchio, Simone Rossi, Luca Cagliero, and Paolo Papotti. 2025. Think2sql: Reinforce llm reasoning capabilities for text2sql. Preprint, arXiv:2504.15077. find
  21. Breno Carvalho Pedroso, Marluce Rodrigues Pereira, and Denilson Alves Pereira. 2025. Performance evaluation of LLMs in the Text-to-SQL task in Portuguese. In Simpósio Brasileiro de Sistemas de Informação (SBSI), pages 260–269. SBC. find
  22. Khanh Trinh Pham, Thu Huong Nguyen, Jun Jo, Quoc Viet Hung Nguyen, and Thanh Tam Nguyen. 2025. Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents. arXiv preprint. ArXiv:2509.24405 [cs]. — [arXiv:2509.24405](https://arxiv.org/abs/2509.24405) find
  23. Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, and Cuiping Li. 2025. Omnisql: Synthesizing high-quality text-to-sql data at scale. Proc. VLDB Endow., 18(11):4695–4709. find
  24. Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan Arik. 2025. Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql. In International Conference on Representation Learni… find
  25. Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, and 1 others. 2023. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems, 36:42330–4…
  26. Mohammadreza Pourreza and Davood Rafiei. 2023. Din-sql: decomposed in-context learning of text-to-sql with self-correction. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  27. Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, and Hangyu Mao. 2024. PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency. Preprint, arXiv:2403.09732. — [arXiv:2403.09732](https://arxiv.org/abs/2403.09732) find
  28. Henrijs Princis, Cristina David, and Alan Mycroft. 2025. Enhancing SQL Query Generation with Neurosymbolic Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(19):19959–19968. find
  29. Jie Lin, Yulong Liang, Jiyan Li, Yi Bai, and Yong Wang. 2024. Nl2sql with partial missing metadata based on multi-view metadata graph compensation and reasoning. Applied Intelligence, 54(2):1511–1524. find
  30. Shu Liu, Alan Zhu, Sumanth Hegde, Shiyi Cao, Shuo Yuan, Samion Suwito, Tyler Griggs, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025a. SkyRL-SQL: Multi-turn SQL Data Agents via RL. In First Workshop on Multi-Turn Interactions in Large Language Models. find
  31. Yifu Liu, Yin Zhu, Yingqi Gao, Zhiling Luo, Xiaoxia Li, Xiaorong Shi, Yuntao Hong, Jinyang Gao, Yu Li, Bolin Ding, and Jingren Zhou. 2025b. Xiyan-sql: A novel multi-generator framework for text-to-sql. Preprint, arXiv:2507.04701. — [arXiv:2507.04701](https://arxiv.org/abs/2507.04701) find
  32. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carneya, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. Openai o1 s…
  33. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022. Language Models are Multilingual Chain-of-Thought Reasoners. https://arxiv.org/abs/2210.03057v1. ↗
  34. Liang Shi, Zhengju Tang, Nan Zhang, Xiaotong Zhang, and Zhi Yang. 2024. A survey on employing large language models for text-to-sql tasks. ACM Computing Surveys. find
  35. Vladislav Shkapenyuk, Divesh Srivastava, Theodore Johnson, and Parisa Ghane. 2025. Automatic Metadata Extraction for Text-to-SQL. arXiv preprint. ArXiv:2505.19988 [cs]. — [arXiv:2505.19988](https://arxiv.org/abs/2505.19988) find
  36. Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis. arXiv preprint. ArXiv:2405.16755 [cs]. — [arXiv:2405.16755](https://arxiv.org/abs/2405.16755) find
  37. Qwen Team. 2025. Qwen3-next-80b-a3b: Qwen3-next: Towards ultimate training & inference efficiency. https://qwen.ai/blog?id=qwen3-next. Accessed: 2025-11-09. ↗
  38. Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-aware schema encoding and linking for text-to-SQL parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7567–7578, Online. Association for…
  39. Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2025a. MAC-SQL: A multi-agent collaborative framework for text-to-SQL. In Proceedings of the 31st International Conference on Computational Linguistics, pages 540–…
  40. Mingyang Wang, Lukas Lange, Heike Adel, Yunpu Ma, Jannik Strötgen, and Hinrich Schuetze. 2025b. Language mixing in reasoning language models: Patterns, impact, and internal causes. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2637–2665, Suzhou, Chi…
  41. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  42. Zhewei Yao, Guoheng Sun, Lukasz Borchmann, Zheyu Shen, Minghang Deng, Bohan Zhai, Hao Zhang, Ang Li, and Yuxiong He. 2025. Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL. arXiv preprint. ArXiv:2505.20315 [cs].
  43. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. Preprint, arXiv:1709.00103.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nobre-2026-language-effects-text-sql-across/