Elamur Research⌕
☰
home / papers / 2026 / SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science

SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science

Paper · 2026 · Wonduk Seo, Juhyeon Lee, Yanjun Shao et al.

Summary

LLM-powered data science agents such as Agent K, AutoKaggle and Data Interpreter usually follow one fixed chain of steps. This limits how many strategies they try and can produce weaker pipelines. SPIO (Sequential Plan Integration and Optimization) tackles this by dividing tabular supervised learning into four stages: preparing the data, engineering features, choosing a model, and tuning its hyperparameters. Code-generation agents first build a baseline for each stage from a data description and a task description. A sequential planning agent then suggests several alternative strategies per stage. It uses data summaries for the early stages and validation scores for the later ones, and it takes earlier plans into account. An LLM optimization agent then either picks one complete pipeline (SPIO-S) or combines predictions from the top-k pipelines (SPIO-E). On regression and classification tasks from Kaggle and OpenML, SPIO beat prior state-of-the-art systems by 5.6% on average. This shows that exploring and merging several solution paths gives more accurate and robust automated analytics than a single-route workflow.

Key points

  • Two operating modes: SPIO-S picks one best end-to-end path, and SPIO-E ensembles the top-k paths. In the ablations, a Top-k = 2 ensemble gave the most dependable improvement, and removing core modules hurt results.
  • Titanic example: GPT-4o reached ACC 0.78 with zero-shot prompting and 0.80 with CoT. SPIO-S raised this to 0.84 and SPIO-E to 0.86.
  • Plan diversity and agreement:
    • The generated plans were embedded with text-embedding-ada-002 and projected with PCA. Preprocessing and feature-engineering plans were widely spread, which suggests the choices adapt to each dataset.
    • In a qualitative evaluation, the LLM's top-ranked plan was chosen 72 out of 100 times.
  • Stability: Across five GPT-4o runs, the main recommendations repeated consistently. For example, on Diamonds every run proposed GBM with Optuna. Only lower-ranked options varied.
  • Scope: The work covers tabular supervised learning only, with whitelisted libraries. Results depend on the LLM backend and prompts, and global optimality is not guaranteed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,267 citations🎓Yale UniversityNew Haven · United States101 works · 13,302 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksMaterials Science · 31 worksMaterials Chemistry · 31 works

Cite

Cite as Wonduk Seo, Juhyeon Lee, Yanjun Shao, Qingshan Zhou, Seunghyun Lee and Yi Bu. 2026. SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1039

@inproceedings{seo2026spio,
  title = {{SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science}},
  author = {Wonduk Seo, Juhyeon Lee, Yanjun Shao, Qingshan Zhou, Seunghyun Lee and Yi Bu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1039},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1039}
}

References

61 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Sonnet Anthropic. 2024. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. URL https://api.semanticscholar. org/CorpusID, 273639283. ↗
  3. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. The journal of machine learning research, 13(1):281–305. find
  4. Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning, volume 4. Springer. find
  5. Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Wenjing Hu, Yuchen Mao, and 1 others. 2024. Spider2-v: How far are multimodal agents from automating data science and engineering workflows? Advances in Neural Information Processing Systems… find
  6. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, and 1 others. 2024. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXiv preprint arXiv:2410.07095.
  7. Liying Cheng, Xingxuan Li, and Lidong Bing. 2023. Is gpt-4 a good data analyst? arXiv preprint arXiv:2305.15038. find
  8. Will Cukierski. 2012. Titanic - machine learning from disaster. https://kaggle.com/competitions/ti tanic. Kaggle. ↗
  9. Sumit Das, Aritra Dey, Akash Pal, and Nabamita Roy. 2015. Applications of artificial intelligence in machine learning: review and prospect. International Journal of Computer Applications, 115(9). find
  10. T De Bie, L De Raedt, J Hernández-Orallo, HH Hoos, P Smyth, and CKI Williams. 2021. Automating data science: Prospects and challenges. 65 (3), 76–87. find
  11. Victor Dibia. 2023. Lida: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models. arXiv preprint arXiv:2303.02927. find
  12. Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni de Paula Lourenco, Jorge Piazentin Ono, Kyunghyun Cho, Claudio Silva, and Juliana Freire. 2021. Alphad3m: Machine learning pipeline synthesis. arXiv preprint arXiv:2111.02508. 22690 10 find
  13. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  14. Nassi Ebadifard, Ajitesh Parihar, Youry Khmelevsky, Gaétan Hains, Albert Wong, and Frank Zhang. 2023. Data extraction, transformation, and loading process automation for algorithmic trading machine learning modelling and performance optimization. arXiv preprint arXiv:2312.12774. — [arXiv:2312.12774](https://arxiv.org/abs/2312.12774) find
  15. Issam El Naqa and Martin J Murphy. 2015. What is machine learning? In Machine learning in radiation oncology: theory and applications, pages 3–11. Springer. find
  16. Thomas Elsken, Jan-Hendrik Metzen, and Frank Hutter. 2017. Simple and efficient architecture search for convolutional neural networks. arXiv preprint arXiv:1711.04528. find
  17. Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. 2020. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505. — [arXiv:2003.06505](https://arxiv.org/abs/2003.06505) find
  18. Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. 2022. Auto-sklearn 2.0: Hands-free automl via meta-learning. Journal of Machine Learning Research, 23(261):1–61. find
  19. Antoine Grosnit, Alexandre Maraval, James Doran, Giuseppe Paolo, Albert Thomas, Refinath Shahul Hameed Nabeezath Beevi, Jonas Gonzalez, Khyati Khandelwal, Ignacio Iacobacci, Abdelhakim Benechehab, and 1 others. 2024. Large language models orchestrating structured reasoning achieve kaggle grandmaster… — [arXiv:2411.03562](https://arxiv.org/abs/2411.03562) find
  20. Yang Gu, Hengyu You, Jian Cao, Muran Yu, Haoran Fan, and Shiyou Qian. 2024. Large language models for constructing and optimizing machine learning workflows: A survey. ACM Transactions on Software Engineering and Methodology. find
  21. Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. 2024. Ds-agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453. find
  22. Md Mahadi Hassan, Alex Knipper, and Shubhra Kanti Karmaker Santu. 2023. Chatgpt as your personal data scientist. arXiv preprint arXiv:2305.13657. find
  23. J Matthew Helm, Andrew M Swiergosz, Heather S Hae-berle, Jaret M Karnuta, Jonathan L Schaffer, Viktor E Krebs, Andrew I Spitzer, and Prem N Ramkumar. 2020. Machine learning and artificial intelligence: definitions, applications, and future directions. Current reviews in musculoskeletal medicine, 13(… find
  24. Noah Hollmann, Samuel Müller, and Frank Hutter. 2023. Large language models for automated data science: Introducing caafe for context-aware automated feature engineering. Advances in Neural Information Processing Systems, 36:44753–44775. find
  25. Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, and 1 others. 2025. Data interpreter: An llm agent for data science. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796–19821.
  26. Addison Howard, Ashley Chow, and Ryan Holbrook. 2022. Spaceship titanic. https://kaggle.com/c ompetitions/spaceship-titanic. Kaggle. ↗
  27. Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2023. Mlagentbench: Evaluating language agents on machine learning experimentation. arXiv preprint arXiv:2310.03302.
  28. Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Deniss Jacenko, and Yuxiang Wu. 2025. Aide: Ai-driven exploration in the space of code. arXiv preprint arXiv:2502.13138. find
  29. Sathvik Joel, Jie Wu, and Fatemeh Fard. 2024. A survey on llm-based code generation for low-resource and domain-specific programming languages. ACM Transactions on Software Engineering and Methodology. find
  30. Wendy Kan. 2016. Ghouls, goblins, and ghosts... boo! https://kaggle.com/competitions/ghouls-g oblins-and-ghosts-boo. Kaggle. ↗
  31. James Max Kanter and Kalyan Veeramachaneni. 2015. Deep feature synthesis: Towards automating data science endeavors. In 2015 IEEE international conference on data science and advanced analytics (DSAA), pages 1–10. IEEE. find
  32. Ambika Kaul, Saket Maheshwary, and Vikram Pudi. 2017. Autolearn—automated feature generation and selection. In 2017 IEEE International Conference on data mining (ICDM), pages 217–226. IEEE. find
  33. Efstathios Kirkos, Charalambos Spathis, and Yannis Manolopoulos. 2008. Support vector machines, decision trees and neural networks for auditor selection. Journal of computational Methods in Sciences and Engineering, 8(3):213–224. find
  34. Hoang Thanh Lam, Johann-Michael Thiebaut, Mathieu Sinn, Bei Chen, Tiep Mai, and Oznur Alkan. 2017. One button machine for automating feature engineering in relational databases. arXiv preprint arXiv:1706.00327. 22691 11 — [arXiv:1706.00327](https://arxiv.org/abs/1706.00327) find
  35. Erin LeDell, Sebastien Poirier, and 1 others. 2020. H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML, volume 2020, page 24. find
  36. Ziming Li, Qianbo Zang, David Ma, Jiawei Guo, Tuney Zheng, Minghao Liu, Xinyao Niu, Yue Wang, Jian Yang, Jiaheng Liu, and 1 others. 2024. Autokaggle: A multi-agent framework for autonomous data science competitions. arXiv preprint arXiv:2410.20424. find
  37. Petar Maymounkov. 2018. Koji: Automating pipelines with mixed-semantics data sources. arXiv preprint arXiv:1901.01908. — [arXiv:1901.01908](https://arxiv.org/abs/1901.01908) find
  38. Anna Montoya and DataCanary. 2016. House prices -advanced regression techniques. https://kaggle.com/competitions/house-prices-advanced-r egression-techniques. Kaggle. ↗
  39. Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, and 1 others. 2025. Mlgym: A new framework and benchmark for advancing ai research agents. arXiv preprint arXiv:2502.14499. find
  40. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474. find
  41. Nikolay O Nikitin, Pavel Vychuzhanin, Mikhail Sarafanov, Iana S Polonskaia, Ilia Revin, Irina V Barabanova, Gleb Maximov, Anna V Kalyuzhnaya, and Alexander Boukhanovsky. 2022. Automated evolutionary approach for the design of composite machine learning pipelines. Future Generation Computer Systems,… find
  42. Randal S Olson and Jason H Moore. 2016. Tpot: A tree-based pipeline optimization tool for automating machine learning. In Workshop on automatic machine learning, pages 66–74. PMLR. find
  43. Walter Reade and Ashley Chow. 2024a. Binary classification with a bank churn dataset. https://kaggle.com/competitions/playground-series-s4e1. Kaggle. ↗
  44. Walter Reade and Ashley Chow. 2024b. Classification with an academic success dataset. https://kaggle.com/competitions/playground-series-s4e6. Kaggle. ↗
  45. Walter Reade and Ashley Chow. 2024c. Multi-class prediction of obesity risk. https://kaggle.com/com petitions/playground-series-s4e2. Kaggle. ↗
  46. Walter Reade and Ashley Chow. 2024d. Steel plate defect prediction. https://kaggle.com/competi tions/playground-series-s4e3. Kaggle. ↗
  47. Walter Reade and Elizabeth Park. 2025a. Backpack prediction challenge. https://kaggle.com/compe titions/playground-series-s5e2. Kaggle. ↗
  48. Walter Reade and Elizabeth Park. 2025b. Binary prediction with a rainfall dataset. https://kaggle.com/competitions/playground-series-s5e3. Kaggle. ↗
  49. Walter Reade and Elizabeth Park. 2025c. Predict podcast listening time. https://kaggle.com/competi tions/playground-series-s5e4. Kaggle. ↗
  50. Alireza Salemi, Mihir Parmar, Palash Goyal, Yiwen Song, Jinsung Yoon, Hamed Zamani, Hamid Palangi, and Tomas Pfister. 2025. Llm-based multi-agent blackboard system for information discovery in data science. arXiv preprint arXiv:2510.01285. — [arXiv:2510.01285](https://arxiv.org/abs/2510.01285) find
  51. Iqbal H Sarker. 2021. Machine learning: Algorithms, real-world applications and research directions. SN computer science, 2(3):160. find
  52. Wonduk Seo, Seungyong Lee, Daye Kang, Zonghao Yuan, and Seunghyun Lee. 2025. Vispath: Automated visualization code synthesis via multi-path reasoning and feedback-driven optimization. arXiv e-prints, pages arXiv–2502. find
  53. Zhenqian Shen, Yongqi Zhang, Lanning Wei, Huan Zhao, and Quanming Yao. 2018. Automated machine learning: From principles to practices. arXiv preprint arXiv:1810.13306. — [arXiv:1810.13306](https://arxiv.org/abs/1810.13306) find
  54. Evan R Sparks, Shivaram Venkataraman, Tomer Kaftan, Michael J Franklin, and Benjamin Recht. 2017. Keystoneml: Optimizing pipelines for large-scale advanced analytics. In 2017 IEEE 33rd international conference on data engineering (ICDE), pages 535– 546. IEEE. find
  55. Richard S Sutton, Andrew G Barto, and 1 others. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge. find
  56. Xiangru Tang, Yuliang Liu, Zefan Cai, Yanjun Shao, Junjie Lu, Yichi Zhang, Zexuan Deng, Helan Hu, Kaikai An, Ruijun Huang, and 1 others. 2023. Ml-bench: Evaluating large language models and agents for machine learning tasks on repository-level code. arXiv preprint arXiv:2311.09835.
  57. Soroosh Tayebi Arasteh, Tianyu Han, Mahshad Lotfinia, Christiane Kuhl, Jakob Nikolas Kather, Daniel Truhn, and Sven Nebelung. 2024. Large language models streamline automated machine learning for clinical studies. Nature Communications, 15(1):1603. find
  58. Chris Thornton, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2013. Auto-weka: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 847–855. 22692 12 find
  59. Joaquin Vanschoren, Jan N Van Rijn, Bernd Bischl, and Luis Torgo. 2014. Openml: networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15(2):49–60. find
  60. Dakuo Wang, Josh Andres, Justin D Weisz, Erick Oduor, and Casey Dugan. 2021. Autods: Towards human-centered automation of data science. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1–12. find
  61. Peiran Wang, Yaoning Yu, Ke Chen, Xianyang Zhan, and Haohan Wang. 2025. Large language model-based data science agent: A survey. arXiv preprint arXiv:2508.02744. — [arXiv:2508.02744](https://arxiv.org/abs/2508.02744) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seo-2026-spio-ensemble-selective-strategies-llm/