Elamur Research⌕
☰
home / papers / 2025 / DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking

DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking

Paper · 2025 · Zhuoqun Li, Haiyang Yu, Xuanang Chen et al.

Summary

Research on retrieval-augmented generation (RAG) has mostly targeted multi-hop and long-form question answering. A practically important case has been left aside: producing a complete, workable engineering design that satisfies several real-world constraints at once, such as building a hospital on swelling soil in a rainy, earthquake-prone region. The authors make two contributions.

The first is SolutionBench. They gathered thousands of peer-reviewed engineering reports from journals in eight fields. GPT-4o, guided by a hand-made template, pulled out requirements, expert solutions, analytical and technical knowledge, and design explanations. The authors then verified the output manually and pooled each domain's knowledge into a shared knowledge base.

The second is SolutionRAG. It grows a tree of candidate solutions, alternating between drafting designs and critiquing them against the constraints ("bi-point thinking"). Node scoring prunes weak branches to keep inference affordable.

SolutionRAG was compared with deep-thinking LLMs without retrieval, standard RAG and multi-round iterative RAG. It achieves state-of-the-art results on SolutionBench. The work frames engineering design as a new, harder RAG task and provides a reusable testbed for it.

Key points

  • SolutionBench spans eight fields: farming, water resources, architecture, telecommunications, aerospace, transportation, mining and environmental engineering.
  • Building the benchmark involved four steps:
    • converting PDFs to text with the marker tool;
    • extracting five fields (requirement, solution, analytical knowledge, technical knowledge, explanation) with GPT-4o;
    • storing the results as JSON;
    • verifying and deduplicating them manually.
  • Ablations show that removing either tree-based exploration or bi-point thinking causes a clear drop, and the two losses are of similar size.
  • Solution quality rises from tree layer 1 to layers 3 and 5. Retained nodes score noticeably higher than pruned ones, which supports the node-evaluation pruning.
  • The system uses off-the-shelf LLMs with no extra training. Because of limited GPU resources, the authors did not tune tree width or depth, and they suggest RL-based training as future work. The code is at github.com/icip-cas/DeepSolution.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🔬Institute of SoftwareBeijing · China21 works · 4,980 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksEngineering · 133 worksMechanical Engineering · 3 works

Cite

Cite as Zhuoqun Li, Haiyang Yu, Xuanang Chen, Hongyu Lin, Yaojie Lu, Fei Huang, Xianpei Han, Yongbin Li and Le Sun. 2025. DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.220

@inproceedings{li2025deepsolution,
  title = {{DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking}},
  author = {Zhuoqun Li, Haiyang Yu, Xuanang Chen, Hongyu Lin, Yaojie Lu, Fei Huang, Xianpei Han, Yongbin Li and Le Sun},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.220},
  url = {https://doi.org/10.18653/v1/2025.acl-long.220}
}

References

38 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Fortus, Joseph Krajcik, Ralph Charles Dershimer, Ronald W Marx, and Rachel Mamlok-Naaman. 2005. Design-based science and real-world problem-solving. International Journal of Science Education, 27(7):855–879. find
  2. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online). International Comm…
  3. Jinhao Jiang, Jiayi Chen, Junyi Li, Ruiyang Ren, Shijie Wang, Wayne Xin Zhao, Yang Song, and Tao Zhang. 2024. Rag-star: Enhancing deliberative reasoning with retrieval augmented verification and refinement. Preprint, arXiv:2412.12881.
  4. Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992, Singapore. Association fo…
  5. David Jonassen, Johannes Strobel, and Chwee Beng Lee. 2006. Everyday problem solving in engineering: Lessons for engineering educators. Journal of engineering education, 95(2):139–151. find
  6. Katharina Kalogerakis, Christian Lüthje, and Cornelius Herstatt. 2010. Developing innovations based on analogies: experience from design and engineering consultants. Journal of Product Innovation Management, 27(3):418–436. find
  7. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.
  8. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. Nv-embed: Improved techniques for training llms as generalist embedding models. Preprint, arXiv:2405.17428.
  9. Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. 2024. RQ-RAG: Learning to refine queries for retrieval augmented generation. In First Conference on Language Modeling. find
  10. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc…
  11. Olivier L De Weck, Daniel Roos, and Christopher L Magee. 2011. Engineering systems: Meeting human needs in a complex technological world. Mit Press. find
  12. Waguih ElMaraghy, Hoda ElMaraghy, Tetsuo Tomiyama, and Laszlo Monostori. 2012. Complexity in engineering design and manufacturing. CIRP annals, 61(2):793–814. find
  13. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. ELI5: Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558–3567, Florence, Italy. Association for Computational Linguistics.
  14. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025a. Search-o1: Agentic search-enhanced large reasoning models. Preprint, arXiv:2501.05366.
  15. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. Preprint, arXiv:2308.03281.
  16. Zhuoqun Li, Xuanang Chen, Haiyang Yu, Hongyu Lin, Yaojie Lu, Qiaoyu Tang, Fei Huang, Xianpei Han, Le Sun, and Yongbin Li. 2025b. StructRAG: Boosting knowledge intensive reasoning of LLMs via inference-time hybrid information structurization. In The Thirteenth International Conference on Learning Rep… find
  17. Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Unified structure generation for universal information extraction. Preprint, arXiv:2203.12277. find
  18. James Mayfield, Eugene Yang, Dawn Lawrie, Sean MacAvaney, Paul McNamee, Douglas W. Oard, Luca Soldaini, Ian Soboroff, Orion Weller, Efsun Kayi, Kate Sanders, Marc Mason, and Noah Hibbler. 2024. On the evaluation of machine-generated reports. In Proceedings of the 47th International ACM SIGIR Confere… find
  19. Madara Ogot and Gul Kremer. 2004. Engineering design: a practical guide. Trafford Publishing. find
  20. OpenAI. 2024a. Gpt-4o system card. arXiv:2410.21276.
  21. Preprint, OpenAI. 2024b. Openai o1 system card. Preprint, arXiv:2412.16720.
  22. Zehan Qi, Rongwu Xu, Zhijiang Guo, Cunxiang Wang, Hao Zhang, and Wei Xu. 2024. long 2 rag: Evaluating long-context & long-form retrieval-augmented generation with key point recall. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4852–4872, Miami, Florida, USA. Associa… find
  23. Team Qwen. 2024. Qwq: Reflect deeply on the boundaries of the unknown. find
  24. Zhengliang Shi, Shuo Zhang, Weiwei Sun, Shen Gao, Pengjie Ren, Zhumin Chen, and Zhaochun Ren. 2024. Generate-then-ground in retrieval-augmented generation for multi-hop question answering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape…
  25. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273–8288, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  26. Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, and Linqi Song. 2024. ProxyQA: An alternative framework for evaluating long-form text generation with large language models. In Proceedings of the 62nd Annual Meeting of the Associ…
  27. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  28. Hieu Tran, Zonghai Yao, Junda Wang, Yifan Zhang, Zhichao Yang, and Hong Yu. 2024. Rare: Retrieval-augmented reasoning enhancement for large language models. Preprint, arXiv:2412.02830.
  29. Minzheng Wang, Longze Chen, Cheng Fu, Shengyi Liao, Xinghua Zhang, Bingli Wu, Haiyang Yu, Nan Xu, Lei Zhang, Run Luo, Yunshui Li, Min Yang, Fei Huang, and Yongbin Li. 2024a. Leave no document behind: Benchmarking long-context llms with extended multi-doc qa. Preprint, arXiv:2406.17419.
  30. Ruobing Wang, Daren Zha, Shi Yu, Qingfei Zhao, Yuxuan Chen, Yixuan Wang, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, Zhiyuan Liu, and Maosong Sun. 2024b. Retriever-and-memory: Towards adaptive note-enhanced retrieval-augmented generation. Preprint, arXiv:2410.08821. — [arXiv:2410.08821](https://arxiv.org/abs/2410.08821) find
  31. Feijie Wu, Zitao Li, Fei Wei, Yaliang Li, Bolin Ding, and Jing Gao. 2025. Talk to right specialists: Routing and planning in multi-agent system for question answering. Preprint, arXiv:2501.07813. — [arXiv:2501.07813](https://arxiv.org/abs/2501.07813) find
  32. Jian Wu, Linyi Yang, Zhen Wang, Manabu Okumura, and Yue Zhang. 2024. Cofca: A step-wise counterfactual multi-hop qa benchmark. Preprint, arXiv:2402.11924. — [arXiv:2402.11924](https://arxiv.org/abs/2402.11924) find
  33. Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi. 2023. A critical evaluation of evaluations for long-form question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3225–3245, Toronto, Canada. Association for C…
  34. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. Preprint, arXiv:1809.09600.
  35. Tian Yu, Shaolei Zhang, and Yang Feng. 2024. Auto-rag: Autonomous retrieval-augmented generation for large language models. arXiv preprint arXiv:2411.19443. — [arXiv:2411.19443](https://arxiv.org/abs/2411.19443) find
  36. Zikang Zhang, Wangjie You, Tianci Wu, Xinrui Wang, Juntao Li, and Min Zhang. 2025. A survey of generative information extraction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4840–4870, Abu Dhabi, UAE. Association for Computational Linguistics. find
  37. Yujia Zhou, Zheng Liu, and Zhicheng Dou. 2024. Boosting the potential of large language models with an intelligent information assistant. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  38. Environment Andrew Zhu, Alyssa Hwang, Liam Dugan, and Chris Callison-Burch. 2024. Fanoutqa: A multi-hop, multi-document question answering benchmark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-deepsolution-boosting-complex-engineering-solution/