Elamur Research⌕
☰
home / papers / 2025 / Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm

Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm

Paper · 2025 · Zhuo Li, Yuhao Du, Xiaoqi Jiao et al.

Summary

Picking a compact, high-value subset of instruction data can cut supervised fine-tuning (SFT) costs for LLMs. Many existing selectors, such as AlpaGasus and DEITA, rate each sample on its own. This approach has three problems: scores often tie, relations among chosen samples are ignored, and diversity-aware variants must scan the whole dataset first.

The authors take inspiration from the Shapley value and reframe selection as a choice problem. At every step, they draw a small random window from the current subset and another from the remaining pool. A prompted LLM then judges which candidate would add the most quality and diversity to the subset. That sample is added greedily, and the loop repeats until the budget is reached. No full traversal or explicit diversity metric is needed.

On Alpaca, a subset under 10% of the data beats training on the full dataset and is competitive with state-of-the-art selectors while needing fewer selections. A medical-domain test supports the method's practical value for cheaper, task-agnostic data curation.

Key points

  • Selection criterion: The method ranks candidates by their marginal contribution to the subset, framed as a greedy arg-max of gain, rather than by standalone pointwise scores. The subset windows A′ and B′ keep each LLM query small.
  • Alpaca results: Roughly 9K selected samples (versus the full 52K) outperform both Full and Random training. The selected subset also shows higher TTR, MTLD, DEITA quality, and reward scores, plus a lower SDI, than AlpaGasus and IFD subsets, while keeping a similar average token length.
  • Medical scaling test: The authors fine-tuned Baichuan2-7B-Chat on HuatuoGPT-sft-data-v1 and evaluated it on the Chinese Medical Benchmark (CMB). Using 20% of the data, the subset reportedly raises the average score by about 7.04 points relative to the Base model and by 4.36 relative to IFD; it also surpasses Full (≈51.3 vs 49.4 avg).
  • Limitations: Results depend on the judging LLM's abilities and biases. The window sizes and prompt design must be tuned by hand.
  • Availability: Code and data are released at github.com/BIRlz/comperative_sample_selection.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shenzhen Research Institute of Big DataShenzhen · China17 works · 1,721 citations🎓Hefei University of TechnologyHefei · China10 works · 1,615 citations🎓Birmingham City UniversityBirmingham · United Kingdom5 works · 26 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Zhuo Li, Yuhao Du, Xiaoqi Jiao, Steven Y. Guo, Yuege Feng, Xiang Wan, Anningzhe Gao and Jinpeng Hu. 2025. Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.270

@inproceedings{li2025add,
  title = {{Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm}},
  author = {Zhuo Li, Yuhao Du, Xiaoqi Jiao, Steven Y. Guo, Yuege Feng, Xiang Wan, Anningzhe Gao and Jinpeng Hu},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.270},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.270}
}

References

56 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2022. In-context examples selection for machine translation. Preprint, arXiv:2212.02437.
  2. Baichuan. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305.
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  4. Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. 2024. Instruction mining: Instruction data selection for tuning large language models. Preprint, arXiv:2307.06290. find
  5. Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. 2024. Alpagasus: Training a better alpaca with fewer data. Preprint, arXiv:2307.08701.
  6. Lin Chen, Moran Feldman, and Amin Karbasi. 2018. Weakly submodular maximization beyond cardinality constraints: Does randomization help greedy? In International Conference on Machine Learning, pages 804–813. PMLR. find
  7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna.lmsys. org (accessed 14 April 2023), 2(3):6. ↗
  8. Marco Cuturi. 2013. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc. find
  9. Chongyuan Dai, Jinpeng Hu, Hongchang Shi, Zhuo Li, Xun Yang, and Meng Wang. 2025. Psyche-r1: Towards reliable psychological llms through unified empathy, expertise, and reasoning. arXiv preprint arXiv:2508.10848.
  10. Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023. Raft: Reward ranked finetuning for generative foundation model alignment. Preprint, arXiv:2304.06767.
  11. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234.
  12. Yuhao Du, Zhuo Li, Pengyu Cheng, Zhihong Chen, Yuejiao Xie, Xiang Wan, and Anningzhe Gao. 2025. Simplify rlhf as reward-weighted sft: A variational method. Preprint, arXiv:2502.11026. — [arXiv:2502.11026](https://arxiv.org/abs/2502.11026) find
  13. Yuhao Du, Zhuo Li, Pengyu Cheng, Xiang Wan, and Anningzhe Gao. 2024. Detecting ai flaws: Target-driven attacks on internal faults in language models. Preprint, arXiv:2408.14853. — [arXiv:2408.14853](https://arxiv.org/abs/2408.14853) find
  14. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Preprint, arXiv:2009.03300.
  15. Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large language models are zero-shot rankers for recommender systems. Preprint, arXiv:2305.08845. find
  16. Jinpeng Hu, DanDan Guo, Yang Liu, Zhuo Li, Zhihong Chen, Xiang Wan, and Tsung-Hui Chang. 2023. A simple yet effective subsequence-enhanced approach for cross-domain ner. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12890–12898. find
  17. Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, and Tsung-Hui Chang. 2022. Graph enhanced contrastive learning for radiology findings summarization. Preprint, arXiv:2204.00203. find
  18. Jinpeng Hu, Hongchang Shi, Chongyuan Dai, Zhuo Li, Peipei Song, and Meng Wang. 2025a. Beyond emotion recognition: A multi-turn multimodal emotion understanding and reasoning benchmark. arXiv preprint arXiv:2508.16859. — [arXiv:2508.16859](https://arxiv.org/abs/2508.16859) find
  19. Jinpeng Hu, Ao Wang, Qianqian Xie, Hui Ma, Zhuo Li, and Dan Guo. 2025b. Agentmental: An interactive multi-agent framework for explainable and adaptive mental health assessment. Preprint, arXiv:2508.11567. — [arXiv:2508.11567](https://arxiv.org/abs/2508.11567) find
  20. Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, and Jaewoo Kang. 2020. Look at the first sentence: Position bias in question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1109–1121, Online. Association for Computational Linguis…
  21. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  22. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024a. Llms-as-judges: A comprehensive survey on llm-based evaluation methods. Preprint, arXiv:2412.05579.
  23. Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024b. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. Preprint, arXiv:2402.00530.
  24. Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. 2024c. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning. Preprint, arXiv:2308.12032.
  25. Zhuo Li, Yuhao Du, Jinpeng Hu, Xiang Wan, and Anningzhe Gao. 2025. Self-instructed derived prompt generation meets in-context learning: Unlocking new potential of black-box LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages…
  26. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2024a. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In The Twelfth International Conference on Learning Representations.
  27. Ziche Liu, Rui Ke, Feng Jiang, and Haizhou Li. 2024b. Take the essence and discard the dross: A rethinking on data selection for fine-tuning large language models. Preprint, arXiv:2406.14115. find
  28. Zifan Liu, Amin Karbasi, and Theodoros Rekatsinas. 2024c. Tsds: Data selection for task-specific model finetuning. Preprint, arXiv:2410.11303. — [arXiv:2410.11303](https://arxiv.org/abs/2410.11303) find
  29. Philip M McCarthy and Scott Jarvis. 2010. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381–392. find
  30. Tai Nguyen and Eric Wong. 2023. example selection with influences. arXiv:2302.11042. — [arXiv:2302.11042](https://arxiv.org/abs/2302.11042) find
  31. In-context Preprint, OpenAI. 2024. Gpt-4 technical report. arXiv:2303.08774.
  32. Preprint, Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2… — Training language models to follow instructions with human feedback
  33. Brian Richards. 1987. Type/token ratios: what do they really tell us? Journal of child language, 14:201–9. find
  34. Lloyd S Shapley. 1951. Notes on the n-person game—ii: The value of an n-person game. find
  35. E. H. Simpson. 1997. Measurement of diversity. Journal of Cardiothoracic and Vascular Anesthesia, 11(6):812–812. find
  36. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. Preprint, arXiv:2210.09261.
  37. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca. ↗
  38. Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. 2023. D4: Improving llm pre-training via document de-duplication and diversification. Preprint, arXiv:2308.12284.
  39. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. Preprint…
  40. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023. Large language models are not fair evaluators. Preprint, arXiv:2305.17926. find
  41. Xidong Wang, Guiming Hardy Chen, Dingjie Song, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Feng Jiang, Jianquan Li, Xiang Wan, Benyou Wang, and Haizhou Li. 2024. Cmb: A comprehensive medical benchmark in chinese. Preprint, arXiv:2308.08833.
  42. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560. find
  43. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. Preprint, arXiv:2109.01652.
  44. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023a. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  45. Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. 2023b. Instructiongpt-4: A 200-instruction paradigm for fine-tuning minigpt-4. Preprint, arXiv:2308.12067. — [arXiv:2308.12067](https://arxiv.org/abs/2308.12067) find
  46. William J. Welch. 1982. Algorithmic complexity: Three np-hard problems in computational statistics. Journal of Statistical Computation and Simulation, 15(1):17–25. find
  47. Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. 2024. Qurating: Selecting high-quality data for training language models. Preprint, arXiv:2402.09739. find
  48. Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024,… find
  49. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. Less: Selecting influential data for targeted instruction tuning. Preprint, arXiv:2402.04333.
  50. Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. 2023. Data selection for language models via importance resampling. Preprint, arXiv:2302.03169. find
  51. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. Preprint, arXiv:2304.12244.
  52. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… find
  53. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? Preprint, arXiv:1905.07830. find
  54. Kaiqi Zhang, Jing Zhao, and Rui Chen. 2024. Koala: Enhancing speculative decoding for llm via multi-layer draft heads with adversarial learning. Preprint, arXiv:2408.08146. — [arXiv:2408.08146](https://arxiv.org/abs/2408.08146) find
  55. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Preprint, arXiv:2306.05685.
  56. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. Lima: Less is more for alignment. Preprint, arXiv:2305.11206. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-add-one-incremental-sample-selection/