Elamur Research⌕
☰
home / papers / 2023 / Platypus: Quick, Cheap, and Powerful Refinement of LLMs

Platypus: Quick, Cheap, and Powerful Refinement of LLMs

Paper · 2023 · Ariel N. Lee, Cole J. Hunter and Nataniel Ruiz

Summary

Adapting large language models to specialised skills usually requires large amounts of data and compute. Lee, Hunter and Ruiz present Platypus, a family of LLaMa-2-based models refined with LoRA adapters on Open-Platypus, a compact instruction set they curated. Open-Platypus draws on 11 open sources focused on STEM and logic. Its core is a modified MATH set enriched with step-by-step PRM800K solutions, and all items use the Alpaca format. The authors remove exact repeats and near-duplicates, then screen the data against benchmark test sets. They manually sort suspicious items into duplicate, gray-area, and similar-but-different cases and discard all three groups. The trained adapters are merged with other fine-tuned models, chosen only if their training data appeared clean. At the time of writing, Platypus models ranked first on Hugging Face's Open LLM Leaderboard while using far less data and compute than competing fine-tunes. The work suggests that careful data selection, leak filtering and adapter merging can partly replace sheer scale. However, the benefit of merging depends heavily on the domain.

Key points

  • Open-Platypus combines 11 public datasets, including PRM800K, MATH, ScienceQA, ReClor, TheoremQA, ARB, airoboros and Openassistant-Guanaco. Roughly 10% of its questions were generated by an LLM, and the dataset is publicly released. Because of source licences, the models are restricted to non-commercial use.
  • A 13B Platypus model can be fine-tuned on a single A100 GPU with about 25k questions in roughly 5 hours.
  • Near-duplicates were identified using SentenceTransformers embeddings with an 80% cosine similarity threshold. When two items matched, the pair with the longer, more detailed answer was kept. The same 80% threshold flagged training items too close to benchmark questions.
  • Only near-exact copies of test items are counted as "true" contamination. Merge candidates were also checked for leaks, including SciQ and dolphin, which were not added to the training set.
  • Merging effects vary by task. For example, adding the Platypus adapter to Camel-Platypus2-70B lowered college-physics performance by 18.42%. The models are English-centric and have not been red-teamed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Boston UniversityBoston · United States30 works · 4,343 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Ariel N. Lee, Cole J. Hunter and Nataniel Ruiz. 2023. Platypus: Quick, Cheap, and Powerful Refinement of LLMs. arXiv:2308.07317.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.07317 · arXiv 2308.07317

@misc{lee2023platypus,
  title = {{Platypus: Quick, Cheap, and Powerful Refinement of LLMs}},
  author = {Ariel N. Lee, Cole J. Hunter and Nataniel Ruiz},
  year = {2023},
  doi = {10.48550/arxiv.2308.07317},
  eprint = {2308.07317},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.07317}
}

References

50 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Aljundi, P. Chakravarty, and T. Tuytelaars. Expert gate: Lifelong learning with a network of experts. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7120–7129, Los Alamitos, CA, USA, jul 2017. IEEE Computer Society. find
  2. E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo. Falcon-40b: an open large language model with state-of-the-art performance, 2023. find
  3. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… — Language Models are Few-Shot Learners
  4. CalderaAI. 30b-lazarus. https://huggingface.co/CalderaAI/30B-Lazarus, 2023. ↗
  5. W. Chen, M. Yin, M. Ku, E. Wan, X. Ma, J. Xu, T. Xia, X. Wang, and P. Lu. Theoremqa: A theorem-driven question answering dataset. arXiv preprint arXiv: 2305.12524, 2023.
  6. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-… — PaLM: Scaling Language Modeling with Pathways
  7. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J.… find
  8. T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv: 2305.14314, 2023. airoboros-gpt4-1.4.1. https://huggingface.co/jondurbin/ ↗
  9. J. Durbin. airoboros-gpt4-1.4.1, 2023. find
  10. ehartford. dolphin. https://huggingface.co/datasets/ehartford/dolphin, 2023. ↗
  11. W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022. find
  12. S. Gunasekar, Y. Zhang, J. Aneja, C. C. T. Mendes, A. D. Giorno, S. Gopi, M. Javaheripi, P. Kauffmann, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, H. S. Behl, X. Wang, S. Bubeck, R. Eldan, A. T. Kalai, Y. T. Lee, and Y. Li. Textbooks are all you need. arXiv: 2306.11644, 2023. find
  13. J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig. Towards a unified view of parameter-efficient transfer learning. In International Conference on Learning Representations, 2022.
  14. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
  15. G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network, 2015.
  16. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre. Training compute-opt… — Training Compute-Optimal Large Language Models
  17. C.-Y. Hsieh, C.-L. Li, C.-K. Yeh, H. Nakhost, Y. Fujii, A. Ratner, R. Krishna, C.-Y. Lee, and T. Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes, 2023.
  18. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
  19. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models, 2020.
  20. N. U. P. R. Lab. leetcode-solutions-python-testgen-gpt4. https://huggingface.co/datasets/nuprl/leetcode-solutions-python-testgen-gpt4, 2023. ↗
  21. K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, and N. Carlini. Deduplicating training data makes language models better, 2022. find
  22. A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra. Solving quantitative reasoning problems with language models, 2022.
  23. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s verify step by step. arXiv preprint arXiv: 2305.20050, 2023.
  24. P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022. find
  25. Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang. Wizardcoder: Empowering code large language models with evol-instruct, 2023. find
  26. D. Mahan, R. Carlow, L. Castricato, N. Cooper, and C. Laforte. Stable beluga models, 2023. find
  27. S. Mangrulkar, S. Gugger, L. Debut, Y. Belkada, and S. Paul. Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft, 2022. ↗
  28. S. Mukherjee, A. Mitra, G. Jawahar, S. Agarwal, H. Palangi, and A. Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv: 2306.02707v1, 2023.
  29. C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang. Is chatgpt a general-purpose natural language processing task solver?, 2023.
  30. N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019.
  31. T. Research. tigerbot-kaggle-leetcodesolutions-en-2k. https://huggingface.co/datasets/TigerResearch/tigerbot-kaggle-leetcodesolutions-en-2k, 2023. ↗
  32. T. Sawada, D. Paleka, A. Havrilla, P. Tadepalli, P. Vidas, A. P. Kranias, J. J. Nay, K. Gupta, and A. Komatsuzaki. Arb: Advanced reasoning benchmark for large language models, 2023.
  33. T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model, 2022. find
  34. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, 2017. find
  35. S. Shen, L. Hou, Y. Zhou, N. Du, S. Longpre, J. Wei, H. W. Chung, B. Zoph, W. Fedus, X. Chen, T. Vu, Y. Wu, W. Chen, A. Webson, Y. Li, V. Zhao, H. Yu, K. Keutzer, T. Darrell, and D. Zhou. Mixture-of-experts meets instruction tuning:a winning combination for large language models, 2023.
  36. K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal, M. Schaekermann, A. Wang, M. Amin, S. Lachgar, P. Mansfield, S. Prakash, B. Green, E. Dominowska, B. A. y Arcas, N. Tomasev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. Barral, D. Webster, G… — Towards Expert-Level Medical Question Answering with Large Language Models
  37. R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto. Stanford alpaca: An instruction-following llama model. GitHub repository, 2023. find
  38. R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic. Galactica: A large language model for science, 2022.
  39. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. Llama: Open and efficient foundation language models, 2023.
  40. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan,… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  41. X. Wang, Z. Hu, P. Lu, Y. Zhu, J. Zhang, S. Subramaniam, A. R. Loomba, S. Zhang, Y. Sun, and W. Wang. Scibench: Evaluating college-level scientific problem-solving abilities of large language models. arXiv: 2307.10635, 2023.
  42. Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560v2, 2023. find
  43. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le. Finetuned language models are zero-shot learners, 2022.
  44. J. Welbl, N. F. Liu, and M. Gardner. Crowdsourcing multiple choice science questions. arXiv: 1707.06209, 2017.
  45. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Transformers: State-of-the-art natural language processing.…
  46. M. Wu, A. Waheed, C. Zhang, M. Abdul-Mageed, and A. F. Aji. Lamini-lm: A diverse herd of distilled models from large-scale instructions. CoRR, abs/2304.14402, 2023.
  47. C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, and D. Jiang. Wizardlm: Empowering large language models to follow complex instructions, 2023. find
  48. R. Yu, S. Liu, and X. Wang. Dataset distillation: A comprehensive review, 2023. find
  49. W. Yu, Z. Jiang, Y. Dong, and J. Feng. Reclor: A reading comprehension dataset requiring logical reasoning. International Conference on Learning Representations (ICLR), April 2020.
  50. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, S. Zhang, G. Ghosh, M. Lewis, L. Zettlemoyer, and O. Levy. Lima: Less is more for alignment. arXiv: 2305.11206, 2023. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lee-2023-platypus-quick-cheap-powerful-refinement/