Elamur Research⌕
☰
home / papers / 2023 / ConPET: Continual Parameter-Efficient Tuning for Large Language Models

ConPET: Continual Parameter-Efficient Tuning for Large Language Models

Paper · 2023 · Chenyang Song, Xu Han, Zheni Zeng et al.

Summary

Adapting large language models to a stream of new tasks is costly when every update changes all weights. Full fine-tuning also tends to erase skills learned on earlier tasks, a problem known as catastrophic forgetting. Song et al. propose ConPET, a framework for continual task adaptation of LLMs in which training cost stays constant as tasks accumulate. It has two variants. Static ConPET upgrades existing memory-based continual learning methods such as EMR in two ways. It replaces full tuning with parameter-efficient tuning (LoRA in the experiments). It also swaps fixed replay from a small memory for hierarchical sampling over all historical data, with a cap on the number of training batches. Dynamic ConPET trains one PET module per task and adds a selector that activates a fixed number of top-scoring modules, plus a cache, much like a lightweight mixture-of-experts. On continual knowledge extraction (new entity or relation types arriving over time), Static ConPET cuts tunable parameters over 3,000× and, across five smaller benchmarks, outperforms a baseline that uses PET alone by a margin of 5 points or more. Dynamic ConPET performs best on the largest dataset. Together they offer an affordable way to keep LLMs current.

Key points

  • The two variants suit different scenarios. Static ConPET works with a single shared PET module and fits smaller benchmarks. Dynamic ConPET separates task-specific PET modules and scales better to many tasks and large data.
  • The dynamic replay strategy draws samples hierarchically from the full history rather than a limited episodic memory. The aim is better data coverage and less over-fitting and forgetting.
  • The PET module selector pre-selects a fixed number of modules, so forward-pass cost does not rise linearly with task count. The paper contrasts this with AdapterCL, whose perplexity-based selection does scale linearly.
  • PET typically trains under 1% of model parameters. The authors cite prior work showing PET tuning of LLaMA-7B takes roughly a quarter of the time of full tuning.
  • The cited knowledge-extraction datasets include Few-NERD, OntoNotes 5.0, BBN, FewRel, TACRED and ACE 2005. Code and data are available at github.com/Raincleared-Song/ConPET.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🏢TencentShenzhen · China134 works · 19,743 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Chenyang Song, Xu Han, Zheni Zeng, Kuai Li, Chen Chen, Zhiyuan Liu, Maosong Sun and Tao Yang. 2023. ConPET: Continual Parameter-Efficient Tuning for Large Language Models. arXiv:2309.14763.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.14763 · arXiv 2309.14763

@misc{song2023conpet,
  title = {{ConPET: Continual Parameter-Efficient Tuning for Large Language Models}},
  author = {Chenyang Song, Xu Han, Zheni Zeng, Kuai Li, Chen Chen, Zhiyuan Liu, Maosong Sun and Tao Yang},
  year = {2023},
  doi = {10.48550/arxiv.2309.14763},
  eprint = {2309.14763},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.14763}
}

References

58 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell 1 — Language Models are Few-Shot Learners https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them 2 https://diskprices.com/ 3 https://cloud.google.com/compute/gpus-pricing et al., “Language models ar… ↗
  2. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, “Finetuned language models are zero-shot learners,” arXiv preprint arXiv:2109.01652, 2021. [Online]. Available: https://arxiv.org/pdf/2109.01652 ↗
  3. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022. [Online]. Available: https://p…
  4. N. Ding, Y. Qin, G. Yang, F. Wei, Z. Yang, Y. Su, S. Hu, Y. Chen, C.-M. Chan, W. Chen et al., “Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models,” arXiv preprint arXiv:2203.06904, 2022. [Online]. Available: https://arxiv.org/pdf/2203.06904.pdf — [arXiv:2203.06904](https://arxiv.org/abs/2203.06904) ↗
  5. X. Jin, D. Zhang, H. Zhu, W. Xiao, S.-W. Li, X. Wei, A. Arnold, and X. Ren, “Lifelong pretraining: Continually adapting language models to emerging corpora,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  6. A. Daruna, M. Gupta, M. Sridharan, and S. Chernova, “Continual learning of knowledge graph embeddings,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1128–1135, 2021. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9343669 ↗
  7. N. Monaikul, G. Castellucci, S. Filice, and O. Rokhlenko, “Continual learning for named entity recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 13 570– 13 577. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/ view/17600/17407 ↗
  8. Y. Qin, S. Hu, Y. Lin, W. Chen, N. Ding, G. Cui, Z. Zeng, Y. Huang, C. Xiao, C. Han et al., “Tool learning with foundation models,” arXiv preprint arXiv:2304.08354, 2023. [Online]. Available: https://arxiv.org/pdf/2304.08354 ↗
  9. K. Zhao, H. Xu, J. Yang, and K. Gao, “Consistent representation learning for continual relation extraction,” in Findings of the Association for Computational Linguistics: ACL 2022, 2022, pp. 3402–3411. [Online]. Available: https://aclanthology.org/2022.findings-acl.268.pdf ↗
  10. G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” Neural Networks, vol. 113, pp. 54–71, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0893608019300231 ↗
  11. A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” arXiv preprint arXiv:1606.04671, 2016. [Online]. Available: https://arxiv.org/pdf/1606.04671.pdf ↗
  12. X. Gu, L. Liu, H. Yu, J. Li, C. Chen, and J. Han, “On the transformer growth for progressive BERT training,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 5174–5180. [Online]. Available: ht… find
  13. D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne, “Experience replay for continual learning,” Advances in Neural Information Processing Systems, vol. 32, 2019. [Online]. Available: https://proceedings.neurips.cc/paper/2019/file/ fa7cdfad1a5aaf8370ebeda47a1ff1c3-Paper.pdf ↗
  14. H. Wang, W. Xiong, M. Yu, X. Guo, S. Chang, and W. Y. Wang, “Sentence embedding alignment for lifelong relation extraction,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pap… find
  15. X. Han, Y. Dai, T. Gao, Y. Lin, Z. Liu, P. Li, M. Sun, and J. Zhou, “Continual relation learning via episodic memory activation and reconsolidation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 6429–6440. [Online]. Available: https://aclantho…
  16. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. [Online]. Available: https://arxiv.org/pdf/1810.04805.pdf ↗
  17. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019. [Online]. Available: https://arxiv.org/pdf/1907.11692.pdf ↗
  18. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in International Conference on Machine Learning. PMLR, 2019, pp. 2790–2799. [Online]. Available: http://proceedings.mlr.press/v97/houlsby1… ↗
  19. X. L. Li and P. Liang, “Prefix-Tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 4582–4597. [O… find
  20. T. Gao, A. Fisch, and D. Chen, “Making pre-trained language models better few-shot learners,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 3816–…
  21. J. Lee, R. Tang, and J. Lin, “What would elsa do? freezing layers during transformer fine-tuning,” arXiv preprint arXiv:1911.03090, 2019. [Online]. Available: https://arxiv.org/pdf/1911.03090.pdf — [arXiv:1911.03090](https://arxiv.org/abs/1911.03090) ↗
  22. M. Zhao, T. Lin, F. Mi, M. Jaggi, and H. Schütze, “Masking as an efficient alternative to finetuning for pretrained language models,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020, pp. 2226–2241. [Online]. Available: https://aclanthology.org/2020.em… ↗
  23. E. B. Zaken, Y. Goldberg, and S. Ravfogel, “Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2022, pp. 1–9. [Online]. Available: https://a…
  24. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021. [Online]. Available: https://arxiv.org/pdf/2106.09685.pdf ↗
  25. Y. Qin, X. Wang, Y. Su, Y. Lin, N. Ding, Z. Liu, J. Li, L. Hou, P. Li, M. Sun et al., “Exploring low-dimensional intrinsic task subspace via prompt tuning,” arXiv preprint arXiv:2110.07867, 2021. [Online]. Available: https://arxiv.org/pdf/2110.07867.pdf — [arXiv:2110.07867](https://arxiv.org/abs/2110.07867) ↗
  26. X. Sun, Y. Ji, B. Ma, and X. Li, “A comparative study between full-parameter and lora-based fine-tuning on chinese instruction data for instruction following large language model,” arXiv preprint arXiv:2304.08109, 2023. [Online]. Available: https://arxiv.org/pdf/2304. 08109.pdf — [arXiv:2304.08109](https://arxiv.org/abs/2304.08109) ↗
  27. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023. [Online]. Available: https://arxiv.org/ pdf/2302.13971.pdf ↗
  28. J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017. [Online]. Avai… find
  29. S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang, “Overcoming catastrophic forgetting by incremental moment matching,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 4655– 4665. [Online]. Available: https://proceedings.neurips.cc/paper/… ↗
  30. A. Chaudhry, P. K. Dokania, T. Ajanthan, and P. H. Torr, “Riemannian walk for incremental learning: Understanding forgetting and intransigence,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 532–547. [Online]. Available: https://openaccess.thecvf.com/content ECCV 201… ↗
  31. Z. Li and D. Hoiem, “Learning without forgetting,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935–2947, 2017. [Online]. Available: https://ieeexplore.ieee.org/ abstract/document/8107520 ↗
  32. J. Zhang, J. Zhang, S. Ghosh, D. Li, S. Tasci, L. Heck, H. Zhang, and C.-C. J. Kuo, “Class-incremental learning via deep model consolidation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020, pp. 1131–1140. [Online]. Available: https://openaccess.thecvf.com/… ↗
  33. T. Chen, I. Goodfellow, and J. Shlens, “Net2Net: Accelerating learning via knowledge transfer,” arXiv preprint arXiv:1511.05641, 2015. [Online]. Available: https://arxiv.org/pdf/1511.05641.pdf — [arXiv:1511.05641](https://arxiv.org/abs/1511.05641) ↗
  34. D. Isele and A. Cosgun, “Selective experience replay for lifelong learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2018. [Online]. Available: https://ojs.aaai.org/index.php/ AAAI/article/view/11595/11454 ↗
  35. C. Qin and S. Joty, “LFPT5: A unified framework for lifelong few-shot language learning based on prompt tuning of T5,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/pdf?id=HCRVf71PMF ↗
  36. A. Razdaibiedina, Y. Mao, R. Hou, M. Khabsa, M. Lewis, and A. Almahairi, “Progressive prompts: Continual learning for language models,” in The Eleventh International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/pdf? id=UJTgQBc91 ↗
  37. A. Madotto, Z. Lin, Z. Zhou, S. Moon, P. A. Crook, B. Liu, Z. Yu, E. Cho, P. Fung, and Z. Wang, “Continual learning in task-oriented dialogue systems,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 7452–7467. [Online]. Available: https://aclanth… find
  38. Q. Gao, C. Zhao, Y. Sun, T. Xi, G. Zhang, B. Ghanem, and J. Zhang, “A unified continual learning framework with general parameter-efficient tuning,” arXiv preprint arXiv:2303.10070, 2023. [Online]. Available: https://arxiv.org/pdf/2303.10070.pdf — [arXiv:2303.10070](https://arxiv.org/abs/2303.10070) ↗
  39. Y. Qin, J. Zhang, Y. Lin, Z. Liu, P. Li, M. Sun, and J. Zhou, “Elle: Efficient lifelong pre-training for emerging data,” in Findings of the Association for Computational Linguistics: ACL 2022, 2022, pp. 2789–2810. [Online]. Available: https://aclanthology.org/2022.findings-acl.220.pdf ↗
  40. Z. Ke, Y. Shao, H. Lin, T. Konishi, G. Kim, and B. Liu, “Continual pre-training of language models,” in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/pdf?id=m GDIItaI3o ↗
  41. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” in International Conference on Learning Representations, 2017. [Online]. Available: https://openreview.net/pdf? id=B1ckMDqlg ↗
  42. D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen, “GShard: Scaling giant models with conditional computation and automatic sharding,” arXiv preprint arXiv:2006.16668, 2020. [Online]. Available: https://arxiv.org/pdf/2006.16668 ↗
  43. M. Lewis, S. Bhosale, T. Dettmers, N. Goyal, and L. Zettlemoyer, “BASE Layers: Simplifying training of large, sparse models,” in International Conference on Machine Learning. PMLR, 2021, pp. 6265–6274. [Online]. Available: http://proceedings.mlr.press/v139/ lewis21a/lewis21a.pdf ↗
  44. S. Roller, S. Sukhbaatar, J. Weston et al., “Hash layers for large sparse models,” Advances in Neural Information Processing Systems, vol. 34, pp. 17 555–17 566, 2021. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/ 2021/file/92bf5e6240737e0326ea59846a83e076-Paper.pdf ↗
  45. W. Fedus, B. Zoph, and N. Shazeer, “Switch Transformers: Scaling to trillion parameter models with simple and efficient sparsity,” The Journal of Machine Learning Research, vol. 23, no. 1, pp. 5232–5270, 2022. [Online]. Available: https://jmlr.org/papers/volume23/21-0998/21-0998.pdf ↗
  46. Z. Zhang, Y. Lin, Z. Liu, P. Li, M. Sun, and J. Zhou, “MoEfication: Transformer feed-forward layers are mixtures of experts,” in Findings of the Association for Computational Linguistics: ACL 2022, 2022, pp. 877–890. [Online]. Available: https://aclanthology.org/2022.findings-acl.71.pdf ↗
  47. S. Gururangan, M. Lewis, A. Holtzman, N. A. Smith, and L. Zettlemoyer, “DEMix layers: Disentangling domains for modular language modeling,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022, pp. 555… find
  48. S. Shen, L. Hou, Y. Zhou, N. Du, S. Longpre, J. Wei, H. W. Chung, B. Zoph, W. Fedus, X. Chen, T. Vu, Y. Wu, W. Chen, A. Webson, Y. Li, V. Zhao, H. Yu, K. Keutzer, T. Darrell, and D. Zhou, “Mixture-of-experts meets instruction tuning: A winning combination for large language models,” arXiv preprint a… — Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
  49. T. Zadouri, A. Üstün, A. Ahmadian, B. Ermiş, A. Locatelli, and S. Hooker, “Pushing mixture of experts to the limit: Extremely parameter efficient moe for instruction tuning,” arXiv preprint arXiv:2309.05444, 2023. [Online]. Available: https://arxiv.org/pdf/2309.05444.pdf ↗
  50. L. B. Soares, N. Fitzgerald, J. Ling, and T. Kwiatkowski, “Matching the blanks: Distributional similarity for relation learning,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 2895–2905. [Online]. Available: https://aclanthology.org/P19-1279.pd… ↗
  51. N. Ding, Y. Chen, X. Han, G. Xu, P. Xie, H.-T. Zheng, Z. Liu, J. Li, and H.-G. Kim, “Prompt-learning for fine-grained entity typing,” arXiv preprint arXiv:2108.10604, 2021. [Online]. Available: https://arxiv.org/pdf/2108.10604.pdf ↗
  52. B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059. [Online]. Available: https://aclanthology.org/ 2021.emnlp-main.243.pdf ↗
  53. N. Ding, G. Xu, Y. Chen, X. Wang, X. Han, P. Xie, H. Zheng, and Z. Liu, “Few-NERD: A few-shot named entity recognition dataset,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volu…
  54. R. Weischedel, M. Palmer, M. Marcus, E. Hovy, S. Pradhan, L. Ramshaw, N. Xue, A. Taylor, J. Kaufman, M. Franchini et al., “OntoNotes release 5.0 ldc2013t19,” Linguistic Data Consortium, Philadelphia, PA, vol. 23, 2013. [Online]. Available: https://catalog.ldc.upenn.edu/LDC2013T19 ↗
  55. R. Weischedel and A. Brunstein, “BBN pronoun coreference and entity type corpus,” Linguistic Data Consortium, Philadelphia, vol. 112, 2005. [Online]. Available: https://catalog.ldc.upenn.edu/LDC2005T33 ↗
  56. X. Han, H. Zhu, P. Yu, Z. Wang, Y. Yao, Z. Liu, and M. Sun, “FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 4803–4809. [Online]. Availabl… find
  57. Y. Zhang, V. Zhong, D. Chen, G. Angeli, and C. D. Manning, “Position-aware attention and supervised data improve slot filling,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 35–45. [Online]. Available: https://nlp.stanford.edu/pubs/zhang2017tacr… ↗
  58. C. Walker, S. Strassel, J. Medero, and K. Maeda, “ACE 2005 multilingual training corpus,” Linguistic Data Consortium, Philadelphia, vol. 57, p. 45, 2006. [Online]. Available: https://catalog.ldc.upenn.edu/ LDC2006T06 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2023-conpet-continual-parameter-efficient-tuning/