Elamur Research
☰
home / papers / 2023 / AugGPT: Leveraging ChatGPT for Text Data Augmentation

AugGPT: Leveraging ChatGPT for Text Data Augmentation

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2302.13007

Summary

Few-shot text classification suffers from scarce and often noisy labeled data in the target domain. Existing text augmentation techniques, such as synonym swaps, random insertion or deletion, back-translation, and masked-LM contextual substitution, tend to either produce samples whose labels may no longer hold (poor faithfulness) or produce samples that barely differ from the originals (poor compactness/diversity). The authors propose AugGPT, which prompts ChatGPT to rewrite every training sentence into several new versions that keep the underlying concept but vary in wording and meaning details. These synthetic sentences are combined with the few-shot originals to fine-tune a BERT-based sentence classifier. Experiments cover both general-domain and medical-domain classification data. AugGPT is compared against state-of-the-art augmentation baselines and reportedly improves classification accuracy by double-digit margins. Analyses of the generated data suggest it is more diverse while staying semantically aligned with its labels. The work shows that instruction-tuned LLMs trained with RLHF can serve as practical, model-agnostic augmenters that reduce the need for manual annotation.

Key points

  • Method: ChatGPT (trained in the InstructGPT style with RLHF) generates multiple paraphrase-like variants per labeled sentence. A BERT classifier is then trained on the original plus augmented samples.
  • Evaluation: The authors test on few-shot text classification in general and medical domains. They also assess the augmented data for faithfulness (label/semantic consistency) and compactness (diversity).
  • Main result: AugGPT shows double-digit accuracy gains over existing augmentation methods and a better distribution of augmented samples.
  • Baselines discussed: The comparison covers character-level methods (typos, OCR/keyboard noise), word-level methods (random swap/deletion, PPDB/WordNet synonyms, embedding and counter-fitted embedding replacement, BERT/DistilBERT/RoBERTa contextual augmentation), and sentence/document-level methods (back-translation, document paraphrasing).
  • Caveat for the library: The experimental section was missing from the supplied text, so specific dataset names and exact scores are not recorded here and should be checked in the original paper.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · AI & machine learning

Cited by

113 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 112 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2302.13007 · arXiv 2302.13007

@misc{liu2023auggpt,
  title = {{AugGPT: Leveraging ChatGPT for Text Data Augmentation}},
  author = {Liu;, Haixing Dai; Zhengliang Liu; Wenxiong Liao; Xiaoke Huang; Yihan Cao; Zihao Wu; Lin Zhao; Shaochen Xu; Wei Liu; Ninghao Liu; Sheng Li; Dajiang Zhu; Hongmin Cai; Lichao Sun; Quanzheng Li; Dinggang Shen; Tianming and Li, Xiang},
  year = {2023},
  doi = {10.48550/arxiv.2302.13007},
  eprint = {2302.13007},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.13007}
}

References

97 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: A survey on few-shot learning,” ACM computing surveys (csur), vol. 53, no. 3, pp. 1–34, 2020. find
  2. W. Yin, “Meta-learning for few-shot natural language processing: A survey,” arXiv preprint arXiv:2007.09604, 2020. — [arXiv:2007.09604](https://arxiv.org/abs/2007.09604) find
  3. C. Wang, J. Wang, M. Qiu, J. Huang, and M. Gao, “Transprompt: Towards an automatic transferable prompting framework for few-shot text classification,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 2792– 2802. find
  4. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. find
  5. H.-y. Lee, S.-W. Li, and N. T. Vu, “Meta learning for natural language processing: A survey,” arXiv preprint arXiv:2205.01500, 2022. find
  6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020. find
  7. B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059. find
  8. X. Han, W. Zhao, N. Ding, Z. Liu, and M. Sun, “Ptr: Prompt tuning with rules for text classification,” AI Open, vol. 3, pp. 182–192, 2022. find
  9. J. Wang, C. Wang, F. Luo, C. Tan, M. Qiu, F. Yang, Q. Shi, S. Huang, and M. Gao, “Towards unified prompt tuning for few-shot text classification,” arXiv preprint arXiv:2205.05313, 2022. find
  10. J. Wei and K. Zou, “Eda: Easy data augmentation techniques for boosting performance on text classification tasks,” arXiv preprint arXiv:1901.11196, 2019. find our card
  11. V. Kumar, H. Glaude, C. de Lichy, and W. Campbell, “A closer look at feature space data augmentation for few-shot intent classification,” in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), 2019, pp. 1–10. find
  12. S. Y. Feng, V. Gangal, J. Wei, S. Chandar, S. Vosoughi, T. Mitamura, and E. Hovy, “A survey of data augmentation approaches for nlp,” arXiv preprint arXiv:2105.03075, 2021. find our card
  13. R. Sennrich, B. Haddow, and A. Birch, “Improving neural machine translation models with monolingual data,” arXiv preprint arXiv:1511.06709, 2015. find
  14. A. Jindal, A. G. Chowdhury, A. Didolkar, D. Jin, R. Sawhney, and R. Shah, “Augmenting nlp models using latent feature interpolations,” in Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 6931–6936. find
  15. C. Shorten, T. M. Khoshgoftaar, and B. Furht, “Text data augmentation for deep learning,” Journal of big Data, vol. 8, pp. 1–34, 2021. find
  16. M. Bayer, M.-A. Kaufhold, and C. Reuter, “A survey on data augmentation for text classification,” ACM Computing Surveys, vol. 55, no. 7, pp. 1–39, 2022. find
  17. B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heinz, and D. Roth, “Recent advances in natural language processing via large pre-trained language models: A survey,” arXiv preprint arXiv:2111.01243, 2021. find our card
  18. C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He et al., “A comprehensive survey on pretrained foundation models: A history from bert to chatgpt,” arXiv preprint arXiv:2302.09419, 2023. — [arXiv:2302.09419](https://arxiv.org/abs/2302.09419) find
  19. Z. Liu, M. He, Z. Jiang, Z. Wu, H. Dai, L. Zhang, S. Luo, T. Han, X. Li, X. Jiang et al., “Survey on natural language processing in medical image analysis.” Zhong nan da xue xue bao. Yi xue ban= Journal of Central South University. Medical Sciences, vol. 47, no. 8, pp. 981–993, 2022. find
  20. S. Wang, Z. Zhao, X. Ouyang, Q. Wang, and D. Shen, “Chatcad: Interactive computer-aided diagnosis on medical image using large language models,” arXiv preprint arXiv:2302.07257, 2023. — [arXiv:2302.07257](https://arxiv.org/abs/2302.07257) find
  21. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems, 2022. find
  22. Y. Belinkov and Y. Bisk, “Synthetic and natural noise both break neural machine translation,” arXiv preprint arXiv:1711.02173, 2017. find
  23. C. Coulombe, “Text Data Augmentation Made Simple By Leveraging NLP Cloud APIs,” Dec. 2018. find
  24. J. Wei and K. Zou, “EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong… find our card
  25. E. Pavlick, P. Rastogi, J. Ganitkevitch, B. Van Durme, and C. Callison-Burch, “Ppdb 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification,” in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th In… find our card
  26. T. Niu and M. Bansal, “Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models,” in Proceedings of the 22nd Conference on Computational Natural Language Learning. Brussels, Belgium: Association for Computational Linguistics, 2018, pp. 486–496. find
  27. G. A. Miller, “Wordnet: a lexical database for english,” Communications of the ACM, vol. 38, no. 11, pp. 39–41, 1995. find our card
  28. W. Y. Wang and D. Yang, “That’s so annoying!!!: A lexical and frame-semantic embedding based data augmentation approach to automatic categorization of annoying behaviors using\# petpeeve tweets,” in Proceedings of the 2015 conference on empirical methods in natural language processing, 2015, pp. 2557… find
  29. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Advances in neural information processing systems, vol. 26, 2013. find
  30. N. Mrkšić, D. Ó Séaghdha, B. Thomson, M. Gašić, L. M. Rojas-Barahona, P.-H. Su, D. Vandyke, T.-H. Wen, and S. Young, “Counter-fitting Word Vectors to Linguistic Constraints,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: H… find
  31. M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, and K.-W. Chang, “Generating Natural Language Adversarial Examples,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, 2018, pp. 2890–2… find our card
  32. S. Kobayashi, “Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). New Orleans, Louisiana: Associa… find our card
  33. V. Kumar, A. Choudhary, and E. Cho, “Data Augmentation Using Pre-trained Transformer Models,” arXiv preprint arXiv:2003.02245, 2020. find our card
  34. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” May 2019. find
  35. L. Sun, C. Xia, W. Yin, T. Liang, P. S. Yu, and L. He, “Mixup-transformer: dynamic data augmentation for nlp tasks,” arXiv preprint arXiv:2010.02394, 2020. find
  36. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” ArXiv, vol. abs/1910.01108, 2019. find
  37. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019. find
  38. R. Sennrich, B. Haddow, and A. Birch, “Improving Neural Machine Translation Models with Monolingual Data,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 86… find
  39. V. Gangal, S. Y. Feng, M. Alikhani, T. Mitamura, and E. Hovy, “Nareor: The narrative reordering problem,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, 2022, pp. 10 645–10 653. find
  40. L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,” IEEE transactions on pattern analysis and machine intelligence, vol. 28, no. 4, pp. 594–611, 2006. find
  41. Y. Ge, Y. Guo, Y.-C. Yang, M. A. Al-Garadi, and A. Sarker, “Few-shot learning for medical text: A systematic review,” arXiv preprint arXiv:2204.14081, 2022. — [arXiv:2204.14081](https://arxiv.org/abs/2204.14081) find
  42. J. Wei, C. Huang, S. Vosoughi, Y. Cheng, and S. Xu, “Few-shot text classification with triplet networks, data augmentation, and curriculum learning,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 202… find
  43. T. Gao, A. Fisch, and D. Chen, “Making pre-trained language models better few-shot learners,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 3816–… find our card
  44. A. Antoniou, H. Edwards, and A. Storkey, “How to train your maml,” arXiv preprint arXiv:1810.09502, 2018. — [arXiv:1810.09502](https://arxiv.org/abs/1810.09502) find
  45. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning. PMLR, 2017, pp. 1126–1135. find
  46. X. Yao, J. Zhu, G. Huo, N. Xu, X. Liu, and C. Zhang, “Model-agnostic multi-stage loss optimization meta learning,” International Journal of Machine Learning and Cybernetics, vol. 12, no. 8, pp. 2349–2363, 2021. find
  47. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” 2018. find
  48. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv preprint arXiv:1910.13461, 2019. find our card
  49. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” The Journal of Machine Learning Research, vol. 21, no. 1, pp. 5485–5551, 2020. find
  50. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” arXiv preprint arXiv:2204.02311, 2022. find
  51. T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé et al., “Bloom: A 176b-parameter open-access multilingual language model,” arXiv preprint arXiv:2211.05100, 2022. find
  52. S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin et al., “Opt: Open pre-trained transformer language models,” arXiv preprint arXiv:2205.01068, 2022. find
  53. S. Longpre, L. Hou, T. Vu, A. Webson, H. W. Chung, Y. Tay, D. Zhou, Q. V. Le, B. Zoph, J. Wei et al., “The flan collection: Designing data and methods for effective instruction tuning,” arXiv preprint arXiv:2301.13688, 2023. find our card
  54. Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain-specific language model pretraining for biomedical natural language processing,” ACM Transactions on Computing for Healthcare (HEALTH), vol. 3, no. 1, pp. 1–23, 2021. find our card
  55. S. Rezayi, H. Dai, Z. Liu, Z. Wu, A. Hebbar, A. H. Burns, L. Zhao, D. Zhu, Q. Li, W. Liu et al., “Clinicalradiobert: Knowledge-infused few shot learning for clinical notes named entity recognition,” in Machine Learning in Medical Imaging: 13th International Workshop, MLMI 2022, Held in Conjunction w… find
  56. C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang, “Is chatgpt a general-purpose natural language processing task solver?” arXiv preprint arXiv:2302.06476, 2023. find our card
  57. W. Jiao, W. Wang, J.-t. Huang, X. Wang, and Z. Tu, “Is chatgpt a good translator? a preliminary study,” arXiv preprint arXiv:2301.08745, 2023. find our card
  58. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung et al., “A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity,” arXiv preprint arXiv:2302.04023, 2023. find our card
  59. Y. Shen, L. Heacock, J. Elias, K. D. Hentel, B. Reig, G. Shih, and L. Moy, “Chatgpt and other large language models are double-edged swords,” p. 230163, 2023. find
  60. F. Antaki, S. Touma, D. Milad, J. El-Khoury, and R. Duval, “Evaluating the performance of chatgpt in ophthalmology: An analysis of its successes and shortcomings,” medRxiv, pp. 2023–01, 2023. find
  61. T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepaño, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo et al., “Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models,” PLOS Digital Health, vol. 2, no. 2, p. e0000198, 2023. find
  62. J. V. Pavlik, “Collaborating with chatgpt: Considering the implications of generative artificial intelligence for journalism and media education,” Journalism & Mass Communication Educator, p. 10776958221149577, 2023. find
  63. D. Baidoo-Anu and L. Owusu Ansah, “Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning,” Available at SSRN 4337484, 2023. find
  64. S. Frieder, L. Pinchetti, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. C. Petersen, A. Chevalier, and J. Berner, “Mathematical capabilities of chatgpt,” arXiv preprint arXiv:2301.13867, 2023. find our card
  65. M. Dowling and B. Lucey, “Chatgpt for (finance) research: The bananarama conjecture,” Finance Research Letters, p. 103662, 2023. find
  66. E. A. van Dis, J. Bollen, W. Zuidema, R. van Rooij, and C. L. Bockting, “Chatgpt: five priorities for research,” Nature, vol. 614, no. 7947, pp. 224–226, 2023. find
  67. R. W. McGee, “Is chat gpt biased against conservatives? an empirical study,” An Empirical Study (February 15, 2023), 2023. find
  68. A. Blum, “Breaking chatgpt with dangerous questions understanding how chatgpt prioritizes safety, context, and obedience,” 2022. find
  69. H. Y. Jabotinsky and R. Sarel, “Co-authoring with an ai? ethical dilemmas and artificial intelligence,” Ethical Dilemmas and Artificial Intelligence (December 15, 2022), 2022. find
  70. T. Susnjak, “Chatgpt: The end of online exam integrity?” arXiv preprint arXiv:2212.09292, 2022. — [arXiv:2212.09292](https://arxiv.org/abs/2212.09292) find
  71. M. Khalil and E. Er, “Will chatgpt get you caught? rethinking of plagiarism detection,” arXiv preprint arXiv:2302.04335, 2023. — [arXiv:2302.04335](https://arxiv.org/abs/2302.04335) find
  72. D. Castelvecchi, “Are chatgpt and alphacode going to replace programmers?” Nature, 2022. find
  73. A. Zarifhonarvar, “Economics of chatgpt: A labor market view on the occupational impact of artificial intelligence,” Available at SSRN 4350925, 2023. find
  74. R. He and J. McAuley, “Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering,” in proceedings of the 25th international conference on world wide web, 2016, pp. 507–517. find
  75. Y. Bao, M. Wu, S. Chang, and R. Barzilay, “Few-shot text classification with distributional signatures,” arXiv preprint arXiv:1908.06039, 2019. — [arXiv:1908.06039](https://arxiv.org/abs/1908.06039) find
  76. S. Wang, X. Liu, B. Liu, and D. Dong, “Sentence-aware adversarial meta-learning for few-shot text classification,” in Proceedings of the 29th International Conference on Computational Linguistics, 2022, pp. 4844–4852. find
  77. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. find
  78. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. find
  79. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. find
  80. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186. find
  81. W. Liao, Z. Liu, H. Dai, Z. Wu, Y. Zhang, X. Huang, Y. Chen, X. Jiang, D. Zhu, T. Liu, S. Li, X. Li, and H. Cai, “Mask-guided bert for few shot text classification,” arXiv preprint arXiv:2302.10447, 2023. — [arXiv:2302.10447](https://arxiv.org/abs/2302.10447) find
  82. S. Rezayi, Z. Liu, Z. Wu, C. Dhakal, B. Ge, C. Zhen, T. Liu, and S. Li, “Agribert: Knowledge-infused agricultural language models for matching food and nutrition,” International Joint Conference on Artificial Intelligence, July 23-29, 2022, Vienna, Austria, 2022. find
  83. E. Ma, “Nlp augmentation,” https://github.com/makcedward/nlpaug, 2019. ↗
  84. J. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, and Y. Qi, “Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 119–126. find our card
  85. N. Ng, K. Yee, A. Baevski, M. Ott, M. Auli, and S. Edunov, “Facebook fair’s wmt19 news translation task submission,” in Proc. of WMT, 2020. find our card
  86. L.-K. Huang, J. Huang, Y. Rong, Q. Yang, and Y. Wei, “Frustratingly easy transferability estimation,” in International Conference on Machine Learning. PMLR, 2022, pp. 9201–9225. find
  87. B. Li, H. Zhou, J. He, M. Wang, Y. Yang, and L. Li, “On the sentence embeddings from pre-trained language models,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 9119–9130. find our card
  88. J. Wang and Y. Dong, “Measurement of text similarity: a survey,” Information, vol. 11, no. 9, p. 421, 2020. find
  89. T. M. Cover, Elements of information theory. John Wiley & Sons, 1999. find
  90. X. Cai, S. Liu, J. Han, L. Yang, Z. Liu, and T. Liu, “Chestxraybert: A pretrained language model for chest radiology report summarization,” IEEE Transactions on Multimedia, pp. 1–1, 2021. find
  91. X. Cai, S. Liu, L. Yang, Y. Lu, J. Zhao, D. Shen, and T. Liu, “Covid-sum: A linguistically enriched scibert-based summarization model for covid-19 scientific papers,” Journal of Biomedical Informatics, vol. 127, p. 103999, 2022. find
  92. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, 2022. find our card
  93. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 684–10 695. find
  94. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021. find
  95. N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” arXiv preprint arXiv:2208.12242, 2022. — [arXiv:2208.12242](https://arxiv.org/abs/2208.12242) find
  96. M. Kosinski, “Theory of mind may have spontaneously emerged in large language models,” arXiv preprint arXiv:2302.02083, 2023. find our card
  97. X. Liu, M. Zhou, G. Shi, Y. Du, L. Zhao, Z. Wu, D. Liu, T. Liu, and X. Hu, “Coupling artificial neurons in bert and biological neurons in the human brain,” in Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI, 2023. find