A Comprehensive Overview of Large Language Models

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · AI & machine learning

Cited by

189 papers of our library cite it (of the 140673 whose bibliographies we read).

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2307.06435 · arXiv 2307.06435

@misc{naveed2023comprehensive,
  title = {{A Comprehensive Overview of Large Language Models}},
  author = {Naveed, Humza and Khan, Asad Ullah and Qiu, Shi and Saqib, Muhammad and Anwar, Saeed and Usman, Muhammad and Akhtar, Naveed and Barnes, Nick and Mian, Ajmal},
  year = {2023},
  doi = {10.48550/arxiv.2307.06435},
  eprint = {2307.06435},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.06435}
}

References

498 references, 169 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Chernyavskiy, D. Ilvovsky, P. Nakov, Transformers:“the end of history” for natural language processing?, in: Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part III 21, Springer, 2021,… find
2. A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for general-purpose language understanding systems, Advances in neural information processing systems 32 (2019). 1, 26, 29 find our card
3. D. Adiwardana, M.-T. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan, Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu, et al., Towards a human-like open-domain chatbot, arXiv preprint arXiv:2001.09977 (2020). 1 find our card
4. B. A. y Arcas, Do large language models understand us?, Daedalus 151 (2) (2022) 183–197. 2 find
5. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are unsupervised multitask learners, OpenAI blog 1 (8) (2019) 9. 2, 7 find
6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901. 2, 6, 7, 8, 9, 16, 18, 23, 24, 25, 34 find
7. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018). 2, 18, 24 find
8. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, in: NAACL-HLT, Association for Computational Linguistics, 2018, pp. 2227–2237. 2 find
9. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, arXiv preprint arXiv:1910.13461 (2019). 2 find our card
10. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, The Journal of Machine Learning Research 21 (1) (2020) 5485–5551. 2, 7, 8, 18, 19, 24, 25, 28, 30, 31 find
11. L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Raffel, mt5: A massively multilingual pre-trained text-to-text transformer, arXiv preprint arXiv:2010.11934 (2020). 2, 7, 8, 24, 25, 28, 30 find
12. Z. Zhang, Y. Gu, X. Han, S. Chen, C. Xiao, Z. Sun, Y. Yao, F. Qi, J. Guan, P. Ke, et al., Cpm-2: Large-scale cost-effective pre-trained language models, AI Open 2 (2021) 216–224. 2, 8, 25 find
13. T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, et al., Bloom: A 176b-parameter open-access multilingual language model, arXiv preprint arXiv:2211.05100 (2022). 2, 4, 9, 11, 23, 24, 25, 30 find
14. S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al., Opt: Open pre-trained transformer language models, arXiv preprint arXiv:2205.01068 (2022). 2, 9, 11, 24, 25 find
15. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., Palm: Scaling language modeling with pathways, arXiv preprint arXiv:2204.02311 (2022). 2, 6, 9, 11, 23, 24, 25 find
16. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, et al., Scaling instruction-finetuned language models, arXiv preprint arXiv:2210.11416 (2022). 2, 7, 11, 16, 17, 22, 24, 25, 28, 31 find
17. V. Sanh, A. Webson, C. Raffel, S. H. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, T. L. Scao, A. Raja, et al., Multitask prompted training enables zero-shot task generalization, arXiv preprint arXiv:2110.08207 (2021). 2, 11, 16, 25, 28, 31 find our card
18. Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Naik, A. Ashok, A. S. Dhanasekaran, A. Arunkumar, D. Stap, et al., Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Langu… find
19. Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, H. Hajishirzi, Self-instruct: Aligning language model with self generated instructions, arXiv preprint arXiv:2212.10560 (2022). 2, 16, 19, 22, 28 find
20. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., Training language models to follow instructions with human feedback, Advances in Neural Information Processing Systems 35 (2022) 27730–27744. 2, 7, 11, 16, 22 find
21. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al., Llama 2: Open foundation and fine-tuned chat models, arXiv preprint arXiv:2307.09288 (2023). 2, 7, 10, 16, 25, 34 find
22. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al., Emergent abilities of large language models, arXiv preprint arXiv:2206.07682 (2022). 2 find our card
23. T. Webb, K. J. Holyoak, H. Lu, Emergent analogical reasoning in large language models, Nature Human Behaviour 7 (9) (2023) 1526–1541. 2 find
24. D. A. Boiko, R. MacKnight, G. Gomes, Emergent autonomous scientific research capabilities of large language models, arXiv preprint arXiv:2304.05332 (2023). 2 find
25. G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, E. Grave, Few-shot learning with retrieval augmented language models, arXiv preprint arXiv:2208.03299 (2022). 2, 18, 19, 34 find
26. D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al., Palm-e: An embodied multimodal language model, arXiv preprint arXiv:2303.03378 (2023). 2, 20, 22, 33 find
27. A. Parisi, Y. Zhao, N. Fiedel, Talm: Tool augmented language models, arXiv preprint arXiv:2205.12255 (2022). 2, 19, 20 find our card
28. B. Zhang, H. Soh, Large language models as zero-shot human models for human-robot interaction, arXiv preprint arXiv:2303.03548 (2023). 2, 33 — [arXiv:2303.03548](https://arxiv.org/abs/2303.03548) find
29. Q. Ye, H. Xu, G. Xu, J. Ye, M. Yan, Y. Zhou, J. Wang, A. Hu, P. Shi, Y. Shi, et al., mplug-owl: Modularization empowers large language models with multimodality, arXiv preprint arXiv:2304.14178 (2023). 2, 22 find our card
30. W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P. Luo, T. Lu, J. Zhou, Y. Qiao, et al., Visionllm: Large language model is also an open-ended decoder for vision-centric tasks, arXiv preprint arXiv:2305.11175 (2023). 2, 22 — [arXiv:2305.11175](https://arxiv.org/abs/2305.11175) find
31. R. Yang, L. Song, Y. Li, S. Zhao, Y. Ge, X. Li, Y. Shan, Gpt4tools: Teaching large language model to use tools via self-instruction, arXiv preprint arXiv:2305.18752 (2023). 2, 19, 22, 23 find
32. E. Saravia, Prompt Engineering Guide, https://github.com/dair-ai/Prompt-Engineering-Guide (12 2022). 2, 7, 18, 34 ↗
33. A. Zeng, X. Liu, Z. Du, Z. Wang, H. Lai, M. Ding, Z. Yang, Y. Xu, W. Zheng, X. Xia, et al., Glm-130b: An open bilingual pre-trained model, arXiv preprint arXiv:2210.02414 (2022). 2, 10, 23, 24, 25 find our card
34. Y. Wang, H. Le, A. D. Gotmare, N. D. Bui, J. Li, S. C. Hoi, Codet5+: Open code large language models for code understanding and generation, arXiv preprint arXiv:2305.07922 (2023). 2, 11, 24, 25 find
35. S. Wang, Y. Sun, Y. Xiang, Z. Wu, S. Ding, W. Gong, S. Feng, J. Shang, Y. Zhao, C. Pang, et al., Ernie 3.0 titan: Exploring larger-scale knowledge enhanced pre-training for language understanding and generation, arXiv preprint arXiv:2112.12731 (2021). 2, 8, 24, 25 find
36. J. Rasley, S. Rajbhandari, O. Ruwase, Y. He, Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 3505– 3506. 2, 5 find
37. S. Rajbhandari, J. Rasley, O. Ruwase, Y. He, Zero: Memory optimizations toward training trillion parameter models, in: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE, 2020, pp. 1–16. 2, 4, 24 find
38. J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, G. Neubig, Towards a unified view of parameter-efficient transfer learning, arXiv preprint arXiv:2110.04366 (2021). 2, 20, 21 find our card
39. Z. Hu, Y. Lan, L. Wang, W. Xu, E.-P. Lim, R. K.-W. Lee, L. Bing, S. Poria, Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models, arXiv preprint arXiv:2304.01933 (2023). 2, 20 find our card
40. B. Lester, R. Al-Rfou, N. Constant, The power of scale for parameter-efficient prompt tuning, arXiv preprint arXiv:2104.08691 (2021). 2, 8, 20, 21 find
41. X. L. Li, P. Liang, Prefix-tuning: Optimizing continuous prompts for generation, arXiv preprint arXiv:2101.00190 (2021). 2, 20, 21 find
42. X. Ma, G. Fang, X. Wang, Llm-pruner: On the structural pruning of large language models, arXiv preprint arXiv:2305.11627 (2023). 2, 22 find our card
43. R. Xu, F. Luo, C. Wang, B. Chang, J. Huang, S. Huang, F. Huang, From dense to sparse: Contrastive pruning for better pre-trained language model compression, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, 2022, pp. 11547–11555. 2, 22 find
44. G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, S. Han, Smoothquant: Accurate and efficient post-training quantization for large language models, in: ICML, Vol. 202 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 38087–38099. 2, 21 find
45. C. Tao, L. Hou, W. Zhang, L. Shang, X. Jiang, Q. Liu, P. Luo, N. Wong, Compression of generative pre-trained language models via quantization, arXiv preprint arXiv:2203.10705 (2022). 2, 21 find
46. A. Pal, D. Karkhanis, M. Roberts, S. Dooley, A. Sundararajan, S. Naidu, Giraffe: Adventures in expanding context lengths in llms, arXiv preprint arXiv:2308.10882 (2023). 2, 17 — [arXiv:2308.10882](https://arxiv.org/abs/2308.10882) find
47. B. Peng, J. Quesnelle, H. Fan, E. Shippole, Yarn: Efficient context window extension of large language models, arXiv preprint arXiv:2309.00071 (2023). 2, 17 find
48. M. Guo, J. Ainslie, D. Uthus, S. Ontanon, J. Ni, Y.-H. Sung, Y. Yang, Longt5: Efficient text-to-text transformer for long sequences, arXiv preprint arXiv:2112.07916 (2021). 2, 18 find our card
49. S. Chen, S. Wong, L. Chen, Y. Tian, Extending context window of large language models via positional interpolation, arXiv preprint arXiv:2306.15595 (2023). 2, 17 find
50. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al., A survey of large language models, arXiv preprint arXiv:2303.18223 (2023). 2, 3, 7 find our card
51. U. Naseem, I. Razzak, S. K. Khan, M. Prasad, A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models, Transactions on Asian and Low-Resource Language Information Processing 20 (5) (2021) 1–35. 2, 3 find
52. B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heinz, D. Roth, Recent advances in natural language processing via large pre-trained language models: A survey, arXiv preprint arXiv:2111.01243 (2021). 2, 3 find our card
53. C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al., A comprehensive survey on pretrained foundation models: A history from bert to chatgpt, arXiv preprint arXiv:2302.09419 (2023). 2, 3 — [arXiv:2302.09419](https://arxiv.org/abs/2302.09419) find
54. Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, Z. Sui, A survey for in-context learning, arXiv preprint arXiv:2301.00234 (2022). 2, 7, 18 find our card
55. J. Huang, K. C.-C. Chang, Towards reasoning in large language models: A survey, arXiv preprint arXiv:2212.10403 (2022). 2, 7, 18 find our card
56. Y. Wang, W. Zhong, L. Li, F. Mi, X. Zeng, W. Huang, L. Shang, X. Jiang, Q. Liu, Aligning large language models with human: A survey, arXiv preprint arXiv:2307.12966 (2023). 2 find our card
57. X. Zhu, J. Li, Y. Liu, C. Ma, W. Wang, A survey on model compression for large language models, arXiv preprint arXiv:2308.07633 (2023). 2 find our card
58. S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, E. Chen, A survey on multimodal large language models, arXiv preprint arXiv:2306.13549 (2023). 2, 22, 23 find our card
59. J. J. Webster, C. Kit, Tokenization as the initial phase in nlp, in: COL-ING 1992 volume 4: The 14th international conference on computational linguistics, 1992. 4 find our card
60. T. Kudo, Subword regularization: Improving neural network translation models with multiple subword candidates, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 66–75. 4 find
61. R. Sennrich, B. Haddow, A. Birch, Neural machine translation of rare words with subword units, in: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016, pp. 1715–1725. 4 find
62. M. Schuster, K. Nakajima, Japanese and korean voice search, in: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2012, pp. 5149–5152. 4 find
63. S. J. Mielke, Z. Alyafeai, E. Salesky, C. Raffel, M. Dey, M. Gallé, A. Raja, C. Si, W. Y. Lee, B. Sagot, et al., Between words and characters: A brief history of open-vocabulary modeling and tokenization in nlp, arXiv preprint arXiv:2112.10508 (2021). 4 find
64. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017). 4, 7 find
65. O. Press, N. Smith, M. Lewis, Train short, test long: Attention with linear biases enables input length extrapolation, in: International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=R8sQPpGCv0 4, 17 ↗
66. J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, Y. Liu, Roformer: Enhanced transformer with rotary position embedding, arXiv preprint arXiv:2104.09864 (2021). 4, 9, 17 find
67. R. Child, S. Gray, A. Radford, I. Sutskever, Generating long sequences with sparse transformers, arXiv preprint arXiv:1904.10509 (2019). 4, 7, 23 find
68. T. Dao, D. Fu, S. Ermon, A. Rudra, C. Ré, Flashattention: Fast and memory-efficient exact attention with io-awareness, Advances in Neural Information Processing Systems 35 (2022) 16344–16359. 4 find
69. K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are universal approximators, Neural networks 2 (5) (1989) 359–366. 4 find
70. V. Nair, G. E. Hinton, Rectified linear units improve restricted boltzmann machines, in: Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807–814. 4 find
71. D. Hendrycks, K. Gimpel, Gaussian error linear units (gelus), arXiv preprint arXiv:1606.08415 (2016). 4 find
72. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting, The journal of machine learning research 15 (1) (2014) 1929–1958. 4 find
73. D. Krueger, T. Maharaj, J. Kramár, M. Pezeshki, N. Ballas, N. R. Ke, A. Goyal, Y. Bengio, A. Courville, C. Pal, Zoneout: Regularizing rnns by randomly preserving hidden activations, arXiv preprint arXiv:1606.01305 (2016). 4 — [arXiv:1606.01305](https://arxiv.org/abs/1606.01305) find
74. N. Shazeer, Glu variants improve transformer, arXiv preprint arXiv:2002.05202 (2020). 4 find
75. Y. N. Dauphin, A. Fan, M. Auli, D. Grangier, Language modeling with gated convolutional networks, in: International conference on machine learning, PMLR, 2017, pp. 933–941. 4 find our card
76. J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization, arXiv preprint arXiv:1607.06450 (2016). 4 find our card
77. B. Zhang, R. Sennrich, Root mean square layer normalization, Advances in Neural Information Processing Systems 32 (2019). 4 find
78. A. Baevski, M. Auli, Adaptive input representations for neural language modeling, arXiv preprint arXiv:1809.10853 (2018). 4 find
79. H. Wang, S. Ma, L. Dong, S. Huang, D. Zhang, F. Wei, Deepnet: Scaling transformers to 1,000 layers, arXiv preprint arXiv:2203.00555 (2022). 4 — [arXiv:2203.00555](https://arxiv.org/abs/2203.00555) find
80. M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, B. Catanzaro, Megatron-lm: Training multi-billion parameter language models using model parallelism, arXiv preprint arXiv:1909.08053 (2019). 4, 5 find
81. "bmtrain: Efficient training for big models.". URL https://github.com/OpenBMB/BMTrain 4, 5 ↗
82. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al., Transformers: State-of-the-art natural language processing, in: Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 3… find our card
83. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, et al., Jax: composable transformations of python+ numpy programs (2018). 5 find
84. S. Li, J. Fang, Z. Bian, H. Liu, Y. Liu, H. Huang, B. Wang, Y. You, Colossal-ai: A unified deep learning system for large-scale parallel training, arXiv preprint arXiv:2110.14883 (2021). 5 find
85. J. He, J. Qiu, A. Zeng, Z. Yang, J. Zhai, J. Tang, Fastmoe: A fast mixture-of-expert training system, arXiv preprint arXiv:2103.13262 (2021). 5 find
86. L. Huawei Technologies Co., Huawei mindspore ai development framework, in: Artificial Intelligence Technology, Springer, 2022, pp. 137– 162. 5 find
87. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems 32 (2019). 5 find our card
88. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al., Tensorflow: a system for large-scale machine learning., in: Osdi, Vol. 16, Savannah, GA, USA, 2016, pp. 265–283. 5 find
89. T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, Z. Zhang, Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems, arXiv preprint arXiv:1512.01274 (2015). 5 find our card
90. W. Fedus, B. Zoph, N. Shazeer, Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, The Journal of Machine Learning Research 23 (1) (2022) 5232–5270. 5, 9 find
91. N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al., Glam: Efficient scaling of language models with mixture-of-experts, in: International Conference on Machine Learning, PMLR, 2022, pp. 5547–5569. 5, 9, 23, 24, 25 find our card
92. X. Ren, P. Zhou, X. Meng, X. Huang, Y. Wang, W. Wang, P. Li, P X. Zhang, A. Podolskiy, G. Arshinov, et al., Pangu- : Towards trillion parameter language model with sparse heterogeneous computing, arXiv preprint arXiv:2303.10845 (2023). 5, 10, 16, 23, 24, 25 find
93. T. Wang, A. Roberts, D. Hesslow, T. Le Scao, H. W. Chung, I. Beltagy, J. Launay, C. Raffel, What language model architecture and pretraining objective works best for zero-shot generalization?, in: International Conference on Machine Learning, PMLR, 2022, pp. 22964–22984. 5 find
94. L. Dong, N. Yang, W. Wang, F. Wei, X. Liu, Y. Wang, J. Gao, M. Zhou, H.-W. Hon, Unified language model pre-training for natural language understanding and generation, Advances in neural information processing systems 32 (2019). 6 find our card
95. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, D. Amodei, Scaling laws for neural language models, arXiv preprint arXiv:2001.08361 (2020). 6 find
96. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al., Training compute-optimal large language models, arXiv preprint arXiv:2203.15556 (2022). 6, 9, 25, 29 find
97. S. Iyer, X. V. Lin, R. Pasunuru, T. Mihaylov, D. Simig, P. Yu, K. Shuster, T. Wang, Q. Liu, P. S. Koura, et al., Opt-iml: Scaling language model instruction meta learning through the lens of generalization, arXiv preprint arXiv:2212.12017 (2022). 7, 11, 16, 17, 22, 25, 28 find our card
98. Z. Sun, Y. Shen, Q. Zhou, H. Zhang, Z. Chen, D. Cox, Y. Yang, C. Gan, Principle-driven self-alignment of language models from scratch with minimal human supervision, arXiv preprint arXiv:2305.03047 (2023). 7, 17 find our card
99. A. Askell, Y. Bai, A. Chen, D. Drain, D. Ganguli, T. Henighan, A. Jones, N. Joseph, B. Mann, N. DasSarma, et al., A general language assistant as a laboratory for alignment, arXiv preprint arXiv:2112.00861 (2021). 7 find our card
100. D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, G. Irving, Fine-tuning language models from human preferences, arXiv preprint arXiv:1909.08593 (2019). 7 find
101. S. Kim, S. J. Joo, D. Kim, J. Jang, S. Ye, J. Shin, M. Seo, The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning, arXiv preprint arXiv:2305.14045 (2023). 7, 16 find
102. Q. Liu, F. Zhou, Z. Jiang, L. Dou, M. Lin, From zero to hero: Examining the power of symbolic tasks in instruction tuning, arXiv preprint arXiv:2304.07995 (2023). 7, 16 find
103. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in Neural Information Processing Systems 35 (2022) 24824–24837. 7, 20, 23 find our card
104. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, D. Zhou, Self-consistency improves chain of thought reasoning in language models, arXiv preprint arXiv:2203.11171 (2022). 7, 20 find our card
105. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models, arXiv preprint arXiv:2305.10601 (2023). 7, 20 find
106. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, S. Gelly, Parameter-efficient transfer learning for nlp, in: International Conference on Machine Learning, PMLR, 2019, pp. 2790–2799. 7, 20 find
107. S. McCandlish, J. Kaplan, D. Amodei, O. D. Team, An empirical model of large-batch training, arXiv preprint arXiv:1812.06162 (2018). 7 find
108. W. Zeng, X. Ren, T. Su, H. Wang, Y. Liao, Z. Wang, X. Jiang, Z. Yang, K. Wang, X. Zhang, et al., Pangu-α : Large-scale autoregressive pre-trained chinese language models with auto-parallel computation, arXiv preprint arXiv:2104.12369 (2021). 8, 23, 24, 25 find
109. S. Yuan, H. Zhao, Z. Du, M. Ding, X. Liu, Y. Cen, X. Zou, Z. Yang, J. Tang, Wudaocorpora: A super large-scale chinese corpora for pre-training language models, AI Open 2 (2021) 65–68. 8, 30 find
110. Y. Sun, S. Wang, S. Feng, S. Ding, C. Pang, J. Shang, J. Liu, X. Chen, Y. Zhao, Y. Lu, et al., Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation, arXiv preprint arXiv:2107.02137 (2021). 8, 25 find our card
111. Z. Dai, Z. Yang, Y. Yang, J. Carbonell, Q. V. Le, R. Salakhutdinov, Transformer-xl: Attentive language models beyond a fixed-length context, arXiv preprint arXiv:1901.02860 (2019). 8 find
112. O. Lieber, O. Sharir, B. Lenz, Y. Shoham, Jurassic-1: Technical details and evaluation, White Paper. AI21 Labs 1 (2021). 8, 24, 25 find
113. Y. Levine, N. Wies, O. Sharir, H. Bata, A. Shashua, Limits to depth efficiencies of self-attention, Advances in Neural Information Processing Systems 33 (2020) 22640–22651. 8, 11 find
114. B. Kim, H. Kim, S.-W. Lee, G. Lee, D. Kwak, D. H. Jeon, S. Park, S. Kim, S. Kim, D. Seo, et al., What changes can large-scale language models bring? intensive study on hyperclova: Billions-scale korean generative pretrained transformers, arXiv preprint arXiv:2109.04650 (2021). 8, 25 find
115. S. Wu, X. Zhao, T. Yu, R. Zhang, C. Shen, H. Liu, F. Li, H. Zhu, J. Luo, L. Xu, et al., Yuan 1.0: Large-scale pre-trained language model in zero-shot and few-shot learning, arXiv preprint arXiv:2110.04725 (2021). 8, 24, 25 find
116. J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, et al., Scaling language models: Methods, analysis & insights from training gopher, arXiv preprint arXiv:2112.11446 (2021). 8, 9, 25, 28 find
117. S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, et al., Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model, arXiv preprint arXiv:2201.11990 (2022). 8, 9, 24, 25 find our card
118. S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang, et al., Gpt-neox-20b: An open-source autoregressive language model, arXiv preprint arXiv:2204.06745 (2022). 9, 23, 24, 25 find our card
119. W. Ben, K. Aran, Gpt-j-6b: A 6 billion parameter autoregressive language model (2021). 9 find
120. P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, et al., Mixed precision training, arXiv preprint arXiv:1710.03740 (2017). 9, 23 find
121. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, arXiv preprint arXiv:1701.06538 (2017). 9, 23 find
122. S. Soltan, S. Ananthakrishnan, J. FitzGerald, R. Gupta, W. Hamza, H. Khan, C. Peris, S. Rawls, A. Rosenbaum, A. Rumshisky, et al., Alexatm 20b: Few-shot learning using a large-scale multilingual seq2seq model, arXiv preprint arXiv:2208.01448 (2022). 9, 23, 24, 25 find
123. R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al., Palm 2 technical report, arXiv preprint arXiv:2305.10403 (2023). 9, 25 find our card
124. Y. Tay, J. Wei, H. W. Chung, V. Q. Tran, D. R. So, S. Shakeri, X. Garcia, H. S. Zheng, J. Rao, A. Chowdhery, et al., Transcending scaling laws with 0.1% extra compute, arXiv preprint arXiv:2210.11399 (2022). 9, 24, 25 find
125. Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, J. Wei, X. Wang, H. W. Chung, D. Bahri, T. Schuster, S. Zheng, et al., Ul2: Unifying language learning paradigms, in: The Eleventh International Conference on Learning Representations, 2022. 9, 10, 24, 25 find our card
126. Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, J. Tang, Glm: General language model pretraining with autoregressive blank infilling, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 320–335. 10 find our card
127. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models, arXiv preprint arXiv:2302.13971 (2023). 10, 23, 25 find
128. M. N. Rabe, C. Staats, Self-attention does not need o(n 2 ) memory, arXiv preprint arXiv:2112.05682 (2021). 10 find
129. V. A. Korthikanti, J. Casper, S. Lym, L. McAfee, M. Andersch, M. Shoeybi, B. Catanzaro, Reducing activation recomputation in large transformer models, Proceedings of Machine Learning and Systems 5 (2023). 10 find our card
130. A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024). 10, 25 find
131. https://mistral.ai/news/mixtral-8x22b/. 10, 25 ↗
132. https://github.com/Snowflake-Labs/snowflake-arctic. 10, 25 ↗
133. https://github.com/xai-org/grok-1. 10 ↗
134. https://x.ai/blog/grok-1.5. 10 ↗
135. G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al., Gemini: a family of highly capable multimodal models, arXiv preprint arXiv:2312.11805 (2023). 10 find our card
136. M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, et al., Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, arXiv preprint arXiv:2403.05530 (2024). 10 find our card
137. B. Adler, N. Agarwal, A. Aithal, D. H. Anh, P. Bhattacharya, A. Brundyn, J. Casper, B. Catanzaro, S. Clay, J. Cohen, et al., Nemotron-4 340b technical report, arXiv preprint arXiv:2406.11704 (2024). 10, 25 find
138. X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, et al., Deepseek llm: Scaling open-source language models with longtermism, arXiv preprint arXiv:2401.02954 (2024). 10, 25 find
139. DeepSeek-AI, A. Liu, B. Feng, B. Wang, B. Wang, B. Liu, C. Zhao, C. Deng, C. Ruan, D. Dai, D. Guo, D. Yang, D. Chen, D. Ji, E. Li, F. Lin, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Xu, H. Yang, H. Zhang, H. Ding, H. Xin, H. Gao, H. Li, H. Qu, J. L. Cai, J. Liang, J. Guo, J. Ni, J. Li, J. Chen, J.… find
140. E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, C. Xiong, Codegen: An open large language model for code with multi-turn program synthesis, arXiv preprint arXiv:2203.13474 (2022). 11, 23, 25, 28 find
141. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al., Evaluating large language models trained on code, arXiv preprint arXiv:2107.03374 (2021). 11, 25, 29, 31 find our card
142. Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, et al., Competition-level code generation with alphacode, Science 378 (6624) (2022) 1092–1097. 11, 23, 25, 29 find our card
143. N. Shazeer, Fast transformer decoding: One write-head is all you need, arXiv preprint arXiv:1911.02150 (2019). 11 find
144. R. Y. Pang, H. He, Text generation by learning from demonstrations, arXiv preprint arXiv:2009.07839 (2020). 11 — [arXiv:2009.07839](https://arxiv.org/abs/2009.07839) find
145. R. Dabre, A. Fujita, Softmax tempering for training neural machine translation models, arXiv preprint arXiv:2009.09372 (2020). 11 — [arXiv:2009.09372](https://arxiv.org/abs/2009.09372) find
146. Y. Wang, W. Wang, S. Joty, S. C. Hoi, Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation, arXiv preprint arXiv:2109.00859 (2021). 11 find
147. R. Li, L. B. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim, et al., Starcoder: may the source be with you!, arXiv preprint arXiv:2305.06161 (2023). 11, 25 find
148. R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, R. Stojnic, Galactica: A large language model for science, arXiv preprint arXiv:2211.09085 (2022). 11, 24, 25, 29 find our card
149. FairScale authors, Fairscale: A general purpose modular pytorch library for high performance and large scale training, https://github.com/ facebookresearch/fairscale (2021). 11 ↗
150. R. Thoppilan, D. De Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, et al., Lamda: Language models for dialog applications, arXiv preprint arXiv:2201.08239 (2022). 11, 25 find our card
151. S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, G. Mann, Bloomberggpt: A large language model for finance, arXiv preprint arXiv:2303.17564 (2023). 11, 25, 33 find our card
152. X. Zhang, Q. Yang, D. Xu, Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters, arXiv preprint arXiv:2305.12002 (2023). 11, 17, 25 find
153. W. Ben, Mesh-transformer-jax: Model-parallel implementation of transformer language model with jax (2021). 12, 24 find
154. N. Muennighoff, T. Wang, L. Sutawika, A. Roberts, S. Biderman, T. L. Scao, M. S. Bari, S. Shen, Z.-X. Yong, H. Schoelkopf, et al., Crosslingual generalization through multitask finetuning, arXiv preprint arXiv:2211.01786 (2022). 16, 25, 28, 31 find our card
155. D. Yin, X. Liu, F. Yin, M. Zhong, H. Bansal, J. Han, K.-W. Chang, Dynosaur: A dynamic growth paradigm for instruction-tuning data curation, arXiv preprint arXiv:2305.14327 (2023). 16 find
156. P. Gao, J. Han, R. Zhang, Z. Lin, S. Geng, A. Zhou, W. Zhang, P. Lu, C. He, X. Yue, et al., Llama-adapter v2: Parameter-efficient visual instruction model, arXiv preprint arXiv:2304.15010 (2023). 16, 24 find
157. Openai. gpt-4 technical report (2023). 16, 35 find our card
158. R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, T. B. Hashimoto, Stanford alpaca: An instruction-following llama model, https://github.com/tatsu-lab/stanford\_alpaca (2023). 16, 25, 28 ↗
159. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, E. P. Xing, Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality (March 2023). URL https://lmsys.org/blog/2023-03-30-vicuna/ 16, 22, 25, 28 ↗
160. B. Peng, C. Li, P. He, M. Galley, J. Gao, Instruction tuning with gpt-4, arXiv preprint arXiv:2304.03277 (2023). 16, 28 find our card
161. T. Liu, B. K. H. Low, Goat: Fine-tuned llama outperforms gpt-4 on arithmetic tasks, arXiv preprint arXiv:2305.14201 (2023). 16 find
162. H. Wang, C. Liu, N. Xi, Z. Qiang, S. Zhao, B. Qin, T. Liu, Huatuo: Tuning llama model with chinese medical knowledge, arXiv preprint arXiv:2304.06975 (2023). 16 find
163. C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, D. Jiang, Wizardlm: Empowering large language models to follow complex instructions, arXiv preprint arXiv:2304.12244 (2023). 16 find our card
164. Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, D. Jiang, Wizardcoder: Empowering code large language models with evol-instruct, arXiv preprint arXiv:2306.08568 (2023). 16, 25 find
165. J. Menick, M. Trebacz, V. Mikulik, J. Aslanides, F. Song, M. Chadwick, M. Glaese, S. Young, L. Campbell-Gillingham, G. Irving, et al., Teaching language models to support answers with verified quotes, arXiv preprint arXiv:2203.11147 (2022). 17 find our card
166. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al., Webgpt: Browser-assisted question-answering with human feedback, arXiv preprint arXiv:2112.09332 (2021). 17, 19, 20, 25, 31 find our card
167. A. Glaese, N. McAleese, M. Tr ebacz, J. Aslanides, V. Firoiu, T. Ewalds, M. Rauh, L. Weidinger, M. Chadwick, P. Thacker, et al., Improving alignment of dialogue agents via targeted human judgements, arXiv preprint arXiv:2209.14375 (2022). 17, 20, 25 find our card
168. R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, C. Finn, Direct preference optimization: Your language model is secretly a reward model, arXiv preprint arXiv:2305.18290 (2023). 17 find
169. H. Dong, W. Xiong, D. Goyal, R. Pan, S. Diao, J. Zhang, K. Shum, T. Zhang, Raft: Reward ranked finetuning for generative foundation model alignment, arXiv preprint arXiv:2304.06767 (2023). 17 find our card
170. Z. Yuan, H. Yuan, C. Tan, W. Wang, S. Huang, F. Huang, Rrhf: Rank responses to align language models with human feedback without tears, arXiv preprint arXiv:2304.05302 (2023). 17 find our card
171. F. Song, B. Yu, M. Li, H. Yu, F. Huang, Y. Li, H. Wang, Preference ranking optimization for human alignment, arXiv preprint arXiv:2306.17492 (2023). 17 find our card
172. H. Liu, C. Sferrazza, P. Abbeel, Languages are rewards: Hindsight fine-tuning using human feedback, arXiv preprint arXiv:2302.02676 (2023). 17 find
173. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al., Constitutional ai: Harmlessness from ai feedback, arXiv preprint arXiv:2212.08073 (2022). 17 find our card
174. Y. Dubois, X. Li, R. Taori, T. Zhang, I. Gulrajani, J. Ba, C. Guestrin, P. Liang, T. B. Hashimoto, Alpacafarm: A simulation framework for methods that learn from human feedback, arXiv preprint arXiv:2305.14387 (2023). 17 find our card
175. C. Si, Z. Gan, Z. Yang, S. Wang, J. Wang, J. Boyd-Graber, L. Wang, Prompting gpt-3 to be reliable, arXiv preprint arXiv:2210.09150 (2022). 17 find our card
176. D. Ganguli, A. Askell, N. Schiefer, T. Liao, K. Lukošiūtė, A. Chen, A. Goldie, A. Mirhoseini, C. Olsson, D. Hernandez, et al., The capacity for moral self-correction in large language models, arXiv preprint arXiv:2302.07459 (2023). 17 find our card
177. A. Wei, N. Haghtalab, J. Steinhardt, Jailbroken: How does llm safety training fail?, arXiv preprint arXiv:2307.02483 (2023). 17 find our card
178. D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al., Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, arXiv preprint arXiv:2209.07858 (2022). 17, 28 find our card
179. S. Casper, J. Lin, J. Kwon, G. Culp, D. Hadfield-Menell, Explore, establish, exploit: Red teaming language models from scratch, arXiv preprint arXiv:2306.09442 (2023). 17 find
180. E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, G. Irving, Red teaming language models with language models, arXiv preprint arXiv:2202.03286 (2022). 17 find our card
181. T. Scialom, T. Chakrabarty, S. Muresan, Fine-tuned language models are continual learners, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 6107–6122. 17 find
182. Z. Shi, A. Lipani, Don’t stop pretraining? make prompt-based fine-tuning powerful learner, arXiv preprint arXiv:2305.01711 (2023). 17 — [arXiv:2305.01711](https://arxiv.org/abs/2305.01711) find
183. H. Gupta, S. A. Sawant, S. Mishra, M. Nakamura, A. Mitra, S. Mashetty, C. Baral, Instruction tuned models are quick learners, arXiv preprint arXiv:2306.05539 (2023). 17 — [arXiv:2306.05539](https://arxiv.org/abs/2306.05539) find
184. H. Chen, Y. Zhang, Q. Zhang, H. Yang, X. Hu, X. Ma, Y. Yanggong, J. Zhao, Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning, arXiv preprint arXiv:2305.09246 (2023). 17 — [arXiv:2305.09246](https://arxiv.org/abs/2305.09246) find
185. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al., Lima: Less is more for alignment, arXiv preprint arXiv:2305.11206 (2023). 17, 25, 28 find
186. C. Han, Q. Wang, W. Xiong, Y. Chen, H. Ji, S. Wang, Lm-infinite: Simple on-the-fly length generalization for large language models, arXiv preprint arXiv:2308.16137 (2023). 17, 18 — [arXiv:2308.16137](https://arxiv.org/abs/2308.16137) find
187. J. Ainslie, T. Lei, M. de Jong, S. Ontañón, S. Brahma, Y. Zemlyanskiy, D. Uthus, M. Guo, J. Lee-Thorp, Y. Tay, et al., Colt5: Faster long-range transformers with conditional computation, arXiv preprint arXiv:2303.09752 (2023). 18 find
188. J. Ding, S. Ma, L. Dong, X. Zhang, S. Huang, W. Wang, F. Wei, Longnet: Scaling transformers to 1,000,000,000 tokens, arXiv preprint arXiv:2307.02486 (2023). 18 find
189. Y. Chen, S. Qian, H. Tang, X. Lai, Z. Liu, S. Han, J. Jia, Longlora: Efficient fine-tuning of long-context large language models, arXiv preprint arXiv:2309.12307 (2023). 18 find our card
190. N. Ratner, Y. Levine, Y. Belinkov, O. Ram, I. Magar, O. Abend, E. Karpas, A. Shashua, K. Leyton-Brown, Y. Shoham, Parallel context windows for large language models, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 6383–64… find
191. W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, F. Wei, Augmenting language models with long-term memory, arXiv preprint arXiv:2306.07174 (2023). 18 find
192. X. Xu, Z. Gou, W. Wu, Z.-Y. Niu, H. Wu, H. Wang, S. Wang, Long time no see! open-domain conversation with long-term persona memory, arXiv preprint arXiv:2203.05797 (2022). 18 find
193. S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark, et al., Improving language models by retrieving from trillions of tokens, in: International conference on machine learning, PMLR, 2022, pp. 2206– 2240. 18, 19, 34 find
194. W. Zhong, L. Guo, Q. Gao, Y. Wang, Memorybank: Enhancing large language models with long-term memory, arXiv preprint arXiv:2305.10250 (2023). 18 find
195. N. Shinn, F. Cassano, B. Labash, A. Gopinath, K. Narasimhan, S. Yao, Reflexion: Language agents with verbal reinforcement learning, arXiv preprint arXiv:2303.11366 14 (2023). 18, 20 find our card
196. C. Hu, J. Fu, C. Du, S. Luo, J. Zhao, H. Zhao, Chatdb: Augmenting llms with databases as their symbolic memory, arXiv preprint arXiv:2306.03901 (2023). 18 — [arXiv:2306.03901](https://arxiv.org/abs/2306.03901) find
197. Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y. Yang, J. Callan, G. Neubig, Active retrieval augmented generation, arXiv preprint arXiv:2305.06983 (2023). 18 find
198. O. Ram, Y. Levine, I. Dalmedigos, D. Muhlgay, A. Shashua, K. Leyton-Brown, Y. Shoham, In-context retrieval-augmented language models, arXiv preprint arXiv:2302.00083 (2023). 18, 34 find
199. X. Li, X. Qiu, Mot: Pre-thinking and recalling enable chatgpt to self-improve with memory-of-thoughts, arXiv preprint arXiv:2305.05181 (2023). 18 — [arXiv:2305.05181](https://arxiv.org/abs/2305.05181) find
200. D. Schuurmans, Memory augmented large language models are computationally universal, arXiv preprint arXiv:2301.04589 (2023). 18 find
201. A. Modarressi, A. Imani, M. Fayyaz, H. Schütze, Ret-llm: Towards a general read-write memory for large language models, arXiv preprint arXiv:2305.14322 (2023). 18 — [arXiv:2305.14322](https://arxiv.org/abs/2305.14322) find
202. S. Robertson, H. Zaragoza, et al., The probabilistic relevance framework: Bm25 and beyond, Foundations and Trends® in Information Retrieval 3 (4) (2009) 333–389. 18 find our card
203. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, D. Zhou, Rationale-augmented ensembles in language models, arXiv preprint arXiv:2207.00747 (2022). 18 find
204. F. Zhang, B. Chen, Y. Zhang, J. Liu, D. Zan, Y. Mao, J.-G. Lou, W. Chen, Repocoder: Repository-level code completion through iterative retrieval and generation, arXiv preprint arXiv:2303.12570 (2023). 18 find
205. B. Wang, W. Ping, P. Xu, L. McAfee, Z. Liu, M. Shoeybi, Y. Dong, O. Kuchaiev, B. Li, C. Xiao, et al., Shall we pretrain autoregressive language models with retrieval? a comprehensive study, arXiv preprint arXiv:2304.06762 (2023). 19 find
206. L. Wang, N. Yang, F. Wei, Learning to retrieve in-context examples for large language models, arXiv preprint arXiv:2307.07164 (2023). 19 find
207. J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, W. Chen, What makes good in-context examples for gpt-3?, arXiv preprint arXiv:2101.06804 (2021). 19 find our card
208. O. Rubin, J. Herzig, J. Berant, Learning to retrieve prompts for in-context learning, arXiv preprint arXiv:2112.08633 (2021). 19 find our card
209. W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, W.-t. Yih, Replug: Retrieval-augmented black-box language models, arXiv preprint arXiv:2301.12652 (2023). 19 find our card
210. O. Rubin, J. Berant, Long-range language modeling with self-retrieval, arXiv preprint arXiv:2306.13421 (2023). 19 — [arXiv:2306.13421](https://arxiv.org/abs/2306.13421) find
211. K. Guu, K. Lee, Z. Tung, P. Pasupat, M. Chang, Retrieval augmented language model pre-training, in: International conference on machine learning, PMLR, 2020, pp. 3929–3938. 19 find
212. S. Hofstätter, J. Chen, K. Raman, H. Zamani, Fid-light: Efficient and effective retrieval-augmented text generation, in: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 1437–1447. 19 find
213. M. Komeili, K. Shuster, J. Weston, Internet-augmented dialogue generation, arXiv preprint arXiv:2107.07566 (2021). 19 find our card
214. A. Lazaridou, E. Gribovskaya, W. Stokowiec, N. Grigorev, Internet-augmented language models through few-shot prompting for open-domain question answering, arXiv preprint arXiv:2203.05115 (2022). 19 find our card
215. D. Gao, L. Ji, L. Zhou, K. Q. Lin, J. Chen, Z. Fan, M. Z. Shou, Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn, arXiv preprint arXiv:2306.08640 (2023). 19 — [arXiv:2306.08640](https://arxiv.org/abs/2306.08640) find
216. P. Lu, B. Peng, H. Cheng, M. Galley, K.-W. Chang, Y. N. Wu, S.-C. Zhu, J. Gao, Chameleon: Plug-and-play compositional reasoning with large language models, arXiv preprint arXiv:2304.09842 (2023). 19, 20, 23 find our card
217. B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, M. T. Ribeiro, Art: Automatic multi-step reasoning and tool-use for large language models, arXiv preprint arXiv:2303.09014 (2023). 19 find
218. C.-Y. Hsieh, S.-A. Chen, C.-L. Li, Y. Fujii, A. Ratner, C.-Y. Lee, R. Krishna, T. Pfister, Tool documentation enables zero-shot tool-usage with large language models, arXiv preprint arXiv:2308.00675 (2023). 19 — [arXiv:2308.00675](https://arxiv.org/abs/2308.00675) find
219. Y. Song, W. Xiong, D. Zhu, C. Li, K. Wang, Y. Tian, S. Li, Restgpt: Connecting large language models with real-world applications via restful apis, arXiv preprint arXiv:2306.06624 (2023). 19 find
220. S. Hao, T. Liu, Z. Wang, Z. Hu, Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings, arXiv preprint arXiv:2305.11554 (2023). 19 find our card
221. S. G. Patil, T. Zhang, X. Wang, J. E. Gonzalez, Gorilla: Large language model connected with massive apis, arXiv preprint arXiv:2305.15334 (2023). 19 find our card
222. Q. Xu, F. Hong, B. Li, C. Hu, Z. Chen, J. Zhang, On the tool manipulation capability of open-source large language models, arXiv preprint arXiv:2305.16504 (2023). 19 find
223. Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al., Toolllm: Facilitating large language models to master 16000+ real-world apis, arXiv preprint arXiv:2307.16789 (2023). 19, 20 find our card
224. Y. Shen, K. Song, X. Tan, D. Li, W. Lu, Y. Zhuang, Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface, arXiv preprint arXiv:2303.17580 (2023). 19, 20, 33 find our card
225. Y. Liang, C. Wu, T. Song, W. Wu, Y. Xia, Y. Liu, Y. Ou, S. Lu, L. Ji, S. Mao, et al., Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis, arXiv preprint arXiv:2303.16434 (2023). 19 — [arXiv:2303.16434](https://arxiv.org/abs/2303.16434) find
226. D. Surís, S. Menon, C. Vondrick, Vipergpt: Visual inference via python execution for reasoning, arXiv preprint arXiv:2303.08128 (2023). 20 find our card
227. A. Maedche, S. Morana, S. Schacht, D. Werth, J. Krumeich, Advanced user assistance systems, Business & Information Systems Engineering 58 (2016) 367–370. 20 find
228. M. Campbell, A. J. Hoane Jr, F.-h. Hsu, Deep blue, Artificial intelligence 134 (1-2) (2002) 57–83. 20 find
229. S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, et al., Metagpt: Meta programming for multi-agent collaborative framework, arXiv preprint arXiv:2308.00352 (2023). 20 find our card
230. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, et al., The rise and potential of large language model based agents: A survey, arXiv preprint arXiv:2309.07864 (2023). 20 find our card
231. L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, et al., A survey on large language model based autonomous agents, arXiv preprint arXiv:2308.11432 (2023). 20 find our card
232. W. Huang, P. Abbeel, D. Pathak, I. Mordatch, Language models as zero-shot planners: Extracting actionable knowledge for embodied agents, in: International Conference on Machine Learning, PMLR, 2022, pp. 9118–9147. 20 find our card
233. S. Hao, Y. Gu, H. Ma, J. J. Hong, Z. Wang, D. Z. Wang, Z. Hu, Reasoning with language model is planning with world model, arXiv preprint arXiv:2305.14992 (2023). 20, 33 find
234. W. Yao, S. Heinecke, J. C. Niebles, Z. Liu, Y. Feng, L. Xue, R. Murthy, Z. Chen, J. Zhang, D. Arpit, et al., Retroformer: Retrospective large language agents with policy gradient optimization, arXiv preprint arXiv:2308.02151 (2023). 20, 33 — [arXiv:2308.02151](https://arxiv.org/abs/2308.02151) find
235. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, P. Sermanet, T. Jackson, N. Brown, L. Luu, S. Levine, K. Hausman, brian ichter, Inner monologue: Embodied reasoning through planning with language models, in: 6th Annual Conference on Robot Lear… find
236. C. Jin, W. Tan, J. Yang, B. Liu, R. Song, L. Wang, J. Fu, Alphablock: Embodied finetuning for vision-language reasoning in robot manipulation, arXiv preprint arXiv:2305.18898 (2023). 20, 33 — [arXiv:2305.18898](https://arxiv.org/abs/2305.18898) find
237. I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, A. Garg, Progprompt: Generating situated robot task plans using large language models, in: 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023, pp. 11523–11530. 20, 33 find our card
238. W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al., Language to rewards for robotic skill synthesis, arXiv preprint arXiv:2306.08647 (2023). 20 — [arXiv:2306.08647](https://arxiv.org/abs/2306.08647) find
239. X. Tang, A. Zou, Z. Zhang, Y. Zhao, X. Zhang, A. Cohan, M. Gerstein, Medagents: Large language models as collaborators for zero-shot medical reasoning, arXiv preprint arXiv:2311.10537 (2023). 20 find
240. A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, et al., Do as i can, not as i say: Grounding language in robotic affordances, in: Conference on Robot Learning, PMLR, 2023, pp. 287–318. 20, 33 find
241. H. Ha, P. Florence, S. Song, Scaling up and distilling down: Language-guided robot skill acquisition, arXiv preprint arXiv:2307.14535 (2023). 20 — [arXiv:2307.14535](https://arxiv.org/abs/2307.14535) find
242. A. Rajvanshi, K. Sikka, X. Lin, B. Lee, H.-P. Chiu, A. Velasquez, Say-nav: Grounding large language models for dynamic planning to navigation in new environments, arXiv preprint arXiv:2309.04077 (2023). 20 — [arXiv:2309.04077](https://arxiv.org/abs/2309.04077) find
243. C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, Y. Su, Llm-planner: Few-shot grounded planning for embodied agents with large language models, arXiv preprint arXiv:2212.04088 (2022). 20 find our card
244. V. S. Dorbala, J. F. Mullen Jr, D. Manocha, Can an embodied agent find your" cat-shaped mug"? llm-based zero-shot object navigation, arXiv preprint arXiv:2303.03480 (2023). 20 — [arXiv:2303.03480](https://arxiv.org/abs/2303.03480) find
245. C. Huang, O. Mees, A. Zeng, W. Burgard, Visual language maps for robot navigation, in: 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023, pp. 10608–10615. 20 find
246. Y. Ding, X. Zhang, C. Paxton, S. Zhang, Task and motion planning with large language models for object rearrangement, arXiv preprint arXiv:2303.06247 (2023). 20, 33 — [arXiv:2303.06247](https://arxiv.org/abs/2303.06247) find
247. X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, J. Tang, Gpt understands, too, arXiv preprint arXiv:2103.10385 (2021). 20, 21 find our card
248. G. Chen, F. Liu, Z. Meng, S. Liang, Revisiting parameter-efficient tuning: Are we really there yet?, arXiv preprint arXiv:2202.07962 (2022). 20 find
249. Y. Wang, S. Mukherjee, X. Liu, J. Gao, A. H. Awadallah, J. Gao, Adamix: Mixture-of-adapter for parameter-efficient tuning of large language models, arXiv preprint arXiv:2205.12410 1 (2) (2022) 4. 20 — [arXiv:2205.12410](https://arxiv.org/abs/2205.12410) find
250. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, arXiv preprint arXiv:2106.09685 (2021). 21, 22, 23 find
251. X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, J. Tang, P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2022, pp. 61–68. 21 find our card
252. A. Razdaibiedina, Y. Mao, R. Hou, M. Khabsa, M. Lewis, A. Almahairi, Progressive prompts: Continual learning for language models, arXiv preprint arXiv:2301.12314 (2023). 21 find
253. Z.-R. Zhang, C. Tan, H. Xu, C. Wang, J. Huang, S. Huang, Towards adaptive prefix tuning for parameter-efficient language model fine-tuning, arXiv preprint arXiv:2305.15212 (2023). 21 find
254. E. B. Zaken, S. Ravfogel, Y. Goldberg, Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models, arXiv preprint arXiv:2106.10199 (2021). 21 find our card
255. T. Dettmers, M. Lewis, Y. Belkada, L. Zettlemoyer, Llm. int8 (): 8-bit matrix multiplication for transformers at scale, arXiv preprint arXiv:2208.07339 (2022). 21, 22 find
256. E. Frantar, S. Ashkboos, T. Hoefler, D. Alistarh, Gptq: Accurate post-training quantization for generative pre-trained transformers, arXiv preprint arXiv:2210.17323 (2022). 21 find
257. X. Wei, Y. Zhang, Y. Li, X. Zhang, R. Gong, J. Guo, X. Liu, Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling, arXiv preprint arXiv:2304.09145 (2023). 21 — [arXiv:2304.09145](https://arxiv.org/abs/2304.09145) find
258. E. Frantar, D. Alistarh, Optimal brain compression: A framework for accurate post-training quantization and pruning, Advances in Neural Information Processing Systems 35 (2022) 4475–4488. 21 find
259. C. Lee, J. Jin, T. Kim, H. Kim, E. Park, Owq: Lessons learned from activation outliers for weight quantization in large language models, arXiv preprint arXiv:2306.02272 (2023). 21 find
260. S. J. Kwon, J. Kim, J. Bae, K. M. Yoo, J.-H. Kim, B. Park, B. Kim, J.-W. Ha, N. Sung, D. Lee, Alphatuning: Quantization-aware parameter-efficient adaptation of large-scale pre-trained language models, arXiv preprint arXiv:2210.03858 (2022). 21 find
261. T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, Qlora: Efficient finetuning of quantized llms, arXiv preprint arXiv:2305.14314 (2023). 21, 22 find
262. Z. Liu, B. Oguz, C. Zhao, E. Chang, P. Stock, Y. Mehdad, Y. Shi, R. Krishnamoorthi, V. Chandra, Llm-qat: Data-free quantization aware training for large language models, arXiv preprint arXiv:2305.17888 (2023). 21, 22 find our card
263. Y. Guo, A. Yao, H. Zhao, Y. Chen, Network sketching: Exploiting binary structure in deep cnns, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 5955–5963. 21 find
264. J. Kim, J. H. Lee, S. Kim, J. Park, K. M. Yoo, S. J. Kwon, D. Lee, Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization, arXiv preprint arXiv:2305.14152 (2023). 22 — [arXiv:2305.14152](https://arxiv.org/abs/2305.14152) find
265. M. Sun, Z. Liu, A. Bair, J. Z. Kolter, A simple and effective pruning approach for large language models, arXiv preprint arXiv:2306.11695 (2023). 22 find
266. Z. Wang, J. Wohlwend, T. Lei, Structured pruning of large language models, arXiv preprint arXiv:1910.04732 (2019). 22 find
267. L. Yin, Y. Wu, Z. Zhang, C.-Y. Hsieh, Y. Wang, Y. Jia, M. Pechenizkiy, Y. Liang, Z. Wang, S. Liu, Outlier weighed layerwise sparsity (owl): A missing secret sauce for pruning llms to high sparsity, arXiv preprint arXiv:2310.05175 (2023). 22 — [arXiv:2310.05175](https://arxiv.org/abs/2310.05175) find
268. C. Tao, L. Hou, H. Bai, J. Wei, X. Jiang, Q. Liu, P. Luo, N. Wong, Structured pruning for efficient generative pre-trained language models, in: Findings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 10880–10895. 22 find
269. J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al., Flamingo: a visual language model for few-shot learning, Advances in Neural Information Processing Systems 35 (2022) 23716–23736. 22 find our card
270. J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, arXiv preprint arXiv:2301.12597 (2023). 22 find our card
271. H. Liu, C. Li, Q. Wu, Y. J. Lee, Visual instruction tuning, arXiv preprint arXiv:2304.08485 (2023). 22 find our card
272. K. Li, Y. He, Y. Wang, Y. Li, W. Wang, P. Luo, Y. Wang, L. Wang, Y. Qiao, Videochat: Chat-centric video understanding, arXiv preprint arXiv:2305.06355 (2023). 22 find our card
273. M. Maaz, H. Rasheed, S. Khan, F. S. Khan, Video-chatgpt: Towards detailed video understanding via large vision and language models, arXiv preprint arXiv:2306.05424 (2023). 22 find our card
274. H. Zhang, X. Li, L. Bing, Video-llama: An instruction-tuned audio-visual language model for video understanding, arXiv preprint arXiv:2306.02858 (2023). 22 find our card
275. X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y. Zou, W. Wang, Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research, arXiv preprint arXiv:2303.17395 (2023). 22 — [arXiv:2303.17395](https://arxiv.org/abs/2303.17395) find
276. C. Lyu, M. Wu, L. Wang, X. Huang, B. Liu, Z. Du, S. Shi, Z. Tu, Macaw-llm: Multi-modal language modeling with image, audio, video, and text integration, arXiv preprint arXiv:2306.09093 (2023). 22 find
277. D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models, arXiv preprint arXiv:2304.10592 (2023). 22 find our card
278. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020). 22 find
279. W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, S. Hoi, Instructblip: Towards general-purpose vision-language models with instruction tuning, arXiv preprint arXiv:2305.06500 (2023). 22 find our card
280. Z. Xu, Y. Shen, L. Huang, Multiinstruct: Improving multi-modal zero-shot learning via instruction tuning, arXiv preprint arXiv:2212.10773 (2022). 22 find
281. Z. Zhao, L. Guo, T. Yue, S. Chen, S. Shao, X. Zhu, Z. Yuan, J. Liu, Chatbridge: Bridging modalities with large language model as a language catalyst, arXiv preprint arXiv:2305.16103 (2023). 22 — [arXiv:2305.16103](https://arxiv.org/abs/2305.16103) find
282. L. Li, Y. Yin, S. Li, L. Chen, P. Wang, S. Ren, M. Li, Y. Yang, J. Xu, X. Sun, et al., M3 it: A large-scale dataset towards multi-modal multilingual instruction tuning, arXiv preprint arXiv:2306.04387 (2023). 22 — [arXiv:2306.04387](https://arxiv.org/abs/2306.04387) find
283. R. Pi, J. Gao, S. Diao, R. Pan, H. Dong, J. Zhang, L. Yao, J. Han, H. Xu, L. K. T. Zhang, Detgpt: Detect what you need via reasoning, arXiv preprint arXiv:2305.14167 (2023). 22 find
284. G. Luo, Y. Zhou, T. Ren, S. Chen, X. Sun, R. Ji, Cheap and quick: Efficient vision-language instruction tuning for large language models, arXiv preprint arXiv:2305.15023 (2023). 22 — [arXiv:2305.15023](https://arxiv.org/abs/2305.15023) find
285. R. Zhang, J. Han, A. Zhou, X. Hu, S. Yan, P. Lu, H. Li, P. Gao, Y. Qiao, Llama-adapter: Efficient fine-tuning of language models with zero-init attention, arXiv preprint arXiv:2303.16199 (2023). 22 find our card
286. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision, in: International Conference on Machine Learning, PMLR, 2023, pp. 28492– 28518. 22 find our card
287. Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, A. Smola, Multimodal chain-of-thought reasoning in language models, arXiv preprint arXiv:2302.00923 (2023). 23 find our card
288. J. Ge, H. Luo, S. Qian, Y. Gan, J. Fu, S. Zhan, Chain of thought prompt tuning in vision language models, arXiv preprint arXiv:2304.07919 (2023). 23 — [arXiv:2304.07919](https://arxiv.org/abs/2304.07919) find
289. C. Wu, S. Yin, W. Qi, X. Wang, Z. Tang, N. Duan, Visual chatgpt: Talking, drawing and editing with visual foundation models, arXiv preprint arXiv:2303.04671 (2023). 23 find our card
290. Z. Yang, L. Li, J. Wang, K. Lin, E. Azarnasab, F. Ahmed, Z. Liu, C. Liu, M. Zeng, L. Wang, Mm-react: Prompting chatgpt for multimodal reasoning and action, arXiv preprint arXiv:2303.11381 (2023). 23 find
291. T. Wang, J. Zhang, J. Fei, Y. Ge, H. Zheng, Y. Tang, Z. Li, M. Gao, S. Zhao, Y. Shan, et al., Caption anything: Interactive image description with diverse multimodal controls, arXiv preprint arXiv:2305.02677 (2023). 23 — [arXiv:2305.02677](https://arxiv.org/abs/2305.02677) find
292. X. Zhu, R. Zhang, B. He, Z. Zeng, S. Zhang, P. Gao, Pointclip v2: Adapting clip for powerful 3d open-world learning, arXiv preprint arXiv:2211.11682 (2022). 23 — [arXiv:2211.11682](https://arxiv.org/abs/2211.11682) find
293. T. Gupta, A. Kembhavi, Visual programming: Compositional visual reasoning without training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14953–14962. 23 find
294. P. Gao, Z. Jiang, H. You, P. Lu, S. C. Hoi, X. Wang, H. Li, Dynamic fusion with intra-and inter-modality attention flow for visual question answering, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6639–6648. 23 find
295. Z. Yu, J. Yu, Y. Cui, D. Tao, Q. Tian, Deep modular co-attention networks for visual question answering, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6281– 6290. 23 find
296. H. You, R. Sun, Z. Wang, L. Chen, G. Wang, H. A. Ayyubi, K.-W. Chang, S.-F. Chang, Idealgpt: Iteratively decomposing vision and language reasoning via large language models, arXiv preprint arXiv:2305.14985 (2023). 23 find
297. R. Zhang, X. Hu, B. Li, S. Huang, H. Deng, Y. Qiao, P. Gao, H. Li, Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15211–15222. 23 find
298. T. Q. Nguyen, J. Salazar, Transformers without tears: Improving the normalization of self-attention, CoRR abs/1910.05895 (2019). 24 find
299. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pre-training approach, arXiv preprint arXiv:1907.11692 (2019). 24, 30 find
300. X. Geng, A. Gudibande, H. Liu, E. Wallace, P. Abbeel, S. Levine, D. Song, Koala: A dialogue model for academic research, Blog post (April 2023). URL https://bair.berkeley.edu/blog/2023/04/03/koala/ 25 ↗
301. L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al., The pile: An 800gb dataset of diverse text for language modeling, arXiv preprint arXiv:2101.00027 (2020). 28, 30 find
302. H. Laurençon, L. Saulnier, T. Wang, C. Akiki, A. Villanova del Moral, T. Le Scao, L. Von Werra, C. Mou, E. González Ponferrada, H. Nguyen, et al., The bigscience roots corpus: A 1.6 tb composite multilingual dataset, Advances in Neural Information Processing Systems 35 (2022) 31809–31826. 28 find
303. Wikipedia. URL https://en.wikipedia.org/wiki/Main\_Page 28 ↗
304. Together Computer, Redpajama: An open source recipe to reproduce llama training dataset (Apr. 2023). URL https://github.com/togethercomputer/ RedPajama-Data 28 ↗
305. O. Honovich, T. Scialom, O. Levy, T. Schick, Unnatural instructions: Tuning language models with (almost) no human labor, arXiv preprint arXiv:2212.09689 (2022). 28 find our card
306. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al., Training a helpful and harmless assistant with reinforcement learning from human feedback, arXiv preprint arXiv:2204.05862 (2022). 28 find
307. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, J. Steinhardt, Measuring massive multitask language understanding, arXiv preprint arXiv:2009.03300 (2020). 26, 29 find our card
308. A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al., Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, arXiv preprint arXiv:2206.04615 (2022). 26, 29 find our card
309. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, S. R. Bowman, Glue: A multi-task benchmark and analysis platform for natural language understanding, arXiv preprint arXiv:1804.07461 (2018). 26, 29 find our card
310. Y. Yao, Q. Dong, J. Guan, B. Cao, Z. Zhang, C. Xiao, X. Wang, F. Qi, J. Bao, J. Nie, et al., Cuge: A chinese language understanding and generation evaluation benchmark, arXiv preprint arXiv:2112.13610 (2021). 29 — [arXiv:2112.13610](https://arxiv.org/abs/2112.13610) find
311. L. Xu, H. Hu, X. Zhang, L. Li, C. Cao, Y. Li, Y. Xu, K. Sun, D. Yu, C. Yu, et al., Clue: A chinese language understanding evaluation benchmark, arXiv preprint arXiv:2004.05986 (2020). 29 find our card
312. L. Xu, X. Lu, C. Yuan, X. Zhang, H. Xu, H. Yuan, G. Wei, X. Pan, X. Tian, L. Qin, et al., Fewclue: A chinese few-shot learning evaluation benchmark, arXiv preprint arXiv:2107.07498 (2021). 29 find
313. E. M. Smith, M. Williamson, K. Shuster, J. Weston, Y.-L. Boureau, Can you put it all together: Evaluating conversational agents’ ability to blend skills, arXiv preprint arXiv:2004.08449 (2020). 29 find our card
314. P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar, et al., Holistic evaluation of language models, arXiv preprint arXiv:2211.09110 (2022). 29 find
315. S. Park, J. Moon, S. Kim, W. I. Cho, J. Han, J. Park, C. Song, J. Kim, Y. Song, T. Oh, et al., Klue: Korean language understanding evaluation, arXiv preprint arXiv:2105.09680 (2021). 29 — [arXiv:2105.09680](https://arxiv.org/abs/2105.09680) find
316. S. Reddy, D. Chen, C. D. Manning, Coqa: A conversational question answering challenge, Transactions of the Association for Computational Linguistics 7 (2019) 249–266. 27, 29 find our card
317. M. T. Pilehvar, J. Camacho-Collados, Wic: 10,000 example pairs for evaluating context-sensitive representations, arXiv preprint arXiv:1808.09121 6 (2018). 27, 29 find our card
318. S. Merity, C. Xiong, J. Bradbury, R. Socher, Pointer sentinel mixture models, arXiv preprint arXiv:1609.07843 (2016). 28, 29 find our card
319. J. W. Rae, A. Potapenko, S. M. Jayakumar, T. P. Lillicrap, Compressive transformers for long-range sequence modelling, arXiv preprint arXiv:1911.05507 (2019). 28, 29 find our card
320. X. Liu, Q. Chen, C. Deng, H. Zeng, J. Chen, D. Li, B. Tang, Lcqmc: A large-scale chinese question matching corpus, in: Proceedings of the 27th international conference on computational linguistics, 2018, pp. 1952–1962. 28, 29 find
321. S. Iyer, N. Dandekar, K. Csernai, First quora dataset release: Question pairs, https://quoradata.quora.com/ First-Quora-Dataset-Release-Question-Pairs. 29 ↗
322. R. Rudinger, J. Naradowsky, B. Leonard, B. Van Durme, Gender bias in coreference resolution, arXiv preprint arXiv:1804.09301 (2018). 29 find our card
323. M.-C. De Marneffe, M. Simons, J. Tonhauser, The commitmentbank: Investigating projection in naturally occurring discourse, in: proceedings of Sinn und Bedeutung, Vol. 23, 2019, pp. 107–124. 29 find our card
324. Z. Li, N. Ding, Z. Liu, H. Zheng, Y. Shen, Chinese relation extraction with multi-grained information and external linguistic knowledge, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4377–4386. 29 find
325. J. Xu, J. Wen, X. Sun, Q. Su, A discourse-level named entity recognition and relation extraction dataset for chinese literature text, arXiv preprint arXiv:1711.07010 (2017). 29 — [arXiv:1711.07010](https://arxiv.org/abs/1711.07010) find
326. J. Chen, Q. Chen, X. Liu, H. Yang, D. Lu, B. Tang, The bq corpus: A large-scale domain-specific chinese corpus for sentence semantic equivalence identification, in: Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 4946–4951. 29 find
327. B. Liu, D. Niu, H. Wei, J. Lin, Y. He, K. Lai, Y. Xu, Matching article pairs with graphical decomposition and convolutions, arXiv preprint arXiv:1802.07459 (2018). 29 find
328. P. Li, W. Li, Z. He, X. Wang, Y. Cao, J. Zhou, W. Xu, Dataset and neural recurrent sequence labeling model for open-domain factoid question answering, arXiv preprint arXiv:1607.06275 (2016). 29 — [arXiv:1607.06275](https://arxiv.org/abs/1607.06275) find
329. N. Peng, M. Dredze, Named entity recognition for chinese social media with jointly trained embeddings, in: Proceedings of the 2015 conference on empirical methods in natural language processing, 2015, pp. 548– 554. 29 find
330. W. Ling, D. Yogatama, C. Dyer, P. Blunsom, Program induction by rationale generation: Learning to solve and explain algebraic word problems, arXiv preprint arXiv:1705.04146 (2017). 29 find
331. R. Weischedel, S. Pradhan, L. Ramshaw, M. Palmer, N. Xue, M. Marcus, A. Taylor, C. Greenberg, E. Hovy, R. Belvin, et al., Ontonotes release 4.0, LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium (2011). 29 find
332. D. Vilares, C. Gómez-Rodríguez, Head-qa: A healthcare dataset for complex reasoning, arXiv preprint arXiv:1906.04701 (2019). 29 find
333. S. L. Blodgett, L. Green, B. O’Connor, Demographic dialectal variation in social media: A case study of african-american english, arXiv preprint arXiv:1608.08868 (2016). 29 find our card
334. N. Mostafazadeh, N. Chambers, X. He, D. Parikh, D. Batra, L. Vanderwende, P. Kohli, J. Allen, A corpus and evaluation framework for deeper understanding of commonsense stories, arXiv preprint arXiv:1604.01696 (2016). 28, 29 find
335. D. Paperno, G. Kruszewski, A. Lazaridou, Q. N. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda, R. Fernández, The lambada dataset: Word prediction requiring a broad discourse context, arXiv preprint arXiv:1606.06031 (2016). 28, 29 find our card
336. B. Hu, Q. Chen, F. Zhu, Lcsts: A large scale chinese short text summarization dataset, arXiv preprint arXiv:1506.05865 (2015). 29 find
337. Z. Shao, M. Huang, J. Wen, W. Xu, X. Zhu, Long and diverse text generation with planning-based hierarchical variational model, arXiv preprint arXiv:1908.06605 (2019). 29 find
338. J. Novikova, O. Dušek, V. Rieser, The e2e dataset: New challenges for end-to-end generation, arXiv preprint arXiv:1706.09254 (2017). 29 find our card
339. C. Zheng, M. Huang, A. Sun, Chid: A large-scale chinese idiom dataset for cloze test, arXiv preprint arXiv:1906.01265 (2019). 29 find
340. Y. Bisk, R. Zellers, J. Gao, Y. Choi, et al., Piqa: Reasoning about physical commonsense in natural language, in: Proceedings of the AAAI conference on artificial intelligence, Vol. 34, 2020, pp. 7432–7439. 28, 29 find our card
341. M. Joshi, E. Choi, D. S. Weld, L. Zettlemoyer, Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, arXiv preprint arXiv:1705.03551 (2017). 28, 29, 31 find our card
342. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, O. Tafjord, Think you have solved question answering? try arc, the ai2 reasoning challenge, arXiv preprint arXiv:1803.05457 (2018). 28, 29, 31 find
343. S. Aroca-Ouellette, C. Paik, A. Roncone, K. Kann, Prost: Physical reasoning of objects through space and time, arXiv preprint arXiv:2106.03634 (2021). 29 — [arXiv:2106.03634](https://arxiv.org/abs/2106.03634) find
344. T. Mihaylov, P. Clark, T. Khot, A. Sabharwal, Can a suit of armor conduct electricity? a new dataset for open book question answering, arXiv preprint arXiv:1809.02789 (2018). 29 find
345. T. C. Ferreira, C. Gardent, N. Ilinykh, C. Van Der Lee, S. Mille, D. Moussallem, A. Shimorina, The 2020 bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020), in: Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web… find
346. C. Xu, W. Zhou, T. Ge, K. Xu, J. McAuley, F. Wei, Blow the dog whistle: A chinese dataset for cant understanding with common sense and world knowledge, arXiv preprint arXiv:2104.02704 (2021). 29 find
347. G. Lai, Q. Xie, H. Liu, Y. Yang, E. Hovy, Race: Large-scale reading comprehension dataset from examinations, arXiv preprint arXiv:1704.04683 (2017). 29 find our card
348. E. Choi, H. He, M. Iyyer, M. Yatskar, W.-t. Yih, Y. Choi, P. Liang, L. Zettlemoyer, Quac: Question answering in context, arXiv preprint arXiv:1808.07036 (2018). 29 find our card
349. M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, J. Berant, Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies, Transactions of the Association for Computational Linguistics 9 (2021) 346–361. 29, 31 find
350. J. Boyd-Graber, B. Satinoff, H. He, H. Daumé III, Besting the quiz master: Crowdsourcing incremental classification games, in: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, 2012, pp. 1290– 1301. 29 find
351. S. Zhang, X. Zhang, H. Wang, J. Cheng, P. Li, Z. Ding, Chinese medical question answer matching using end-to-end character-level multi-scale cnns, Applied Sciences 7 (8) (2017) 767. 29 find
352. S. Zhang, X. Zhang, H. Wang, L. Guo, S. Liu, Multi-scale attentive interaction networks for chinese medical question answer selection, IEEE Access 6 (2018) 74061–74071. 29 find
353. C. Xu, J. Pei, H. Wu, Y. Liu, C. Li, Matinf: A jointly labeled large-scale dataset for classification, question answering and summarization, arXiv preprint arXiv:2004.12302 (2020). 29 find
354. K. Sakaguchi, R. L. Bras, C. Bhagavatula, Y. Choi, Winogrande: An adversarial winograd schema challenge at scale, Communications of the ACM 64 (9) (2021) 99–106. 27, 29 find
355. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, Y. Choi, Hellaswag: Can a machine really finish your sentence?, arXiv preprint arXiv:1905.07830 (2019). 29 find
356. M. Roemmele, C. A. Bejan, A. S. Gordon, Choice of plausible alternatives: An evaluation of commonsense causal reasoning., in: AAAI spring symposium: logical formalizations of commonsense reasoning, 2011, pp. 90–95. 29 find
357. H. Levesque, E. Davis, L. Morgenstern, The winograd schema challenge, in: Thirteenth international conference on the principles of knowledge representation and reasoning, 2012. 27, 29 find
358. A. Talmor, J. Herzig, N. Lourie, J. Berant, Commonsenseqa: A question answering challenge targeting commonsense knowledge, arXiv preprint arXiv:1811.00937 (2018). 29, 31 find
359. M. Sap, H. Rashkin, D. Chen, R. LeBras, Y. Choi, Socialiqa: Commonsense reasoning about social interactions, arXiv preprint arXiv:1904.09728 (2019). 29 find our card
360. K. Sun, D. Yu, D. Yu, C. Cardie, Investigating prior knowledge for challenging chinese machine reading comprehension, Transactions of the Association for Computational Linguistics 8 (2020) 141–155. 29 find
361. S. Zhang, X. Liu, J. Liu, J. Gao, K. Duh, B. Van Durme, Record: Bridging the gap between human and machine commonsense reading comprehension, arXiv preprint arXiv:1810.12885 (2018). 29 find our card
362. P. Rajpurkar, J. Zhang, K. Lopyrev, P. Liang, Squad: 100,000+ questions for machine comprehension of text, arXiv preprint arXiv:1606.05250 (2016). 29, 31 find our card
363. C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, K. Toutanova, Boolq: Exploring the surprising difficulty of natural yes/no questions, arXiv preprint arXiv:1905.10044 (2019). 29, 31 find our card
364. P. Rajpurkar, R. Jia, P. Liang, Know what you don’t know: Unanswerable questions for squad, arXiv preprint arXiv:1806.03822 (2018). 29, 31 find our card
365. D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, M. Gardner, Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs, arXiv preprint arXiv:1903.00161 (2019). 29, 31 find our card
366. I. Dagan, O. Glickman, B. Magnini, The pascal recognising textual entailment challenge, in: Machine learning challenges workshop, Springer, 2005, pp. 177–190. 29, 31 find
367. Y. Chang, M. Narang, H. Suzuki, G. Cao, J. Gao, Y. Bisk, Webqa: Multihop and multimodal qa, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16495–16504. 29, 31 find
368. Y. Cui, T. Liu, Z. Chen, W. Ma, S. Wang, G. Hu, Dataset for the first evaluation on chinese machine reading comprehension, arXiv preprint arXiv:1709.08299 (2017). 29 find
369. Y. Cui, T. Liu, W. Che, L. Xiao, Z. Chen, W. Ma, S. Wang, G. Hu, A span-extraction dataset for chinese machine reading comprehension, arXiv preprint arXiv:1810.07366 (2018). 29, 31 find
370. Y. Cui, T. Liu, Z. Yang, Z. Chen, W. Ma, W. Che, S. Wang, G. Hu, A sentence cloze dataset for chinese machine reading comprehension, arXiv preprint arXiv:2004.03116 (2020). 29 find
371. Y. Li, T. Liu, D. Li, Q. Li, J. Shi, Y. Wang, Character-based bilstm-crf incorporating pos and dictionaries for chinese opinion target extraction, in: Asian Conference on Machine Learning, PMLR, 2018, pp. 518–533. 29 find
372. D. Khashabi, S. Chaturvedi, M. Roth, S. Upadhyay, D. Roth, Looking beyond the surface: A challenge set for reading comprehension over multiple sentences, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find our card
373. T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, et al., Natural questions: a benchmark for question answering research, Transactions of the Association for Computational Linguistics 7 (2019) 453–466. 29 find our card
374. C. C. Shao, T. Liu, Y. Lai, Y. Tseng, S. Tsai, Drcd: A chinese machine reading comprehension dataset, arXiv preprint arXiv:1806.00920 (2018). 29 find
375. W. He, K. Liu, J. Liu, Y. Lyu, S. Zhao, X. Xiao, Y. Liu, Y. Wang, H. Wu, Q. She, et al., Dureader: a chinese machine reading comprehension dataset from real-world applications, arXiv preprint arXiv:1711.05073 (2017). 29 find our card
376. H. Tang, J. Liu, H. Li, Y. Hong, H. Wu, H. Wang, Dureaderrobust: A chinese dataset towards evaluating the robustness of machine reading comprehension models, arXiv preprint arXiv:2004.11142 (2020). 29 find
377. J. Welbl, N. F. Liu, M. Gardner, Crowdsourcing multiple choice science questions, arXiv preprint arXiv:1707.06209 (2017). 29 find our card
378. C. Xiong, Z. Dai, J. Callan, Z. Liu, R. Power, End-to-end neural ad-hoc ranking with kernel pooling, in: Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval, 2017, pp. 55–64. 29 find
379. A. Peñas, E. Hovy, P. Forner, Á. Rodrigo, R. Sutcliffe, R. Morante, Qa4mre 2011-2013: Overview of question answering for machine reading evaluation, in: Information Access Evaluation. Multilinguality, Multimodality, and Visualization: 4th International Conference of the CLEF Initiative, CLEF 2013, V… find
380. S. Lim, M. Kim, J. Lee, Korquad1. 0: Korean qa dataset for machine reading comprehension, arXiv preprint arXiv:1909.07005 (2019). 29 — [arXiv:1909.07005](https://arxiv.org/abs/1909.07005) find
381. C. Xiao, H. Zhong, Z. Guo, C. Tu, Z. Liu, M. Sun, Y. Feng, X. Han, Z. Hu, H. Wang, et al., Cail2018: A large-scale legal dataset for judgment prediction, arXiv preprint arXiv:1807.02478 (2018). 29 find our card
382. D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, et al., Measuring coding challenge competence with apps, arXiv preprint arXiv:2105.09938 (2021). 29, 31 find our card
383. Y. Wang, X. Liu, S. Shi, Deep neural solver for math word problems, in: Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 845–854. 29, 31 find
384. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al., Training verifiers to solve math word problems, arXiv preprint arXiv:2110.14168 (2021). 29, 31 find our card
385. J. Austin, A. Odena, M. I. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. J. Cai, M. Terry, Q. V. Le, C. Sutton, Program synthesis with large language models, CoRR abs/2108.07732 (2021). 29 find our card
386. F. Shi, M. Suzgun, M. Freitag, X. Wang, S. Srivats, S. Vosoughi, H. W. Chung, Y. Tay, S. Ruder, D. Zhou, et al., Language models are multilingual chain-of-thought reasoners, arXiv preprint arXiv:2210.03057 (2022). 29 find our card
387. S. Roy, D. Roth, Solving general arithmetic word problems, arXiv preprint arXiv:1608.01413 (2016). 29 find our card
388. S.-Y. Miao, C.-C. Liang, K.-Y. Su, A diverse corpus for evaluating and developing english math word problem solvers, arXiv preprint arXiv:2106.15772 (2021). 29 find
389. R. Koncel-Kedziorski, S. Roy, A. Amini, N. Kushman, H. Hajishirzi, Mawps: A math word problem repository, in: Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies, 2016, pp. 1152–1157. 29 find
390. A. Patel, S. Bhattamishra, N. Goyal, Are nlp models really able to solve simple math word problems?, arXiv preprint arXiv:2103.07191 (2021). 29 find our card
391. Y. Lai, C. Li, Y. Wang, T. Zhang, R. Zhong, L. Zettlemoyer, W.-t. Yih, D. Fried, S. Wang, T. Yu, Ds-1000: A natural and reliable benchmark for data science code generation, in: International Conference on Machine Learning, PMLR, 2023, pp. 18319–18345. 29 find
392. J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al., Program synthesis with large language models, arXiv preprint arXiv:2108.07732 (2021). 29 find our card
393. Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, D. Kiela, Adversarial nli: A new benchmark for natural language understanding, arXiv preprint arXiv:1910.14599 (2019). 29, 31 find
394. A. Williams, N. Nangia, S. R. Bowman, A broad-coverage challenge corpus for sentence understanding through inference, arXiv preprint arXiv:1704.05426 (2017). 29 find
395. R. T. McCoy, E. Pavlick, T. Linzen, Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference, arXiv preprint arXiv:1902.01007 (2019). 29 find our card
396. J. Liu, L. Cui, H. Liu, D. Huang, Y. Wang, Y. Zhang, Logiqa: A challenge dataset for machine reading comprehension with logical reasoning, arXiv preprint arXiv:2007.08124 (2020). 29 find our card
397. P. Lewis, B. Oğuz, R. Rinott, S. Riedel, H. Schwenk, Mlqa: Evaluating cross-lingual extractive question answering, arXiv preprint arXiv:1910.07475 (2019). 29 find our card
398. A. Conneau, G. Lample, R. Rinott, A. Williams, S. R. Bowman, H. Schwenk, V. Stoyanov, Xnli: Evaluating cross-lingual sentence representations, arXiv preprint arXiv:1809.05053 (2018). 29, 31 find
399. Y. Yang, Y. Zhang, C. Tar, J. Baldridge, Paws-x: A cross-lingual adversarial dataset for paraphrase identification, arXiv preprint arXiv:1908.11828 (2019). 29, 31 find our card
400. S. Narayan, S. B. Cohen, M. Lapata, Don’t give me the details, just the summary!, Topic-Aware Convolutional Neural Networks for Extreme Summarization. ArXiv, abs (1808). 29 find
401. E. M. Ponti, G. Glavaš, O. Majewska, Q. Liu, I. Vulić, A. Korhonen, Xcopa: A multilingual dataset for causal commonsense reasoning, arXiv preprint arXiv:2005.00333 (2020). 29 find our card
402. A. Tikhonov, M. Ryabinin, It’s all in the heads: Using attention heads as a baseline for cross-lingual transfer in commonsense reasoning, arXiv preprint arXiv:2106.12066 (2021). 29 find
403. J. H. Clark, E. Choi, M. Collins, D. Garrette, T. Kwiatkowski, V. Nikolaev, J. Palomaki, Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages, Transactions of the Association for Computational Linguistics 8 (2020) 454–470. 29 find our card
404. T. Scialom, P.-A. Dray, S. Lamprier, B. Piwowarski, J. Staiano, Mlsum: The multilingual summarization corpus, arXiv preprint arXiv:2004.14900 (2020). 29 find
405. S. Lin, J. Hilton, O. Evans, Truthfulqa: Measuring how models mimic human falsehoods, arXiv preprint arXiv:2109.07958 (2021). 29, 32 find our card
406. I. Augenstein, C. Lioma, D. Wang, L. C. Lima, C. Hansen, C. Hansen, J. G. Simonsen, Multifc: A real-world multi-domain dataset for evidence-based fact checking of claims, arXiv preprint arXiv:1909.03242 (2019). 29 find our card
407. J. Thorne, A. Vlachos, C. Christodoulopoulos, A. Mittal, Fever: a large-scale dataset for fact extraction and verification, arXiv preprint arXiv:1803.05355 (2018). 29 find our card
408. I. Mollas, Z. Chrysopoulou, S. Karlos, G. Tsoumakas, Ethos: an online hate speech detection dataset, arXiv preprint arXiv:2006.08328 (2020). 29, 32 — [arXiv:2006.08328](https://arxiv.org/abs/2006.08328) find
409. M. Nadeem, A. Bethke, S. Reddy, Stereoset: Measuring stereotypical bias in pretrained language models, arXiv preprint arXiv:2004.09456 (2020). 29, 32 find our card
410. A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, S. R. Bowman, Bbq: A hand-built bias benchmark for question answering, arXiv preprint arXiv:2110.08193 (2021). 29 find our card
411. J. Zhao, T. Wang, M. Yatskar, V. Ordonez, K.-W. Chang, Gender bias in coreference resolution: Evaluation and debiasing methods, arXiv preprint arXiv:1804.06876 (2018). 29 find our card
412. N. Nangia, C. Vania, R. Bhalerao, S. R. Bowman, Crows-pairs: A challenge dataset for measuring social biases in masked language models, arXiv preprint arXiv:2010.00133 (2020). 29 find our card
413. S. Gehman, S. Gururangan, M. Sap, Y. Choi, N. A. Smith, Realtoxicityprompts: Evaluating neural toxic degeneration in language models, arXiv preprint arXiv:2009.11462 (2020). 29 find our card
414. D. Borkan, L. Dixon, J. Sorensen, N. Thain, L. Vasserman, Nuanced metrics for measuring unintended bias with real data for text classification, in: Companion proceedings of the 2019 world wide web conference, 2019, pp. 491–500. 29 find
415. O. Bojar, R. Chatterjee, C. Federmann, Y. Graham, B. Haddow, M. Huck, A. J. Yepes, P. Koehn, V. Logacheva, C. Monz, et al., Findings of the 2016 conference on machine translation, in: Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, 2016, pp. 131–198. 29 find our card
416. B. Loïc, B. Magdalena, B. Ondřej, F. Christian, G. Yvette, G. Roman, H. Barry, H. Matthias, J. Eric, K. Tom, et al., Findings of the 2020 conference on machine translation (wmt20), in: Proceedings of the Fifth Conference on Machine Translation, Association for Computational Linguistics„ 2020, pp. 1… find
417. W. Li, F. Qi, M. Sun, X. Yi, J. Zhang, Ccpm: A chinese classical poetry matching dataset, arXiv preprint arXiv:2106.01979 (2021). 29 — [arXiv:2106.01979](https://arxiv.org/abs/2106.01979) find
418. E. Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, J. Weston, Wizard of wikipedia: Knowledge-powered conversational agents, arXiv preprint arXiv:1811.01241 (2018). 29 find
419. H. Rashkin, E. M. Smith, M. Li, Y.-L. Boureau, Towards empathetic open-domain conversation models: A new benchmark and dataset, arXiv preprint arXiv:1811.00207 (2018). 29 find
420. E. Dinan, V. Logacheva, V. Malykh, A. Miller, K. Shuster, J. Urbanek, D. Kiela, A. Szlam, I. Serban, R. Lowe, et al., The second conversational intelligence challenge (convai2), in: The NeurIPS’18 Competition: From Machine Learning to Intelligent Conversations, Springer, 2020, pp. 187–208. 29 find our card
421. H. Zhou, C. Zheng, K. Huang, M. Huang, X. Zhu, Kdconv: A chinese multi-domain dialogue dataset towards multi-turn knowledge-driven conversation, arXiv preprint arXiv:2004.04100 (2020). 29 find
422. L. CO, Iflytek: a multiple categories chinese text classifier. competition official website (2019). 29 find
423. J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, J. Blackburn, The pushshift reddit dataset, in: Proceedings of the international AAAI conference on web and social media, Vol. 14, 2020, pp. 830–839. 30 find
424. A. Fan, Y. Jernite, E. Perez, D. Grangier, J. Weston, M. Auli, Eli5: Long form question answering, arXiv preprint arXiv:1907.09190 (2019). 31 find our card
425. Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Arunkumar, A. Ashok, A. S. Dhanasekaran, A. Naik, D. Stap, et al., Benchmarking generalization via in-context instructions on 1,600+ language tasks, arXiv preprint arXiv:2204.07705 (2022). 31 find
426. T. Xie, C. H. Wu, P. Shi, R. Zhong, T. Scholak, M. Yasunaga, C.-S. Wu, M. Zhong, P. Yin, S. I. Wang, et al., Unifiedskg: Unifying and multi-tasking structured knowledge grounding with text-to-text language models, arXiv preprint arXiv:2201.05966 (2022). 31 find our card
427. Q. Ye, B. Y. Lin, X. Ren, Crossfit: A few-shot learning challenge for cross-task generalization in nlp, arXiv preprint arXiv:2104.08835 (2021). 31 find
428. V. Aribandi, Y. Tay, T. Schuster, J. Rao, H. S. Zheng, S. V. Mehta, H. Zhuang, V. Q. Tran, D. Bahri, J. Ni, et al., Ext5: Towards extreme multi-task scaling for transfer learning, arXiv preprint arXiv:2111.10952 (2021). 31 find
429. A. Williams, N. Nangia, S. Bowman, A broad-coverage challenge corpus for sentence understanding through inference, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Association f… find
430. Y. Zhang, J. Baldridge, L. He, PAWS: Paraphrase adversaries from word scrambling, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Lingui… find our card
431. C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, D. Yang, Is chat-GPT a general-purpose natural language processing task solver?, in: The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https://openreview.net/forum?id=u03xn1COsO 32 ↗
432. M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili, et al., Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects, TechRxiv (2023). 32 find
433. X. L. Dong, S. Moon, Y. E. Xu, K. Malik, Z. Yu, Towards next-generation intelligent assistants leveraging llm techniques, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 5792–5793. 32 find
434. K. Pandya, M. Holia, Automating customer service using langchain: Building custom open-source gpt chatbot for organizations, arXiv preprint arXiv:2310.05421 (2023). 32 — [arXiv:2310.05421](https://arxiv.org/abs/2310.05421) find
435. J. Li, B. Hui, G. Qu, B. Li, J. Yang, B. Li, B. Wang, B. Qin, R. Cao, R. Geng, et al., Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls, arXiv preprint arXiv:2305.03111 (2023). 32 find our card
436. A. Rao, J. Kim, M. Kamineni, M. Pang, W. Lie, M. D. Succi, Evaluating chatgpt as an adjunct for radiologic decision-making, medRxiv (2023) 2023–02. 32 find
437. M. Benary, X. D. Wang, M. Schmidt, D. Soll, G. Hilfenhaus, M. Nassir, C. Sigler, M. Knödler, U. Keller, D. Beule, et al., Leveraging large language models for decision support in personalized oncology, JAMA Network Open 6 (11) (2023) e2343689–e2343689. 32 find
438. C. M. Chiesa-Estomba, J. R. Lechien, L. A. Vaira, A. Brunet, G. Cam-maroto, M. Mayo-Yanez, A. Sanchez-Barrueco, C. Saga-Gutierrez, Exploring the potential of chat-gpt as a supportive tool for sialendoscopy clinical decision making and patient information support, European Archives of Oto-Rhino-Laryn… find
439. S. Montagna, S. Ferretti, L. C. Klopfenstein, A. Florio, M. F. Pengo, Data decentralisation of llm-based chatbot systems in chronic disease self-management, in: Proceedings of the 2023 ACM Conference on Information Technology for Social Good, 2023, pp. 205–212. 32 find
440. D. Bill, T. Eriksson, Fine-tuning a llm using reinforcement learning from human feedback for a therapy chatbot application (2023). 32 find
441. M. Abbasian, I. Azimi, A. M. Rahmani, R. Jain, Conversational health agents: A personalized llm-powered agent framework, arXiv preprint arXiv:2310.02374 (2023). 32 — [arXiv:2310.02374](https://arxiv.org/abs/2310.02374) find
442. K. V. Lemley, Does chatgpt help us understand the medical literature?, Journal of the American Society of Nephrology (2023) 10–1681. 32 find
443. S. Pal, M. Bhattacharya, S.-S. Lee, C. Chakraborty, A domain-specific next-generation large language model (llm) or chatgpt is required for biomedical engineering and research, Annals of Biomedical Engineering (2023) 1–4. 32 find
444. Y. Du, S. Zhao, Y. Chen, R. Bai, J. Liu, H. Wu, H. Wang, B. Qin, The calla dataset: Probing llms’ interactive knowledge acquisition from chinese medical literature, arXiv preprint arXiv:2309.04198 (2023). 32 — [arXiv:2309.04198](https://arxiv.org/abs/2309.04198) find
445. A. Abd-Alrazaq, R. AlSaad, D. Alhuwail, A. Ahmed, P. M. Healy, S. Latifi, S. Aziz, R. Damseh, S. A. Alrazak, J. Sheikh, et al., Large language models in medical education: Opportunities, challenges, and future directions, JMIR Medical Education 9 (1) (2023) e48291. 32 find
446. A. B. Mbakwe, I. Lourentzou, L. A. Celi, O. J. Mechanic, A. Dagan, Chatgpt passing usmle shines a spotlight on the flaws of medical education (2023). 32 find
447. S. Ahn, The impending impacts of large language models on medical education, Korean Journal of Medical Education 35 (1) (2023) 103. 32 find
448. E. Waisberg, J. Ong, M. Masalkhi, A. G. Lee, Large language model (llm)-driven chatbots for neuro-ophthalmic medical education, Eye (2023) 1–3. 32 find
449. G. Deiana, M. Dettori, A. Arghittu, A. Azara, G. Gabutti, P. Castiglia, Artificial intelligence and public health: Evaluating chatgpt responses to vaccination myths and misconceptions, Vaccines 11 (7) (2023) 1217. 32 find
450. L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. E. Tozzi, C. Rizzo, Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health, Frontiers in Public Health 11 (2023) 1166120. 32 find
451. N. L. Rane, A. Tawde, S. P. Choudhary, J. Rane, Contribution and performance of chatgpt and other large language models (llm) for scientific and research advancements: a double-edged sword, International Research Journal of Modernization in Engineering Technology and Science 5 (10) (2023) 875–899. 3… find
452. W. Dai, J. Lin, H. Jin, T. Li, Y.-S. Tsai, D. Gašević, G. Chen, Can large language models provide feedback to students? a case study on chatgpt, in: 2023 IEEE International Conference on Advanced Learning Technologies (ICALT), IEEE, 2023, pp. 323–325. 32 find
453. E. Kasneci, K. Seßler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier, et al., Chatgpt for good? on opportunities and challenges of large language models for education, Learning and individual differences 103 (2023) 102274. 32 find our card
454. N. Rane, Enhancing the quality of teaching and learning through chatgpt and similar large language models: Challenges, future prospects, and ethical considerations in education, Future Prospects, and Ethical Considerations in Education (September 15, 2023) (2023). 32 find
455. J. C. Young, M. Shishido, Investigating openai’s chatgpt potentials in generating chatbot’s dialogue for english as a foreign language learning, International Journal of Advanced Computer Science and Applications 14 (6) (2023). 32 find
456. J. Irons, C. Mason, P. Cooper, S. Sidra, A. Reeson, C. Paris, Exploring the impacts of chatgpt on future scientific work, SocArXiv (2023). 32 find
457. P. G. Schmidt, A. J. Meir, Using generative ai for literature searches and scholarly writing: Is the integrity of the scientific discourse in jeopardy?, arXiv preprint arXiv:2311.06981 (2023). 32 — [arXiv:2311.06981](https://arxiv.org/abs/2311.06981) find
458. Y. Zheng, H. Y. Koh, J. Ju, A. T. Nguyen, L. T. May, G. I. Webb, S. Pan, Large language models for scientific synthesis, inference and explanation, arXiv preprint arXiv:2310.07984 (2023). 33 — [arXiv:2310.07984](https://arxiv.org/abs/2310.07984) find
459. B. Aczel, E.-J. Wagenmakers, Transparency guidance for chatgpt usage in scientific writing, PsyArXiv (2023). 33 find
460. S. Altmäe, A. Sola-Leyva, A. Salumets, Artificial intelligence in scientific writing: a friend or a foe?, Reproductive BioMedicine Online (2023). 33 find
461. S. Imani, L. Du, H. Shrivastava, Mathprompter: Mathematical reasoning using large language models, arXiv preprint arXiv:2303.05398 (2023). 33 find our card
462. Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, C. Zhou, Scaling relationship on learning mathematical reasoning with large language models, arXiv preprint arXiv:2308.01825 (2023). 33 find our card
463. K. Yang, A. M. Swope, A. Gu, R. Chalamala, P. Song, S. Yu, S. Godil, R. Prenger, A. Anandkumar, Leandojo: Theorem proving with retrieval-augmented language models, arXiv preprint arXiv:2306.15626 (2023). 33 find our card
464. K. M. Collins, A. Q. Jiang, S. Frieder, L. Wong, M. Zilka, U. Bhatt, T. Lukasiewicz, Y. Wu, J. B. Tenenbaum, W. Hart, et al., Evaluating language models for mathematics through interactions, arXiv preprint arXiv:2306.01694 (2023). 33 find
465. Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu, et al., Summary of chatgpt-related research and perspective towards the future of large language models, Meta-Radiology (2023) 100017. 33 find
466. J. Drápal, H. Westermann, J. Savelka, Using large language models to support thematic analysis in empirical legal studies, arXiv preprint arXiv:2310.18729 (2023). 33 find
467. J. Savelka, K. D. Ashley, M. A. Gray, H. Westermann, H. Xu, Explaining legal concepts with augmented large language models (gpt-4), arXiv preprint arXiv:2306.09525 (2023). 33 find
468. N. Guha, J. Nyarko, D. E. Ho, C. Ré, A. Chilton, A. Narayana, A. Chohlas-Wood, A. Peters, B. Waldon, D. N. Rockmore, et al., Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models, arXiv preprint arXiv:2308.11462 (2023). 33 find our card
469. J. Cui, Z. Li, Y. Yan, B. Chen, L. Yuan, Chatlaw: Open-source legal large language model with integrated external knowledge bases, arXiv preprint arXiv:2306.16092 (2023). 33 find our card
470. H. Yang, X.-Y. Liu, C. D. Wang, Fingpt: Open-source financial large language models, arXiv preprint arXiv:2306.06031 (2023). 33 find our card
471. Y. Li, S. Wang, H. Ding, H. Chen, Large language models in finance: A survey, in: Proceedings of the Fourth ACM International Conference on AI in Finance, 2023, pp. 374–382. 33 find
472. A. Lykov, D. Tsetserukou, Llm-brain: Ai-driven fast generation of robot behaviour tree based on large language model, arXiv preprint arXiv:2305.19352 (2023). 33 — [arXiv:2305.19352](https://arxiv.org/abs/2305.19352) find
473. E. Billing, J. Rosén, M. Lamb, Language models for human-robot interaction, in: ACM/IEEE International Conference on Human-Robot Interaction, March 13–16, 2023, Stockholm, Sweden, ACM Digital Library, 2023, pp. 905–906. 33 find
474. Y. Ye, H. You, J. Du, Improved trust in human-robot collaboration with chatgpt, IEEE Access (2023). 33 find
475. Y. Ding, X. Zhang, C. Paxton, S. Zhang, Leveraging commonsense knowledge from large language models for task and motion planning, in: RSS 2023 Workshop on Learning for Task and Motion Planning, 2023. 33 find
476. J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, T. Funkhouser, Tidybot: Personalized robot assistance with large language models, arXiv preprint arXiv:2305.05658 (2023). 33 — [arXiv:2305.05658](https://arxiv.org/abs/2305.05658) find
477. E. Strubell, A. Ganesh, A. McCallum, Energy and policy considerations for deep learning in nlp, arXiv preprint arXiv:1906.02243 (2019). 34 find our card
478. E. M. Bender, T. Gebru, A. McMillan-Major, S. Shmitchell, On the dangers of stochastic parrots: Can language models be too big?, in: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 2021, pp. 610–623. 34 find
479. C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals, Understanding deep learning (still) requires rethinking generalization, Communications of the ACM 64 (3) (2021) 107–115. 34 find
480. M. Tänzer, S. Ruder, M. Rei, Memorisation versus generalisation in pre-trained language models, arXiv preprint arXiv:2105.00828 (2021). 34 find
481. S. M. West, M. Whittaker, K. Crawford, Discriminating systems, AI Now (2019) 1–33. 34 find
482. K. Valmeekam, A. Olmo, S. Sreedharan, S. Kambhampati, Large language models still can’t plan (a benchmark for llms on planning and reasoning about change), arXiv preprint arXiv:2206.10498 (2022). 34 find our card
483. Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al., Siren’s song in the ai ocean: A survey on hallucination in large language models, arXiv preprint arXiv:2309.01219 (2023). 34 find our card
484. A. Webson, E. Pavlick, Do prompt-based models really understand the meaning of their prompts?, arXiv preprint arXiv:2109.01247 (2021). 34 find our card
485. O. Shaikh, H. Zhang, W. Held, M. Bernstein, D. Yang, On second thought, let’s not think step by step! bias and toxicity in zero-shot reasoning, arXiv preprint arXiv:2212.08061 (2022). 34 find our card
486. B. C. Das, M. H. Amini, Y. Wu, Security and privacy challenges of large language models: A survey, arXiv preprint arXiv:2402.00888 (2024). 34 find
487. X. Liu, H. Cheng, P. He, W. Chen, Y. Wang, H. Poon, J. Gao, Adversarial training for large neural language models, ArXiv (April 2020). URL https://www.microsoft.com/en-us/research/ publication/adversarial-training-for-large-neural-language-models/ 34 ↗
488. E. Shayegani, M. A. A. Mamun, Y. Fu, P. Zaree, Y. Dong, N. Abu-Ghazaleh, Survey of vulnerabilities in large language models revealed by adversarial attacks (2023). arXiv:2310.10844. 34 find
489. X. Xu, K. Kong, N. Liu, L. Cui, D. Wang, J. Zhang, M. Kankanhalli, An llm can fool itself: A prompt-based adversarial attack (2023). arXiv: 2310.13345. 34 — [arXiv:2310.13345](https://arxiv.org/abs/2310.13345) find
490. H. Zhao, H. Chen, F. Yang, N. Liu, H. Deng, H. Cai, S. Wang, D. Yin, M. Du, Explainability for large language models: A survey (2023). arXiv:2309.01029. 35 find our card
491. S. Huang, S. Mamidanna, S. Jangam, Y. Zhou, L. H. Gilpin, Can large language models explain themselves? a study of llm-generated self-explanations (2023). arXiv:2310.11207. 35 — [arXiv:2310.11207](https://arxiv.org/abs/2310.11207) find
492. H. Brown, K. Lee, F. Mireshghallah, R. Shokri, F. Tramèr, What does it mean for a language model to preserve privacy?, in: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 2280–2292. 35 find
493. R. Plant, V. Giuffrida, D. Gkatzia, You are what you write: Preserving privacy in the era of large language models, arXiv preprint arXiv:2204.09391 (2022). 35 — [arXiv:2204.09391](https://arxiv.org/abs/2204.09391) find
494. W. Niu, Z. Kong, G. Yuan, W. Jiang, J. Guan, C. Ding, P. Zhao, S. Liu, B. Ren, Y. Wang, Real-time execution of large-scale language models on mobile (2020). arXiv:2009.06823. 35 find
495. C. Guo, J. Tang, W. Hu, J. Leng, C. Zhang, F. Yang, Y. Liu, M. Guo, Y. Zhu, Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization, in: Proceedings of the 50th Annual International Symposium on Computer Architecture, 2023, pp. 1–15. 35 find
496. B. Meskó, E. J. Topol, The imperative for regulatory oversight of large language models (or generative ai) in healthcare, npj Digital Medicine 6 (1) (2023) 120. 35 find
497. J. Zhang, X. Ji, Z. Zhao, X. Hei, K.-K. R. Choo, Ethical considerations and policy implications for large language models: Guiding responsible development and deployment, arXiv preprint arXiv:2308.02678 (2023). 35 — [arXiv:2308.02678](https://arxiv.org/abs/2308.02678) find
498. J. Mökander, J. Schuett, H. R. Kirk, L. Floridi, Auditing large language models: a three-layered approach, AI and Ethics (2023) 1–31. 35 find