Elamur Research⌕
☰
home / papers / 2023 / The Generative AI Paradox: "What It Can Create, It May Not Understand"

The Generative AI Paradox: "What It Can Create, It May Not Understand"

Paper · 2023 · Peter West, Ximing Lu, Nouha Dziri et al.

Summary

Today's generative systems, such as GPT4 or Midjourney, can produce text and images that rival expert work, yet they still make elementary mistakes that ordinary people would avoid. The authors call this the Generative AI Paradox. Because models are trained to imitate expert outputs directly, their ability to generate need not rest on understanding the same material and can even exceed it. Humans tend to show the reverse pattern, with comprehension usually coming before expert production.

To test this, the paper defines understanding in two ways:

  • Selective setting: the model must choose the correct answer in a discriminative version, such as multiple choice, of a task it can already solve generatively.
  • Interrogative setting: the model must answer questions about content it generated itself.

The authors compare GPT3.5 and GPT4, along with image generators and vision-language models, against human baselines. Models often match or beat people at generation but fall behind them in both forms of understanding. The authors conclude that AI capabilities should not be interpreted through human-intelligence analogies.

Key points

  • Formal hypothesis: if a human and a model score equally on generation for a task, the human's understanding score should exceed the model's by more than a margin ϵ. The paper tests two sub-hypotheses, one for the selective setting and one for the interrogative setting.
  • Language benchmarks cover commonsense, NLI, dialogue, summarization and QA: CSQA, SocialIQA, HellaSwag, PIQA, α-NLI/α-NLG, WaNLI, δ-NLI, FaithDial, DREAM, Mutual+, XSUM, Topioca and RACE.
  • Generation vs. discrimination: for GPT4, discrimination accuracy rises much less steeply with generation quality than it does for humans. Humans also cope better with adversarial hard negatives, and the human–model gap grows as tasks become harder.
  • Interrogative probing: GPT-4 misreads details of its own stories. In one example it claims only one design was sketched in a story where it had written about years of sketching designs. It also gives answers that contradict each other across follow-up questions.
  • Proposed causes and implications: the authors point to training objectives and the scale and nature of training input as possible explanations. They recommend treating models as a counterpoint to human cognition rather than a parallel to it.

Note: the middle of the supplied text was missing, so exact accuracy figures and full vision-model details are not included.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger and Yejin Choi. 2023. The Generative AI Paradox: "What It Can Create, It May Not Understand". arXiv:2311.00059.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2311.00059 · arXiv 2311.00059

@misc{west2023generative,
  title = {{The Generative AI Paradox: "What It Can Create, It May Not Understand"}},
  author = {Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger and Yejin Choi},
  year = {2023},
  doi = {10.48550/arxiv.2311.00059},
  eprint = {2311.00059},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.00059}
}

References

60 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vaibhav Adlakha, Shehzaad Dhuliawala, Kaheer Suleman, Harm de Vries, and Siva Reddy. Topiocqa: Open-domain conversational question answering with topic switching. Transactions of the Association for Computational Linguistics, 10:468–483, 2022.
  2. Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. Do language models know when they’re hallucinating references? arXiv preprint arXiv:2305.18248, 2023.
  3. Patricia A Alexander. The development of expertise: The journey from acclimation to proficiency. Educational researcher, 32(8):10–14, 2003. find
  4. Jacob Andreas. Models of meaning? The 11th Joint Conference on Lexical and Computational Semantics at NAACL, 2022. find
  5. Konstantine Arkoudas. Gpt-4 can’t reason. arXiv preprint arXiv:2308.03762, 2023.
  6. Emily Bender. High resource languages vs low resource languages. The Gradient, 2019. URL https://thegradient.pub/ the-benderrule-on-naming-the-languages-we-study-and-why-it-matters/ \#fn4. ↗
  7. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on”a is b”fail to learn”b is a”. 2023. URL https://api.semanticscholar.org/CorpusID:262083829. ↗
  8. David C Berliner. Expertise: The wonder of exemplary performances. Creating powerful thinking in teachers and students, pp. 161–186, 1994. find
  9. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. Abductive commonsense reasoning. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Byg1v1HKDB. ↗
  10. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432–7439, 2020.
  11. Marc Brysbaert. How many words do we read per minute? a review and meta-analysis of reading rate. Journal of Memory and Language, 109:104047, 2019. ISSN 0749-596X. doi: https://doi.org/10.1016/j.jml.2019.104047. URL https://www.sciencedirect.com/science/ article/pii/S0749596X19300786. — [doi:10.1016/j.jml.2019.104047](https://doi.org/10.1016/j.jml.2019.104047) ↗
  12. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  13. Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers. 2022. find
  14. Wikipedia contributors. Wikipedia:size of wikipedia - wikipedia. URL https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia\#:˜:text=As% 20of%2022%20September%202023,of%20all%20pages%20on%20Wikipedia. ↗
  15. Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang, and Ming Zhou. MuTual: A dataset for multi-turn dialogue reasoning. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1406–1416, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020…
  16. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.
  17. Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar Zaiane, Mo Yu, Edoardo M Ponti, and Siva Reddy. FaithDial: A Faithful Benchmark for Information-Seeking Dialogue. Transactions of the Association for Computational Linguistics, 10:1473–1490, 12 2022. doi: 10.1162/tacl a 00529.
  18. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jian, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D Hwang, et al. Faith and fate: Limits of transformers on compositionality. arXiv preprint arXiv:2305.18654, 2023.
  19. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031, 2021. doi: 10.1162/tacl a 00410. URL ht… find
  20. Allyson Ettinger. What bert is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48, Jan 2020. ISSN 2307-387X. doi: 10.1162/tacl a 00298. URL http://dx.doi.org/10.1162/ tacl_a_00298.
  21. Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao, and Hoifung Poon. Self-verification improves few-shot clinical information extraction. arXiv preprint arXiv:2306.00024, 2023.
  22. Fernand Gobet. Understanding expertise: A multi-disciplinary approach. Bloomsbury Publishing, 2017. find
  23. Google. Bard. https://bard.google.com, 2023. Accessed before: 2023-09-28. ↗
  24. Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. arXiv preprint arXiv:2303.11897, 2023.
  25. Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. arXiv preprint arXiv:2307.06350, 2023. find
  26. Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/ zenodo.5143773. If you use this softwar… find
  27. Midjourney Inc. Midjourney. https://midjourney.com, 2023. Accessed before: 2023-09-28. ↗
  28. Myeongjun Jang, Deuk Sin Kwon, and Thomas Lukasiewicz. BECEL: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 3680–3696, Gyeongju, Republic of Korea, October 2022. International Committee on Computational… find
  29. Nora Kassner and Hinrich Schütze. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. Association for Computational Linguistics, 2020.
  30. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 785–794, Copenhagen, Denmark, September 2017. Association for Computatio…
  31. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022.
  32. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023.
  33. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
  34. Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. WANLI: Worker and AI collaboration for natural language inference dataset creation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 6826–6847, Abu Dhabi, United Arab Emirates, December 2022. Association for C…
  35. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. find
  36. Microsoft. Bingchat. https://bing.com/chat, 2023. Accessed before: 2023-09-28. ↗
  37. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797–1807, Brussels, Belgium, October-No…
  38. OpenAI. ChatGPT: Optimizing language models for dialogue, 2022. find
  39. OpenAI. GPT-4 technical report, 2023. URL https://arxiv.org/pdf/2303.08774.pdf. ↗
  40. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476, 2023.
  41. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763. PMLR, 2021…
  42. Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. Studying the inductive biases of rnns with synthetic variations of natural languages. In North American Chapter of the Association for Computational Linguistics, 2019. URL https://api.semanticscholar.org/CorpusID: 80628431. ↗
  43. Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. On the systematicity of probing contextualized word representations: The case of hypernymy in BERT. In Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, pp. 88–102, Barcelon…
  44. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. Are red roses red? evaluating consistency of question-answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6174–6184, Florence, Italy, July 2019. Association for Computational Linguis… find
  45. Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, and Yejin Choi. Thinking like a skeptic: Defeasible inference in natural language. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4661–4675, Online, Novemb… find
  46. Mihaela Sabin, Karen H Jin, and Adrienne Smith. Oral exams in shift to remote learning. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, pp. 666– 672, 2021. find
  47. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35…
  48. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EM… find
  49. Maarten Sap, Ronan LeBras, Daniel Fried, and Yejin Choi. Neural theory-of-mind? on the limits of social intelligence in large lms. arXiv preprint arXiv:2210.13312, 2022. find
  50. Minjoon Seo, Tom Kwiatkowski, Ankur P Parikh, Ali Farhadi, and Hannaneh Hajishirzi. Phrase-indexed question answering: A new challenge for scalable document comprehension. In EMNLP, 2018. find
  51. Gala Stojnić, Kanishk Gandhi, Shannon Yasuda, Brenden M Lake, and Moira R Dillon. Commonsense psychology in human infants and machines. Cognition, 235:105406, 2023. find
  52. Shane Storks, Qiaozi Gao, Yichi Zhang, and Joyce Chai. Tiered reasoning for intuitive physics: Toward verifiable commonsense language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4902–4918, Punta Cana, Dominican Republic, November 2021. Association for… find
  53. Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. DREAM: A challenge data set and models for dialogue-based reading comprehension. Transactions of the Association for Computational Linguistics, 7:217–231, 2019. doi: 10.1162/tacl a 00264. URL https://aclanthology.org/Q19-1014. ↗
  54. Nigar M Shafiq Surameery and Mohammed Y Shakor. Use chat gpt to solve programming bugs. International Journal of Information Technology & Computer Engineering (IJITC) ISSN: 2455-5290, 3(01):17–22, 2023. find
  55. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume…
  56. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJ4km2R5t7. ↗
  57. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019.
  58. Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik Narasimhan. COLLIE: Systematic construction of constrained text generation tasks, 2023.
  59. Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. — [arXiv:2206.10789](https://arxiv.org/abs/2206.10789) find
  60. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791–4800, Florence, Italy, July 2019. Association for Computational Linguisti… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/west-2023-generative-ai-paradox-create-may/