Elamur Research
☰
home / papers / 2023 / Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts

Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2305.13300

Summary

Tool- and retrieval-augmented LLMs often receive external evidence that contradicts what they learned during pre-training. This paper asks how such models behave when retrieved text conflicts with their internal knowledge, which the authors call counter-memory. Earlier studies built conflicts through word-level edits such as entity substitution or negation. Those edits produce incoherent passages, and models may simply ignore them.

The authors instead propose a five-step framework:

  • They elicit each model's closed-book answer together with its supporting explanation.
  • They derive a counter-answer and have ChatGPT write a coherent passage that supports it.
  • They keep only high-quality pairs, filtered by NLI-based entailment checks and answer-consistency checks.
  • They then test models zero-shot, giving them either counter-memory alone or both kinds of evidence together.

The experiments use POPQA and STRATEGYQA with eight LLMs. Two seemingly opposite behaviours emerge. When counter-memory is the only evidence, models readily accept it, which means malicious tools could mislead them. When evidence that agrees with their memory is also present, models show strong confirmation bias. These findings matter for building safe tool-augmented systems and generative search engines.

Key points

  • Datasets: POPQA has about 14K entity-centric questions built from Wikidata triples, with popularity measured by Wikipedia page views. STRATEGYQA contains True/False questions that require multi-step reasoning.
  • Models tested: ChatGPT, GPT-4, PaLM2, Qwen-7B, Llama2-7B, Llama2-70B, Vicuna-7B and Vicuna-33B. In closed-book mode, GPT-4 answered 50.8% of POPQA and 77.3% of STRATEGYQA correctly.
  • Contrast with earlier work: The finding that LLMs accept coherent counter-memory contradicts prior conclusions of model stubbornness, such as Longpre et al., 2021.
  • Prompt selection: Ten instruction variants were compared for stability before the final prompts were chosen.
  • Resources: Code and data are released at github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

158 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 153 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.13300 · arXiv 2305.13300

@misc{xie2023adaptive,
  title = {{Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts}},
  author = {Xie, Jian and Zhang, Kai and Chen, Jiangjie and Lou, Renze and Su, Yu},
  year = {2023},
  doi = {10.48550/arxiv.2305.13300},
  eprint = {2305.13300},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.13300}
}

References

72 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alibaba. Qwen, 2023. URL https://github.com/QwenLM/Qwen-7B/blob/main/tech\_memo.md. ↗
  2. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023. find our card
  3. AutoGPT. Autogpt, 2023. URL https://github.com/Significant-Gravitas/AutoGPT. ↗
  4. Parishad BehnamGhader, Santiago Miret, and Siva Reddy. Can retriever-augmented language models reason? the blame game between the retriever and the language model. arXiv preprint arXiv:2212.09146, 2022. URL https://arxiv.org/abs/2212.09146. ↗
  5. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In Proceedings of ICML, 2022. URL https://proceedi… find
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Proceedings of NeurIPS, 2020. URL https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8a… ↗
  7. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In Proceedings of USENIX Security Symposium, 2021. URL https… find our card
  8. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In Proceedings of ICLR, 2023. URL https://openreview.net/forum?id=TatRHT\_1cK. ↗
  9. Hung-Ting Chen, Michael Zhang, and Eunsol Choi. Rich knowledge sources bring complex knowledge conflicts: Recalibrating models to reflect conflicting evidence. In Proceedings of EMNLP, pp. 2292–2307, 2022. URL https://aclanthology.org/2022.emnlp-main.146. ↗
  10. Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li, and Yanghua Xiao. Say what you mean! large language models speak too positively about negative commonsense knowledge. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9890–9… find
  11. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022. URL https: //arxiv.org/abs/2204.02311. find
  12. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of ACL, 2022. URL https://aclanthology.org/2022. acl-long.581. ↗
  13. Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of EMNLP, 2021. URL https://aclanthology.org/2021.emnlp-main.522. ↗
  14. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of ACL, 2021. URL https://aclanthology.org/2021.tacl-1.60/. ↗
  15. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627, 2023. find our card
  16. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of ACL, 2021. URL https://aclanthology.org/2021.tacl-1.21/. ↗
  17. Reto Gubelmann and Siegfried Handschuh. Context matters: A pragmatic study of plms’ negation understanding. In Proceedings of ACL, 2022. URL https://aclanthology.org/2022.acl-long.315/. ↗
  18. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In Proceedings of ICML, pp. 3929–3938, 2020. URL https://dl.acm. org/doi/abs/10.5555/3524938.3525306. ↗
  19. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. In Proceedings of ICLR, 2021. URL https://openreview.net/forum?id=XPZIaotutsD. ↗ our card
  20. Gautier Izacard and Édouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of EACL, 2021. URL https://aclanthology.org/2021. eacl-main.74. ↗ our card
  21. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 2023. URL https://dl.acm.org/doi/10.1145/3571730. ↗ our card
  22. Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of ACL, 2020. URL https://aclanthology.org/2020.tacl-1.28. ↗ our card
  23. Nora Kassner, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. Beliefbank: Adding memory to a pre-trained language model for a systematic notion of belief. In Proceedings of EMNLP, 2021. URL https://aclanthology.org/2021.emnlp-main.697. ↗
  24. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. In Proceedings of ICLR, 2020. URL https://openreview.net/forum?id=HklBjCEKvH. ↗
  25. Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Tomáš Kočiský, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. Mind the gap: Assessing temporal generalization in neural language m… find
  26. Xiang Lorraine Li, Adhiguna Kuncoro, Jordan Hoffmann, Cyprien de Masson d’Autume, Phil Blunsom, and Aida Nematzadeh. A systematic investigation of commonsense knowledge in large language models. In Proceedings of EMNLP, 2022. URL https://aclanthology.org/2022. emnlp-main.812/. ↗
  27. Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models. In Proceedings of EMNLP, 2020. URL https://aclanthology.org/2020.emnlp-main.557. ↗ our card
  28. Adam Liska, Tomas Kocisky, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, D’Autume Cyprien De Masson, Tim Scholtes, Manzil Zaheer, Susannah Young, et al. Streamingqa: A benchmark for adaptation to new knowledge over time in question answering models. In Proceedings of ICML, 2022. URL… find
  29. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In Proceedings of EMNLP, 2021. URL https://aclanthology.org/2021.emnlp-main.565. ↗ our card
  30. Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842, 2023. URL https://arxiv.org/abs/2304.09842. ↗ our card
  31. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of ACL, 2022. URL https://aclanthology.org/2022.acl-long.556. ↗ our card
  32. Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam, and Noah A Smith. Time waits for no one! analysis and challenges of temporal misalignment. In Proceedings of NAACL, 2022. URL https://aclanthology.org/2022.naacl-main.435/. ↗
  33. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511, 2022. URL https://arxiv.org/abs/2212.10511. ↗
  34. Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Proceedings of NeurIPS, 2022. URL https://openreview.net/forum?id=-h6WAS6eE4. ↗
  35. Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In Proceedings of ICLR, 2023. URL https://openreview.net/forum?id=MkbcAHIYgyS. ↗
  36. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. URL https: //arxiv.org/abs/2112.0933… find our card
  37. Raymond S Nickerson. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology, 2(2):175–220, 1998. URL https://journals.sagepub.com/doi/abs/10.1037/1089-2680.2.2.175?journalCode=rgpa. — [doi:10.1037/1089-2680.2.2.175?journalcode=rgpa](https://doi.org/10.1037/1089-2680.2.2.175?journalcode=rgpa) ↗
  38. Tong Niu and Mohit Bansal. Adversarial over-sensitivity and over-stability strategies for dialogue models. In Proceedings of CoNLL, 2018. URL https://aclanthology.org/K18-1047/. ↗
  39. OpenAI. Chatgpt, 2022. URL https://openai.com/blog/chatgpt. ↗
  40. OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774. ↗ our card
  41. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In Proceedings of NeurIPS, 2022. URL https://proceedings.neurips.cc/paper\_files/p… ↗
  42. Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. Contraqa: Question answering under contradicting contexts. arXiv preprint arXiv:2110.07803, 2021. URL https://arxiv.org/abs/2110.07803. — [arXiv:2110.07803](https://arxiv.org/abs/2110.07803) ↗
  43. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of EMNLP-IJCNLP, 2019. URL https://aclanthology.org/D19-1250. ↗ our card
  44. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. How context affects language models’ factual predictions. In Proceedings of AKBC, 2020. URL https://openreview.net/forum?id=025X0zPfn. ↗ our card
  45. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, et al. Tool learning with foundation models. arXiv preprint arXiv:2304.08354, 2023. URL https://arxiv.org/abs/2304.08354. ↗ our card
  46. Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad. In Proceedings of ACL, 2018. URL https://arxiv.org/abs/1806.03822. ↗ our card
  47. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. In-context retrieval-augmented language models. arXiv preprint arXiv:2302.00083, 2023. URL https://arxiv.org/abs/2302.00083. ↗
  48. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of EMNLP, 2019. URL https://arxiv.org/abs/1908.10084. ↗
  49. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of EMNLP, 2020. URL https://aclanthology.org/2020.emnlp-main.437/. ↗
  50. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. URL https://arxiv.org/abs/2302.04761. ↗ our card
  51. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. arXiv preprint arXiv:2302.00093, 2023a. URL https://arxiv.org/abs/2302.00093. ↗ our card
  52. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652, 2023b. URL https://arxiv.org/abs/2302.00083. ↗ our card
  53. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Findings of EMNLP, 2021. URL https://aclanthology. org/2021.findings-emnlp.320/. find our card
  54. Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Lee Boyd-Graber, and Lijuan Wang. Prompting GPT-3 to be reliable. In Proceedings of ICLR, 2023. URL https://openreview.net/forum?id=98p5x51L5af. ↗ our card
  55. Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. Recitation-augmented language models. In Proceedings of ICLR, 2023. URL https://openreview.net/forum?id=-cqvvvb-NkI. ↗
  56. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. URL https: //arxiv.org/abs/2302.13971. find
  57. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b. find
  58. Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, et al. Freshllms: Refreshing large language models with search engine augmentation. arXiv preprint arXiv:2310.03214, 2023. find our card
  59. Boshi Wang, Xiang Yue, and Huan Sun. Can chatgpt defend the truth? automatic dialectical evaluation elicits llms’ deficiencies in reasoning. arXiv preprint arXiv:2305.13160, 2023. — [arXiv:2305.13160](https://arxiv.org/abs/2305.13160) find
  60. Cunxiang Wang, Pai Liu, and Yue Zhang. Can generative pre-trained language models serve as knowledge bases for closed-book qa? In Proceedings of ACL-IJCNLP, 2021. URL https: //aclanthology.org/2021.acl-long.251/. find
  61. Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. In Proceedings of NAACL, 2022. URL https://aclanthology.org/2022. naacl-main.341. ↗ our card
  62. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In Proceedings of ICLR, 2023. URL https://arxiv.org/abs/2210.03629. ↗ our card
  63. Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. Generate rather than retrieve: Large language models are strong context generators. In Proceedings of ICLR, 2023. URL https://openreview.net/forum?id=fB0hRu9GZUS. ↗ our card
  64. Xiang Yue, Boshi Wang, Kai Zhang, Ziru Chen, Yu Su, and Huan Sun. Automatic evaluation of attribution by large language models. arXiv preprint arXiv:2305.06311, 2023. URL https: //arxiv.org/abs/2305.06311. find
  65. Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. GLM-130b: An open bilingual pre-trained model. In Proceedings of ICL… find our card
  66. Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual memorization in neural language models. arXiv preprint arXiv:2112.12938, 2021. URL https://arxiv.org/abs/2112.12938. — [arXiv:2112.12938](https://arxiv.org/abs/2112.12938) ↗
  67. Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, and Joseph E Gonzalez. The wisdom of hindsight makes language models better instruction followers. arXiv preprint arXiv:2302.05206, 2023. — [arXiv:2302.05206](https://arxiv.org/abs/2302.05206) find
  68. Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, et al. Retrieving multimodal information for augmented generation: A survey. arXiv preprint arXiv:2303.10868, 2023. URL https://arxiv.org/abs/2303.10868. ↗
  69. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023. find
  70. Zexuan Zhong, Tao Lei, and Danqi Chen. Training language models with memory augmentation. In Proceedings of EMNLP, 2022. URL https://aclanthology.org/2022.emnlp-main.382. ↗
  71. Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. Context-faithful prompting for large language models. arXiv preprint arXiv:2303.11315, 2023. URL https://arxiv.org/abs/2303.11315. ↗
  72. A PPENDIX Within this supplementary material, we elaborate on the following aspects: • Appendix A: Discussions • Appendix B: Experimental Setup Details • Appendix C: Prompts List find