Elamur Research⌕
☰
home / papers / 2023 / Can LLM-Generated Misinformation Be Detected?

Can LLM-Generated Misinformation Be Detected?

Paper · 2023 · Canyu Chen and Kai Shu

Summary

The paper asks whether misinformation written by large language models is more dangerous than misinformation written by people. It treats danger as detection difficulty: if a model-written text has the same meaning as a human-written fake but is harder to flag, it likely has a more deceptive style. The authors first propose a taxonomy that classifies LLM-generated misinformation along five axes. They then group realistic ways of producing it into three families: unintentional hallucination, arbitrary generation, and controllable generation. They check whether ChatGPT refuses each approach. Next, they compare how hard LLM-produced and human-written false content is to detect, using the same human annotators and zero-shot detectors (for example, GPT-4 with chain-of-thought prompting). The texts were produced by ChatGPT, Llama2, and Vicuna through paraphrasing, rewriting, and open-ended expansion. In general, the model-generated versions fooled both people and automated detectors more often. This matters for moderation and fact-checking, and the authors also outline countermeasures across the full LLM lifecycle.

Key points

  • The taxonomy has five dimensions: types (such as fake news, rumors, conspiracy theories, clickbait, and cherry-picking), domains (such as healthcare, politics, and finance), sources, intents (unintentional or intentional), and errors (such as Total Fabrication, Outdated Information, and False Context).
  • There are three generation families. Hallucination Generation (HG) is unintentional. Arbitrary Misinformation Generation (AMG) comes in totally and partially arbitrary forms. Controllable Misinformation Generation (CMG) covers Paraphrase, Rewriting, Open-ended Generation, and Information Manipulation.
  • Each method was tried 100 times on ChatGPT. AMG prompts were mostly refused, with attack success rates of about 5% and 9%. HG and most CMG methods got through almost every time (about 100%), and Information Manipulation succeeded about 87% of the time.
  • The generators were ChatGPT, Llama2-7b/13b/70b, and Vicuna-7b/13b/33b. Detection was evaluated in a zero-shot setting to mimic real-world use.
  • The central finding is that misinformation written by LLMs, with the same meaning as human-written misinformation, is harder for both humans and detectors to identify.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Illinois Institute of TechnologyChicago · United States18 works · 2,097 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksInformation Systems · 1,730 worksSociology and Political Science · 626 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Canyu Chen and Kai Shu. 2023. Can LLM-Generated Misinformation Be Detected?. arXiv:2309.13788.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.13788 · arXiv 2309.13788

@misc{chen2023can,
  title = {{Can LLM-Generated Misinformation Be Detected?}},
  author = {Canyu Chen and Kai Shu},
  year = {2023},
  doi = {10.48550/arxiv.2309.13788},
  eprint = {2309.13788},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.13788}
}

References

110 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ankit Aich, Souvik Bhattacharya, and Natalie Parde. Demystifying neural fake news via linguistic feature-based interpretation. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 6586–6599, Gyeongju, Republic of Korea, 2022. International Committee on Computational… find
  2. Markus Anderljung, Joslyn Barnhart, Anton Korinek, Jade Leung, Cullen O’Keefe, Jess Whittlestone, Shahar Avin, Miles Brundage, Justin Bullock, Duncan Cass-Beggs, Ben Chang, Tantum Collins, Tim Fist, Gillian Hadfield, Alan Hayes, Lewis Ho, Sara Hooker, Eric Horvitz, Noam Kolt, Jonas Schuett, Yonadav… — [arXiv:2307.03718](https://arxiv.org/abs/2307.03718) find
  3. Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco,… — Foundational Challenges in Assuring Alignment and Safety of Large Language Models
  4. Abolfazl Asudeh, Hosagrahar Visvesvaraya Jagadish, You Wu, and Cong Yu. On detecting cherry-picked trendlines. Proceedings of the VLDB Endowment, 13(6):939–952, 2020. find
  5. Navid Ayoobi, Sadat Shahriar, and Arjun Mukherjee. The looming threat of fake and llm-generated linkedin profiles: Challenges and opportunities for detection and prevention. In Proceedings of the 34th ACM Conference on Hypertext and Social Media, pp. 1–10, 2023. find
  6. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv: Ar…
  7. Clark Barrett, Brad Boyd, Ellie Burzstein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, Kathleen Fisher, Tatsunori Hashimoto, Dan Hendrycks, Somesh Jha, Daniel Kang, Florian Kerschbaum, Eric Mitchell, John Mitchell, Zulfikar Ramzan,… find
  8. Pranjal Bhardwaj, Krishna Yadav, Hind Alsharif, and Rania Anwar Aboalela. Gan-based unsupervised learning approach to generate and detect fake news. In International Conference on Cyber Security, Privacy and Networking, pp. 384–396. Springer, 2021. find
  9. Meghana Moorthy Bhat and Srinivasan Parthasarathy. How effectively can machines defend against machine-generated fake news? an empirical study. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pp. 48–53, Online, 2020. Association for Computational Linguistics. doi: 10.1… find
  10. Mars Gokturk Buchholz. Assessing the effectiveness of gpt-3 in detecting false political statements: A case study on the liar dataset. arXiv preprint arXiv: 2306.08190, 2023. — [arXiv:2306.08190](https://arxiv.org/abs/2306.08190) find
  11. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar… find
  12. Canyu Chen and Kai Shu. Combating misinformation in the age of llms: Opportunities and challenges. arXiv preprint arXiv: 2311.05656, 2023.
  13. Canyu Chen, Haoran Wang, Matthew A. Shapiro, Yunyu Xiao, Fei Wang, and Kai Shu. Combating health misinformation in social media: Characterization, detection, intervention, and open issues. ARXIV.ORG, 2022. doi: 10.48550/arXiv.2211.05289. — [doi:10.48550/arxiv.2211.05289](https://doi.org/10.48550/arxiv.2211.05289) find
  14. Yimin Chen, Niall J Conroy, and Victoria L Rubin. Misleading online content: recognizing clickbait as" false news". In Proceedings of the 2015 ACM on workshop on multimodal deception detection, pp. 15–19, 2015. find
  15. Ziwei Chen, Linmei Hu, Weixin Li, Yingxia Shao, and Liqiang Nie. Causal intervention and counterfactual reasoning for multi-modal fake news detection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 627– 638, Toronto, Canada, 20… find
  16. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/. ↗
  17. Limeng Cui and Dongwon Lee. Coaid: Covid-19 healthcare misinformation dataset. arXiv preprint arXiv: 2006.00885, 2020. find
  18. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langu…
  19. Li Du, Yequan Wang, Xingrun Xing, Yiqun Ya, Xiang Li, Xin Jiang, and Xuezhi Fang. Quantifying and attributing the hallucination of large language models via association analysis. arXiv preprint arXiv: 2309.05217, 2023. — [arXiv:2309.05217](https://arxiv.org/abs/2309.05217) find
  20. Yibing Du, Antoine Bosselut, and Christopher D. Manning. Synthetic disinformation attacks on automated fact verification systems. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twel… find
  21. AAAI Press, 2022. URL https://ojs.aaai.org/index.php/AAAI/article/view/21302. ↗
  22. Ziv Epstein, Mengying C Fang, Antonio A Arechar, and David G Rand. What label should be applied to content produced by generative ai?, 2023. URL osf.io/preprints/psyarxiv/v4mfz. find
  23. Yi Fung, Kung-Hsiang Huang, Preslav Nakov, and Heng Ji. The battlefront of combating misinformation and coping with media bias. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Lan… find
  24. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan… — Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
  25. Yuan Gao, Xiang Wang, Xiangnan He, Huamin Feng, and Yongdong Zhang. Rumor detection with self-supervised learning on texts and social graph. Frontiers Comput. Sci., 2022. doi: 10.48550/arXiv.2204.08838. — [doi:10.48550/arxiv.2204.08838](https://doi.org/10.48550/arxiv.2204.08838) find
  26. Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. Generative language models and automated influence operations: Emerging threats and potential mitigations. ARXIV.ORG, 2023. doi: 10.48550/arXiv.2301.04246. — [doi:10.48550/arxiv.2301.04246](https://doi.org/10.48550/arxiv.2301.04246) find
  27. Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206, 2022. doi: 10.1162/tacl_a_00454. URL https://aclanthology.org/2022.tacl-1.11. ↗
  28. Ahmed Abdeen Hamed. Improving detection of chatgpt-generated fake science using real publication text: Introducing xfakebibs a supervised learning network algorithm. 2023. find
  29. Hans W. A. Hanley and Zakir Durumeric. Machine-made media: Monitoring the mobilization of machine-generated articles on misinformation and mainstream news websites. arXiv preprint arXiv: 2305.09820, 2023. — [arXiv:2305.09820](https://arxiv.org/abs/2305.09820) find
  30. Peter Henderson, E. Mitchell, Christopher D. Manning, Dan Jurafsky, and Chelsea Finn. Self-destructing models: Increasing the costs of harmful dual uses of foundation models. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 2022. doi: 10.1145/3600211.3604690. find
  31. Lewis Ho, Joslyn Barnhart, Robert Trager, Yoshua Bengio, Miles Brundage, Allison Carnegie, Rumman Chowdhury, Allan Dafoe, Gillian Hadfield, Margaret Levi, and Duncan Snidal. International institutions for advanced ai. arXiv preprint arXiv: 2307.04699, 2023. — [arXiv:2307.04699](https://arxiv.org/abs/2307.04699) find
  32. Xuming Hu, Zhijiang Guo, Junzhe Chen, Lijie Wen, and Philip S. Yu. Mr2: A benchmark for multimodal retrieval-augmented rumor detection in social media. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, pp. 2901–2912, New Yo… — [doi:10.1145/3539618.3591896](https://doi.org/10.1145/3539618.3591896) find
  33. Kung-Hsiang Huang, Kathleen McKeown, Preslav Nakov, Yejin Choi, and Heng Ji. Faking fake news for real fake news detection: Propaganda-loaded training data generation. arXiv preprint arXiv: Arxiv-2203.05386, 2022. find
  34. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv: 2309.00614, 2023.
  35. Yiqiao Jin, Xiting Wang, Ruichao Yang, Yizhou Sun, Wei Wang, Hao Liao, and Xing Xie. Towards fine-grained reasoning for fake news detection. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 202… find
  36. AAAI Press, 2022. URL https://ojs.aaai.org/index.php/AAAI/article/view/20517. ↗
  37. Takeshi Kojima, S. Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Neural Information Processing Systems, 2022.
  38. Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Soheil Feizi, and Hima Lakkaraju. Certifying llm safety against adversarial prompting. arXiv preprint arXiv: 2309.02705, 2023.
  39. Srijan Kumar and Neil Shah. False information on web and social media: A survey. ArXiv preprint, abs/1804.08559, 2018. URL https://arxiv.org/abs/1804.08559. ↗
  40. Ksenia Lagutina, Nadezhda Lagutina, Elena Boychuk, Inna Vorontsova, Elena Shliakhtina, Olga Belyaeva, Ilya Paramonov, and P.G. Demidov. A survey on stylometric text features. In 2019 25th Conference of Open Innovations Association (FRUCT), pp. 184–195, 2019. doi: 10.23919/FRUCT48121.2019.8981504. — [doi:10.23919/fruct48121.2019.8981504](https://doi.org/10.23919/fruct48121.2019.8981504) find
  41. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning. arXiv preprint arXiv: Arxiv-2304.05613, 2023.
  42. Thai Le, Suhang Wang, and Dongwon Lee. MALCOM: generating malicious comments to attack neural fake news detection models. In Claudia Plant, Haixun Wang, Alfredo Cuzzocrea, Carlo Zaniolo, and Xindong Wu (eds.), 20th IEEE International Conference on Data Mining, ICDM 2020, Sorrento, Italy, November 17… find
  43. IEEE, 2020. doi: 10.1109/ICDM50108.2020.00037. URL https://doi.org/10.1109/ICDM50108.2020.00037. — [doi:10.1109/icdm50108.2020.00037](https://doi.org/10.1109/icdm50108.2020.00037) ↗
  44. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXiv preprint arXiv: 2305.11747, 2023a.
  45. Quanzhi Li, Qiong Zhang, Luo Si, and Yingchi Liu. Rumor detection on social media: Datasets, methods and opportunities. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pp. 66–75, Hong Kong, China, 2019. Associatio… find
  46. Xinyi Li, Yongfeng Zhang, and Edward C. Malthouse. A preliminary study of chatgpt on news recommendation: Personalization, provider fairness, fake news. arXiv preprint arXiv: 2306.10702, 2023b. — [arXiv:2306.10702](https://arxiv.org/abs/2306.10702) find
  47. Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. Evaluating the logical reasoning ability of chatgpt and gpt-4. arXiv preprint arXiv: 2304.03439, 2023.
  48. Benjamin A Lyons, Jacob M Montgomery, Andrew M Guess, Brendan Nyhan, and Jason Reifler. Overconfidence in news judgments is associated with false news susceptibility. Proceedings of the National Academy of Sciences, 118(23):e2019527118, 2021. find
  49. Abdurahman Maarouf, Dominik Bär, Dominique Geissler, and Stefan Feuerriegel. Hqp: A human-annotated dataset for detecting online propaganda. arXiv preprint arXiv: 2304.14931, 2023. find
  50. Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. A survey on computational propaganda detection. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 4… — [doi:10.24963/ijcai.2020/672](https://doi.org/10.24963/ijcai.2020/672) find
  51. Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. Augmented language models: a survey. arXiv preprint arXiv: 2302.07842, 2023.
  52. Akhtar Mubashara, Schlichtkrull Michael, Guo Zhijiang, Cocarascu Oana, Simperl Elena, and Vlachos Andreas. Multimodal automated fact-checking: A survey. arXiv preprint arXiv: 2305.13507, 2023. find
  53. Tempestt Neal, Kalaivani Sundararajan, Aneez Fatima, Yiming Yan, Yingfei Xiang, and Damon Woodard. Surveying stylometry techniques and applications. ACM Comput. Surv., 50(6), 2017. ISSN 0360-0300. doi: 10.1145/3132039. URL https://doi.org/10.1145/3132039. — [doi:10.1145/3132039](https://doi.org/10.1145/3132039) ↗
  54. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training l… — Training language models to follow instructions with human feedback
  55. Artidoro Pagnoni, Martin Graciarena, and Yulia Tsvetkov. Threat scenarios and best practices to detect neural fake news. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 1233–1249, Gyeongju, Republic of Korea, 2022. International Committee on Computational Lingui… find
  56. Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang. On the risk of misinformation pollution with large language models. arXiv preprint arXiv: 2305.13661, 2023.
  57. Ajeet Ram Pathak, Aditee Mahajan, Keshav Singh, Aishwarya Patil, and Anusha Nair. Analysis of techniques for rumor detection in social media. Procedia Computer Science, 167:2286–2296, 2020. find
  58. Kellin Pelrine, Meilina Reksoprodjo, Caleb Gupta, Joel Christoph, and Reihaneh Rabbany. Towards reliable misinformation mitigation: Generalization, uncertainty, and gpt-4. arXiv preprint arXiv: 2305.14928, 2023. find
  59. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3419–3448, Abu Dhabi, U…
  60. Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 3391–3401, Santa Fe, New Mexico, USA, 2018. Association for Computational Linguistics. URL https://ac…
  61. Mary Phuong, Matthew Aitchison, Elliot Catt, Sarah Cogan, Alexandre Kaskasoli, Victoria Krakovna, David Lindner, Matthew Rahtz, Yannis Assael, Sarah Hodkinson, Heidi Howard, Tom Lieberum, Ramana Kumar, Maria Abi Raad, Albert Webson, Lewis Ho, Sharon Lin, Sebastian Farquhar, Marcus Hutter, Gregoire D… find
  62. Huachuan Qiu, Shuai Zhang, Anqi Li, Hongliang He, and Zhenzhong Lan. Latent jailbreak: A benchmark for evaluating text safety and output robustness of large language models. arXiv preprint arXiv: 2307.08487, 2023. — [arXiv:2307.08487](https://arxiv.org/abs/2307.08487) find
  63. Maribeth Rauh, John Mellor, Jonathan Uesato, Po-Sen Huang, Johannes Welbl, Laura Weidinger, Sumanth Dathathri, Amelia Glaese, Geoffrey Irving, Iason Gabriel, William Isaac, and Lisa Anne Hendricks. Characteristics of harmful text: Towards rigorous benchmarking of language models. In NeurIPS, 2022. U… find
  64. Jonas Schuett, Noemi Dreksler, Markus Anderljung, David McCaffary, Lennart Heim, Emma Bluemke, and Ben Garfinkel. Towards best practices in agi safety and governance: A survey of expert opinion. arXiv preprint arXiv: 2305.07153, 2023. — [arXiv:2305.07153](https://arxiv.org/abs/2305.07153) find
  65. Tal Schuster, Roei Schuster, Darsh J. Shah, and Regina Barzilay. The limitations of stylometry for detecting machine-generated fake news. Computational Linguistics, 46(2):499–510, 2020. doi: 10.1162/coli_a_00380. URL https://aclanthology.org/2020.cl-2.8. ↗
  66. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv: 2308.03825, 2023.
  67. Qiang Sheng, Juan Cao, Xueyao Zhang, Rundong Li, Danding Wang, and Yongchun Zhu. Zoom out and observe: News environment perception for fake news detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4543–4556, Dublin, Irela… find
  68. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. Fake news detection on social media: A data mining perspective. SIGKDD Explor., 19(1):22–36, 2017. doi: 10.1145/3137597.3137600. URL https://doi.org/10.1145/3137597.3137600. — [doi:10.1145/3137597.3137600](https://doi.org/10.1145/3137597.3137600) ↗
  69. Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. defend: Explainable fake news detection. In Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis (eds.), Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,… find
  70. ACM, 2019. doi: 10.1145/3292500.3330935. URL https://doi.org/10.1145/3292500.3330935. — [doi:10.1145/3292500.3330935](https://doi.org/10.1145/3292500.3330935) ↗
  71. Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data, 8(3):171–188, 2020. find
  72. Kai Shu, Yichuan Li, Kaize Ding, and Huan Liu. Fact-enhanced synthetic news generation. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in A… find
  73. AAAI Press, 2021a. URL https://ojs.aaai.org/index.php/AAAI/article/view/17629. ↗
  74. Kai Shu, Guoqing Zheng, Yichuan Li, Subhabrata Mukherjee, Ahmed Hassan Awadallah, Scott Ruston, and Huan Liu. Early detection of fake news with multi-source weak social supervision. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, Septemb… find
  75. Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, and Apostol Vassilev. Evaluating… — [arXiv:2306.05949](https://arxiv.org/abs/2306.05949) find
  76. Giovanni Spitale, Nikola Biller-Andorno, and Federico Germani. Ai model gpt-3 (dis)informs us better than humans. Science Advances, 9(26):eadh1850, 2023. doi: 10.1126/sciadv.adh1850. URL https://www.science.org/doi/abs/10.1126/sciadv.adh1850. — [doi:10.1126/sciadv.adh1850](https://doi.org/10.1126/sciadv.adh1850) ↗
  77. Harald Stiff and Fredrik Johansson. Detecting computer-generated disinformation. International Journal of Data Science and Analytics, 13(4):363–383, 2022. find
  78. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bhavya Kailkhura, Caiming Xiong, Chao Zhang, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang… — TrustLLM: Trustworthiness in Large Language Models
  79. Reuben Tan, Bryan Plummer, and Kate Saenko. Detecting cross-modal inconsistency to defend against neural fake news. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2081–2106, Online, 2020. Association for Computational Linguistics. doi: 10.18653… find
  80. Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen, and Guilin Qi. Evaluation of chatgpt as a question answering system for answering complex questions. arXiv preprint arXiv: Arxiv-2303.07992, 2023. find
  81. Max Tegmark and Steve Omohundro. Provably safe systems: the only path to controllable agi. arXiv preprint arXiv: 2309.01933, 2023. — [arXiv:2309.01933](https://arxiv.org/abs/2309.01933) find
  82. SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313, 2024.
  83. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  84. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579–2605, 2008. URL http://jmlr.org/papers/v9/vandermaaten08a. html. ↗
  85. Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert… — Introducing v0.5 of the AI Safety Benchmark from MLCommons
  86. Juraj Vladika and F. Matthes. Scientific fact-checking: A survey of resources and approaches. Annual Meeting of the Association for Computational Linguistics, 2023. doi: 10.48550/arXiv.2305.16859. find
  87. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. Decodingtrust: A comprehensive assessment of trus… — DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
  88. Haoran Wang, Yingtong Dou, Canyu Chen, Lichao Sun, Philip S. Yu, and Kai Shu. Attacking fake news detectors via manipulating news social engagement. The Web Conference, 2023b. doi: 10.1145/3543507.3583868. — [doi:10.1145/3543507.3583868](https://doi.org/10.1145/3543507.3583868) find
  89. Jia Wang, Min Gao, Yinqiu Huang, Kai Shu, and Hualing Yi. Find: Fine-grained discrepancy-based fake news detection enhanced by event abstract generation. Computer Speech & Language, 78: 101461, 2023c. find
  90. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? arXiv preprint arXiv: 2307.02483, 2023.
  91. Liang Wu, Fred Morstatter, Kathleen M Carley, and Huan Liu. Misinformation in social media: definition, manipulation, and detection. ACM SIGKDD explorations newsletter, 21(2):80–90, 2019. find
  92. Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv: 2306.13063, 2023. find
  93. Jing Yao, Xiaoyuan Yi, Xiting Wang, Jindong Wang, and Xing Xie. From instructions to intrinsic human values - a survey of alignment goals for big models. arXiv preprint arXiv: 2308.12014, 2023. — [arXiv:2308.12014](https://arxiv.org/abs/2308.12014) find
  94. Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, and Meng Jiang. A survey of knowledge-enhanced text generation. arXiv preprint arXiv: 2010.04389, 2020. find
  95. Zhenrui Yue, Huimin Zeng, Yang Zhang, Lanyu Shang, and Dong Wang. Metaadapt: Domain adaptive few-shot misinformation detection via meta learning. arXiv preprint arXiv: 2305.12692, 2023. find
  96. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Process…
  97. Haopeng Zhang, Xiao Liu, and Jiawei Zhang. Extractive summarization via chatgpt for faithful summary generation. arXiv preprint arXiv: 2304.04193, 2023a. find
  98. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. How language model hallucinations can snowball. arXiv preprint arXiv: 2305.13534, 2023b.
  99. Tianhua Zhang, Hongyin Luo, Yung-Sung Chuang, Wei Fang, Luc Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox, Helen Meng, and James Glass. Interpretable unified language checking. arXiv preprint arXiv: Arxiv-2304.03728, 2023c.
  100. Xichen Zhang and Ali A Ghorbani. An overview of online fake news: Characterization, detection, and discussion. Information Processing & Management, 57(2):102025, 2020. find
  101. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren’s song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv: 2309.01219, 2023d…
  102. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. Safetybench: Evaluating the safety of large language models with multiple choice questions. ArXiv preprint, abs/2309.07045, 2023e. URL https://arxiv.org/abs/2309.07045. ↗
  103. Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. Defending large language models against jailbreaking attacks through goal prioritization. ArXiv preprint, abs/2311.09096, 2023f. URL https://arxiv.org/abs/2311.09096. ↗
  104. Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473, 2024.
  105. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large l… — A Survey of Large Language Models
  106. Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–20, 2023. find
  107. Weikang Zhou, Xiao Wang, Limao Xiong, Han Xia, Yingshuang Gu, Mingxu Chai, Fukang Zhu, Caishuang Huang, Shihan Dou, Zhiheng Xi, Rui Zheng, Songyang Gao, Yicheng Zou, Hang Yan, Yifan Le, Ruohui Wang, Lijun Li, Jing Shao, Tao Gui, Qi Zhang, and Xuanjing Huang. Easyjailbreak: A unified framework for ja… — [arXiv:2403.12171](https://arxiv.org/abs/2403.12171) find
  108. Xinyi Zhou and Reza Zafarani. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53(5):1–40, 2020. find
  109. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv: 2307.15043, 2023.
  110. Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. Detection and resolution of rumours in social media: A survey. ACM Comput. Surv., 51(2):32:1–32:36, 2018. doi: 10.1145/3161603. URL https://doi.org/10.1145/3161603. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2023-llm-generated-misinformation-detected/