Elamur Research⌕
☰
home / papers / 2026 / Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation

Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation

Paper · 2026 · Jiaying Wu, Zihang Fu, Haonan Wang et al.

Summary

Community Notes on X lets volunteers attach context to misleading posts, but the process is slow. The authors studied 30.8K English health-related notes and found that corrections usually arrive after public attention has peaked. Most notes never get enough votes to be shown at all. To speed this up, they propose CROWDNOTES+, an LLM-based framework with two modes. In the first, humans supply evidence links and an LLM turns them into a note. In the second, the model plans queries, retrieves and selects evidence, and writes the note itself. A three-stage hierarchical check scores outputs for evidence relevance, correctness and overall helpfulness. The authors also release HEALTHNOTES, a benchmark of crowd-rated notes, and HEALTHJUDGE, a fine-tuned helpfulness evaluator. Across 15 LLMs, generated notes beat human-written ones on correctness, helpfulness and evidence quality. The analysis also shows that human voters often reward fluent writing over factual accuracy. The authors frame LLMs as assistants to human contributors, not as autonomous moderators.

Key points

  • Health notes take a median of 10.4 hours to appear after a post is flagged. A Helpful/Not Helpful verdict takes another 7.2 hours, for 17.6 hours in total. About 87.9% of notes stay stuck at "Needs More Ratings".
  • The dataset covers 30,791 notes on 25,484 posts, collected up to 4 August 2025. Health topics were filtered with Lingshu-32B, which agreed with GPT-4.1 on 99.2% of a 1,000-note check sample.
  • Spikes in flagged posts track real-world events such as vaccine news and policy debates. A spike is a day more than 2.5 SD above a 28-day rolling mean.
  • HEALTHNOTES contains about 1.2K notes with crowd-confirmed verdicts. It is released only under controlled, research-only access. Code is available at github.com/jiayingwu19/CrowdNotesPlus.
  • Limitations: the work covers only English and only health. Automated retrieval sometimes picks weakly grounded or lower-credibility sources. Upstream detection of misleading posts is not modeled.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore180 works · 34,304 citations🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🔬Institute of Computing TechnologyBeijing · China51 works · 3,411 citations🎓Yunnan UniversityKunming · China17 works · 40 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov and Min-Yen Kan. 2026. Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.233

@inproceedings{wu2026beyond,
  title = {{Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation}},
  author = {Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov and Min-Yen Kan},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.233},
  url = {https://doi.org/10.18653/v1/2026.acl-long.233}
}

References

45 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Funmi Adebesin, Hanlie Smuts, Tendani Mawela, George Maramba, Marie Hattingh, and 1 others. 2023. The role of social media in health misinformation and disinformation during the COVID-19 pandemic: bibliometric analysis. JMIR infodemiology, 3(1):e48620. find
  2. Jennifer Allen, Antonio A Arechar, Gordon Pennycook, and David G Rand. 2021. Scaling up fact-checking using the wisdom of crowds. Science advances, 7(36):eabf4393. find
  3. Anthropic. 2025. Introducing claude 4. https://www.anthropic.com/news/claude-4. ↗
  4. Yuwei Chuai, Moritz Pilarski, Thomas Renault, David Restrepo-Amariles, Aurore Troussel-Clément, Gabriele Lenzini, and Nicolas Pröllochs. 2024a. Community-based fact-checking reduces the spread of misleading posts on social media. arXiv preprint arXiv:2409.08781. — [arXiv:2409.08781](https://arxiv.org/abs/2409.08781) find
  5. Yuwei Chuai, Haoye Tian, Nicolas Pröllochs, and Gabriele Lenzini. 2024b. Did the roll-out of community notes reduce engagement with misinformation on X/Twitter? Proc. ACM Hum.-Comput. Interact., 8(CSCW2). find
  6. Soham De, Michiel A. Bakker, Jay Baxter, and Martin Saveski. 2025. Supernotes: Driving consensus in crowd-sourced fact-checking. In Proceedings of the ACM on Web Conference 2025, page 3751–3761. find
  7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  8. OpenAI. 2025b. Introducing OpenAI o3 and o4-mini. https://openai.com/index/ introducing-o3-and-o4-mini/. ↗
  9. Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023. Fact-checking complex claims with program-guided reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6981–700…
  10. Jan Pfänder and Sacha Altay. 2025. Spotting false news and doubting true news: a systematic review and meta-analysis of news judgements. Nature human behaviour, pages 1–12. find
  11. Max Glockner, Yufang Hou, Preslav Nakov, and Iryna Gurevych. 2024. Missci: Reconstructing fallacies in misrepresented science. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4372– 4405, Bangkok, Thailand. find
  12. Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2018. A stylometric inquiry into hyperpartisan and fake news. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 231–240, Melbourne, Australia.
  13. William Godel, Zeve Sanderson, Kevin Aslett, Jonathan Nagler, Richard Bonneau, Nathaniel Persily, and Joshua A Tucker. 2021. Moderating with the mob: Evaluating the efficacy of real-time crowdsourced fact-checking. Journal of Online Trust and Safety, 1(1). find
  14. Thomas Renault, David Restrepo Amariles, and Aurore Troussel. 2024. Collaboratively adding context to social media posts reduces the sharing of false news. arXiv preprint arXiv:2404.02803. — [arXiv:2404.02803](https://arxiv.org/abs/2404.02803) find
  15. Google. 2025. Gemini 2.5 pro. https://deepmind.google/technologies/gemini/pro/. ↗
  16. Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2024. Bad actor, good advisor: Exploring the role of large language models in fake news detection. In Proceedings of the AAAI conference on artificial intelligence, pages 22105–22113. find
  17. Md Saiful Islam, Tonmoy Sarkar, Sazzad Hossain Khan, Abu-Hena Mostofa Kamal, SM Murshid Hasan, Alamgir Kabir, Dalia Yeasmin, Mohammad Ariful Islam, Kamal Ibne Amin Chowdhury, Kazi Selim Anwar, and 1 others. 2020. COVID-19–related infodemic and its impact on public health: A global social media analy… find
  18. Haiwen Li, Soham De, Manon Revel, Andreas Haupt, Brad Miller, Keith Coleman, Jay Baxter, Martin Saveski, and Michiel A Bakker. 2025. Scaling human judgment in community notes with LLMs. arXiv preprint arXiv:2506.24118. — [arXiv:2506.24118](https://arxiv.org/abs/2506.24118) find
  19. Cameron Martel, Jennifer Allen, Gordon Pennycook, and David G Rand. 2024. Crowds can effectively identify misinformation at scale. Perspectives on Psychological Science, 19(2):477–488. find
  20. Mistral AI Team. 2024. Un ministral, des ministraux. https://mistral.ai/news/ministraux. ↗
  21. OpenAI. 2025a. Introducing GPT-4.1 in the API. https://openai.com/index/gpt-4-1/. ↗
  22. Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2024. ARES: An automated evaluation framework for retrieval-augmented generation systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo… find
  23. Rodrygo L. T. Santos, Craig Macdonald, and Iadh Ounis. 2015. Search result diversification. Found. Trends Inf. Retr., 9(1):1–90. find
  24. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, and 1 others. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. find
  25. Maryam Shahbazi and Deborah Bunker. 2024. Social media trust: Fighting misinformation in the time of crisis. International Journal of Information Management, 77:102780. find
  26. Gautam Kishore Shahi, Anne Dirkson, and Tim A Ma-jchrzak. 2021. An exploratory study of COVID-19 misinformation on Twitter. Online social networks and media, 22:100104. find
  27. Jane B Singer. 2023. Closing the barn door? Fact-checkers as retroactive gatekeepers of the COVID-19 “infodemic”. Journalism & Mass Communication Quarterly, 100(2):332–353. find
  28. Sahajpreet Singh, Jiaying Wu, Svetlana Churina, and Kokil Jaidka. 2025. On the limitations of LLM-synthesized social media misinformation moderation. In Proceedings of the ICLR 2025 Workshop ICBINB. find
  29. Isaac Slaughter, Axel Peytavin, Johan Ugander, and Martin Saveski. 2025. Community notes reduce engagement with and diffusion of false information online. Proceedings of the National Academy of Sciences, 122(38):e2503413122. find
  30. Herun Wan, Jiaying Wu, Minnan Luo, Zhi Zeng, and Zhixiong Su. 2025. Truth over tricks: Measuring and mitigating shortcut learning in misinformation detection. arXiv preprint arXiv:2506.02350. — [arXiv:2506.02350](https://arxiv.org/abs/2506.02350) find
  31. Yuyan Wang, Cheenar Banerjee, Samer Chucri, Fabio Soldo, Sriraj Badam, Ed H. Chi, and Minmin Chen. 2025. Beyond item dissimilarities: Diversifying by intent in recommender systems. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, page 2672–2681. find
  32. Stefan Wojcik, Sophie Hilgard, Nick Judd, Delia Mocanu, Stephen Ragain, MB Hunzaker, Keith Coleman, and Jay Baxter. 2022. Birdwatch: Crowd wisdom and bridging algorithms can inform understanding and reduce the spread of misinformation. arXiv preprint arXiv:2210.15723. — [arXiv:2210.15723](https://arxiv.org/abs/2210.15723) find
  33. Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, and Xue Liu. 2024a. Result Diversification in Search and Recommendation: A Survey . IEEE Transactions on Knowledge & Data Engineering, 36(10):5354–5373. find
  34. Jiaying Wu, Jiafeng Guo, and Bryan Hooi. 2024b. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 3367–3378. find
  35. Jiaying Wu and Bryan Hooi. 2023. Decor: Degree-corrected social graph refinement for fake news detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 2582–2593. find
  36. Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, and 1 others. 2025. Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning. arXiv preprint arXiv:2506.07044. — [arXiv:2506.07044](https://arxiv.org/abs/2506.07044) find
  37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  38. Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, and Zhaofeng Liu. 2025. Evaluation of retrieval-augmented generation: A survey. In Big Data, pages 102–120. find
  39. Hengran Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. 2024. Are large language models good at utility judgments? In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1941–1951. find
  40. Xuan Zhang and Wei Gao. 2023. Towards LLM-based fact verification on news claims with a hierarchical step-by-step prompting method. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Compu… find
  41. Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, and Kai Shu. 2021. Mining dual emotion for fake news detection. In Proceedings of the Web Conference 2021, page 3465–3476. find
  42. Xinyi Zhou, Ashish Sharma, Amy X Zhang, and Tim Althoff. 2024. Correcting misinformation on social media with a large language model. arXiv preprint arXiv:2403.11169. — [arXiv:2403.11169](https://arxiv.org/abs/2403.11169) find
  43. Jiaying Wu, Shen Li, Ailin Deng, Miao Xiong, and Bryan Hooi. 2023. Prompt-and-align: Prompt-based social alignment for few-shot fake news detection. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, page 2726–2736. find
  44. Amelie Wuehrl, Dustin Wright, Roman Klinger, and Isabelle Augenstein. 2024. Understanding fine-grained distortions in reports of scientific findings. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6175–6191, Bangkok, Thailand. xAI. 2025. Grok 4. https://x.ai/news/grok-… ↗
  45. Rui Xing, Preslav Nakov, Timothy Baldwin, and Jey Han Lau. 2026. COMMUNITYNOTES: A dataset for exploring the helpfulness of fact-checking explanations. In Findings of the Association for Computational Linguistics: EACL 2026, Rabat, Morocco.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-beyond-crowd-llm-augmented-community/