Elamur Research⌕
☰
home / papers / 2026 / Improving Public Health Safety in Low-Resource Languages Using a Human-Verified Health Misinformation Corpus and Large Language Models

Improving Public Health Safety in Low-Resource Languages Using a Human-Verified Health Misinformation Corpus and Large Language Models

Paper · 2026 · Sujal Maharjan, Astha Shrestha, Laxmi Thapa et al.

Summary

Automatic detection of health misinformation is barely studied for low-resource languages such as Nepali, mainly because trustworthy evaluation data are missing. The authors build Nep-Health-Misinfo, a human-verified Nepali corpus in Devanagari script. It is derived from four English benchmarks: CoAID, COVID-19, Monkeypox-V1 and Monkeypox-V2. The sources were cleaned and deduplicated, machine-translated, and then post-edited by native Nepali annotators following a Machine Translation Post-Editing (MTPE) protocol.

Evaluating NMT systems on this material reveals a "translation asymmetry": truthful health statements are translated much more faithfully than deceptive ones. The authors then benchmark seven open-weight LLMs under 10B parameters in zero-shot and few-shot settings. Few-shot exemplars are selected either at random or through K-means centroids. Few-shot prompting brings clear gains, with Qwen2.5-7B-Instruct performing best. Random exemplar selection tends to reach higher peaks than centroid-based selection.

An error analysis exposes shared model weaknesses. Models are swayed by authoritative-sounding sources, and they misclassify factual LGBTQ+ health news. The work provides a needed safety benchmark for a South Asian language.

Key points

  • Corpus: Nep-Health-Misinfo contains 4,924 unique instances labeled real/fake. It was produced via MTPE with two native Nepali annotators aware of medical context. User handles were anonymized as @MENTION.
  • Translation asymmetry: state-of-the-art NMT reaches BLEU 43.21 on factual health text. On deceptive narratives, BLEU falls to 19.11 and TER reaches 62.42.
  • Models tested: Qwen2.5-7B-Instruct, Gemma-3-4B-IT, Ministral-8B-Instruct, Falcon3-7B-Instruct, Phi-4-Mini-Instruct, Mistral-7B-Instruct-v0.3 and Falcon-H1-3B-Instruct.
  • Best result: Qwen2.5-7B-Instruct reaches a top Macro F1 of 0.8488 with few-shot prompting, up from 0.7188 zero-shot on the same data.
  • Failure analysis: 14 samples were missed by every model. The failure modes are Authority Bias, Pseudo-Science and Safety-Alignment Bias.
  • Limitations: the corpus has Western-origin source bias and is text-only. Code and data are available on GitHub (SUJAL390/Nep-Health-Misinfo-CHIPSAL-LREC).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations🎓Virginia TechBlacksburg · United States36 works · 43,036 citations🎓Tribhuvan UniversityKathmandu · Nepal5 works · 15 citations🎓IIMS CollegeKathmandu · Nepal5 works · 5 citations🎓O. P. Jindal Global UniversitySonīpat · India3 works · 22 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 worksLow-resource languages · 267 works

Cite

Cite as Sujal Maharjan, Astha Shrestha, Laxmi Thapa, Sweta Poudel, Shuvam Shiwakoti, Rabin Thapa, Kritesh Rauniyar and Surendrabikram Thapa. 2026. Improving Public Health Safety in Low-Resource Languages Using a Human-Verified Health Misinformation Corpus and Large Language Models. Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5obqozzwbr2d

@inproceedings{maharjan2026improving,
  title = {{Improving Public Health Safety in Low-Resource Languages Using a Human-Verified Health Misinformation Corpus and Large Language Models}},
  author = {Sujal Maharjan, Astha Shrestha, Laxmi Thapa, Sweta Poudel, Shuvam Shiwakoti, Rabin Thapa, Kritesh Rauniyar and Surendrabikram Thapa},
  booktitle = {Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026)},
  year = {2026},
  doi = {10.63317/5obqozzwbr2d},
  url = {https://doi.org/10.63317/5obqozzwbr2d}
}

References

28 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. U Acharya. 2023. Promoting digital literacy with scarce resources. Managing the Misinformation Effect—The State of Fact-Checking in Asia. International Federation of Journalists. find
  2. Daniel Alexander Alber, Zihao Yang, Anton Alyakin, Eunice Yang, Sumedha Rai, Aly A Valliani, Jeff Zhang, Gabriel R Rosenbaum, Ashley K Amend-Thomas, David B Kurland, et al. 2025. Medical large language models are vulnerable to data-poisoning attacks. Nature Medicine, 31(2):618–626. find
  3. Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. 2025. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775.
  4. Sirisha Bojjireddy, Soon Ae Chun, and James Geller. 2021. Machine learning approach to detect fake news, misinformation in covid-19 pandemic. In Proceedings of the 22nd Annual International Conference on Digital Government Research, pages 575–578. find
  5. Matteo Cinelli, Walter Quattrociocchi, Alessandro Galeazzi, Carlo Michele Valensise, Emanuele Brugnoli, Ana Lucia Schmidt, Paola Zola, Fabiana Zollo, and Antonio Scala. 2020. The covid-19 social media infodemic. Scientific reports, 10(1):16598. find
  6. Stephen Crone. 2022. Monkeypox misinformation: Twitter dataset. find
  7. Limeng Cui and Dongwon Lee. 2020. Coaid: Covid-19 healthcare misinformation dataset. arXiv preprint arXiv:2006.00885. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  9. Gunther Eysenbach et al. 2008. Medicine 2.0: social networking, collaboration, participation, apo-mediation, and openness. Journal of medical Internet research, 10(3):e1030. find
  10. Jay Gala, Pranjal A. Chitale, Raghavan AK, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar, Janki Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, and Anoop Kunchukuttan. 2023. Indictrans2: Towards high-quality and accessible machine translatio…
  11. Plan Ghimire and Pranjal Shrestha. 2025. Bilingual fake-news detection in low-resource media: A transformer-based framework for nepali– english content. Journal of Innovations in Engineering Education, 8(1):133–138. find
  12. Saroj Giri, Shiva Ram Dam, Rajesh Kamar, and Suraj Basant Tulachan. 2024. Fake news detection using convolutional neural networks. Technical Journal, 4(1):64–68. find
  13. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):…
  14. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the nlp world. arXiv preprint arXiv:2004.09095.
  15. Anusha Kamath, Kanishk Singla, Rakesh Paul, Raviraj Bhuminand Joshi, Utkarsh Vaidya, Sanjay Singh Chauhan, and Niranjan Wartikar. 2025. Benchmarking hindi llms: A new suite of datasets and a comparative analysis. In Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standa… find
  16. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  18. Ayush Maheshwari, Kaushal Sharma, Vivek Patel, and Aditya Maheshwari. 2025. Indicparam: Benchmark to evaluate llms on low-resource indic languages. arXiv preprint arXiv:2512.00333. — [arXiv:2512.00333](https://arxiv.org/abs/2512.00333) find
  19. Miriam J Metzger. 2007. Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Journal of the American society for information science and technology, 58(13):2078– 2091. find
  20. Parth Patwa, Shivam Sharma, Srinivas Pykl, Vineeth Guptha, Gitanjali Kumari, Md Shad Akhtar, Asif Ekbal, Amitava Das, and Tanmoy Chakraborty. 2021. Fighting an infodemic: Covid-19 fake news dataset. In International Workshop on Combating On line Ho st ile Posts in Regional Languages dur ing Emerge n… find
  21. Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, and Partha Talukdar. 2024. Indicgenbench: A multilingual benchmark to evaluate generation capabilities of llms on indic languages. arXiv preprint arXiv:2404.16816. find
  22. Briony Swire-Thompson and David Lazer. 2020. Public health and online misinformation: challenges and recommendations. Annual review of public health, 41:433–451. find
  23. Surendrabikram Thapa, Kritesh Rauniyar, Hariram Veeramani, Aditya Shah, Imran Razzak, and Usman Naseem. 2024. Did you tell a deadly lie? evaluating large language models for health misinformation identification. In International Conference on Web Information Systems Engineering, pages 391–405. Sprin… find
  24. Emily K Vraga and Leticia Bode. 2020. Defining misinformation and understanding its bounded nature: Using expertise and evidence for describing misinformation. Political Communication, 37(1):136–144. find
  25. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  26. John Zarocostas. 2020. How to fight an infodemic. The lancet, 395(10225):676. find
  27. Zhihao Zhang, Yiran Zhang, Xiyue Zhou, Liting Huang, Imran Razzak, Preslav Nakov, and Usman Naseem. 2025. From generation to detection: A multimodal multi-task dataset for benchmarking health misinformation. arXiv preprint arXiv:2505.18685. find
  28. Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. 2023. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI conference on human factors in computing systems, pages 1–20. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maharjan-2026-improving-public-health-safety-low/