Elamur Research⌕
☰
home / papers / 2026 / Logic Matters in Lightweight Hallucination Classification for RAG System

Logic Matters in Lightweight Hallucination Classification for RAG System

Paper · 2026 · Ningyuan Yang and Kaizhu Huang

Summary

Retrieval-augmented generation can still produce unsupported answers. Cheap NLI-based checkers often miss these errors when the evidence needed to confirm a claim is spread across several retrieved passages.

Yang and Huang propose a modular, training-free detector with three stages:

  1. It splits documents and answers into short factual chunks, using a rule-based filter to drop non-factual sentences.
  2. It links the chunks in a semantic-similarity graph and groups related ones using betweenness centrality.
  3. It scores each group with a relevance module plus an off-the-shelf NLI discriminator, then merges the results into a single hallucination verdict.

The authors stress that "logic" here means semantic coherence across documents, not formal inference. They also release HotPotQA-Derived, a multi-hop benchmark built from HotPotQA. It keeps retrieved documents separate and is split into three difficulty tiers.

A ~0.5B configuration reaches 82.4% accuracy at 85 ms. A ~1.5B configuration reaches 85.6%, outperforming the 11B TrueTeacher. The results suggest that better evidence grouping, rather than bigger models, can make hallucination checking practical on modest hardware.

Key points

  • Config A (~0.5B parameters): 82.4% accuracy on HotPotQA-Derived at 85 ms with a 2.1 GB memory footprint. It beats all sub-1B baselines by more than 30%.
  • Config B (~1.5B parameters): 85.6% accuracy at 165 ms. It surpasses TrueTeacher (11B) while being about 7× smaller and 1.7× faster.
  • The approach works with different NLI models:
    • Six of seven tested discriminators (e.g., HHEM-2.1-Open, DeBERTa-v3-base, AlignScore-Large, MiniCheck-FT5) gain between +6.7% and +29.9%.
    • Only TrueTeacher dips slightly (−0.9%).
  • Ablation on the bridge-hard split shows the graph module matters most. Removing Semantic Evidence Aggregation costs 19.6 points while saving only 18 ms, and removing any two modules drops accuracy to roughly 55% or lower.
  • Additional results and limitations:
    • The method achieves 72.2% F1 on RAGTruth and competitive results on HaluBench and HaluEval.
    • The sentence encoder is all-MiniLM-L6-v2.
    • Performance is weaker on the RAGTruth Data-to-Text (table-based) subset.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Duke Kunshan UniversityKunshan · China2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

Cite

Cite as Ningyuan Yang and Kaizhu Huang. 2026. Logic Matters in Lightweight Hallucination Classification for RAG System. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.73

@inproceedings{yang2026logic,
  title = {{Logic Matters in Lightweight Hallucination Classification for RAG System}},
  author = {Ningyuan Yang and Kaizhu Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.73},
  url = {https://doi.org/10.18653/v1/2026.acl-long.73}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Forrest Bao, Miaoran Li, Rogger Luo, and Ofer Mendelevitch. 2024. HHEM-2.1-Open. find
  2. Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert. 2025. Ragas: Automated evaluation of retrieval augmented generation. Preprint, arXiv:2309.15217.
  3. Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. 2023. Trueteacher: Learning factual consistency evaluation with large language models. Preprint, arXiv:2305.11171.
  4. Assaf Gerner, Netta Madvil, Nadav Barak, Alex Zaik-man, Jonatan Liberman, Liron Hamra, Rotem Brazi-lay, Shay Tsadok, Yaron Friedman, Neal Harow, Noam Bresler, Shir Chorev, and Philip Tannor. 2025. Grounded in context: Retrieval-based method for hallucination detection. Preprint, arXiv:2504.15771. — [arXiv:2504.15771](https://arxiv.org/abs/2504.15771) find
  5. Michelle Girvan and Mark EJ Newman. 2002. Community structure in social and biological networks. Proceedings of the National Academy of Sciences, 99(12):7821–7826. find
  6. Yuntong Hu, Zhihan Lei, Zheng Zhang, Bo Pan, Chen Ling, and Liang Zhao. 2024. GRAG: Graph retrieval-augmented generation. arXiv preprint arXiv:2405.16506. find
  7. Sean Lee, Rui Huang, Aamir Shakir, and Julius Lipp. 2025. Baked-in brilliance: Reranking meets rl with mxbai-rerank-v2. find
  8. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. CoRR, abs/2005.11401.
  9. Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, Singapore. Association for Computa…
  10. Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2023. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. arXiv preprint arXiv:2401.00396.
  11. Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, and Rebecca Qian. 2024. Lynx: An open source hallucination evaluation model. Preprint, arXiv:2407.08488. — [arXiv:2407.08488](https://arxiv.org/abs/2407.08488) find
  12. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  13. Hithesh Sankararaman, Mohammed Nasheed Yasin, Tanner Sorensen, Alessandro Di Bari, and Andreas Stolcke. 2024. Provenance: A light-weight fact-checker for retrieval augmented LLM generation output. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Tra… find
  14. Alex Shan, John Bauer, and Christopher D. Manning. 2025. Osiris: A lightweight open-source hallucination detection system. Preprint, arXiv:2505.04844. find
  15. Qiongkai Xu. 2025. GRADA: Graph-based reranking against adversarial documents attack. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22244–22266. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-logic-matters-lightweight-hallucination-classification/