Elamur Research⌕
☰
home / papers / 2026 / Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding

Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding

Paper · 2026 · Weixu Zhang, Fanghua Ye, Qiang Gao et al.

Summary

When LLMs answer questions or summarize from a supplied passage, they often fall back on knowledge memorized in pretraining. The result is fluent text that contradicts or ignores the input, known as faithfulness hallucination. The authors propose Context-Fidelity Boosting (CFB), an inference-only decoding method. It borrows the logit-biasing trick from text watermarking, but instead of favoring a "green" vocabulary subset, it raises the logits of tokens that occur in the source context. There are three variants, each more adaptive than the last:

  • Static: a constant bonus.
  • Context-aware: the bonus is scaled by the Jensen–Shannon divergence between next-token distributions computed with and without the context.
  • Token-aware: the scaled bonus is redistributed across tokens using attention over source positions and semantic similarity.

The method was tested on summarization (CNN-DM) and question answering (including NQ-Synth) with several open-source LLMs. CFB consistently improves faithfulness metrics at low extra cost, compared with decoding baselines such as CAD, AdaCAD and COIECD. The model itself is never modified, so CFB fits easily into context-grounded pipelines like RAG. However, it does require access to logits.

Key points

  • Requirements: CFB is purely decoding-time, with no retraining or architecture edits. The token-aware variant additionally needs attention or embedding signals, which rules out black-box APIs.
  • Ablation results: With Llama3-8B on CNN-DM, the full token-aware variant reached 35.81 ROUGE-L, 94.31 FactKB and 89.38 BERT-P.
    • Removing semantic similarity caused a severe collapse.
    • Removing JSD scaling hurt all metrics.
    • Removing attention caused only a small drop.
  • Hyperparameters: The best relevance weights were λ1 = 0.6 (attention) and λ2 = 0.4 (similarity). Moderate boost values δ work best on CNN-DM, while NQ-Synth tolerates a wider range.
  • Case study: In a knowledge-conflict example (Great Wall length, 21,196 vs. 8,850 km), greedy decoding and COIECD reverted to the memorized figure, while CFB reported the context's value.
  • Code: Released at github.com/weixuzhang/CFB. The paper appears in Findings of ACL 2026 (Tencent Hunyuan, McGill/MILA and others).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Université de MontréalMontreal · Canada68 works · 79,382 citations🎓McGill UniversityMontreal · Canada87 works · 11,027 citations🎓Wuhan UniversityWuhan · China35 works · 1,127 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Weixu Zhang, Fanghua Ye, Qiang Gao, Jian Li, Haolun Wu, Yuxing Tian, Sijing Duan, Nan Du and Xiaolong Li. 2026. Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2121

@inproceedings{zhang2026context,
  title = {{Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding}},
  author = {Weixu Zhang, Fanghua Ye, Qiang Gao, Jian Li, Haolun Wu, Yuxing Tian, Sijing Duan, Nan Du and Xiaolong Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2121},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2121}
}

References

38 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zhongwu Chen, Chengjin Xu, Dingmin Wang, Zhen Huang, Yong Dou, and Jian Guo. 2025. Rulerag: Rule-guided retrieval-augmented generation with language models for question answering. Preprint, arXiv:2410.22353. — [arXiv:2410.22353](https://arxiv.org/abs/2410.22353) find
  2. Giwon Hong, Aryo Pradipta Gema, Rohit Saxena, Xiaotang Du, Ping Nie, Yu Zhao, Laura Perez-Beltrachini, Max Ryabinin, Xuanli He, Clémentine Fourrier, and Pasquale Minervini. 2024. The hallucinations leaderboard - an open effort to measure hallucinations in large language models. CoRR, abs/2404.05904. — [arXiv:2404.05904](https://arxiv.org/abs/2404.05904) find
  3. Bairu Hou, Yang Zhang, Jacob Andreas, and Shiyu Chang. 2024. A probabilistic framework for LLM hallucination detection via belief tree propagation. CoRR, abs/2406.06950.
  4. Minda Hu, Bowei He, Yufei Wang, Liangyou Li, Chen Ma, and Irwin King. 2024. Mitigating large language model hallucination with faithful finetuning. CoRR, abs/2406.11267. — [arXiv:2406.11267](https://arxiv.org/abs/2406.11267) find
  5. Le Huang, Hengzhi Lan, Zijun Sun, Chuan Shi, and Ting Bai. 2024. Emotional rag: Enhancing role-playing agents through emotional retrieval. Preprint, arXiv:2410.23041. — [arXiv:2410.23041](https://arxiv.org/abs/2410.23041) find
  6. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. CoRR, abs/2311.05232.
  7. Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James R. Glass. 2024. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces…
  8. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2024. A watermark for large language models. Preprint, arXiv:2301.10226.
  9. Jiaxi Cui, Munan Ning, Zongjian Li, Bohua Chen, Yang Yan, Hao Li, Bin Ling, Yonghong Tian, and Li Yuan. 2024. Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model. Preprint, arXiv:2306.16092.
  10. Jean Lee, Nicholas Stevens, and Soyeon Caren Han. 2025. Large language models in finance (finllms). Neural Computing and Applications. find
  11. Shangbin Feng, Vidhisha Balachandran, Yuyang Bai, and Yulia Tsvetkov. 2023. Factkb: Generalizable factuality evaluation using language models enhanced with factual knowledge. Preprint, arXiv:2305.08281. find
  12. Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Teare, Beatrice Alex, Pasquale Minervini, and Amrutha Saseendran. 2024. Decore: Decoding by contrasting retrieval heads to mitigate hallucinations. CoRR, abs/2410.18860. find
  13. Noah Golowich and Ankur Moitra. 2024. Edit distance robust watermarks for language models. Preprint, arXiv:2406.02633. — [arXiv:2406.02633](https://arxiv.org/abs/2406.02633) find
  14. Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel-Eskin, and Mohit Bansal. 2024. Fundamental problems with model editing: How should rational belief revision work in llms? CoRR, abs/2406.19354. find
  15. Philippe Laban, Alexander R. Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024. Summary of a haystack: A challenge to long-context llms and rag systems. Preprint, arXiv:2407.01370.
  16. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  17. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2024a. A semantic invariant robust watermark for large language models. Preprint, arXiv:2310.06356. — [arXiv:2310.06356](https://arxiv.org/abs/2310.06356) find
  18. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2024b. A semantic invariant robust watermark for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  19. Yantao Liu, Zijun Yao, Xin Lv, Yuchen Fan, Shulin Cao, Jifan Yu, Lei Hou, and Juanzi Li. 2024c. Untangle the KNOT: Interweaving conflicting knowledge and reasoning skills in large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resour… find
  20. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Short Papers, NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 783– 791. As…
  21. Yepeng Liu and Yuheng Bu. 2024. Adaptive text watermark for large language models. Preprint, arXiv:2401.13927. — [arXiv:2401.13927](https://arxiv.org/abs/2401.13927) find
  22. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063, Online and Punta Cana, Dominican Re…
  23. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… find
  24. Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. 2024. Faitheval: Can your language model stay faithful to context, even if "the moon is made of marshmallows". CoRR, abs/2410.03727. — [arXiv:2410.03727](https://arxiv.org/abs/2410.03727) find
  25. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don‘t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–1807, Brussels, Belgium. As…
  26. Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2024. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu…
  27. Zexuan Qiu, Zijing Ou, Bin Wu, Jingjing Li, Aiwei Liu, and Irwin King. 2024. Entropy-based decoding for retrieval-augmented large language models. CoRR, abs/2406.17519. find
  28. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083, Vancouver, Canada. Association for Compu… find
  29. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. find
  30. S. M. Towhidul Islam Tonmoy, S. M. Mehedi Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. 2024. A comprehensive survey of hallucination mitigation techniques in large language models. CoRR, abs/2401.01313.
  31. Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. 2024. Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge. CoRR, abs/2409.07394. find
  32. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  33. Kevin Wu, Eric Wu, and James Y. Zou. 2024. Clasheval: Quantifying the tug-of-war between an llm’s internal prior and external evidence. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, Decem…
  34. Dingkang Yang, Dongling Xiao, Jinjie Wei, Mingcheng Li, Zhaoyu Chen, Ke Li, and Lihua Zhang. 2024. Improving factuality in large language models via decoding-time hallucinatory and truthful comparators. CoRR, abs/2408.12325. — [arXiv:2408.12325](https://arxiv.org/abs/2408.12325) find
  35. Xiaowei Yuan, Zhao Yang, Yequan Wang, Shengping Liu, Jun Zhao, and Kang Liu. 2024. Discerning and resolving knowledge conflicts through adaptive decoding with contextual information-entropy constraint. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virt… find
  36. Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu, Jianfeng Gao, Tuo Zhao, Dan Roth, and Hao Cheng. 2024. Model tells itself where to attend: Faithfulness meets automatic attention steering. CoRR, abs/2409.10790. find
  37. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find
  38. Yinghao Zhu, Changyu Ren, Shiyun Xie, Shukai Liu, Hangyuan Ji, Zixiang Wang, Tao Sun, Long He, Zhoujun Li, Xi Zhu, and Chengwei Pan. 2024. Realm: Rag-driven enhancement of multimodal electronic health records analysis via large language models. Preprint, arXiv:2402.07016. — [arXiv:2402.07016](https://arxiv.org/abs/2402.07016) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-context-fidelity-boosting-enhancing-faithful/