Elamur Research⌕
☰
home / papers / 2026 / Generating Effective CoT Traces for Mitigating Causal Hallucination

Generating Effective CoT Traces for Mitigating Causal Hallucination

Paper · 2026 · Yiheng Zhao and Jun Yan

Summary

Small language models (≤1.5B parameters) often claim that two events are causally linked when they are not. This is a serious problem for event causality identification (ECI). Existing ECI corpora provide only yes/no labels, and no Chain-of-Thought (CoT) training data exist for this task. To measure the bias, the authors define the Causal Hallucination Rate (CHR): accuracy on causal pairs minus accuracy on non-causal pairs.

Using CHR, they re-test three heuristics from math-reasoning distillation and find that none of them carries over directly:

  • Perplexity: choosing the lowest-perplexity trace does not help.
  • Length: longer traces with more semantic explanation and more reasoning steps work better, even for small models.
  • Rewriting: having the target model rewrite traces helps only when perplexity does not rise.

From these findings they build a pipeline that produces long, explanation-rich traces aligned with the target model. They then fine-tune Qwen2.5-1.5B and Llama3.2-1B on traces built from EventStoryLine. The fine-tuned models show far lower CHR and higher mean accuracy. These gains carry over to other datasets, to harder document-level inputs, and to misleading prompts. This suggests that well-designed CoT data can make compact models more reliable at causal reasoning.

Key points

  • CHR metric: CHR = Acc_causal − Acc_non-causal. Positive values signal causal hallucination, and negative values signal a tilt toward predicting "no cause." The untuned Qwen2.5-1.5B scores about 84.55 on EventStoryLine.
  • Effect of trace source and length: In the criteria analysis, traces from Llama3.1-8B (longer) cut CHR to about 34, compared with about 60 for shorter Qwen2.5-7B traces. Even longer "thinking-style" traces that were also rewritten reached 24.42.
  • Document-level generalization: On inter-sentence pairs in EventStoryLine, CHR fell from 54.52% to 1.41% for Qwen2.5-1.5B and from 93.94% to 4.51% for Llama3.2-1B. Mean accuracy rose in both cases.
  • Cross-dataset generalization: Gains also held on Causal-TimeBank and MAVEN-ERE, where CHR dropped from roughly 68–85 to roughly 4–11.
  • Robustness: Prompts that falsely asserted or denied causality barely affected the fine-tuned models. For Qwen2.5-1.5B on EventStoryLine, mean accuracy moved only from 66.00 to 63.08.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Yiheng Zhao and Jun Yan. 2026. Generating Effective CoT Traces for Mitigating Causal Hallucination. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.264

@inproceedings{zhao2026generating,
  title = {{Generating Effective CoT Traces for Mitigating Causal Hallucination}},
  author = {Yiheng Zhao and Jun Yan},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.264},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.264}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Ruichu Cai, Shengyin Yu, Jiahao Zhang, Wei Chen, Boyan Xu, and Keli Zhang. 2025. Dr. eci: Infusing large language models with causal knowledge for decomposed reasoning in event causality identification. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9346–9375… find
  3. Tommaso Caselli and Piek Vossen. 2017. The event StoryLine corpus: A new benchmark for causal and temporal relation extraction. In Proceedings of the Events and Stories in the News Workshop, pages 77– 86, Vancouver, Canada. Association for Computational Linguistics. find
  4. Qing Cheng, Zefan Zeng, Xingchen Hu, Yuehang Si, and Zhong Liu. 2025. A survey of event causality identification: Taxonomy, challenges, assessment, and prospects. ACM Computing Surveys, 58(3):1– 37. find
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  6. Jinglong Gao, Xiao Ding, Bing Qin, and Ting Liu. 2023. Is chatgpt a good causal reasoner? a comprehensive evaluation. arXiv preprint arXiv:2305.07375. find
  7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  8. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. find
  9. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — Qwen2.5-Coder Technical Report
  10. Viet Dac Lai, Amir Pouran Ben Veyseh, Minh Van Nguyen, Franck Dernoncourt, and Thien Huu Nguyen. 2022. Meci: A multilingual dataset for event causality identification. In Proceedings of the 29th international conference on computational linguistics, pages 2346–2356. find
  11. Bo Li, Mingda Wang, Shikun Zhang, and Wei Ye. 2026a. Instruction data selection via answer divergence. Preprint, arXiv:2604.10448. find
  12. Bo Li, Shikun Zhang, and Wei Ye. 2026b. Data selection for multi-turn dialogue instruction tuning. Preprint, arXiv:2604.07892. find
  13. Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, and Radha Poovendran. 2025. Small models struggle to learn from strong reasoners. arXiv preprint arXiv:2502.12143. find
  14. Paramita Mirza and Sara Tonelli. 2016. Catena: Causal and temporal relation extraction from natural language texts. In The 26th international conference on computational linguistics, pages 64–75. ACL. find
  15. Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, and 1 others. 2022. Maven-ere: A unified large-scale dataset for event coreference, temporal, causal, and subevent relation extraction. arXiv preprint arXiv:2211.07342.
  16. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  17. Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024. Self-distillation bridges distribution gap in language model fine-tuning. arXiv preprint arXiv:2402.13669.
  18. Zefan Zeng, Xingchen Hu, Qing Cheng, Weiping Ding, Wentao Li, and Zhong Liu. 2025. Zero-shot event causality identification via multi-source evidence fuzzy aggregation with large language models. arXiv preprint arXiv:2506.05675. — [arXiv:2506.05675](https://arxiv.org/abs/2506.05675) find
  19. Dylan Zhang, Qirun Dai, and Hao Peng. 2025. The best instruction-tuning data are those that fit. arXiv preprint arXiv:2502.04194. — [arXiv:2502.04194](https://arxiv.org/abs/2502.04194) find
  20. Yiheng Zhao and Jun Yan. 2025. Can llms identify event causality more accurately through debate? a systematic assessment of llms’ factuality and reasoning. In 2025 IEEE Symposium on Trustworthy, Explainable and Responsible Computational Intelligence (CITREx), pages 1–8. IEEE. find
  21. Xiang Zou, Xuanhong Li, Po Hu, and Ming Dong. 2025. Mrbalance: A framework for enhancing event causality identification in multi-agent debates via role assignment. Knowledge-Based Systems, page 114470. find
  22. Training Set Statistics Fold Total Pos Neg 1 2 3 4 5 7021 7491 7378 7629 6697 2179 2122 2128 2213 2094 4842 5369 5250 5416 4603 Total Pos Neg 1 2 3 4 5 4108 3244 3326 2404 4916 630 648 696 528 682 3478 2596 2630 1876 4234 find
  23. Dataset Total Pos Neg Causal-TimeBank MAVEN-ERE 7656 19642 298 3359 7358 16283 *Table 11: Statistics of the Causal-TimeBank and MAVEN-ERE datasets. “Pos” denotes causal event pairs, and “Neg” denotes non-causal event pairs.* Data Statistics Neg EventStoryLine Training Framework Epochs Per-device Bat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-generating-effective-cot-traces-mitigating/