uir-cis at SemEval-2026 Task 12: Mitigating Prior-Induced Hallucinations in Retrieval-Augmented Reasoning via Precision-Oriented Decoding
Paper · 2026 · Chiyao Zhou, Zebing Wang, Kexin Deng et al.
Summary
The uir-cis team tackles SemEval-2026 Task 12, Abductive Event Reasoning (AER). Systems must choose the direct cause(s) of an observed event from candidate options, using retrieved documents that may be incomplete or noisy. The metric is unforgiving. A correct proper subset earns 0.5, but including any wrong option zeroes the instance.
The authors found that their model tended to "over-select": it attached plausible distractors based on prior knowledge rather than the evidence. Their pipeline works as follows:
- A BGE cross-encoder reranker picks the top 5 documents, with context capped at 512 tokens.
- A LoRA-adapted Qwen2.5-32B-Instruct, running on two RTX 4090 GPUs, reasons over them with zero-shot CoT.
Their main contribution, Precision-Oriented Decoding (POD), lowers the sampling temperature to 0.45 and takes a majority vote across 9 reasoning paths. This lifted the test score from 0.769 to 0.802. A recall-oriented union ensemble with high-temperature outputs dropped to 0.761. The takeaway is that under zero-tolerance scoring, suppressing false positives matters more than recovering missed causes.
Key points
- Score progression: baseline (T=0.7, K=3) scored 0.769, POD (T=0.45, K=9) scored 0.802, and the union ensemble scored 0.761. POD reached precision 0.889 and recall 0.922. The ensemble had the highest recall (0.934) but lower precision (0.870).
- Ablation: lowering temperature alone (K=1) gave 0.75, and voting alone at T=0.70 (K=9) plateaued at 0.78. Only the combination reached 0.80.
- Error analysis: zero-score over-selection errors fell from 133 (ensemble) to 106 (POD), roughly a 20% reduction.
- Caveats:
- Table 1 scores for GPT-4o, Claude 3.5 Sonnet, Llama-3-8B and Mixtral are described as "expected ranges", not actual runs.
- The retriever is named inconsistently in the paper (BGE-Reranker vs. BGE-M3).
- Limitation: running 9 decoding passes through a 32B model makes the system slow. Speculative decoding is suggested as future work.
Institutions
Fields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works
🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksInformation retrieval & search · 473 works
Cite
Cite as Chiyao Zhou, Zebing Wang, Kexin Deng, Yaru Zhao, Lin Deng and Binyang Li. 2026. uir-cis at SemEval-2026 Task 12: Mitigating Prior-Induced Hallucinations in Retrieval-Augmented Reasoning via Precision-Oriented Decoding. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).
@inproceedings{zhou2026uir,
title = {{uir-cis at SemEval-2026 Task 12: Mitigating Prior-Induced Hallucinations in Retrieval-Augmented Reasoning via Precision-Oriented Decoding}},
author = {Chiyao Zhou, Zebing Wang, Kexin Deng, Yaru Zhao, Lin Deng and Binyang Li},
booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
year = {2026},
doi = {10.18653/v1/2026.semeval-1.415},
url = {https://doi.org/10.18653/v1/2026.semeval-1.415}
}References
- Pengfei Cao, Mingxuan Yang, Yubo Chen, Chenlong Zhang, Mingxuan Liu, Kang Liu, and Jun Zhao. 2026. Semeval-2026 task 12: Abductive event reasoning: Towards real-world event causal inference for large Case Study: Diagnostics of the “Over-Selection” Hallucination (Q-2518) find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-uir-cis-semeval-2026-task/