📄 Disentangling Reasoning Logic to Resolve Explicit Knowledge Conflicts

Summary

When retrieved documents give opposing answers to the same question, an LLM has to decide which source to trust. The authors call these explicit knowledge conflicts. They argue that existing fusion decoders and prompting heuristics fail for two reasons: contradictory evidence gets tangled together (logic entanglement), and conflict types vary too much for fixed rules (case heterogeneity).

Their framework, KCR (Knowledge Conflict Reasoning), works in two stages. First, it extracts the key entity and relation from the query and then rewrites each candidate answer's context into separate reasoning traces. These traces come in two forms: textual entity–relation chains and paths through a local knowledge graph. Second, it trains the backbone model with RLVR using two rewards. A logic coherence reward pulls the model toward the traces of the correct answer and away from the spurious ones. A logic consistency reward penalizes mixing mutually exclusive facts.

The task follows the ConflictQA setup. With KCR, a 7B model beats frameworks built on GPT-4o and GPT-5.1. This suggests that structured reasoning training, rather than model scale, can drive reliable conflict adjudication in RAG.

Key points

• Task setup: each input has a query, two conflicting candidate answers, and a supporting context for each, following ConflictQA (Xie et al., 2024).
• Disentanglement stage:
  • A frozen Qwen2.5-7B does zero-shot entity and relation extraction, with popQA entity labels used where available.
  • GPT-4o-mini builds the local knowledge graphs.
  • Only graph paths that contain the query entity or relation are kept.
• Training stage: RLVR with group sampling and logic coherence plus logic consistency rewards. The authors chose RL over SFT because SFT tends to memorize specific conflict patterns.
• Main results: on the Inter-Context Knowledge Conflict task, the best KCR variant beats the strongest prior framework by between +2.4% and +24.3% in the reported comparison. All results are averaged over three runs.
• Ablations and limitations:
  • Both rewards turn out to be necessary.
  • Gains are largest on tasks with strict logical chains, such as popQA.
  • The method depends on external APIs for graph extraction and costs more to train than SFT.
  • Evaluation covers only controlled, non-adversarial settings.
  • Code is released at github.com/zhengxianda/KCR.