📄 Enforcing Transitivity in Coreference Resolution

Summary

Pairwise coreference classifiers judge each pair of mentions separately. They then merge links after the fact, which can yield implausible chains. For example, "John Smith" and "Jane Smith" may end up in one entity only because each is linked to "Smith". Finkel and Manning argue that earlier ILP-based coreference work, notably Denis and Baldridge (2007), never encoded transitivity, although ILP suits such constraints well. They train a logistic classifier on all mention pairs. At test time, they pass its log-probabilities to an ILP solver whose triple-wise constraints allow only consistent clusterings, so the most probable legal partition is selected. Experiments on MUC-6 and the ACE Phase 2 NWIRE and BNEWS portions compare this against a Soon-style baseline and a D&B-style baseline. Enforcing transitivity improves results on most metrics, with the largest gains on ACE NWIRE. The study also questions the MUC scorer, which rewards trivial over-merging. Overall, the work shows that global consistency constraints are a cheap and useful addition to pairwise coreference models.

Key points

• Method: The ILP objective is the summed log-probability of the pairwise decisions. For every mention triple, an inequality forces the third pair to be coreferent whenever the other two are. The problems were solved with lp_solve.
• Data: The paper uses MUC-6 (formal train/test) and ACE Phase 2 NWIRE and BNEWS. NPAPER was dropped because some long documents were too large for lp_solve, since ILP is NP-hard.
• Gains: On ACE NWIRE, the b³ F1 score rose by up to 3.6% and cluster f-measure by up to 16.5%. The ILP model beat both baselines on cluster F, Rand index and VOI across all three corpora.
• MUC scorer: Placing all mentions in one entity on MUC-6 gives 100% recall, 78.9% precision and 88.2% F1. The ILP system scored lower on MUC than the D&B-style baseline because its clusters are smaller.
• Features and tools: Features follow Soon et al. (2001), including string and head match, agreement, appositive and NE features. NE tags come from Finkel et al. (2005) and POS tags from Toutanova et al. (2003).