📄 Supervised Within-Document Event Coreference using Information Propagation

Summary

The paper addresses within-document event coreference, which means deciding when two event mentions in one text describe the same real-world event. The authors first survey earlier work. They argue that past results cannot be compared fairly because studies differ in how they define coreference, use partially annotated corpora such as ECB, rely on gold ACE arguments and attributes, or use data that is not public. They then build a supervised pairwise system. A Random Forest classifier in Weka scores mention pairs, and BestLink clustering groups them. The system uses lexical, discourse, syntactic and semantic features, along with event arguments (agent, patient, location). After clustering, argument information is passed between coreferent mentions, and the handling differs by argument type. For locations, the system adds world knowledge from DBpedia and Geonames. Experiments on the IC corpus show that arguments and propagation improve most metrics. On ACE 2005, the system scores above previously reported systems, although the authors stress that these comparisons are only approximate. Pairwise F stays near 50%, so the task remains far from solved.

Key points

• Data: The main dataset is the IC domain corpus of 65 documents, split into 40 for training/development and 25 for testing. Full-coreference inter-annotator agreement is 0.614 (Fleiss's kappa). The system is also evaluated on ACE 2005, using 539 documents for training and 60 for testing.
• Pipeline and resources:
  • Arguments come from the Fanse semantic parser, using PropBank ARG0/ARG1.
  • Locations come from Stanford NER and are disambiguated with DBpedia Spotlight, then DBpedia, with Geonames as a fallback. This lets the system link, for example, Istanbul to Turkey.
  • The feature set includes 105 semantic features.
• IC results: Adding propagation raises pairwise F from 51.60 to 53.11, MUC F to 61.44, and BLANC F to 75.06. The gain in recall comes with some loss of precision. In practice, propagation ran for only one iteration.
• ACE results: The system reaches MUC F 50.98, B³ F 89.38 and CEAF-e F 86.47. Bejan & Harabagiu (2010) reported 45.1, 83.8 and 76.7 on the same metrics.
• Clustering: BestLink consistently beats plain transitive closure. Errors often come from over-merging mentions that share a head word and from imperfect argument extraction.