📄 Using Cross-Entity Inference to Improve Event Extraction

Summary

The paper tackles ACE event extraction: detecting event triggers and types, finding event participants, and labeling their roles. Earlier transductive approaches such as cross-event inference rely on relations between events. Those relations are hard to extract, so they often supply misleading or missing evidence. The authors instead build on a simple intuition: entities with similar backgrounds, such as two presidential contenders, tend to take part in similar events in similar roles.

The method has four stages:

1. Each entity is used as a search query against large-scale resources such as the Web.
2. The retrieved documents are turned into a descriptive profile of the entity's type.
3. Entities are grouped by profile similarity, using CLUTO clustering during training and K-nearest neighbors at test time.
4. Known training events are propagated to test sentences in three steps: trigger and type first, then arguments, then roles.

On ACE 2005 data, this improves a sentence-level baseline by roughly 8.6 F points for triggers and almost 12 F points for arguments and roles. It also beats cross-event inference on argument and role classification. The work shows that well-characterized entities can carry much of the signal needed for event detection.

Key points

• The task follows ACE 2005, with its 33 event subtypes treated as flat, independent types. The system uses the gold ACE entity labels rather than running its own name identification.
• Mean F-scores rose over the sentence-level baseline as follows:
  • triggers: 59.74 to 68.33
  • arguments: 41.29 to 53.15
  • roles: 36.46 to 48.36
• Compared with cross-event inference, argument F was higher by 2.87 points and role F by 3.81 points (mean performance). Trigger F was slightly lower (68.33 vs 68.79), mainly because of weaker recall: only sentences containing at least one entity mention were treated as candidates.
• Automatic clustering produced 129 entity subtypes. Spot checks of 10 of them found at least 19.2% noisy members each, yet manually cleaned clusters gave only small gains (trigger F 69.15 vs 68.33). Using the coarse ACE entity types instead caused a sharp drop (trigger F 51.34).
• Two human annotators were evaluated on 40 ACE newswire texts. The one who knew the extraction rules reached about 75 trigger F, while the uninformed one scored higher on arguments than on triggers.

Interesting ideas

• I1. Web-derived entity profiles: Treating each entity as a search query and summarizing what comes back gives a cheap background profile even for rare names. This could support any task where entity semantics matter, such as relation extraction or entity linking for low-resource languages.
• I2. Analogy-based transfer through similar entities: Labels move from training events to test events via entity similarity, as in "Obama beats McCain" borrowing the Elect reading from "Bush beats McCain." This avoids depending on an intermediate structure that is itself error-prone to extract, such as event-to-event relations.
• I3. Cascaded inference: Type and trigger are decided first, then arguments, then roles, with each step conditioned on the earlier ones. The human study hints that arguments could also be found independently of triggers, which is a useful caution when designing pipelines.
• I4. Clean-versus-noisy clustering ablation: Comparing automatic clusters, hand-cleaned clusters and coarse gold types showed two things. Moderate cluster noise barely hurt, while granularity mattered a lot. This kind of check is worth reusing for any method built on induced clusters.
• I5. Candidate filtering as a recall ceiling: Requiring an entity mention in a sentence silently capped recall. It is a reminder to measure how much gold data a candidate-selection filter excludes before any modeling starts.