📄 Preemptive Information Extraction using Unrestricted Relation Discovery

Summary

Traditional Information Extraction (IE) needs heavy manual work for every new scenario: someone must choose the target event type, collect relevant documents and build extraction patterns, without knowing whether the task is feasible. Shinyama and Sekine propose Preemptive Information Extraction, which builds every plausible IE table ahead of time so that adapting IE becomes a matter of searching existing tables. The core method, Unrestricted Relation Discovery, works as follows:

• News is crawled from several sources and grouped into "basic clusters" of articles about the same event.
• Entities are linked across those articles with cross-document coreference.
• Each entity is described by "basic patterns" taken from GLARF predicate-argument structures.
• Basic clusters are grouped into "metaclusters" (tables) when several entity pairs share weighted patterns and the texts are similar enough in vocabulary.

From about 28k articles the system produced 101 tables with at least three rows. Of 48 sampled tables, 75% were judged consistent. This early, fully unsupervised result suggests that relation tables can be discovered without any scenario specification.

Key points

• Data scale: 28,009 source articles formed 5,543 basic clusters. These yielded 643,767 basic pattern tokens (7,990 types), 302 metaclusters, and 101 metaclusters with three or more rows.
• Evaluation: 36 of 48 randomly chosen tables (75%) were consistent, and 118 of 161 rows (73%) matched the evaluator's description.
• Recall estimate: the largest table, on hurricanes, was compared with a hand-built list and recovered 6 of 9 storms. Hurricane names were mostly separated from person names, even though the NE tagger did not distinguish them.
• Error analysis of 10 wrong rows:
  • 4 came from coreference mistakes.
  • 1 came from a parse error.
  • 4 came from distant patterns that belonged to different events within the same cluster.
  • 1 had an unclear cause.
• Basic patterns come from GLARF (Meyers et al.). GLARF regularizes constructions such as participials and coordination, and it converts easily into a word-level directed graph.

Interesting ideas

• I1. Treat IE adaptation as retrieval. Building candidate relation tables in advance turns the costly task of developing an extractor into simply looking up an existing table. This could suit any domain where requirements arrive unpredictably.
• I2. Use the pattern as the column. Syntactic context around an entity serves both as a clustering feature and as a label for the entity's role. Entities from different documents therefore line up into the same column without any predefined schema.
• I3. Pool comparable articles to fight sparsity. Merging several reports of the same event, and linking their entities with coreference, gives each entity far more context than a single article would. This trick is useful in low-resource settings, such as smaller-language news.
• I4. Add a topical gate that ignores names. Requiring bag-of-words similarity between documents, with names excluded, keeps unrelated events with shared surface patterns apart (e.g., a politician vs. a storm "flying to" a place). Down-weighting very frequent patterns such as "say" adds further protection against spurious matches.
• I5. Evaluate open-ended output by describability. A judge writes one specific templated sentence ($1, $2) for each table and checks how many rows fit it. This gives a cheap, interpretable consistency measure when no gold schema exists.