📄 Automatic Acquisition of Domain Knowledge for Information Extraction

Summary

Adapting an Information Extraction system to a new event scenario requires finding the many ways the target events are phrased. That search normally means reading or annotating large amounts of text. The authors propose ExDisco, an unsupervised bootstrapping procedure that needs only a handful of user-supplied seed patterns and a raw, unlabelled domain corpus.

The corpus is parsed with a Conexor FDG dependency parser. Names, numbers and dates are normalized, and each clause is turned into a subject–verb–object tuple. ExDisco then alternates between two steps:

• splitting documents into relevant and non-relevant sets using the current patterns;
• promoting the candidate pattern whose occurrences are most concentrated in the relevant set.

Document relevance is graded rather than binary, so evidence from later, less trusted patterns counts for less.

On the MUC-6 Management Succession scenario, the Proteus extractor with discovered (lightly reviewed) patterns outperformed a version tuned by hand for a month and came close to the heavily developed current system. Document-filtering quality fell within the range of supervised or hand-built MUC-6 systems. This shows that much costly corpus analysis can be automated.

Key points

• Training data was 9,224 Wall Street Journal articles. Parsing produced about 440,000 clause tuples, of which roughly 215,000 were distinct.
• Seeds were tiny:
  • Management Succession: company + appoint-type verb + person, and person + resign-type verb.
  • Mergers/Acquisitions: buy/purchase + company, and company + merge.
• Candidate patterns are scored by their precision on relevant documents multiplied by the log of their relevant-document count. Overly frequent and very rare patterns are filtered out before scoring.
• Text filtering was evaluated over 100 iterations against the MUC-6 training and test sets (100 documents each) and against 200 hand-judged Acquisition documents. Recall/precision was comparable to MUC-6 participants.
• Unlike Riloff (1996), ExDisco learns complete multi-slot event patterns and requires no hand-classified relevant documents.

Interesting ideas

• I1. Learn patterns and relevant documents together. Each is used to refine the other, which breaks the circularity of needing relevant text to find patterns and patterns to find relevant text. This mutual-bootstrapping setup suits any low-resource task where labelled examples are scarce, such as Ukrainian-language domain extraction.
• I2. Graded document relevance. Seed-matched documents count fully, while documents reached through newly learned patterns receive inherited, lower confidence. This damps semantic drift during bootstrapping and is easy to add to other self-training loops.
• I3. Break sparse tuples into argument pairs. Rare full tuples are scored as pairs (e.g., verb–object or subject–object), and the missing slot is filled afterwards with high-scoring words. This yields reliable statistics and, as a side effect, word classes (e.g., verbs linking company and person).
• I4. Normalize before mining. Replacing names, numbers and dates with class tokens, and collapsing passives and relative clauses into one form, makes surface variants countable as a single pattern. It is a cheap step that strengthens almost any corpus pattern-mining effort.
• I5. Use the by-product as a free metric. The procedure's document relevance ranking can be scored directly as a text-filtering task. This gives a way to monitor a bootstrapping run without building full extraction gold data.