📄 LCC-WSD: System Description for English Coarse Grained All Words Task at SemEval 2007

Summary

The paper describes the Word Sense Disambiguation system that Language Computer Corporation entered in the SemEval 2007 English Coarse Grained All Words task. The authors ask how well standard supervised learners do when senses are coarse rather than fine grained. Coarse senses should be easier to separate, which makes WSD more useful for downstream applications. They trained a separate classifier per word with Maximum Entropy and SVM (LIBSVM). Training data were pooled from SemCor, the Senseval 2 and 3 all-words and lexical-sample corpora, and OMWE 1.0. The features were topical, syntactic and semantic, and some semantic features came from eXtended WordNet (XWN) glosses annotated with semantic relations. They grew the feature set in stages, then distilled a global set through greedy per-word feature selection. Adding XWN glosses as extra training data did not raise precision, so the glosses were used only for words with no other examples. WordNet first sense covered the remaining words. The submission reached 81.446% precision at full coverage, which placed it in the top five (best system: 83.208%).

Key points

• Preprocessing used in-house tools for tokenization, NER, compound concept detection and an in-house Collins parser. Gold POS and lemmas came from the test file, and a WordNet lookup was also applied.
• Features covered several kinds of evidence:
  • word and lemma context and collocations in a (-3,3) window;
  • keywords and bigrams over nearby sentences;
  • verb mode and voice, parse-tree parent, the first non-VP ancestor, and a main-clause flag;
  • parents of neighbouring words;
  • named entities within ±5 sentences;
  • XWN AGENT/THEME gloss signatures.
• Tested on the Senseval 2 lexical sample, ME scored 76.03%, 75.86% and 76.03% on the first three feature sets, and 77.56% on the selected global set. SVM stayed between 71.36% and 73.30%, so ME was chosen.
• Tested on Senseval 2 all words, training on SemCor alone gave 79.61%, XWN glosses alone gave 57.21%, and both together gave 79.44%.
• The final system combined three sources:
  • ME models for 852 words (1715 instances);
  • XWN-trained models for 50 words (70 instances);
  • first-sense fallback for 484 words.

Interesting ideas

• I1. Greedy forward feature selection was run per word with 10-fold cross-validation, but only on a subset because it was slow; the features chosen most often were then pooled into one global set reused for every word. This distilling trick makes an expensive search affordable.
• I2. The authors noticed that adding features left accuracy flat and concluded that each word needs a different useful subset. Measuring each added feature group separately exposed this.
• I3. Each target word gets a semantic signature: the words linked to it as AGENT or THEME across relation-tagged dictionary glosses. A binary feature then records whether each signature word appears in the current sentence.
• I4. Before merging a noisier, stylistically different source (gloss text) into training, the authors tested it alone and in combination. It did not help, so they confined it to words that had no other data.
• I5. A tiered back-off design guarantees full coverage: a rich classifier where data exist, a weaker-source model where only that source exists, and the WordNet first sense otherwise.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Run costly feature or rule searches on a small sample of items, then promote the features that win most often into a shared default for the full training run.
• I2 (learn): Evaluate every new feature group as its own ablation step, so that useless additions get caught instead of silently diluting the parser or tagger.
• I3 (world): Build sense signatures for WordNet senses from role-tagged glosses, so a sense choice can be explained by which typical agents or themes actually occur in the text.
• I4 (learn): Treat silver or out-of-genre data as a fallback tier only, unless a controlled comparison shows that mixing it in improves held-out scores.
• I5 (world): Resolve word senses through an explicit cascade that records which tier decided each case, so every sense assignment stays traceable and never goes missing.