📄 HIT-IR-WSD: A WSD System for English Lexical Sample Task

Summary

The paper describes HIT-IR-WSD, a supervised word sense disambiguation system from the Information Retrieval Lab at Harbin Institute of Technology, created for SemEval 2007's Task 11, which tested disambiguation of English words in a lexical sample setting. The authors frame sense selection as per-word classification. They train a libsvm classifier using only the official training data, which covers 20 nouns and 20 adjectives in the web-sourced track. Each instance becomes a binary vector built from four groups of cues:

• lemmatized context words,
• POS tags of nearby words, marked by side,
• lemma collocations in a fixed window, also marked by side,
• dependency information linking the target to its head, taken from MaltParser.

On a held-out development split, they compare every combination of these groups, tune the SVM's C parameter and test several kernels. The linear kernel clearly wins. The final system scored 81.9% micro-averaged on the test set, the best result among participating runs. The work shows that a careful feature design with a simple linear classifier is enough for strong lexical-sample WSD.

Key points

• Toolchain: WordNet morph for root forms, SVMTool for Penn Treebank POS tags, MaltParser for dependencies and libsvm for multi-class classification. No extra training data was used.
• Development set: 1,781 instances, about half the size of the remaining training portion. All 15 feature-group combinations were evaluated on it.
• Context words alone reached 80.3%, and the best combinations reached 82.9%. Syntax alone gave 72.6% and POS alone 70.4%. Syntactic features gave inconsistent gains, which the authors partly attribute to parser errors.
• Kernel comparison: the linear kernel scored 82.9%, while RBF, polynomial and sigmoid each scored 68.3% with default settings. Polynomial kernels improved as their degree approached 1.
• Accuracy kept rising with more training data. POS tagging and parsing took about 2 hours, while SVM training and prediction took under 5 minutes.

Interesting ideas

• I1. Ablate every subset of feature groups, labelled with a bit vector (e.g. 1011), so that both the value of each group and its interactions with the others become visible.
• I2. Describe a word's sense context through its dependency head: the head's lemma, the head's POS, the relation label and the head's direction.
• I3. Build the stop-word list empirically, keeping a candidate only if removing it measurably helps the dev score.
• I4. Use side-marked lemma collocations in a ±5 window that keep function words, since order and closeness carry information that unordered context loses.
• I5. Diagnose the model with a learning curve and a kernel-degree sweep, which show whether more data or more model capacity is the better investment.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Run all on/off combinations of evidence sources (parser witnesses, VerbNet constraints, coreference cues) against a fixed dev set, to see which ones actually pull their weight.
• I2 (frames): Use the UD head and deprel of an ambiguous noun or adjective as structured evidence when choosing its WordNet sense for a frame slot, and keep a record of which arc decided the choice.
• I3 (learn): Apply the same "admit each item only if it helps" test to self-induced repair rules or filter lists, so that no rule enters without its own measured gain.
• I4 (tokens): Feed lemma-plus-side features near ambiguous tokens into the tagger and lemmatizer as an explainable local cue.
• I5 (learn): Plot parser and reader accuracy against the amount of silver data to decide when gathering more witness-agreed data stops paying off.