🔀 Word sense disambiguation
Word sense disambiguation (WSD) identifies which meaning of an ambiguous word is intended in a given context. It is usually framed as picking one entry from a fixed sense inventory, such as the synsets of WordNet.
What it is
Many frequent words carry several meanings. Bank can name a financial institution or the side of a river, and plant can be a living organism or a factory. A WSD system reads the surrounding text and assigns the appropriate meaning to each occurrence. Ide and Véronis described it as an "intermediate" task. It is seldom an end in itself, but machine translation, information retrieval, content analysis, parsing and speech processing all depend on resolving ambiguity (Ide & Véronis 1998).
The task requires an inventory that lists the possible senses. In English this is most often WordNet, where each sense is a synset with a short definition (gloss). WordNet draws very fine distinctions, and human annotators disagree often. Reported inter-annotator agreement on the Senseval-3 all-words test set was 72.5%, which places a practical ceiling near 70% for fine-grained WSD (Navigli 2006). Coarser alternatives exist for this reason:
- WordNet senses can be clustered by mapping them onto the coarser, hierarchically organised senses of the Oxford Dictionary of English (Navigli 2006).
- Nouns and verbs can be tagged with the 41 lexicographer "supersense" classes (Ciaramita & Altun 2006).
Evaluations come in two settings. In the lexical-sample setting, only a few selected words are tested. In the all-words setting, every content word in running text is disambiguated. A common reference point is the first-sense baseline, which always picks the sense WordNet lists first. For English, that is the sense most frequent in hand-tagged text (Agirre et al. 2014). A sibling task, word sense induction, discovers senses from raw text without any inventory (Brody & Lapata 2009; Manandhar et al. 2010).
The key idea
All approaches exploit the same intuition: the words around an ambiguous word reveal its meaning. Methods differ in where the evidence about each sense comes from. Knowledge-based methods use dictionaries, thesauri or lexical graphs. Supervised methods learn from sense-tagged examples. Unsupervised methods rely on untagged corpora (Ide & Véronis 1998).
The classic knowledge-based mechanism is the Lesk algorithm (Lesk, 1986). It chooses the sense whose dictionary definition shares the most words with the context. Its simplified form is:
ŝ = argmax over s in Senses(w) of | Gloss(s) ∩ Context(w) |
Here w is the target word and Senses(w) is its set of candidate senses in the inventory. Gloss(s) is the set of content words in the definition of sense s, and Context(w) is the set of content words around w. The bars denote the size of the intersection, and ŝ is the chosen sense.
As a worked example, take "the pine cone fell from the evergreen tree." Suppose cone has two glosses: "solid body narrowing to a point from a round base" and "fruit of certain evergreen trees." The context words {pine, fell, evergreen, tree} share nothing with the first gloss. They share evergreen with the second, and also tree once trees is reduced to its lemma, so the fruit sense wins.
The original method matches the definitions of each sense of the target against the definitions of the surrounding words. The simplified version compares glosses directly with the context. Both fail when definitions and context use different words for the same thing. Later work softens this by enlarging glosses with related senses and scoring them by distributional similarity instead of exact overlap (Basile et al. 2014).
Two other core mechanisms recur in the field:
- Decision lists rank contextual clues by log-likelihood and label each instance using only the single strongest matching clue (Yarowsky 1994).
- Graph methods turn candidate senses into vertices linked by lexical relations. They then run PageRank, commonly with the damping factor set to 0.85, so that senses supported by the rest of the text rise to the top (Mihalcea et al. 2004; Mihalcea 2005).
History and founding papers
The problem was first raised in machine translation, in Weaver's memorandum. Kaplan found in 1950 that translators resolved ambiguity about as well from two words on each side as from the whole sentence. Bar-Hillel's 1960 "box in the pen" argument held that automatic disambiguation was out of reach, and this view fed into the ALPAC report of 1966. Masterman's Roget-based system, which looked for overlapping thesaurus heads, anticipated knowledge-based WSD (Ide & Véronis 1998).
Statistical methods arrived in the early 1990s. Brown and colleagues defined senses by the translations they predict, learned them from the Canadian Hansard, and cut a translation system's error rate by 13% (Brown et al. 1991). Yarowsky trained one model per Roget category on encyclopedia text and reached 92% accuracy on 12 polysemous words (Yarowsky 1992). Dagan and Itai counted syntactic co-occurrences in a target-language corpus, testing on Hebrew and German source text. On the Hebrew–English data, their method decided 70% of ambiguous words and was correct in 91% of those decisions (Dagan & Itai 1994).
Yarowsky's 1995 bootstrapping algorithm became an influential, minimally supervised method. It needs only a few seed collocations and otherwise learns from untagged text. It grows a decision list from those seeds, guided by two regularities:
- One sense per discourse: in hand-tagged data, repeated occurrences of a word in one document kept the majority sense about 99.8% of the time.
- One sense per collocation: nearby words are strong, consistent signals of sense.
With the discourse constraint added, the method reached 96.5%. That edged past the 96.1% of a supervised decision list (Yarowsky 1995).
In 1997, Resnik and Yarowsky criticised the lack of shared evaluation. They proposed probability-sensitive scoring and inventories limited to distinctions that languages lexicalise differently. These ideas shaped the later shared tasks (Resnik & Yarowsky 1997).
The Senseval campaigns, later renamed SemEval, then provided common data:
| Benchmark | Representative result |
|---|---|
| Senseval-1/2 lexical sample | Linear SVM with four knowledge sources: 79.2% recall on Senseval-1, 65.4% on Senseval-2 (Lee & Ng 2002) |
| Senseval-3 all-words | First-sense baseline of 62.40%; only 4 of 26 systems beat it (Ciaramita & Altun 2006) |
| SemEval-2010 induction | 100 target words; metrics disagreed on rankings (Manandhar et al. 2010) |
Knowledge-based work during the same period moved to graphs over WordNet. One example is PageRank over sense graphs (Mihalcea et al. 2004). Another is sequence labeling with gloss-overlap edges, which reached 54.2% on Senseval-2 (Mihalcea 2005). Navigli and Lapata compared connectivity measures and found local measures such as KPP (Key Player Problem) most reliable (Navigli & Lapata 2007). Personalized PageRank over the full WordNet graph, released as UKB, also belongs to this line (Agirre et al. 2014).
How it is used today
Neural supervised systems now dominate English all-words WSD. They are trained mainly on SemCor, a part of the Brown Corpus hand-tagged with WordNet senses (Basile et al. 2014). They are evaluated in a unified framework that merges the Senseval and SemEval datasets (Raganato et al. 2017). Scores are usually given as F1, the harmonic mean of precision and recall. The leading systems combine contextual encoders with knowledge from the inventory:
- GlossBERT pairs each context with every candidate gloss as a sentence-pair task. It reached 77.0 F1 on the combined test sets (Huang et al. 2019).
- LMMS spreads BERT-derived sense vectors through WordNet to cover all of its senses, so a nearest-neighbour match works without a most-frequent-sense fallback (Loureiro & Jorge 2019).
- EWISER propagates scores along WordNet edges and was the first to exceed 80 F1 on the combined benchmark (Bevilacqua & Navigli 2020).
- A gloss-informed bi-encoder was evaluated on the same English all-words benchmark. On less frequent senses, it cut error by 31.1% relative to prior work, which is 15.6 F1 above the nearest rival (Blevins & Zettlemoyer 2020).
Knowledge-based methods remain useful where annotated data are lacking, because they need only a wordnet (Agirre et al. 2014). They also extend naturally to joint entity linking and disambiguation (Moro et al. 2014).
In translation, WSD helped once senses were redefined as target phrases learned from aligned corpora. This idea was developed in word-translation work (Vickrey et al. 2005) and then in phrase-based and hierarchical systems (Carpuat & Wu 2007; Chan et al. 2007). Parallel text can also replace manual sense tagging as a source of training data (Ng et al. 2003). For German→English and German→French neural translation, the ContraWSD contrastive test found baseline accuracy of about 70%, with weak handling of rare senses (Gonzales et al. 2017). Induced senses are also used to cluster and diversify web search results (Marco & Navigli 2013).
Open questions
Granularity remains unresolved. If fine WordNet distinctions cap human agreement, it is unclear what the right inventory is for a given application (Navigli 2006).
The long tail is a second problem. The bi-encoder still scores more than 40 F1 points higher on most-frequent-sense examples than on less frequent ones (Blevins & Zettlemoyer 2020).
Evaluation is still debated. Exact-match accuracy ignores calibrated uncertainty (Resnik & Yarowsky 1997), and conclusions about sense induction depend heavily on the chosen metric (Manandhar et al. 2010).
Extrinsic value is uncertain. Early studies found mixed gains in information retrieval (Ide & Véronis 1998). In translation, significant gains came only after senses were recast as translations (Carpuat & Wu 2007; Chan et al. 2007). Sense-aware neural systems improved only modestly and left BLEU essentially unchanged (Gonzales et al. 2017).
Coverage is a further concern. Fixed inventories miss newly emerging senses, which induction methods can help detect (Lau et al. 2012). Many languages also lack frequency-ordered sense lists, so the first-sense baseline is far weaker there than in English (Agirre et al. 2014).
Key papers
Related concepts
- Lexical semantics — Lexical semantics is the study of word meaning. It asks what words and multiword expressions denote, how a word's senses are organized, how…
- Semantic similarity — Semantic similarity is a graded measure of how close two linguistic units, such as words, concepts, sentences or documents, are in meaning.…
- Word embeddings — Word embeddings are dense, real-valued vectors that represent the words of a vocabulary as points in a shared geometric space. They are…
- Semantic role labeling — Semantic role labeling is the task of finding, for each predicate in a sentence, the phrases that act as its participants and labeling each…
More papers
Topics: 🔀 Word sense disambiguation · 📖 Lexical semantics & word embeddings
Shelves: 🗣️ Linguistics & languages · 💬 Natural language processing
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/word-sense-disambiguation/