📖 Lexical semantics
Lexical semantics is the study of word meaning. It asks what words and multiword expressions denote, how a word's senses are organized, how words relate to one another through synonymy, hypernymy, antonymy or meronymy, and how meaning shifts with context and over time.
What it is
Lexical semantics is the branch of semantics that takes the word, rather than the sentence or discourse, as its unit of analysis. It deals with several recurring problems:
- Polysemy is a single form carrying several related senses.
- Homonymy is a single form carrying unrelated meanings.
- Near-synonymy covers words that share a core sense but differ in nuance.
- Hyponymy and hypernymy link specific terms to general ones.
- Antonymy links opposites, and meronymy links parts to wholes.
- Meaning varies across contexts and changes across historical periods.
In computational work the field has two complementary traditions. One builds explicit lexical resources, in which senses and relations are written down by lexicographers. The other induces meaning from patterns of word use in large corpora.
Similarity, as in car and automobile, is a narrower notion than relatedness. Relatedness also takes in part–whole links, opposites and loose functional or associative ties. Treating the two as interchangeable has misled evaluation (Budanitsky & Hirst 2006).
Near-synonyms such as error, mistake, blunder and slip show a second difficulty. They differ in denotation, implication, attitude and style, so a single "sense" label is too coarse to choose between them (Edmonds & Hirst 2002).
The key idea
The central working assumption is that meaning can be captured as structure. In computational work, two forms of structure dominate.
In the relational view, a word sense is a node in a network. In WordNet, a hand-built English lexical database, sets of synonymous senses (synsets) are linked by hypernymy, meronymy, antonymy and other pointers (Miller 1994). Closeness of meaning is then read off the graph, for instance through path length or the information content of shared ancestors.
In the distributional view, which goes back to Harris (1954) and Firth (1957), a word is represented by the contexts it occurs in. Two words are similar to the degree that their context vectors point in the same direction:
sim(u, v) = cos(u, v) = Σᵢ uᵢ·vᵢ / ( √(Σᵢ uᵢ²) · √(Σᵢ vᵢ²) )
Here u and v are the vectors for two words. Each component uᵢ or vᵢ is a weighted co-occurrence count, or a learned embedding coordinate, for context dimension i. The sum Σᵢ runs over all dimensions. The denominator divides by the two vector lengths, so the score reflects the direction of the vectors rather than their length (magnitude). This normalisation does not remove every frequency effect (see Open questions).
The same quantity, applied to one word's vectors from two time periods, becomes a measure of semantic change. The spaces for the periods must first be aligned, for example with orthogonal Procrustes (Hamilton et al. 2016). One minus the cosine between a word's aligned vectors in consecutive decades is the standard "global" change score. The same authors also propose a local alternative that tracks changes in a word's nearest-neighbour similarities (Hamilton et al. 2016).
The two views are complementary. On the RG and WordSim-353 benchmarks, two methods stood out. A personalized-PageRank method over the WordNet graph was the best of the resource-based methods, and a Web-scale distributional method was the best of the corpus-based ones. A supervised combination of the two outperformed either alone (Agirre et al. 2009).
History and founding papers
Linguistic lexical semantics has long roots, including structuralist field theory, Cruse (1986) on lexical relations, Fillmore's frame semantics and Pustejovsky (1995) on the generative lexicon.
One major strand of computational work took shape around WordNet, described above (Miller 1994). Its format was reused for German in GermaNet, which was built from scratch rather than translated. GermaNet added features such as a pointer for regular polysemy (animal/meat) and flagged artificial nodes (Hamp & Feldweg 1997). Later work linked many wordnets into an open multilingual network (Bond & Foster 2013).
WordNet was soon enriched with further layers. One layer added subject-field codes, labelling about 96% of WordNet 1.6 noun synsets with 115 domain codes (Magnini & Cavaglià 2000). Another added affective labels, covering 2,874 synsets and 4,787 words (Strapparava & Valitutti 2004). Frame-semantic annotation of German in the SALSA corpus had a first release planned for mid-2006. It held roughly 20,000 annotated predicate instances, drawn from about 500 verbs plus some deverbal nouns (Burchardt et al. 2006).
In parallel, corpus methods appeared. Lexical co-occurrence statistics were used to resolve structural ambiguity in parsing (Hindle & Rooth 1993). Monosemous WordNet relatives served as a source of automatically harvested sense-training data (Leacock et al. 1998). Noun taxonomies with hypernym labels were induced from raw Wall Street Journal text, yielding 20,014 nouns under 654 internal nodes (Caraballo 1999). Unknown nouns were assigned to WordNet's 26 noun supersense classes (Ciaramita & Johnson 2003).
Budanitsky and Hirst compared five WordNet-based relatedness measures on malapropism detection. They found the Jiang–Conrath information-content measure the strongest (Budanitsky & Hirst 2006).
How it is used today
Context-sensitive representation is now central. Single-vector models average over all uses of a word. Structured vector spaces instead combine a word's vector with the selectional preferences of its syntactic neighbours (Erk & Padó 2008). Multi-prototype models cluster each word's occurrences into several sense-like vectors (Reisinger & Mooney 2010).
A theoretical account holds that ordinary embeddings already store a polysemous word as a linear mixture of sense vectors. That mixture can be recovered by sparse coding into roughly 2,000 shared "discourse atoms" (Arora et al. 2018).
Pretrained language models extend this line of work. One approach starts from BERT contextual embeddings of the sense-annotated examples in SemCor. It then propagates them through WordNet's structure to obtain sense vectors for all 117,659 synsets of WordNet 3.0. These vectors support full-coverage disambiguation by simple nearest-neighbour matching (Loureiro & Jorge 2019).
Probing across six languages tells a mixed story. Static vectors distilled from monolingual BERT match or beat fastText on lexical similarity, word analogy and relation prediction, though the best extraction setup differs by language and task. Lexical knowledge sits mostly in the lower layers (Vulić et al. 2020).
Relation extraction remains active. Supervised classifiers led to a "selective" version of the distributional inclusion hypothesis. It holds that a hypernym's contexts include a hyponym's only over relevant dimensions (Roller et al. 2014). HypeNET integrated LSTM-encoded dependency paths with distributional vectors and gained up to 14 F1 points over either signal alone (Shwartz et al. 2016). Word-class lattices extract definitions and their genus terms from Web text (Navigli & Velardi 2010).
Lexical units larger than one token are handled by general-purpose multiword-expression taggers. Trained on English Web Treebank reviews annotated for MWEs, including gappy ones, such a tagger reaches close to 60% F1 under a matching-based measure proposed by its authors (Schneider et al. 2014).
Lexical semantic knowledge also feeds downstream systems. In coreference resolution, for example, WordNet and Wikipedia features mainly improved the resolution of common nouns (Ponzetto & Strube 2006).
Diachronic lexical semantics has grown into a distinct subfield. An early distributional study compared 1960s and 1990s vectors from Google Books bigrams by cosine. This score correlated with human change ratings at 0.386, against 0.301 for a frequency signal (Gulordava & Baroni 2011).
Aligned decade-level embeddings later suggested two statistical regularities. First, frequent words drift more slowly. Second, once frequency is controlled for, polysemous words change faster. Polysemy was approximated by a word's clustering coefficient in a co-occurrence network, a measure of contextual diversity rather than a count of senses (Hamilton et al. 2016). Control-corpus experiments later attributed this second regularity largely to frequency (Dubossarsky et al. 2017).
Clustering BERT usage vectors traces narrowing and broadening without any sense inventory (Giulianelli et al. 2020).
Evaluation of this subfield rests on human judgements of how related two uses of a word are. The DURel framework (Schlechtweg et al. 2018) collects such judgements. They underlie the SemEval-2020 benchmark for four languages, to which 33 teams submitted 186 systems (Schlechtweg et al. 2020).
Open questions
What counts as a sense? Induced clusters do not line up with lexicographic senses. In one multi-prototype model they came out finer than WordNet's senses and included implausible groupings, although the number of clusters there is a chosen parameter (Reisinger & Mooney 2010). Usage-based annotation avoids fixing senses at all (Schlechtweg et al. 2018). It remains unsettled whether discrete inventories or graded similarity should be the primary object of study.
Coverage of relations is a second gap. In Morris's reading experiments, about 60% of the lexical relations perceived by readers were "non-classical." Some relatedness is built ad hoc in context, and fixed resources do not capture it (Budanitsky & Hirst 2006).
Measurement artefacts form a third problem. Matched control corpora reproduce much of the apparent link between frequency, polysemy and change. This suggests that cosine-based change scores are inherently biased by frequency (Dubossarsky et al. 2017). On the shared change benchmark, static type embeddings beat contextual ones but were strongly frequency-biased. Some systems' predictions correlated with frequency above 0.8 (Schlechtweg et al. 2020). Separating cultural shift from regular linguistic drift is still only partly solved (Hamilton et al. 2016).
Finally, it is unclear how much explicit lexical knowledge large language models actually contain. The best ways of extracting it vary by language and task (Vulić et al. 2020). Whether such models can replace curated resources, or will keep needing them as anchors, is an open empirical question.
Key papers
Related concepts
- Word sense disambiguation — Word sense disambiguation (WSD) identifies which meaning of an ambiguous word is intended in a given context. It is usually framed as…
- Distributional semantics — Distributional semantics is the study and modelling of meaning through the statistics of linguistic context. Words, phrases or sentences…
- Semantic similarity — Semantic similarity is a graded measure of how close two linguistic units, such as words, concepts, sentences or documents, are in meaning.…
- Lexicography — Lexicography is the discipline of compiling, describing and designing dictionaries and related lexical resources. It covers both the…
More papers
Topics: 📖 Lexical semantics & word embeddings · 🔀 Word sense disambiguation
Shelves: 🗣️ Linguistics & languages
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/lexical-semantics/