📄 SlangNet: A WordNet like resource for English Slang

Summary

Tools built on standard lexicons such as Princeton WordNet miss the meanings that internet users keep inventing. Some are entirely new words (swag, turnt). Others are new senses of old ones, such as "sick" meaning excellent. Urban Dictionary records many of these, but its unmoderated, crowd-written entries are redundant, opinionated and full of errors, so they cannot serve directly as a lexicon.

The authors build SlangNet, a slang resource that plugs into WordNet. They find candidate slang in a crawled Reddit corpus in two ways: frequent words that WordNet lacks, and known words whose Urban Dictionary tags match none of the WordNet glosses. Annotators then work in a dedicated tool, SAnE. For each item they either link it to an existing synset or write a new, formally worded synset with a POS tag and examples.

Once SlangNet is added to WordNet, unmodified Lesk and Extended Lesk (run through PyWSD) can select slang senses. The authors report better accuracy than earlier work that used raw Urban Dictionary definitions. The resource targets noisy web text in retrieval, web mining and, later, sentiment analysis.

Key points

• The bootstrap corpus is about 5.4 million Reddit comment sentences. The released resource holds roughly 3000 slang words, and the authors plan continuous crawling to keep it current.
• Urban Dictionary supplies the top-voted definition, examples and similar-word tags. Its scale is cited as 18 million unique readers and about seven million entries. Annotators assign POS manually because automatic tagging was unreliable on this noisy text. Acronyms like LOL or BTW receive a special ABR tag.
• Reddit validation covered 100 sentences each for cool, own and insane. The SlangNet sense accounted for 97%, 56% and 88% of uses respectively. Two annotators double-tagged 50 sentences for insane and reached a kappa of 0.84.
• WSD accuracy, simple Lesk / Extended Lesk:
  • cool: 0.15 / 0.17 (the authors blame its many fine-grained senses)
  • insane: 0.38 / 0.54
  • own: 0.45 / 0.51
• On the novel-sense dataset of Cook et al. (2014), simple Lesk scored 0.31 for follower, 0.19 for poster and 0.42 for text. Adapted Lesk was skipped because semantic relations were not yet added.

Interesting ideas

• I1. Corpus words that recur often but are absent from the reference lexicon are flagged as likely neologisms, so lexicon misses become a cheap candidate generator.
• I2. A new sense of a known word is detected when its related-word tags overlap with none of the existing sense glosses but do match the crowd definition. The mismatch itself is the signal.
• I3. Annotators first try to attach slang to an existing synset and create a new one only when nothing fits. This keeps the addition compatible, so WordNet-based algorithms run on it unchanged.
• I4. Before scoring the disambiguator, the authors measured how often the slang sense actually occurs in real text (97% for cool), which exposes how strong a trivial majority-sense baseline would be.
• I5. Items that resist normal POS assignment, such as chat acronyms, get their own explicit tag instead of being forced into an ill-fitting class.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Lemma lookups that fail repeatedly in the en crate could be counted and queued as reviewable "unknown word" candidates rather than silently treated as noise.
• I2 (world): When compiling the WordNet-based ontology, a contextual cue that fits no stored sense could trigger an explicit "possible unlisted sense" note instead of forcing the nearest synset.
• I3 (world): Domain or informal vocabulary could be added as an overlay on the compiled ontology, so verb classes, frames and lookups keep working without special cases.
• I4 (learn): Every sense or role evaluation could report the corpus sense distribution and a most-frequent-sense control alongside the system score, so gains are not illusory.
• I5 (tokens): A clearly marked category for LOL- or BTW-like tokens would let the tagger and downstream parser handle them deliberately and explainably, instead of mis-tagging them as nouns or interjections at random.