Elamur Research
☰
home / papers / 2003 / Supersense Tagging of Unknown Nouns in WordNet

Supersense Tagging of Unknown Nouns in WordNet

What it is about

The paper extends named-entity classification to common nouns. Each unknown noun is assigned one of 26 supersenses, the lexicographer classes that WordNet uses to organise its noun entries (person, location, event, quantity and so on). The classifier is an averaged multiclass perceptron over features from the context and the syntax, trained on about 787,000 noun instances from a 40-million-word parsed corpus. Nouns with more than one supersense were removed from training. Because the WordNet glosses and examples add about 66,800 training instances, the authors can test whether dictionary text helps. It does: the model trained on the glosses beats the same model without them on both test sets, and the paper also reports that a most-frequent-label baseline ("person") scores in the low twenties. The paper also argues for an evaluation on new nouns, not cross-validation.

Ideas

For Mova

The touch points are onto (WordNet lexicographer files), en/src/ptag.rs (averaged perceptron, the same learner family) and en/src/tok.rs for tokens. The chapter is languages/en/11-nouns.md. The paper's corpus and WordNet test sets are NC-tagged, so only the method and tests use them; the build can use WordNet itself (open) and EWT (CC BY-SA). Measure first the "person" baseline and the perceptron on the EWT unknown nouns.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W03-1022.pdf

Summary

The paper addresses nouns missing from WordNet. Placing them at the exact synset is very hard, so the authors aim for something coarser: predicting a noun's supersense, one of WordNet's 26 lexicographer classes (person, artifact, location, and so on). This extends named-entity classification from proper nouns to common nouns, using a richer label set.

Training instances come from the parsed Bllip corpus, restricted to unambiguous nouns. Each instance is described by collocation, spelling and syntactic-context features and fed to an averaged multiclass perceptron. The authors also turn WordNet's own glosses (definitions and example sentences) into extra labelled training data.

For evaluation, they train on WordNet 1.6 and test on nouns that first appear in WordNet 1.71, and compare this with a cross-validation-style test. Adding the gloss data improves accuracy on both test sets, with statistically significant gains. It helps more than adding a similar amount of corpus data.

This matters because it shows a dictionary can serve as its own source of annotated data. It also gives a realistic, reproducible benchmark for lexical acquisition.

Key points

  • Label set: the 26 WordNet noun lexicographer classes. Among new WordNet 1.71 nouns, the three most frequent classes cover just over half of tokens, and nine cover 90%. Common nouns are therefore spread over classes far more evenly than named entities.

  • Scale of the problem: in Bllip (40M words, Wall Street Journal), nouns unknown to WordNet 1.6 have a relative frequency of about 0.0054. That is roughly one unknown noun every eight sentences.

  • Training data:

    • 787,186 instances from Bllip, after removing ambiguous nouns (72% of tokens, 28.9% of types).
    • 66,841 instances from WordNet definitions.
    • 6,147 instances from WordNet example sentences.
    • About 400K of the Bllip instances were sampled because of memory limits.

  • Model: an averaged multiclass perceptron (Crammer and Singer style, averaged as in Collins 2002). Training stopped at about 10 epochs, with the stopping point tuned on the other test set. A word type's label is the majority vote over the predictions for its tokens.

  • Results (token / type accuracy):

    ModelTest 1.71Test 1.6
    Most-frequent-label baseline20.0 / 27.824.1 / 21.0
    AP-B-5535.9 / 50.747.4 / 47.7
    AP-B-6536.1 / 50.847.9 / 48.3
    AP-B-55+WN36.9 / 52.952.3 / 53.4

    AP-B-55 and AP-B-65 are trained on 55% and 65% of the Bllip data; AP-B-55+WN adds the WordNet gloss data to the 55% set.

Interesting ideas

  • Treat dictionary glosses as annotated corpus: each definition or example sentence becomes a training instance labelled with its synset's coarse class. A single gloss says little, but evidence piles up at the coarse-label level. This could apply to any language with a WordNet-like resource or an explanatory dictionary, for example to bootstrap Ukrainian semantic taggers.
  • Version-based evaluation: train on an older release of a resource and test on entries added in a newer one. This simulates real lexical acquisition better than random cross-validation. It is also easy to standardize when the label inventory stays fixed across releases.
  • Classify at a coarse but natural granularity: predicting roughly 26 established classes instead of tens of thousands of synsets keeps the task learnable. It also reuses a taxonomy built by lexicographers rather than an ad hoc one. This is a useful middle ground for extending any large hierarchy.
  • Clean, sense-specific data can beat more data: the gloss data outperformed comparable or larger amounts of corpus data. The authors attribute this to two things: it is less noisy, and it covers polysemous words that had to be dropped from the corpus data. This suggests investing in disambiguated or pertinent examples before scaling raw volume.
  • Type-level decisions by token voting: each occurrence is classified separately, and the word gets the majority label. The authors report this worked better than merging all contexts into one aggregate pattern per word. It is a simple design worth trying for lexicon induction.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1119355.1119377

@inproceedings{ciaramita2003supersense,
  title = {{Supersense Tagging of Unknown Nouns in WordNet}},
  author = {Ciaramita, Massimiliano and Johnson, Mark},
  booktitle = {Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing},
  year = {2003},
  doi = {10.3115/1119355.1119377},
  url = {https://doi.org/10.3115/1119355.1119377}
}

References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. BLLIP. 2000. 1987-1989 WSJ Corpus Release 1. Linguistic Data Consortium. find
  2. E. Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics. find
  3. M. Ciaramita. 2002. Boosting Automatic Lexical Acquisition with Morphological Information. In Proceedings of the Workshop on Unsupervised Lexical Acquisition, ACL-02. find
  4. M. Collins and Y. Singer. 1999. Unsupervised Models for Named Entity Classification. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. find
  5. M. Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-02), pages 1–8. find our card
  6. K. Crammer and Y. Singer. 2002. Ultraconservative Online Algorithms for Multiclass Problems. Technical Report [2001-18], School of Computer Science and Engineering, Hebrew University, Jerusalem, Israel. find
  7. C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. find
  8. Y. Freund and R. Schapire. 1999. Large Margin Classification Using the Perceptron Algorithm. Machine Learning, 37. find
  9. R. Granger. 1977. FOUL-UP: A Program that Figures Out Meanings of Words from Context. In Proceedings of the Fifth International Joint Conference on Artificial Intelligence. find
  10. M. Hearst. 1992. Automatic Acquisition of Hyponyms from Large Text Corpora. In Proceedings of the 14th International Conference on Computational Linguistics. find
  11. M. Pasca and S.H. Harabagiu. 2001. The Informative Role of WordNet in Open-Domain Question Answering. In NAACL 2001 Workshop on WordNet and Other Lexical Resources: Applications, Extensions and Customizations. find
  12. J. Pustejovsky, A. Rumshisky, and J. Casta È Ç o. 2002. Rerendering Semantic Ontologies: Automatic Extensions to UMLS through Corpus Analytics. In In Proceedings of REC 2002 Workshop on Ontologies and Lexical Knowledge Bases. find
  13. E. Riloff and R. Jones. 1999. Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping. In Proceedings of the Sixteenth National Conference on Artificial Intelligence. find
  14. B. Roark and E. Charniak. 1998. Noun-Phrase Co-Occurrence Statistics for Semi-Automatic Semantic Lexicon Construction. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics. find
  15. U. Zernik. 1991. Introduction. In U. Zernik, editor, Lexical Acquisition: Exploiting On-line Resources to Build a Lexicon. Lawrence Erlbaum Associates. find