Elamur Research⌕
☰
home / papers / 1999 / Inducing a Semantically Annotated Lexicon via EM-Based Clustering

Inducing a Semantically Annotated Lexicon via EM-Based Clustering

Paper · 1999 · Mats Rooth, Stefan Riezler, Detlef Prescher et al.

Summary

Most earlier work on learning selectional restrictions depends on a fixed semantic taxonomy such as an entailment hierarchy. Such resources exist for only a few languages, and their suitability for describing word meaning is debatable. Rooth et al. instead derive semantic slot labels for subcategorization frames directly from unannotated text. They parse the British National Corpus using Carroll and Rooth's (1998) probabilistic grammar, a context-free one lexicalized on heads, and extract verb–noun pairs for subject and object relations. They then fit a latent class model with EM, in which a hidden class jointly generates a verb-slot and a noun. A second EM step assigns class labels to the slots of whole frames. The induced classes are often semantically coherent, for example scalar change or communicative action. The models are tested on a pseudo-disambiguation task. The authors also show how the labels can serve as relation constants in decompositional lexical entries, which captures alternations such as transitive versus intransitive increase. The method needs no hand-built hierarchy, so it can be applied to any language that has a large corpus, morphology tools and a robust parser.

Key points

  • Training data: 1,280,715 verb–noun pair tokens (608,850 types), taken from maximal-probability parses of the British National Corpus (117 million words).
  • Model: p(v,n) is a sum over classes c of p(c)·p(v|c)·p(n|c). Each "verb" is really a verb paired with a slot: as:s (intransitive subject), aso:s (transitive subject) or aso:o (transitive object).
  • Example classes: in a 35-class model, class 17 groups scalar-change verbs (increase, fall, rise, reduce) with things that move along scales. Class 5 links communicative-action verbs with human agents.
  • Evaluation setup:
    • 3,000 pairs were held out, leaving 1,178,698 training tokens.
    • Each held-out noun was paired with a frequency-sampled verb that never occurs with it, giving 1,337 test triples.
    • Only words with frequency between 30 and 3,000 were kept.
    • 3 × 10 × 6 models were compared, varying starting values, number of classes and number of iterations.
  • Linguistic outcome: transitive and intransitive increase share the component R17 ∧ increase17. This means the diathesis alternation was learned. The authors expect a near-complete lexicon to need a few hundred classes.

Interesting ideas

  • I1. Slot-specific verb units: Clustering a verb together with its grammatical slot, rather than the bare verb, lets one model capture how different argument positions select different noun types. The same trick could suit Ukrainian, where case marking makes slots easy to identify.
  • I2. Pseudo-disambiguation as cheap evaluation: The model must pick the real verb over a never-seen distractor verb for a held-out noun, so no gold annotation is needed. This is a reusable way to test any selectional-preference or embedding model.
  • I3. Frequency filters on test items: Restricting test words to a middle frequency band keeps the test away from both sparse noise and trivially frequent words. This is a useful design choice for any lexical benchmark built automatically.
  • I4. Sense-indexed lexical roots: Tagging a word's root with its induced class index keeps homonyms apart, such as decline a proposal versus decline five percent. Related uses of one sense still share structure, which could help any sense-aware lexicon.
  • I5. Probabilistic outputs for reuse: Because the classes come from a simple probabilistic model, the learned entries can be plugged into head-lexicalized parsers or language models. This makes the induced lexicon a component that other systems can use, not just a stand-alone resource.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of StuttgartStuttgart · Germany75 works · 46,113 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 works

Cite

Cite as Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll and Franz Beil. 1999. Inducing a Semantically Annotated Lexicon via EM-Based Clustering. Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1034678.1034703

@inproceedings{rooth1999inducing,
  title = {{Inducing a Semantically Annotated Lexicon via EM-Based Clustering}},
  author = {Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll and Franz Beil},
  booktitle = {Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics},
  year = {1999},
  doi = {10.3115/1034678.1034703},
  url = {https://doi.org/10.3115/1034678.1034703}
}

References

17 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. 1970. A maximization technique occuring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics, 41(1):164-171. find
  2. Franz Beil, Glenn Carroll, Detlef Prescher, Stefan Riezler, and Mats Rooth. 1999. Inside-outside estimation of a lexicalized PCFG for German. In Proceedings of the 37th Annual Meeting of the A CL, Maryland. find
  3. Glenn Carroll and Mats Rooth. 1998. Valence induction with a head-lexicalized PCFG. In Proceedings of EMNLP-3, Granada. find
  4. Ido Dagan, Lillian Lee, and Fernando Pereira. to appear. Similarity-based models of word cooccurence probabilities. Machine Learning. find
  5. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(B):1-38. find
  6. Jerry A. Fodor. 1998. Concepts : Where Cogni-tire Science Went Wrong. Oxford Cognitive Science Series, Oxford. find
  7. K. Hale and S.J. Keyser. 1993. Argument structure and the lexical expression of syntactic relations. In K. Hale and S.J. Keyser, editors, The View from Building 20. MIT Press, Cambridge, MA. find
  8. Thomas Hofmann and Jan Puzicha. 1998. Unsupervised learning from dyadic data. Technical Report TR-98-042, International Computer Science Insitute, Berkeley, CA. find
  9. Murat Kural. 1996. Verb Incorporation and Elementary Predicates. Ph.D. thesis, University of California, Los Angeles. find
  10. Beth Levin. 1993. English Verb Classes and Alternations. A Preliminary Investigation. The University of Chicago Press, Chicago/London. find
  11. Fernando Pereira, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of english words. In Proceedings of the 31th Annual Meeting of the A CL, Columbus, Ohio.
  12. Philip Resnik. 1993. Selection and information: A class-bases approach to lexical relationships. Ph.D. thesis, University of Pennsylvania, CIS Department. find
  13. Francecso Ribas. 1994. An experiment on learning appropriate selectional restrictions from a parsed corpus. In Proceedings of COLING-9~, Kyoto, Japan. find
  14. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1998. EM-based clustering for NLP applications. In Inducing Lexicons with the EM Algorithm. AIMS Report 4(3), Institut fiir Maschinelle Sprachverarbeitung, Universit~t Stuttgart. find
  15. Mats Rooth. Ms. Two-dimensional clusters in grammatical relations. In Symposium on Representation and Acquisition of Lexical Knowledge: Polysemy, Ambiguity, and Generativity. AAAI 1995 Spring Symposium Series, Stanford University. find
  16. Lawrence K. Saul and Fernando Pereira. 1997. Aggregate and mixed-order Markov models for statistical language processing. In Proceedings of EMNLP-2. find
  17. Helmut Schuhmacher. 1986. Verben in Feldern. Valenzw5rterbuch zur Syntax und Semantik deutscher Verben. de Gruyter, Berlin. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rooth-1999-inducing-semantically-annotated-lexicon-em/