Elamur Research⌕
☰
home / papers / 2017 / Unsupervised Induction of Compositional Types for English Adjective-Noun Pairs

Unsupervised Induction of Compositional Types for English Adjective-Noun Pairs

Paper · 2017 · Wiebke Petersen and Oliver Hellwig

Summary

The paper asks whether adjectives can be grouped by how they modify nouns, using only co-occurrence statistics and no linguistic theory. The authors take a frame-based view: a noun is a structure of attributes and values. An adjective either fills a value (for example, COLOR in "red ball") or adds new attributes and constraints (relational or modal adjectives such as "fake meat"). They extracted about one million adjective–noun pairs from parsed 2013 English news text. On this data they compared four unsupervised clusterers: a k-means baseline, Rooth-LDA, Lex-LDA, and their own bidirectional clustering (BidirClus). Each was scored against HeiPLAS, a gold set of attribute–adjective–noun triples, using RI and ARI. All three specialised models clearly beat k-means. BidirClus came out on top among the truly unsupervised systems, because the LDA models had priors tuned on HeiPLAS itself. A manual review found that many apparent "errors" were sensible frame-level groupings that WordNet-derived HeiPLAS splits too finely. This motivates a dedicated frame-semantic gold standard.

Key points

  • Corpus: the 2013 statmt.org English news dump, parsed with the Stanford CoreNLP dependency parser.
    • JJ followed by NN/NE/NNS lemma sequences gave 1,048,653 A+N pairs.
    • These cover 8,392 adjectives, 17,560 nouns and 430,256 distinct combinations, with a matrix density of about 0.0029.
  • Gold standard: HeiPLAS (Hartung, 2015), built from WordNet attributes.
    • It has 1,598 triples with 849 adjectives, 923 nouns and 253 attributes.
    • Attributes serve as class labels, but the set covers only value-specifying "basic" adjectives.
  • Model setup:
    • Rooth-LDA and Lex-LDA use k = 250 topics, a number chosen with the gap statistic on GloVe adjective vectors.
    • Their priors were grid-searched to maximise ARI on HeiPLAS, which leaks supervision into the models.
    • Re-runs with uninformed priors of 0.01 scored slightly lower than the tuned versions and fell below BidirClus.
  • BidirClus:
    • It alternately clusters adjectives (by their nouns) and nouns (by their adjectives) in a VSM.
    • Distance combines Jaccard distance with an LDA (k = 15) topic term.
    • Each round merges the top 5% most similar rows via transitive closure, using majority-vote cluster vectors.
  • Error analysis:
    • 32 adjective pairs were judged correct from a frame view but split by HeiPLAS, such as exceptional/extraordinary and current/future.
    • The main weakness of BidirClus is that its clusters are disjoint, so it cannot handle polysemy. For example, "right" ends up with "left" rather than with "appropriate".

Interesting ideas

  • I1. When hyperparameters are tuned on the evaluation gold, rerun with neutral priors to measure how much of the score comes from that leakage.
  • I2. Before blaming the model, look at the cases where it disagrees with the gold. Some "errors" turn out to be reasonable coarser groupings that a lexicographically fine-grained gold splits apart.
  • I3. Cluster two interacting vocabularies in alternation, so that adjective classes sharpen noun classes and vice versa. This lets sparse co-occurrence data reinforce itself.
  • I4. Assign the latent class to each adjective–noun pair rather than to the adjective alone. A polysemous adjective can then belong to different classes depending on its noun ("red pen" has two possible attributes).
  • I5. Let a per-adjective parameter switch between a lexicalised (idiom-like) mode and a compositional mode. This separates fixed phrases from genuinely compositional modification.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Add an "uninformed settings" run as a standard negative control whenever a component's thresholds were tuned on the same test set it is scored on.
  • I2 (learn): Sort evaluation disagreements into "model wrong" versus "gold too fine for our purpose", especially where the gold comes from WordNet senses.
  • I3 (world): Induce coarse attribute/value-range groups for adjectives and matching noun types from parsed amod pairs, as an inspectable supplement to the WordNet-based ontology.
  • I4 (frames): Decide which noun attribute an adjective fills per amod edge, using the noun as context, instead of keeping one fixed attribute per adjective lemma.
  • I5 (world): Flag A+N pairs that behave as lexicalised units, so the world builder stores them as single entities rather than an attribute plus a value.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Wiebke Petersen and Oliver Hellwig. 2017. Unsupervised Induction of Compositional Types for English Adjective-Noun Pairs. Proceedings of the 12th International Conference on Computational Semantics (IWCS) –- Short papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{petersen2017unsupervised,
  title = {{Unsupervised Induction of Compositional Types for English Adjective-Noun Pairs}},
  author = {Wiebke Petersen and Oliver Hellwig},
  booktitle = {Proceedings of the 12th International Conference on Computational Semantics (IWCS) –- Short papers},
  year = {2017},
  url = {https://aclanthology.org/W17-6932/}
}

References

32 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anderson, C. and S. Löbner (2017). Roles and the lexical semantics of role-denoting relational adjectives. Abstract submitted to 12th International Tbilisi Symposium on Logic, Language, and Computation. find
  2. Baroni, M. and R. Zamparelli (2010). Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of EMNLP, Boston, pp. 1183–1193. find
  3. Blei, D. M., A. Y. Ng, and M. I. Jordan (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993–1022. ↗
  4. Bojar, O., C. Buck, C. Federmann, B. Haddow, P. Koehn, J. Leveling, C. Monz, P. Pecina, M. Post, H. Saint-Amand, R. Soricut, L. Specia, and A. Tamchyna (2014). Findings of the 2014 Workshop on Statistical Machine Translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 12–58.
  5. Boleda, G. (2007). Automatic Acquisition of Semantic Classes for Adjectives. Ph. D. thesis, Pompeu Fabra University. find
  6. Dima, C. (2016). On the compositionality and semantic interpretation of English noun compounds. In Proceedings of the 1st Workshop on Representation Learning for NLP, pp. 27–39. find
  7. Dima, C. and E. Hinrichs (2015). Automatic noun compound interpretation using deep neural networks and word embeddings. In Proceedings of the 11th International Conference on Computational Semantics, pp. 173–183. find
  8. Dixon, R. W. (2010). Where have all the adjectives gone? In Where have All the Adjectives Gone? And Other Essays in Semantics and Syntax, pp. 1–62. Berlin: De Gruyter Mouton. find
  9. Guevara, E. (2010). A regression model of adjective-noun compositionality in distributional semantics. In Proceedings of the 2010 Workshop on Geometrical Models of Natural Language Semantics, pp. 33–37.
  10. Hartung, M. (2015). Distributional Semantic Models of Attribute Meaning in Adjectives and Nouns. Ph. D. thesis, University of Heidelberg. ↗
  11. Hartung, M. and A. Frank (2011). Exploring supervised LDA models for assigning attributes to adjective-noun phrases. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 540–551. find
  12. Hartung, M., F. Kaupmann, S. Jebbara, and P. Cimiano (2017). Learning compositionality functions on word embeddings for modelling attribute meaning in adjective-noun phrases. In Proceedings of the 15th Meeting of the European Chapter of the ACL (EACL). find
  13. Hubert, L. and P. Arabie (1985). Comparing partitions. Journal of classification 2(1), 193–218. find
  14. Hundsnurscher, F. and J. Splett (1982). Semantik der Adjektive im Deutschen: Analyse der semantischen Relationen. Westdeutscher Verlag. find
  15. Löbner, S. (2014). Evidence for frames from human language. In T. Gamerschlag, D. Gerland, R. Osswald, and W. Petersen (Eds.), Frames and Concept Types, Volume 94 of Studies in Linguistics and Philosophy, pp. 23–67. Springer International Publishing. ↗
  16. Manning, C. D., M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky (2014). The Stanford CoreNLP natural language processing toolkit. In ACL System Demonstrations, pp. 55–60.
  17. Mikolov, T., I. Sutskever, K. Chen, G. S. Corrado, and J. Dean (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pp. 3111–3119. ↗
  18. Mitchell, J. and M. Lapata (2008). Vector-based models of semantic composition. In Proceedings of 46th Annual Meeting of the ACL, pp. 236–244. find
  19. Navigli, R. (2009). Word sense disambiguation: A survey. ACM Comput. Surv. 41, 10:1–10:69. find
  20. Pennington, J., R. Socher, and C. D. Manning (2014). GloVe: Global vectors for word representation. In Proceedings of the 2014 EMNLP, pp. 1532–1543.
  21. Petersen, W. (2015/2007). Representation of concepts as frames. In T. Gamerschlag, D. Gerland, R. Osswald, and W. Petersen (Eds.), Meaning, Frames, and Conceptual Representation, Volume 2 of Studies in Language and Cognition, pp. 43 – 67. Düsseldorf University Press. Reprint with comments. Originally published 2007 in The Baltic International Yearbook of Cognition, Logic and Communication, Vol. 2. find
  22. Petersen, W. and O. Hellwig (2016). Exploring the value space of attributes: Unsupervised bidirectional clustering of adjectives in German. In Proceedings of the COLING, pp. 2839–2848. find
  23. Pustejovsky, J. (1995). The Generative Lexicon. Cambridge, MA: MIT Press. ↗
  24. Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66, 846–850. ↗
  25. Raskin, V. and S. Nirenburg (1995). Lexical semantics of adjectives. New Mexico State University, Computing Research Laboratory Technical Report, MCCS-95-288. find
  26. Rooth, M., S. Riezler, D. Prescher, G. Carroll, and F. Beil (1999). Inducing a semantically annotated lexicon via EM-based clustering. In Proceedings of the 37th Annual Meeting of the ACL, pp. 104–111. Association for Computational Linguistics.
  27. Séaghdha, D. and A. Korhonen (2014). Probabilistic distributional semantics with latent variable models. Computational Linguistics 40(3), 587–631. ↗
  28. Séaghdha, D. O. (2010). Latent variable models of selectional preference. In Proceedings of the 48th Annual Meeting of the ACL, pp. 435–444.
  29. Socher, R., C. C. Lin, C. Manning, and A. Y. Ng (2011). Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning, pp. 129–136. find
  30. Tibshirani, R., G. Walther, and T. Hastie (2001). Estimating the number of data clusters via the Gap statistic. Journal of the Royal Statistical Society B 63, 411–423. find
  31. Tratz, S. and E. Hovy (2010). A taxonomy, dataset, and classifier for automatic noun compound interpretation. In Proceedings of the 48th Annual Meeting of the ACL, pp. 678–687. find
  32. Turney, P. D. and P. Pantel (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research 37(1), 141–188. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/petersen-2017-unsupervised-induction-compositional-types-english/