Elamur Research⌕
☰
home / papers / 2018 / Extracting Commonsense Properties from Embeddings with Limited Human Guidance

Extracting Commonsense Properties from Embeddings with Limited Human Guidance

Paper · 2018 · Yiben Yang, Larry Birnbaum, Ji-Ping Wang et al.

Summary

The paper addresses how to learn commonsense comparisons between objects along one property, such as whether one thing is heavier or rounder than another. This knowledge is rarely stated directly in text, and earlier methods relied on many hand-annotated frames for each property. The authors propose PCE (Property Comparison from Embeddings). It maps pre-trained embeddings of two objects into a space defined by label words: an adjective and its antonym for the two poles, plus "similar". This design lets the model handle properties it never saw during training (zero-shot). A four-way variant adds a "not applicable" output to detect nonsensical comparisons. The authors also introduce an active learning strategy that synthesizes uncertain queries for annotators. On VERB PHYSICS, PCE outperforms the earlier F&C system while using far less annotation. They also release a new dataset, PROPERTY COMMON SENSE. The work shows that embeddings plus minimal supervision can produce useful commonsense knowledge at low annotation cost.

Key points

  • On VERB PHYSICS, PCE with Google 1B LSTM embeddings reaches 0.76 overall test accuracy, compared with 0.70 for F&C. F&C needs roughly 188 frame labels per property; PCE needs none. GloVe and Word2vec embeddings were also tested.
  • In zero-shot evaluation (one property held out), PCE beats random and Emb-Similarity baselines. Using both poles matters: on speed, two-pole PCE scores 0.58 versus 0.32 for the one-pole variant.
  • PROPERTY COMMON SENSE covers 32 properties, 689 objects (drawn from the McRae Feature Norms plus named entities), and 3148 labeled triples. Inter-annotator agreement is Cohen's Kappa 0.64, with disagreements mostly about "similar" and N/A judgments.
  • On the four-way task, PCE achieves 0.67 accuracy with Word2vec or LSTM embeddings, compared with 0.51 for the majority-class baseline.
  • Synthesis-based active learning outperforms random, least-confident, and expected-model-change baselines. Code and data are available at github.com/yangyiben/PCE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Northwestern UniversityEvanston · United States49 works · 3,654 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksComputer Vision and Pattern Recognition · 3,621 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Yiben Yang, Larry Birnbaum, Ji-Ping Wang and Doug Downey. 2018. Extracting Commonsense Properties from Embeddings with Limited Human Guidance. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-2102

@inproceedings{yang2018extracting,
  title = {{Extracting Commonsense Properties from Embeddings with Limited Human Guidance}},
  author = {Yiben Yang, Larry Birnbaum, Ji-Ping Wang and Doug Downey},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2102},
  url = {https://doi.org/10.18653/v1/p18-2102}
}

References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Benjamin Van Durme and Lenhart Schubert. 2008. Open knowledge extraction through compositional language processing. In Proceedings of the 2008 Conference on Semantics in Text Processing, pages 239–254. Association for Computational Linguistics. find
  2. Eric B Baum and Kenneth Lang. 1992. Query learning can work poorly when a human oracle is used. In International joint conference on neural networks, volume 8, page 8. find
  3. W. Cai, Y. Zhang, and J. Zhou. 2013. Maximizing expected model change for active learning in regression. In 2013 IEEE 13th International Conference on Data Mining, pages 51–60. find
  4. Lucy Vanderwende. 2005. Volunteers created the web. In AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors, pages 84–90. find
  5. Aron Culotta and Andrew McCallum. 2005. Reducing labeling effort for structured prediction tasks. In AAAI, pages 746–751. AAAI Press / The MIT Press. find
  6. Maxwell Forbes and Yejin Choi. 2017. Verb physics: Relative physical knowledge of actions and objects. arXiv preprint arXiv:1706.03799. find
  7. Jonathan Gordon and Benjamin Van Durme. 2013. Reporting bias and knowledge acquisition. In Proceedings of the 2013 workshop on Automated knowledge base construction, pages 25–30. ACM. find
  8. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410. find
  9. Ken McRae, George S. Cree, Mark S. Seidenberg, and Chris Mcnorgan. 2005. Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37(4):547–559. find
  10. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pa…
  11. Mark Palatucci, Dean Pomerleau, Geoffrey E Hinton, and Tom M Mitchell. 2009. Zero-shot learning with semantic output codes. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1410–1418. Curran Associate… find
  12. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543.
  13. Lenhart Schubert. 2002. Can we derive general world knowledge from texts? In Proceedings of the second international conference on Human Language Technology Research, pages 94–97. Morgan Kaufmann Publishers Inc. find
  14. B. Settles. 2009. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2018-extracting-commonsense-properties-embeddings-limited/