Elamur Research⌕
☰
home / papers / 2021 / Dimensions of Commonsense Knowledge

Dimensions of Commonsense Knowledge

Paper · 2021 · Filip Ilievski, Alessandro Oltramari, Kaixin Ma et al.

Summary

Commonsense knowledge is spread across many resources, such as ConceptNet, ATOMIC, WordNet, Wikidata and Visual Genome. These resources differ in format, purpose and vocabulary, which makes them hard to combine. Recent text-centric resources also tend to give up explicit semantics, so it is hard to tell what each one covers or how it helps downstream reasoning.

The authors survey a broad set of these sources, including commonsense KGs, general KGs, lexical and visual resources, and corpora/LMs, with a focus on their relations. Working bottom-up, they group these relations into 13 knowledge dimensions. They then use the dimensions to re-annotate the Commonsense Knowledge Graph (CSKG), which merges seven of the surveyed sources.

The enriched graph supports four analyses: per-source coverage, cross-source overlap, a comparison with LM-based edge clustering, and the effect of each dimension on zero-shot commonsense QA. The QA results show that temporal and desire/goal knowledge help most, while distinctness and lexical knowledge contribute little.

The work offers a shared vocabulary for aligning knowledge with tasks, and it exposes gaps in both current resources and current benchmarks.

Key points

  • Among the 13 dimensions are temporal, comparative, quality, utility, creation, spatial and part-whole knowledge, along with similarity, lexical, distinctness, desire/goal and relational-other. The enriched CSKG is publicly released.
  • Sources vary widely in how many relations they define. For example, ConceptNet has 34, ATOMIC has 9, Wikidata has about 6.7k, and Visual Genome has 42,374. Some sources cover many dimensions shallowly, while others cover few dimensions in depth.
  • Fact overlap between sources stays sparse even after relations are mapped to dimensions. This motivates automated node resolution across graphs.
  • Unsupervised LM-based edge clusters agree poorly overall with the dimension-based grouping. Some clusters, however, are dominated by one or two dimensions.
  • For the QA experiments, a language model is pre-trained on knowledge from each dimension and then evaluated zero-shot on two commonsense QA benchmarks. The results suggest that spatial and distinctness knowledge are under-represented in current evaluations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States633 works · 195,217 citations🎓University of Southern CaliforniaLos Angeles · United States183 works · 37,526 citations🎓Rensselaer Polytechnic InstituteTroy · United States24 works · 5,186 citations🏢Robert Bosch (United States)Farmington Hills · United States7 works · 5,006 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L. McGuinness and Pedro Szekely. 2021. Dimensions of Commonsense Knowledge. arXiv:2101.04640 (Knowledge-Based Systems 2021).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2101.04640 · arXiv 2101.04640

@misc{ilievski2021dimensions,
  title = {{Dimensions of Commonsense Knowledge}},
  author = {Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L. McGuinness and Pedro Szekely},
  year = {2021},
  doi = {10.48550/arxiv.2101.04640},
  eprint = {2101.04640},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.04640}
}

References

94 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aristotle, Jones, R., Ross, W., 2012. The Metaphysics. CreateSpace Independent Publishing Platform. URL: https://books.google.com/ books?id=-D3JtAEACAAJ . ↗
  2. Asher, N., Morreau, M., 1990. Commonsense entailment: A modal theory of nonmonotonic reasoning, in: European Workshop on Logics in Artificial Intelligence, Springer. pp. 1–30. find
  3. Baker, C.F., Fillmore, C.J., Lowe, J.B., 1998. The berkeley framenet project, in: 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pp. 86–90.
  4. Banerjee, P., Baral, C., 2020. Self-supervised knowledge triplet learning for zero-shot question answering, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online. pp. 151–162. URL: https://www.aclweb.org/… ↗
  5. Bhagavatula, C., Bras, R.L., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., tau Yih, W., Choi, Y., 2020. Abductive commonsense reasoning, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id=Byg1v1HKDB . ↗
  6. Bhakthavatsalam, S., Anastasiades, C., Clark, P., 2020a. Genericskb: A knowledge base of generic statements. arXiv preprint arXiv:2005.00660 . find
  7. Bhakthavatsalam, S., Richardson, K., Tandon, N., Clark, P., 2020b. Do dogs have whiskers? a new knowledge base of haspart relations. arXiv preprint arXiv:2006.07510 . find
  8. Bisk, Y., Zellers, R., LeBras, R., Gao, J., Choi, Y., 2020. Piqa: Reasoning about physical commonsense in natural language., in: AAAI, pp. 7432–7439.
  9. Boratko, M., Li, X.L., Das, R., O’Gorman, T.J., Le, D., McCallum, A., 2020. Protoqa: A question answering dataset for prototypical common-sense reasoning, in: EMNLP. find
  10. Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Çelikyilmaz, A., Choi, Y., 2019. Comet: Commonsense transformers for automatic knowledge graph construction, in: ACL.
  11. Brentano, F., 2014. Psychology from an empirical standpoint. Routledge. find
  12. Cambria, E., Li, Y., Xing, F.Z., Poria, S., Kwok, K., 2020. Senticnet 6: Ensemble application of symbolic and subsymbolic ai for sentiment analysis. Proceedings of the 29th ACM International Conference on Information & Knowledge Management . find
  13. Cambria, E., Song, Y., Wang, H., Howard, N., 2012. Semantic multidimensional scaling for open-domain sentiment analysis. IEEE intelligent systems 29, 44–51. find
  14. Casati, R., Varzi, A.C., et al., 1999. Parts and places: The structures of spatial representation. Mit Press. find
  15. Chen, J., Liu, J., 2011. Combining conceptnet and wordnet for word sense disambiguation, in: Proceedings of 5th International Joint Conference on Natural Language Processing, pp. 686–694. find
  16. Davison, J., Feldman, J., Rush, A., 2019. Commonsense knowledge mining from pretrained models, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computat…
  17. Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2019. BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long…
  18. Dodge, E.K., Hong, J., Stickles, E., 2015. Metanet: Deep semantic automatic metaphor analysis, in: Proceedings of the Third Workshop on Metaphor in NLP, pp. 40–49. find
  19. Elazar, Y., Zhang, H., Goldberg, Y., Roth, D., 2021. Back to square one: Bias detection, training and commonsense disentanglement in the winograd schema. arXiv preprint arXiv:2104.08161 . find
  20. Ettinger, A., 2020. What bert is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics 8, 34–48.
  21. Forbus, K.D., 1997. Qualitative reasoning. find
  22. Gangemi, A., Alam, M., Asprino, L., Presutti, V., Recupero, D.R., 2016. Framester: A wide coverage linguistic linked data hub, in: European Knowledge Acquisition Workshop, Springer. pp. 239–254. find
  23. Gangemi, A., Guarino, N., Masolo, C., Oltramari, A., Schneider, L., 2002. Sweetening ontologies with dolce, in: International Conference on Knowledge Engineering and Knowledge Management, Springer. pp. 166–181. find
  24. Gentzen, G., 1934. Investigations into logical deduction. translation printed in m. szabo the collected papers of gerhard gentzen. find
  25. Gibson, E.J., Pick, A.D., et al., 2000. An ecological approach to perceptual learning and development. Oxford University Press, USA. find
  26. Gordon, A.S., Hobbs, J.R., 2017. A formal theory of commonsense psychology: How people think people think. Cambridge University Press. find
  27. Grice, H.P., 1975. Logic and conversation, in: Speech acts. Brill, pp. 41–58. find
  28. Guha, R.V., Brickley, D., Macbeth, S., 2016. Schema. org: evolution of structured data on the web. Communications of the ACM 59, 44– 51. find
  29. Hartigan, J.A., Wong, M.A., 1979. Algorithm as 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics) 28, 100–108. find
  30. Hayes, P.J., 1990. The naive physics manifesto, in: Boden, M.A. (Ed.), The Philosophy of Artificial Intelligence. Oxford University Press. Oxford readings in philosophy, pp. 171–205. find
  31. Hicks, G.D., 1904. Idealism and the problem of knowledge and existence, in: Proceedings of the Aristotelian Society, JSTOR. pp. 136– 178. find
  32. Ilievski, F., Bosselut, A., Razniewski, S., Kejriwal, M., 2021a. Commonsense knowledge acquisition and representation, in: AAAI Tutorials. find
  33. Ilievski, F., Szekely, P., Kejriwal, M., 2020a. Commonsense knowledge graphs, in: The International Semantic Web Conference (ISWC) Tutorials. find
  34. Ilievski, F., Szekely, P., Schwabe, D., 2020b. Commonsense knowledge in wikidata. Wikidata workshop . find
  35. Ilievski, F., Szekely, P.A., Zhang, B., 2021b. CSKG: the commonsense knowledge graph, in: Verborgh, R., Hose, K., Paulheim, H., Champin, P., Maleshkova, M., Corcho, Ó., Ristoski, P., Alam, M. (Eds.), The Semantic Web - 18th International Conference, ESWC 2021, Virtual Event, June 6-10, 2021, Proceed… find
  36. Ilievski, F., Vossen, P., Van Erp, M., 2017. Hunger for contextual knowledge and a road map to intelligent entity linking, in: International Conference on Language, Data and Knowledge, Springer, Cham. pp. 143–149. find
  37. Jeretic, P., Warstadt, A., Bhooshan, S., Williams, A., 2020. Are natural language inference models IMPPRESsive? Learning IM-Plicature and PRESupposition, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online. pp…
  38. Ji, S., Pan, S., Cambria, E., Marttinen, P., Philip, S.Y., 2021. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems . find
  39. Jiang, Z., Xu, F.F., Araki, J., Neubig, G., 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics 8, 423–438. URL: https://www.aclweb.org/anthology/2020.tacl-1.28 , doi: 10.1162/tacl_a_00324 . ↗
  40. Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.J., Shamma, D.A., et al., 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision 123, 32– 73.
  41. de Lacalle, M.L., Laparra, E., Aldabe, I., Rigau, G., 2016. A multilingual predicate matrix, in: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pp. 2662–2668.
  42. Lascarides, A., Asher, N., 1993. Temporal interpretation, discourse relations and commonsense entailment. Linguistics and philosophy 16, 437–493. find
  43. Lenat, D.B., 1995. Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM 38, 33–38. find
  44. Levine, J., 1983. Materialism and qualia: The explanatory gap. Pacific philosophical quarterly 64, 354–361. find
  45. Lewis, P.S.H., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., Kiela, D., 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks, in: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (Eds.), Adva…
  46. Li, X.L., Liang, P., 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv:2101.00190 . find
  47. Lin, B.Y., Chen, X., Chen, J., Ren, X., 2019. KagNet: Knowledge-aware graph networks for commonsense reasoning, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Associa… find
  48. Lin, B.Y., Lee, S., Khanna, R., Ren, X., 2020a. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, On…
  49. Lin, Z., Madotto, A., Fung, P., 2020b. Exploring versatile generative language model via parameter-efficient transfer learning, in: Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics, Online. pp. 441– 459. URL: https://www.aclweb.org/anth… ↗
  50. Liu, H., Singh, P., 2004. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal 22, 211–226. find
  51. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V., 2019. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692 .
  52. Ma, K., Francis, J., Lu, Q., Nyberg, E., Oltramari, A., 2019. Towards generalizable neuro-symbolic systems for commonsense question answering, in: Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, Association for Computational Linguistics, Hong Kong, China. p…
  53. Ma, K., Ilievski, F., Francis, J., Bisk, Y., Nyberg, E., Oltramari, A., 2021. Knowledge-driven data construction for zero-shot evaluation in commonsense question answering, in: AAAI. find
  54. MacLachlan, G.L., Reid, I., 1994. Framing and interpretation. Melbourne University Press. find
  55. McCarthy, J., et al., 1960. Programs with common sense. RLE and MIT computation center. find
  56. Miller, G.A., 1998. WordNet: An electronic lexical database. MIT press. find
  57. Miller, G.A., 2003. The cognitive revolution: a historical perspective. Trends in cognitive sciences 7, 141–144. find
  58. van Miltenburg, E., 2016. Stereotyping and bias in the flickr30k dataset, in: Edlund, J., Heylen, D., Paggio, P. (Eds.), Proceedings of Multimodal Corpora: Computer vision and language processing (MMC 2016), pp. 1–4. URL: http://www.lrec-conf.org/proceedings/ lrec2016/workshops/LREC2016Workshop-MCC-… ↗
  59. Moore, R.C., 1982. The role of logic in knowledge representation and commonsense reasoning. SRI International. Artificial Intelligence Center. find
  60. Mostafazadeh, N., Kalyanpur, A., Moon, L., Buchanan, D., Berkowitz, L., Biran, O., Chu-Carroll, J., 2020. GLUCOSE: GeneraLized and COntextualized story explanations, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Ling…
  61. Navigli, R., Ponzetto, S.P., 2012. Babelnet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence 193, 217–250. find
  62. Niles, I., Pease, A., 2001. Towards a standard upper ontology, in: Proceedings of the international conference on Formal Ontology in Information Systems-Volume 2001, pp. 2–9. find
  63. Onyeka, E., Varde, A., Anu, V., Tandon, N., Daramola, O., 2020. Using commonsense knowledge and text mining for implicit requirements localization. 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) , 935–940. find
  64. Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., Miller, A., 2019. Language models as knowledge bases?, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP…
  65. Plummer, B.A., Wang, L., Cervantes, C.M., Caicedo, J.C., Hockenmaier, J., Lazebnik, S., 2016. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. International Journal of Computer Vision , 1–20. find
  66. Puri, M., Varde, A., Dong, B., 2018. Pragmatics and semantics to connect specific local laws with public reactions. 2018 IEEE International Conference on Big Data (Big Data) , 5433–5435. find
  67. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 9. find
  68. Razniewski, S., Tandon, N., Varde, A.S., 2021. Information to wisdom: Commonsense knowledge extraction and compilation, in: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 1143–1146. find
  69. Roget, P.M., 2020. Roget’s Thesaurus. Good Press. find
  70. Rohde, D.L., Gonnerman, L.M., Plaut, D.C., 2006. An improved model of semantic similarity based on lexical co-occurrence. Communications of the ACM 8, 116. find
  71. Romero, J., Razniewski, S., Pal, K., Z. Pan, J., Sakhadeo, A., Weikum, G., 2019. Commonsense properties from query logs and question answering forums, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1411– 1420. find
  72. Sap, M., Le Bras, R., Allaway, E., Bhagavatula, C., Lourie, N., Rashkin, H., Roof, B., Smith, N.A., Choi, Y., 2019a. Atomic: An atlas of machine commonsense for if-then reasoning, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3027–3035. find
  73. Sap, M., Rashkin, H., Chen, D., Le Bras, R., Choi, Y., 2019b. Social IQa: Commonsense reasoning about social interactions, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNL… find
  74. Sap, M., Shwartz, V., Bosselut, A., Choi, Y., Roth, D., 2020. Commonsense reasoning for natural language processing, in: ACL. find
  75. Schuler, K.K., 2005. Verbnet: A broad-coverage, comprehensive verb lexicon . find
  76. Segers, R., Vossen, P., Rospocher, M., Serafini, L., Laparra, E., Rigau, G., 2015. Eso: A frame based ontology for events and implied situations. Proceedings of MAPLEX 2015. find
  77. Shin, T., Razeghi, Y., IV, R.L.L., Wallace, E., Singh, S., 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts, in: Webber, B., Cohn, T., He, Y., Liu, Y. (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP…
  78. Shwartz, V., West, P., Le Bras, R., Bhagavatula, C., Choi, Y., 2020a. Unsupervised commonsense question answering with self-talk, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online. pp. 4615– 4629. URL… find
  79. Shwartz, V., West, P., Le Bras, R., Bhagavatula, C., Choi, Y., 2020b. Unsupervised commonsense question answering with self-talk, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online. pp. 4615– 4629. URL… find
  80. Singh, P., Lin, T., Mueller, E.T., Lim, G., Perkins, T., Zhu, W.L., 2002. Open mind common sense: Knowledge acquisition from the general public, in: OTM Confederated International Conferences" On the Move to Meaningful Internet Systems", Springer. pp. 1223–1237. find
  81. Speer, R., Chin, J., Havasi, C., 2017. Conceptnet 5.5: An open multilingual graph of general knowledge, in: Thirty-First AAAI Conference on Artificial Intelligence. find
  82. Storks, S., Gao, Q., Chai, J., 2019. Commonsense reasoning for natural language understanding: A survey of benchmarks, resources, and approaches. ArXiv abs/1904.01172.
  83. Talmor, A., Herzig, J., Lourie, N., Berant, J., 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and…
  84. Tandon, N., De Melo, G., Weikum, G., 2017. Webchild 2.0: Fine-grained commonsense knowledge distillation, in: Proceedings of ACL 2017, System Demonstrations, pp. 115–120. find
  85. Tandon, N., Varde, A.S., de Melo, G., 2018. Commonsense knowledge in machine intelligence. ACM SIGMOD Record 46, 49–52. find
  86. Tanon, T.P., Weikum, G., Suchanek, F.M., 2020. Yago 4: A reasonable knowledge base. The Semantic Web 12123, 583 – 596. find
  87. Vrandečić, D., Krötzsch, M., 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM 57, 78–85.
  88. Williams, B., Lieberman, H., Winston, P.H., 2017. Understanding stories with large-scale common sense., in: COMMONSENSE. find
  89. Winston, M.E., Chaffin, R., Herrmann, D., 1987. A taxonomy of part-whole relations. Cognitive science 11, 417–444. find
  90. Wu, W., Li, H., Wang, H., Zhu, K.Q., 2012. Probase: A probabilistic taxonomy for text understanding, in: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 481– 492. find
  91. Yang, W., Wang, X., Farhadi, A., Gupta, A., Mottaghi, R., 2018. Visual semantic navigation using scene priors. arXiv preprint arXiv:1810.06543 . — [arXiv:1810.06543](https://arxiv.org/abs/1810.06543) find
  92. Zellers, R., Bisk, Y., Farhadi, A., Choi, Y., 2019. From recognition to cognition: Visual commonsense reasoning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6720–6731. find
  93. Zhang, H., Zhao, X., Song, Y., 2020. WinoWhy: A deep diagnosis of essential commonsense knowledge for answering Winograd schema challenge, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online. pp. 5736–5745. UR… find
  94. Zhou, P., Khanna, R., Lee, S., Lin, B.Y., Ho, D., Pujara, J., Ren, X., 2020. Rica: Evaluating robust inference capabilities based on commonsense axioms. arXiv preprint arXiv:2005.00782 . find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ilievski-2021-dimensions-commonsense-knowledge/