Elamur Research⌕
☰
home / papers / 2021 / Refined Commonsense Knowledge from Large-Scale Web Contents

Refined Commonsense Knowledge from Large-Scale Web Contents

Paper · 2021 · Tuan-Phong Nguyen, Simon Razniewski, Julien Romero et al.

Summary

Commonsense knowledge bases such as ConceptNet, WebChild, TupleKB and Quasimodo store knowledge as flat subject-predicate-object triples. This format cannot separate refined subjects (a diesel bus versus an electric one), cannot link equivalent or more specific statements, and cannot record when or where a statement holds. These resources also tend to sacrifice either precision or coverage.

The authors present ASCENT++, a two-phase pipeline that builds a richer commonsense KB from the C4 web crawl. In the extraction phase, dependency-parse rules produce OpenIE-style assertions, including ones about subgroups and aspects of a subject. A supervised classifier then labels semantic facets cued by prepositional phrases and adverbs. In the consolidation phase, pages are filtered for each subject by embedding similarity to a reference Wikipedia article, which separates homonyms. Assertions are clustered with SentenceTransformers, mapped onto the ConceptNet schema, and ranked by learned typicality and saliency scores.

Human judgments show better precision and relative recall than automatically built competitors. Question-answering experiments also show that the KB usefully supports language models.

Key points

  • The input corpus is C4, about 365 million English web pages. Extraction targets 10,000 salient ConceptNet concepts and yields roughly 2 million assertions.
  • Assertions carry facets of eight types, such as temporal and spatial validity. Each assertion also receives typicality and saliency scores, and a canonicalized ConceptNet-schema version is provided.
  • Measured on each subject's ten highest-ranked assertions, ASCENT++ improves on the earlier ASCENT KB by 25% in typicality and 14.6% in saliency, and its relative recall is 36.6% greater.
  • The extrinsic QA evaluation covered three settings and six CSKBs. ASCENT++ significantly beat language models without commonsense knowledge in two settings and ranked first or second among the CSKBs in all three.
  • The data, code and a web interface are released at ascentpp.mpi-inf.mpg.de.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Max Planck Institute for InformaticsSaarbrücken · Germany22 works · 4,687 citations🎓Télécom SudParisÉvry-Courcouronnes · France3 works · 343 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Tuan-Phong Nguyen, Simon Razniewski, Julien Romero and Gerhard Weikum. 2021. Refined Commonsense Knowledge from Large-Scale Web Contents. arXiv:2112.04596 (IEEE Transactions on Knowledge and Data Engineering, 2022).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/tkde.2022.3206505 · arXiv 2112.04596

@misc{nguyen2021refined,
  title = {{Refined Commonsense Knowledge from Large-Scale Web Contents}},
  author = {Tuan-Phong Nguyen, Simon Razniewski, Julien Romero and Gerhard Weikum},
  year = {2021},
  doi = {10.1109/tkde.2022.3206505},
  eprint = {2112.04596},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/tkde.2022.3206505}
}

References

85 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. McCarthy, Programs with common sense. RLE and MIT computation center, 1960. find
  2. E. A. Feigenbaum, “Knowledge engineering,” Annals of the New York Academy of Sciences, 1984. find
  3. D. B. Lenat, “Cyc: A large-scale investment in knowledge infrastructure,” CACM, 1995. find
  4. “The OpenCyc Platform,” https://web.archive.org/web/20160330064354/http://www.opencyc.org/, accessed: 2022-05-16. ↗
  5. R. Speer and C. Havasi, “ConceptNet 5: A large semantic network for relational knowledge,” Theory and Applications of Natural Language Processing, 2012. find
  6. N. Tandon, G. de Melo, F. M. Suchanek, and G. Weikum, “WebChild: harvesting and organizing commonsense knowledge from the web,” in WSDM, 2014. find
  7. B. D. Mishra, N. Tandon, and P. Clark, “Domain-targeted, high precision knowledge extraction,” TACL, 2017.
  8. J. Romero, S. Razniewski, K. Pal, J. Z. Pan, A. Sakhadeo, and G. Weikum, “Commonsense properties from query logs and question answering forums,” in CIKM, 2019. find
  9. K. Panton, C. Matuszek, D. B. Lenat, D. Schneider, M. Witbrock, N. Siegel, and B. Shepard, “Common sense reasoning - from cyc to intelligent assistant,” in Ambient Intelligence in Everyday, 2006. find
  10. H. Lin, L. Sun, and X. Han, “Reasoning with heterogeneous knowledge for commonsense machine comprehension,” in EMNLP, 2017. find
  11. B. Y. Lin, X. Chen, J. Chen, and X. Ren, “Kagnet: Knowledge-aware graph networks for commonsense reasoning,” in EMNLP-IJCNLP, 2019. find
  12. J. Xia, C. Wu, and M. Yan, “Incorporating relation knowledge into commonsense reading comprehension with multi-task learning,” in CIKM, 2019. find
  13. B. Y. Lin, W. Zhou, M. Shen, P. Zhou, C. Bhagavatula, Y. Choi, and X. Ren, “CommonGen: A constrained text generation challenge for generative commonsense reasoning,” in Findings of EMNLP, 2020. find
  14. F. Ilievski, P. Szekely, and D. Schwabe, “Commonsense knowledge in Wikidata,” in Wikidata workshop, 2020. find
  15. M. Sap, R. LeBras, E. Allaway, C. Bhagavatula, N. Lourie, H. Rashkin, B. Roof, N. A. Smith, and Y. Choi, “Atomic: An atlas of machine commonsense for if-then reasoning,” in AAAI, 2018. find
  16. H. Zhang, D. Khashabi, Y. Song, and D. Roth, “Transomcs: From linguistic graphs to commonsense knowledge,” in IJCAI, 2020. find
  17. P. West, C. Bhagavatula, J. Hessel, J. D. Hwang, L. Jiang, R. L. Bras, X. Lu, S. Welleck, and Y. Choi, “Symbolic knowledge distillation: from general language models to commonsense models,” in NAACL, 2022.
  18. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, 2020.
  19. T.-P. Nguyen, S. Razniewski, and G. Weikum, “Advanced semantics for commonsense knowledge extraction,” in WWW, 2021. find
  20. N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in EMNLP, 2019.
  21. T. B. Brown et al., “Language models are few-shot learners,” in NeurIPS, 2020.
  22. P. Singh, T. Lin, E. T. Mueller, G. Lim, T. Perkins, and W. L. Zhu, “Open mind common sense: Knowledge acquisition from the general public,” in OTM Confederated International Conferences, 2002. find
  23. H. Liu and P. Singh, “Conceptnet—a practical commonsense reasoning tool-kit,” BT technology journal, 2004. find
  24. J. Gordon, B. V. Durme, and L. K. Schubert, “Learning from the web: Extracting general world knowledge from noisy text,” in AAAI Workshops, 2010. find
  25. J. D. Hwang, C. Bhagavatula, R. L. Bras, J. Da, K. Sakaguchi, A. Bosselut, and Y. Choi, “(Comet-)Atomic 2020: On symbolic and neural commonsense knowledge graphs,” in AAAI, 2021.
  26. H. Zhang, X. Liu, H. Pan, Y. Song, and C. W.-K. Leung, “Aser: A large-scale eventuality knowledge graph,” in WWW, 2020. find
  27. H. Zhang, X. Liu, H. Pan, H. Ke, J. Ou, T. Fang, and Y. Song, “ASER: Towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities,” Artificial Intelligence, 2022. find
  28. F. Ilievski, P. Szekely, and B. Zhang, “Cskg: The commonsense knowledge graph,” in ESWC, 2021. find
  29. S. Bhakthavatsalam, C. Anastasiades, and P. Clark, “Genericskb: A knowledge base of generic statements,” arXiv:2005.00660, 2020. find
  30. T.-P. Nguyen, S. Razniewski, and G. Weikum, “Inside ASCENT: Exploring a deep commonsense knowledge base and its usage in question answering,” in ACL: System Demonstrations, 2021. find
  31. O. Etzioni, M. J. Cafarella, D. Downey, S. Kok, A. Popescu, T. Shaked, S. Soderland, D. S. Weld, and A. Yates, “Web-scale information extraction in knowitall: (preliminary results),” in WWW, 2004. find
  32. R. Snow, D. Jurafsky, and A. Y. Ng, “Semantic taxonomy induction from heterogenous evidence,” in ACL, 2006. find
  33. R. Girju, A. Badulescu, and D. I. Moldovan, “Automatic discovery of part-whole relations,” Comput. Linguistics, 2006. find
  34. P. Pantel and M. Pennacchiotti, “Espresso: Leveraging generic patterns for automatically harvesting semantic relations,” in ACL, 2006.
  35. M. Pasca and B. V. Durme, “Weakly-supervised acquisition of open-domain classes and class attributes from web documents and query logs,” in ACL, 2008. find
  36. S. P. Ponzetto and M. Strube, “Taxonomy induction based on a collaboratively built knowledge repository,” Artif. Intell., 2011. find
  37. W. Wu, H. Li, H. Wang, and K. Q. Zhu, “Probase: a probabilistic taxonomy for text understanding,” in SIGMOD, 2012. find
  38. S. Hertling and H. Paulheim, “WebIsALOD: Providing hypernymy relations extracted from the web as linked open data,” in ISWC, 2017. find
  39. G. A. Miller, “Wordnet: A lexical database for English,” CACM, 1995.
  40. J. Seitner, C. Bizer, K. Eckert, S. Faralli, R. Meusel, H. Paulheim, and S. P. Ponzetto, “A large database of hypernymy relations extracted from the web,” in LREC, 2016.
  41. B. Liu, W. Guo, D. Niu, J. Luo, C. Wang, Z. Wen, and Y. Xu, “GIANT: scalable creation of a web-scale ontology,” in SIGMOD, 2020. find
  42. N. Tandon, C. Hariman, J. Urbani, A. Rohrbach, M. Rohrbach, and G. Weikum, “Commonsense in parts: Mining part-whole relations from the web and image tags,” in AAAI, 2016.
  43. S. Bhakthavatsalam, K. Richardson, N. Tandon, and P. Clark, “Do dogs have whiskers? a new knowledge base of haspart relations,” arXiv:2006.07510, 2020. find
  44. D. M. Gabbay, Many-Dimensional Modal Logics: Theory and Applications. Elsevier North Holland, 2003. find
  45. A. Hogan et al., “Knowledge graphs,” ACM Computing Surveys (CSUR), 2021.
  46. L. Schubert, “Can we derive general world knowledge from texts,” in HLT, 2002. find
  47. M. Dragoni, S. Poria, and E. Cambria, “Ontosenticnet: A commonsense ontology for sentiment analysis,” IEEE Intell. Syst., 2018. find
  48. S. Zhang, R. Rudinger, K. Duh, and B. Van Durme, “Ordinal common-sense inference,” TACL, 2017.
  49. T. Chen, Z. Jiang, A. Poliak, K. Sakaguchi, and B. Van Durme, “Uncertain natural language inference,” in ACL, 2020.
  50. Y. Chalier, S. Razniewski, and G. Weikum, “Joint reasoning for multi-faceted commonsense knowledge,” in AKBC, 2020.
  51. M. Palmer, D. Gildea, and N. Xue, Semantic Role Labeling, ser. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2010. find
  52. J. Clarke, V. Srikumar, M. Sammons, and D. Roth, “An NLP curator (or: How i learned to stop worrying and love NLP pipelines),” in LREC, 2012. find
  53. G. Stanovsky, J. Michael, L. Zettlemoyer, and I. Dagan, “Supervised open information extraction,” in NAACL, 2018. find
  54. R. E. Prasojo, M. Kacimi, and W. Nutt, “Stuffie: Semantic tagging of unlabeled facets using fine-grained information extraction,” in CIKM, 2018. find
  55. M. Cetto, C. Niklaus, A. Freitas, and S. Handschuh, “Graphene: Semantically-linked propositions in open information extraction,” in COLING, 2018. find
  56. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in NAACL, 2019.
  57. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. find
  58. F. Petroni, T. Rocktäschel, P. Lewis, A. Bakhtin, Y. Wu, A. H. Miller, and S. Riedel, “Language models as knowledge bases?” in EMNLP, 2019.
  59. A. Bosselut, H. Rashkin, M. Sap, C. Malaviya, A. Çelikyilmaz, and Y. Choi, “COMET: commonsense transformers for automatic knowledge graph construction,” in ACL, 2019.
  60. J. Davison, J. Feldman, and A. M. Rush, “Commonsense knowledge mining from pretrained models,” in EMNLP, 2019.
  61. Z. Bouraoui, J. Camacho-Collados, and S. Schockaert, “Inducing relational knowledge from bert,” in AAAI, 2020.
  62. T.-P. Nguyen and S. Razniewski, “Materialized knowledge bases from commonsense transformers,” in Proceedings of the First Workshop on Commonsense Representation and Reasoning, 2022. find
  63. R. Navigli, “Word sense disambiguation: A survey,” ACM Comput. Surv., 2009. find
  64. J. Gordon and L. K. Schubert, “Quantificational sharpening of commonsense knowledge,” in Commonsense Knowledge, AAAI Fall Symposium, 2010. find
  65. V. Shwartz and C. Waterson, “Olive oil is made of olives, baby oil is made for babies: Interpreting noun compounds using paraphrases in a neural model,” in NAACL, 2018. find
  66. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try ARC, the AI2 reasoning challenge,” arXiv:1803.05457, 2018.
  67. J. Gordon and B. V. Durme, “Reporting bias and knowledge acquisition,” in AKBC, 2013. find
  68. J. Romero and S. Razniewski, “Inside quasimodo: Exploring construction and usage of commonsense knowledge,” in CIKM, 2020. find
  69. Y. Elazar, A. Mahabal, D. Ramachandran, T. Bedrax-Weiss, and D. Roth, “How large are lions? inducing distributions over quantitative attributes,” in ACL, 2019. find
  70. Y. Feng, X. Chen, B. Y. Lin, P. Wang, J. Yan, and X. Ren, “Scalable multi-hop relational reasoning for knowledge-aware question answering,” in EMNLP, 2020.
  71. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in NeurIPS, 2013.
  72. S. Wang, L. Thompsondo, and M. Iyyer, “Phrase-BERT: Improved phrase embeddings from BERT with an application to corpus exploration,” in EMNLP, 2021. find
  73. T. Pellissier Tanon, G. Weikum, and F. Suchanek, “Yago 4: A reasonable knowledge base,” in ESWC, 2020. find
  74. S. Soderland, J. Gilmer, R. Bart, O. Etzioni, and D. S. Weld, “Open information extraction to kbp relations in 3 hours,” in TAC, 2013. find
  75. L. Galárraga, G. Heitz, K. Murphy, and F. M. Suchanek, “Canonicalizing open knowledge bases,” in CIKM, 2014. find
  76. R. A. Putri, G. Hong, and S.-H. Myaeng, “Aligning open ie relations and kb relations using a siamese network based on word embedding,” in IWCS, 2019. find
  77. D. Zhang, S. Mukherjee, C. Lockard, X. L. Dong, and A. McCallum, “Openki: Integrating open information extraction and knowledge bases with relation inference,” in NAACL, 2019. find
  78. R. Navigli and S. P. Ponzetto, “Babelnet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network,” Artificial intelligence, 2012. find
  79. N. Mehrabi, P. Zhou, F. Morstatter, J. Pujara, X. Ren, and A. Galstyan, “Lawyers are dishonest? quantifying representational harms in commonsense knowledge resources,” in EMNLP, 2021. find
  80. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv:1907.11692, 2019.
  81. J. L. Fleiss and J. Cohen, “The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability,” Educational and psychological measurement, 1973. find
  82. B. J. Devereux, L. K. Tyler, J. Geertzen, and B. Randall, “The centre for speech, language and the brain (CSLB) concept property norms,” Behavior research methods, 2014. find
  83. K. Guu, K. Lee, Z. Tung, P. Pasupat, and M.-W. Chang, “Realm: Retrieval-augmented language model pre-training,” in ICML, 2020.
  84. F. Petroni, P. Lewis, A. Piktus, T. Rocktäschel, Y. Wu, A. H. Miller, and S. Riedel, “How context affects language models’ factual predictions,” in AKBC, 2020. — How Context Affects Language Models' Factual Predictions
  85. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite Bert for self-supervised learning of language representations,” in ICLR, 2020.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2021-refined-commonsense-knowledge-large-scale/