Elamur Research⌕
☰
home / papers / 2018 / Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikidata, and YAGO

Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikidata, and YAGO

Summary

The large open cross-domain knowledge graphs (KGs) DBpedia, Freebase, OpenCyc, Wikidata and YAGO are widely used, yet no one has compared them thoroughly. The authors from KIT fill this gap. They build on the classic data quality model of Wang et al. and its extensions for Linked Data by Bizer and Zaveri. On this basis, 34 quality criteria are formulated. They are grouped into dimensions, and the dimensions into four categories: intrinsic, contextual, representational and accessibility quality. Each criterion is turned into a metric with a value between 0 and 1. The overall score of a graph is computed as a weighted combination of the metrics, with the weights chosen by the user to suit their task. For all five KGs, the authors gathered key statistics via SPARQL queries and computed the metrics on the full data rather than on samples. In the summary table, Wikidata has the highest average score and OpenCyc the lowest. The work provides a practical tool for selecting a knowledge graph in research and industrial projects, and the authors have made all data and detailed results available online.

Key points

  • Selection of graphs: datasets tagged "crossdomain" were taken from the datahub.io catalog, and Wikidata was added separately. Closed graphs (Google Knowledge Graph, Knowledge Vault) were not considered. Resources available only via an API without dumps (WolframAlpha, Facebook Graph) were likewise excluded.
  • Evaluation framework: the dimensions of Wang et al. are supplemented with the dimensions consistency and verifiability (following Bizer), as well as license and interlinking (following Zaveri). Each metric yields a value in the range [0, 1], and these are aggregated into a weighted fulfillment degree h(g).
  • Average scores, unweighted / weighted: Wikidata — 0.738 / 0.742, DBpedia — 0.708 / 0.718, YAGO — 0.625 / 0.646, Freebase — 0.605 / 0.575, OpenCyc — 0.498 / 0.516. The weighted score was computed with example weights from 0 to 3.
  • Individual metrics, assuming the column order in the extracted table has been reconstructed correctly: OpenCyc scores 0 for labels in multiple languages (m_Lang). Freebase and OpenCyc have no SPARQL endpoint (m_SPARQL = 0).
  • The statistics were obtained via SPARQL queries, so the authors do not depend on the N-Triples serialization, as Duan et al. do. To assess domain coverage, the most frequent classes were manually mapped to domains (people, geography, etc.), and duplicate entities within a single domain were removed. The data are published at km.aifb.kit.edu.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Knowledge graphs & ontologies

Cite

Cite as Michael Färber, Frederic Bartscherer, Carsten Menne and Achim Rettinger. 2018. Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikidata, and YAGO. Semantic Web Journal 9(1).

The paper itself, not this page. BibTeX from the library's record of the work.

@article{farber2018linked,
  title = {{Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikidata, and YAGO}},
  author = {Michael Färber, Frederic Bartscherer, Carsten Menne and Achim Rettinger},
  journal = {Semantic Web Journal 9(1)},
  year = {2018},
  url = {https://www.semantic-web-journal.net/content/linked-data-quality-dbpedia-freebase-opencyc-wikidata-and-yago}
}

References

42 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Acosta, E. Simperl, F. Flöck, and M.-E. Vidal. HARE: A Hybrid SPARQL Engine to Enhance Query Answers via Crowdsourcing. In K-CAP2015: The 8th International Conference on Knowledge Capture. International Conference on Knowledge Capture, ACM, 2015. find
  2. M. Acosta, A. Zaveri, E. Simperl, D. Kontokostas, S. Auer, and J. Lehmann. Crowdsourcing linked data quality assessment. In The Semantic Web–ISWC 2013, pages 260–276. Springer, 2013. find
  3. M. Acosta, A. Zaveri, E. Simperl, D. Kontokostas, F. Flöck, and J. Lehmann. Detecting Linked Data Quality Issues via Crowdsourcing: A DBpedia Study. Semantic Web, 2016. find
  4. S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives. DBpedia: A Nucleus for a Web of Open Data. In Proceedings of the 6th International Semantic Web Conference and 2nd Asian Semantic Web Conference, ISWC 2007/ASWC 2007, pages 722–735. Springer, 2007. find
  5. S. Auer, J. Lehmann, A.-C. Ngonga Ngomo, and A. Zaveri. Introduction to Linked Data and Its Lifecycle on the Web. In Reasoning Web. Semantic Technologies for Intelligent Data Access, volume 8067 of Lecture Notes in Computer Science, pages 1–90. Springer Berlin Heidelberg, 2013. find
  6. C. Batini, C. Cappiello, C. Francalanci, and A. Maurino. Methodologies for Data Quality Assessment and Improvement. ACM Comput. Surv., 41(3):16:1–16:52, July 2009. find
  7. S. Bechhofer, F. van Harmelen, J. Hendler, I. Horrocks, D. L. McGuinness, and P. F. Patel-Schneider. OWL Web Ontology Language Reference. https://www.w3.org/TR/2004/REC-owl-ref-20040210, 2004. [Online; accessed 06-Apr-2016]. M. Färber et al. / Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikid… ↗
  8. T. Berners-Lee. Linked Data. http://www.w3.org/DesignIssues/LinkedData.html, 2006. [Online; accessed 28-Feb-2016]. ↗
  9. T. Berners-Lee. Linked Data Is Merely More Data. http://www.w3.org/DesignIssues/LinkedData.html, 2006. [Online; accessed 28-02-2016]. ↗
  10. T. Berners-Lee, J. Hendler, and O. Lassila. The semantic web. Scientific American, 284(5):29–37, 5 2001. find
  11. C. Bizer. Quality Driven Information Filtering: In the Context of Web Based Information Systems. VDM Publishing, 2007. find
  12. C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak, and S. Hellmann. DBpedia–A crystallization point for the Web of Data. Web Semantics: science, services and agents on the world wide web, 7(3):154–165, 2009. find
  13. P. Buneman, S. Khanna, and T. Wang-Chiew. Why and where: A characterization of data provenance. In Database Theory—ICDT 2001, pages 316–330. Springer, 2001. find
  14. X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang. Knowledge Vault: A Web-scale Approach to Probabilistic Knowledge Fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pages 601–610, N… find
  15. S. Duan, A. Kementsietsidis, K. Srinivas, and O. Udrea. Apples and oranges: a comparison of RDF benchmarks and real RDF datasets. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, pages 145–156. ACM, 2011. find
  16. B. Ell, D. Vrandečić, and E. Simperl. Proceedings of the 10th International Semantic Web Conference (ISWC 2011), chapter Labels in the Web of Data, pages 162–176. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011. find
  17. C. Fellbaum. WordNet – An Electronic Lexical Database. MIT Press, 1998. find
  18. A. Flemming. Qualitätsmerkmale von Linked Data-veröffentlichenden Datenquellen (Quality characteristics of linked data publishing datasources). Diploma Thesis, Humboldt University of Berlin, http://www.dbis.informatik.hu-berlin.de/fileadmin/research/papers/diploma_seminar_thesis/Diplomarbeit_Annika_… ↗
  19. G. Freedman and E. Reynolds. Enriching basal reader lessons with semantic webbing. Reading Teacher, 33(6):677–684, 1980. find
  20. C. Fürber and M. Hepp. SWIQA – A Semantic Web Information Quality Assessment Framework. In Proceedings of the 19th European Conference on Information Systems (ECIS2011), volume 15, page 19, 2011. find
  21. R. Guns. Tracing the Origins of the Semantic Web. Journal of the American Society for Information Science and Technology, 64(10):2173–2181, 2013. find
  22. H. Halpin, P. J. Hayes, J. P. McCusker, D. L. McGuinness, and H. S. Thompson. The Semantic Web – ISWC 2010: 9th International Semantic Web Conference, ISWC 2010, Shanghai, China, chapter When owl:sameAs Isn’t the Same: An Analysis of Identity in Linked Data, pages 305–320. Springer Berlin Heidelberg… find
  23. O. Hassanzadeh, M. J. Ward, M. Rodriguez-Muro, and K. Srinivas. Understanding a large corpus of web tables through matching with knowledge bases: an empirical study. In Proceedings of the 10th International Workshop on Ontology Matching collocated with the 14th International Semantic Web Conference… find
  24. T. Heath and C. Bizer. Linked data: Evolving the web into a global data space. Synthesis lectures on the semantic web: theory and technology, 1(1):1–136, 2011. find
  25. J. Hoffart, F. Suchanek, K. Berberich, E. Kelham, G. de Melo, G. Weikum, F. Suchanek, G. Kasneci, M. Ramanath, and A. Pease. Yago2: A spatially and temporally enhanced knowledge base from wikipedia. Commun. ACM, 52(4):56–64, 2009. find
  26. A. Hogan, A. Harth, A. Passant, S. Decker, and A. Polleres. Weaving the Pedantic Web. Proceedings of the WWW2010 Workshop on Linked Data on the Web, 628, 2010. find
  27. A. Hogan, J. Umbrich, A. Harth, R. Cyganiak, A. Polleres, and S. Decker. An empirical survey of linked data conformance. Web Semantics: Science, Services and Agents on the World Wide Web, 14:14–44, 2012. find
  28. P. Jain, P. Hitzler, K. Janowicz, and C. Venkatramani. There’s No Money in Linked Data. http://corescholar.libraries.wright.edu/cse/240, 2013. accessed July 20, 2015. ↗
  29. J. M. Juran, F. Gryna, and R. Bingham. Quality Control Handbook. McGraw-Hill, New York, 1974. find
  30. G. Kobilarov, T. Scott, Y. Raimond, S. Oliver, C. Sizemore, M. Smethurst, C. Bizer, and R. Lee. Media Meets Semantic Web – How the BBC Uses DBpedia and Linked Data to Make Connections. In Proceedings of the 6th European Semantic Web Conference on The Semantic Web: Research and Applications, ESWC 200… find
  31. D. Kontokostas, P. Westphal, S. Auer, S. Hellmann, J. Lehmann, R. Cornelissen, and A. Zaveri. Test-driven evaluation of linked data quality. In Proceedings of the 23rd international conference on World Wide Web, pages 747–758. ACM, 2014. find
  32. M. Mecella, M. Scannapieco, A. Virgillito, R. Baldoni, T. Catarci, and C. Batini. Managing data quality in cooperative information systems. In On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE, pages 486–502. Springer, 2002. find
  33. O. Medelyan and C. Legg. Integrating Cyc and Wikipedia: Folksonomy meets rigorously defined common-sense. In Wikipedia and Artificial Intelligence: An Evolving Synergy, Papers from the 2008 AAAI Workshop, page 65, 2008. find
  34. F. Naumann. Quality-driven query answering for integrated information systems, volume 2261. Springer Science & Business Media, 2002. find
  35. L. L. Pipino, Y. W. Lee, and R. Y. Wang. Data quality assessment. Communications of the ACM, 45(4):211–218, 2002. find
  36. E. Sandhaus. Semantic Technology at the New York Times: Lessons Learned and Future Directions. In Proceedings of the 9th International Semantic Web Conference on The Semantic Web - Volume Part II, ISWC’10, pages 355–355, Berlin, Heidelberg, 2010. Springer-Verlag. find
  37. T. P. Tanon, D. Vrandečić, S. Schaffert, T. Steiner, and L. Pintscher. From freebase to wikidata: The great migration. In World Wide Web Conference, 2016. find
  38. S. Tartir, I. B. Arpinar, M. Moore, A. P. Sheth, and M. Färber et al. / Linked Data Quality of DBpedia, Freebase, OpenCyc, Wikidata, and YAGO B. Aleman-meza. OntoQA: Metric-based ontology quality analysis. In IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogen… find
  39. R. Y. Wang, M. P. Reddy, and H. B. Kon. Toward quality data: An attribute-based approach. Decision Support Systems, 13(3):349–372, 1995. find
  40. R. Y. Wang and D. M. Strong. Beyond accuracy: What data quality means to data consumers. Journal of management information systems, 12(4):5–33, 1996. find
  41. A. Zaveri, D. Kontokostas, M. A. Sherif, L. Bühmann, M. Morsey, S. Auer, and J. Lehmann. User-driven quality evaluation of dbpedia. In Proceedings of the 9th International Conference on Semantic Systems, pages 97–104. ACM, 2013. find
  42. A. Zaveri, A. Rula, A. Maurino, R. Pietrobon, J. Lehmann, and S. Auer. Quality assessment for linked data: A survey. Semantic Web, 7(1):63–93, 2015. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/farber-2018-linked-data-quality-dbpedia-freebase/