Elamur Research⌕
☰
home / papers / 2018 / T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples

T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples

Summary

Many NLP tasks require corpora in which sentences are aligned with knowledge base (KB) triples. These include relation extraction, knowledge base population, question answering, and text generation from facts. Existing resources of this kind have shortcomings: they are too small, cover only a few predicates, or are of unknown quality. The authors created T-REx, a large dataset of alignments between Wikipedia article abstracts and Wikidata triples. Their pipeline detects and links entities, recognizes dates, resolves coreference, and searches the text for predicate names. The alignments themselves are built by three distant supervision variants: NoSub, AllEnt, and SPO. As a result, about 11 million triples are linked to 3.09 million texts. This is roughly two orders of magnitude more than in the largest previous dataset, and 2.5 times as many predicates are covered. The advantage also holds for rare relations. Quality was assessed via crowdsourcing: the best method correctly aligned 97.8 % of the checked triples. The resource is open, so it can be used to train models that cover a broad range of relations.

Key points

  • Data sources: the English part of the DBpedia Abstracts corpus (4.6 million documents) and the Wikidata truthy dump (144 million triples). The final T-REx contains 11 million triples linked to 3.09 million abstracts (6.2 million sentences) and more than 600 unique predicates (NoSub covered 642).
  • Pipeline: entities are linked by DBpedia Spotlight, temporal expressions are recognized by SUtime, and coreference is resolved by Stanford CoreNLP together with a heuristic for pronouns. Predicates are searched for in the text by their names and synonyms from the KB.
  • Three alignment methods:
    • NoSub assumes that the subject of a paragraph is the main entity of the article, even when it is not explicitly mentioned.
    • AllEnt aligns every pair of entities in a sentence. It yields the most alignments, 11.1 million.
    • SPO additionally requires that the predicate itself be found in the sentence.
  • Quality evaluation: 2,600 alignments from 700 abstracts. Each was checked by at least 5 crowdsourced annotators. The best precision is 97.8 %. The main sources of errors are nested relations, incorrect logical inference, and entity linking errors.
  • Access: the dataset is distributed under the CC BY-SA 4.0 license (w3id.org/t-rex) in JSON and NIF 2.0 formats. The pipeline code is released under the MIT license.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Question answering Knowledge graphs & ontologies

Cite

Cite as Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest and Elena Simperl. 2018. T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{elsahar2018t,
  title = {{T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples}},
  author = {Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest and Elena Simperl},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  url = {https://aclanthology.org/L18-1544/}
}

References

14 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Augenstein, I., Maynard, D., and Ciravegna, F. (2016). Distantly supervised web relation extraction for knowledge base population. Semantic Web, 7(4):335–349. Brümmer, M., Dojchinovski, M., and Hellmann, S. (2016). Dbpedia abstracts: A large-scale, open, multilingual nlp training corpus. In Proceed… find
  2. Chang, A. X. and Manning, C. (2012). Sutime: A library for recognizing and normalizing time expressions. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey, may. Demeester, T., Rocktäschel, T., and Riedel, S. (2016). Lifted rule injection… find
  3. Gabrilovich, E., Ringgaard, M., and Subramanya, A. (2013). FACC1: Freebase annotation of ClueWeb corpora, Version 1, June. Hellmann, S., Lehmann, J., Auer, S., and Brümmer, M. (2013). Integrating NLP using linked data. In The Semantic Web -ISWC 2013 - 12th International Semantic Web Conference, Syd… find
  4. Hewlett, D., Lacoste, A., Jones, L., Polosukhin, I., Fandrianto, A., Han, J., Kelcey, M., and Berthelot, D. (2016). Wikireading: A novel large-scale language understanding task over wikipedia. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Augus… find
  5. Ji, H. and Grishman, R. (2011). Knowledge Base Population: Successful Approaches and Challenges. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA, pages 1148–1158. find
  6. Lebret, R., Grangier, D., and Auli, M. (2016). Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1203–1213. our card
  7. Li, X., Strassel, S., Ji, H., Griffitt, K., and Ellis, J. (2012). Linguistic resources for entity linking evaluation: from monolingual to cross-lingual. In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Eight International Conference on Language Resources and Evaluation… find
  8. Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., and McClosky, D. (2014). The stanford corenlp natural language processing toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, System D… our card
  9. Martin, T., Botschen, F., Nagesh, A., and McCallum, A. (2016). Call for discussion: Building a new standard dataset for relation extraction tasks. In Proceedings of the 5th Workshop on Automated Knowledge Base Construction, AKBC@NAACL-HLT 2016, San Diego, CA, USA, June 17, 2016, pages 92–96. Mendes,… find
  10. ACM. Mintz, M., Bills, S., Snow, R., and Jurafsky, D. (2009). Distant supervision for relation extraction without labeled data. In Proc. of ACL 2009, pages 1003–1011. find
  11. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). Squad: 100, 000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383–2392. our card
  12. Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., and Gamon, M. (2015). Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages… our card
  13. Xu, K., Reddy, S., Feng, Y., Huang, S., and Zhao, D. (2016). Question answering on freebase via relation extraction and textual evidence. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. our card
  14. Yao, L., Haghighi, A., Riedel, S., and McCallum, A. (2011). Structured relation discovery using generative models. In Proc. of EMNLP 2011. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elsahar-2018-t-rex-large-scale-alignment/