Elamur Research⌕
☰
home / papers / 2018 / Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning

Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning

Paper · 2018 · Pengda Qin, Weiran Xu and William Yang Wang

Summary

Distant supervision labels training sentences automatically by aligning a knowledge base (Freebase) with raw text. This produces many wrongly labeled positives. Earlier neural extractors either keep one sentence per entity pair or soft-weight sentences with attention. Neither approach helps when every sentence for an entity pair is wrong.

The authors make a hard decision instead. They train one policy-gradient reinforcement learning agent per relation type. For each distantly labeled positive sentence, the agent chooses to remove it or retain it. Removed sentences are not discarded but moved into the negative set. The reward is the change in F1 of a relation classifier retrained on the cleaned data, measured on a held-out validation split.

The method does not depend on the choice of classifier. Applied to the NYT–Freebase (Riedel) dataset, it improved PR curves and AUC for both the extractor from Zeng et al. (2015) and the one from Lin et al. (2016). It works as a plug-in denoising step for any distant supervision pipeline.

Key points

  • State, actions and reward:
    • The state joins the current sentence vector (word and position embeddings), weighted more heavily, with the average vector of previously removed sentences, which approximates an MDP.
    • The actions are {remove, retain}.
    • The reward is α·(F1ᵢ − F1ᵢ₋₁) between epochs.
  • Training setup:
    • Agents are pre-trained and then retrained.
    • A policy network was chosen over DQN because the reward only becomes available after a full pass over the training data.
    • Per-relation agents outperformed a single shared agent.
  • Evaluation: Experiments used the New York Times (Riedel et al., 2010) dataset. Gains for both the PCNN-style model (Zeng et al., 2015) and the attention model (Lin et al., 2016) were statistically significant (t-test, p < 0.05).
  • Where false positives occur: The detected false positives cluster in entity pairs with few sentences. Pairs with 1–5 sentences account for 55.9%, 32.0%, 3.7%, 4.4% and 0.7% of them, respectively. The authors attribute this to imperfect alignment between Freebase and NYT.
  • Example: The agent flagged (Sami Moubayed, Syria) as a false positive for /people/person/place_of_birth.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Santa BarbaraSanta Barbara · United States35 works · 9,287 citations🎓Beijing University of Posts and TelecommunicationsBeijing · China22 works · 32,364 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 109 works

Cite

Cite as Pengda Qin, Weiran Xu and William Yang Wang. 2018. Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1199

@inproceedings{qin2018robust,
  title = {{Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning}},
  author = {Pengda Qin, Weiran Xu and William Yang Wang},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1199},
  url = {https://doi.org/10.18653/v1/p18-1199}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866. — [arXiv:1708.05866](https://arxiv.org/abs/1708.05866) find
  2. Razvan Bunescu and Raymond J Mooney. 2005. Subsequence kernels for relation extraction. In NIPS, pages 171–178. find
  3. Meng Fang, Yuan Li, and Trevor Cohn. 2017. Learning how to active learn: A deep reinforcement learning approach. arXiv preprint arXiv:1708.02383. find
  4. Jun Feng, Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. 2018. Reinforcement learning for relation classification from noisy data. find
  5. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 363–370. Association for Computational Linguistic…
  6. Zhou GuoDong, Su Jian, Zhang Jie, and Zhang Min. 2005. Exploring various knowledge in relation extraction. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 427–434. Association for Computational Linguistics.
  7. Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2009. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of the Workshop on Seman…
  8. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1… find
  9. Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao, et al. 2017. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In AAAI, pages 3060–3066. find
  10. ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 1003–1011. Association for Computational Linguistics. find
  11. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. find
  12. Thien Huu Nguyen and Ralph Grishman. 2015. Event detection and domain adaptation with convolutional neural networks. In ACL (2), pages 365–371.
  13. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Machine Learning and Knowledge Discovery in Databases, pages 148–163. Springer. find
  14. Benjamin Roth, Tassilo Barth, Michael Wiegand, and Dietrich Klakow. 2013. A survey of noise reduction methods for distant supervision. In Proceedings of the 2013 workshop on Automated knowledge base construction, pages 73–78. ACM. find
  15. Cicero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. arXiv preprint arXiv:1504.06580.
  16. Yatian Shen and Xuanjing Huang. 2016. Attention-based convolutional neural network for semantic relation extraction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. find
  17. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489. find
  18. Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 455– 46… find
  19. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In ACL (1). find
  20. Kun Xu, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2015. Semantic relation classification via convolutional neural networks with simple negative sampling. arXiv preprint arXiv:1506.07650. find
  21. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2003. Kernel methods for relation extraction… find
  22. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), Lisbon, Portugal, pages 17–21. find
  23. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. 2014. Relation classification via convolutional deep neural network. In COLING, pages 2335–2344.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qin-2018-robust-distant-supervision-relation-extraction/