Elamur Research⌕
☰
home / papers / 2022 / RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction

RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction

Paper · 2022 · Yew Ken Chia, Lidong Bing, Soujanya Poria et al.

Summary

Most relation extraction systems need large annotated datasets and only handle a fixed set of relation types. This paper introduces Zero-Shot Relation Triplet Extraction (ZeroRTE). In this setting, a model must pull complete (head entity, tail entity, relation label) triplets out of raw sentences. The relation labels at test time never appear in training. Earlier zero-shot settings assumed gold entities or a given head entity, so ZeroRTE is the harder task.

The authors' framework, RelationPrompt, treats the problem as data synthesis. A language model is fine-tuned to act as a Relation Generator. It receives a relation label as a prompt and writes a sentence plus an entity pair in a structured text format. A separate Relation Extractor is then trained on this synthetic data for the unseen labels. Each generated sentence holds only one triplet, so the authors add Triplet Search Decoding to recover several triplets per sentence at inference.

On reorganized FewRel and Wiki-ZSL, RelationPrompt beats prior zero-shot relation classification (ZeroRC) methods and ZeroRTE baselines. The work matters because it gives a benchmark and a reference point for low-resource relation extraction.

Key points

  • New task: ZeroRTE is presented as the first setting that extracts full relation triplets with no annotated examples of the test relations. The seen and unseen label sets are disjoint.
  • Benchmark data: The benchmark is built from reorganized FewRel and Wiki-ZSL. Code and data are released at github.com/declare-lab/RelationPrompt.
  • Structured prompt templates:
    • The generator maps "Relation: <Label>" to a context sentence plus head and tail entities.
    • The extractor maps a sentence to head, tail and relation.
    • The same extractor also supports ZeroRC.
  • Triplet Search Decoding: This method lets an extractor trained only on single-triplet synthetic samples output multiple triplets. The authors describe it as effective and interpretable.
  • Analysis and limitations:
    • Generated samples are judged reasonable and diverse.
    • Performance is also studied as a function of the amount of synthetic data per label.
    • A noted weakness is that generated entity spans sometimes fit the prompted relation poorly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Singapore University of Technology and DesignSingapore · Singapore33 works · 10,760 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

Cite

Cite as Yew Ken Chia, Lidong Bing, Soujanya Poria and Luo Si. 2022. RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction. Findings of the Association for Computational Linguistics: ACL 2022.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-acl.5

@inproceedings{chia2022relationprompt,
  title = {{RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction}},
  author = {Yew Ken Chia, Lidong Bing, Soujanya Poria and Luo Si},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.5},
  url = {https://doi.org/10.18653/v1/2022.findings-acl.5}
}

References

43 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proc. of ACL. our card
  2. Rami Aly, Andreas Vlachos, and Ryan McDonald. 2021. Leveraging type descriptions for zero-shot named entity recognition and classification. In Proc. of ACL. find
  3. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proc. of ACL. our card
  4. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2001. A neural probabilistic language model. In Proc. of NeurIPS. our card
  5. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proc. of AAAI. find
  6. Chih-Yao Chen and Cheng-Te Li. 2021. Zs-bert: Towards zero-shot relation extraction with attribute representation learning. In Proc. of NAACL. our card
  7. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. CoRR, arXiv:2107.13586. our card
  8. Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021. Template-based named entity recognition using BART. In Findings of ACL-IJCNLP. our card
  9. Bosheng Ding, Linlin Liu, Lidong Bing, Canasai Kruengkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si, and Chunyan Miao. 2020. DAGA: Data augmentation with a generation approach for low-resource tagging tasks. In Proc. of EMNLP. our card
  10. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proc. of ICLR. find
  11. Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Generating training data with language models: Towards zero-shot language understanding. CoRR, arXiv:2202.04538. our card
  12. Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proc. of EMNLP. find
  13. Tapas Nayak and Hwee Tou Ng. 2020. Effective modeling of encoder-decoder architecture for joint entity and relation extraction. In Proc. of AAAI. find
  14. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. CoRR, abs/1503.02531. our card
  15. Abiola Obamuyide and Andreas Vlachos. 2018. Zero-shot relation classification as textual entailment. In Proc. of FEVER. find
  16. Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Sebastian Riedel, and Clare Voss. 2018. Zero-shot transfer learning for event extraction. In Proc. of ACL. our card
  17. Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cicero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2020. Structured prediction as translation between augmented natural languages. In Proc. of ICLR. find
  18. Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. 2017. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In Proc. of AAAI. find
  19. Yannis Papanikolaou and Andrea Pierleoni. 2020. DARE: data augmented relation extraction with GPT-2. CoRR, arXiv:2004.13845. — [arXiv:2004.13845](https://arxiv.org/abs/2004.13845) find
  20. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. CoRR, arXiv:1909.05858. our card
  21. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proc. of EMNLP-IJCNLP. our card
  22. Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic relations. In Proc. of NAACL. our card
  23. Raul Puri and Bryan Catanzaro. 2019. Zero-shot text classification with generative language models. CoRR, arXiv:1912.10165. — [arXiv:1912.10165](https://arxiv.org/abs/1912.10165) find
  24. Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, and Hyung Won Chung. 2021. Neural data augmentation via example extrapolation. CoRR, arXiv:2102.01335. — [arXiv:2102.01335](https://arxiv.org/abs/2102.01335) find
  25. Pushpankar Kumar Pushp and Muktabh Mayank Srivastava. 2017. Train once, test anywhere: Zero-shot learning for text classification. CoRR, arXiv:1712.05972. — [arXiv:1712.05972](https://arxiv.org/abs/1712.05972) find
  26. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proc. of CoNLL. find
  27. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI. find
  28. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. find
  29. Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Proc. of CHI. our card
  30. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proc. of EMNLP. find
  31. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomás Kociský, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In Proc. of ICLR. our card
  32. Alisa Smirnova and Philippe Cudré-Mauroux. 2018. Relation extraction using distant supervision: A survey. CSUR, 51(5):1:35. find
  33. Jue Wang and Wei Lu. 2020. Two are better than one: Joint entity and relation extraction with table-sequence encoders. In Proc. of EMNLP. our card
  34. Wei Wang, Vincent W. Zheng, Han Yu, and Chunyan Miao. 2019. A survey of zero-shot learning: Settings, methods, and applications. TIST, 10(2):1:37. find
  35. Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proc. of EMNLP-IJCNLP. our card
  36. Shanchan Wu and Yifan He. 2019. Enriching pre-trained language model with entity information for relation classification. In Proc. of CIKM. find
  37. Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. 2020. Unsupervised data augmentation for consistency training. In Proc. of NeurIPS. our card
  38. Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2016. Question answering on freebase via relation extraction and textual evidence. In Proc. of ACL. our card
  39. Lu Xu, Yew Ken Chia, and Lidong Bing. 2021. Learning span-level interactions for aspect sentiment triplet extraction. In Proc. of ACL. our card
  40. Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. 2022. Zerogen: Efficient zero-shot learning via dataset generation. CoRR, arXiv:2202.07922. our card
  41. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In Proc. of ICML. our card
  42. Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021. Meta-tuning language models to answer prompts better. In Findings of EMNLP. find
  43. Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proc. of NAACL. *Figure 7: Additional sentence samples from the datasets. The head and tail entities are shown in blue and orange, respectively.* *Figure 8: Additional synthetic samples from the g… our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chia-2022-relationprompt-leveraging-prompts-generate-synthetic/