Elamur Research⌕
☰
home / papers / 2026 / A Study on Building Efficient Zero-Shot Relation Extraction Models

A Study on Building Efficient Zero-Shot Relation Extraction Models

Summary

Zero-shot relation extraction (RE) labels the link between two entity mentions using only textual side-information about relation types, such as names, aliases and descriptions, instead of annotated examples. The authors focus on a practical retrieval scenario, such as a journalist mining a large but fixed news archive with newly defined relation types. This setting imposes two requirements that earlier work rarely meets. First, utterances must be encoded once and cached (offline encoding). That requires late interaction and rules out marking head and tail mentions in the input. Second, irrelevant candidates must be discarded, which calls for a rejection mechanism. A new typology of zero-shot RE models shows that no recent system satisfies both requirements out of the box. The authors modify three open-source state-of-the-art systems (EMMA, RE-Matching, AlignRE) to support offline encoding and add three rejection strategies drawn from prior research. Experiments on FewRel and WikiZSL, with and without rejection, show that every model loses robustness under these realistic constraints. AlignRE still performs best overall, which offers practical guidance for scalable relation mining.

Key points

  • Two constraints are formalised: "on-the-fly" zero-shot classification, where the set of target relation types arrives with each query, and offline encoding, where text representations are precomputed and stored.
  • Inline entity markup (e.g. <E1>…</E1>) usually gives the best RE accuracy but blocks offline encoding. The typology also finds no late-interaction zero-shot RE model with a native rejection mechanism.
  • The adapted models are EMMA (Li et al., 2024a), RE-Matching (Zhao et al., 2023) and AlignRE (Li et al., 2024b). Each is combined with three generic, model-agnostic rejection options.
  • Evaluation uses FewRel and WikiZSL with tailored rejection procedures. Re-TACRED and NYT were set aside because they lack type side-information, and FewRel 2.0 does not match the on-the-fly scenario.
  • Generative LLM approaches are excluded because they cannot cache input representations. Code is released at gitlab.inria.fr/huthomas/zsre-models-adaptation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Information extraction

Cite

Cite as Hugo THOMAS, Caio Corro, Guillaume Gravier and Pascale Sébillot. 2026. A Study on Building Efficient Zero-Shot Relation Extraction Models. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5ndtgoxeph5j

@inproceedings{thomas2026study,
  title = {{A Study on Building Efficient Zero-Shot Relation Extraction Models}},
  author = {Hugo THOMAS, Caio Corro, Guillaume Gravier and Pascale Sébillot},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5ndtgoxeph5j},
  url = {https://doi.org/10.63317/5ndtgoxeph5j}
}

References

48 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maristella Agosti, Giorgio Di Nunzio, Stefano Marchesin, Gianmaria Silvello, et al. 2018. A relation extraction approach for clinical decision support. In Proceedings of the CIKM 2018 Workshops co-located with 27th ACM International Conference on Information and Knowledge Management (CIKM 2018); 12t… find
  2. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 2895–2905, Florence, Italy. Associati… our card
  3. Marília Barandas, Duarte Folgado, Ricardo Santos, Raquel Simão, and Hugo Gamboa. 2022. Uncertainty-based rejection in machine learning: Implications for model development and interpretability. Electronics, 11(3):396. find
  4. Elisa Bassignana and Barbara Plank. 2022. What do you mean by relation extraction? A survey on datasets and study on scientific relation classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL 2022), pages 67–83, Dubl… find
  5. Jack Boylan, Chris Hokamp, and Demian Gholipour Ghalandari. 2025. GLiREL - Generalist model for zero-shot relation extraction. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2025), page… find
  6. Vivien Cabannes, Alessandro Rudi, and Francis Bach. 2020. Structured prediction with partial labelling through the infimum loss. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), pages 1230–1239. PMLR. find
  7. Chih-Yao Chen and Cheng-Te Li. 2021. ZS-BERT: Towards zero-shot relation extraction with attribute representation learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2021), pages 3470–3479,… find
  8. Zi-Yuan Chen, Chih-Hung Chang, Yi-Pei Chen, Jijnasa Nayak, and Lun-Wei Ku. 2019. UHop: An unrestricted-hop relation extraction framework for knowledge-based question answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human… find
  9. Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si. 2022. RelationPrompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction. In Proceedings of Findings of the Association for Computational Linguistics: ACL 2022, pages 45–57, Dublin, Ireland. Association for C… find
  10. Caio Corro. 2024. A fast and sound tagging method for discontinuous named-entity recognition. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), pages 19506–19518, Miami, Florida, USA. Association for Computational Linguistics. find
  11. Timothee Cour, Ben Sapp, and Ben Taskar. 2011. Learning from partial labels. Journal of Machine Learning Research, 12(42):1501–1536. find
  12. Koby Crammer and Yoram Singer. 2002. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265–292. find
  13. Shumin Deng, Yubo Ma, Ningyu Zhang, Yixin Cao, and Bryan Hooi. 2024. Information extraction in low-resource scenarios: Survey and perspective. In Proceedings of the 15th IEEE International Conference on Knowledge Graphs (ICKG 2024), pages 33–49, Abu Dhabi, United Arab Emirates. IEEE. find
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… our card
  15. Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2019. FewRel 2.0: Towards more challenging few-shot relation classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural L… find
  16. Maryam Habibi, Leon Weber, Mariana Neves, David Luis Wiegandt, and Ulf Leser. 2017. Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics, 33(14):i37–i48. find
  17. Hassan Hamad, Abhinav Kumar Thakur, Nijil Kolleri, Sujith Pulikodan, and Keith Chugg. 2024. FIRE: A dataset for financial relation extraction. In Proceedings of Findings of the Association for Computational Linguistics: NAACL 2024, pages 3628–3642, Mexico City, Mexico. Association for Computational… find
  18. Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018. FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), p… find
  19. Tingting Hang, Shuting Liu, Jun Feng, Hamza Dji-gal, and Jun Huang. 2025. Few-shot relation extraction based on prompt learning: A taxonomy, survey, challenges and future directions. ACM Computing Surveys, 58(2):40. find
  20. Kilian Hendrickx, Lorenzo Perini, Dries Van der Plas, Wannes Meert, and Jesse Davis. 2024. Machine learning with a reject option: A survey. Machine Learning, 113(5):3073–3110. find
  21. Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), pages 2410–2421, Toulon, France. ICLR. find
  22. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over BERT. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020), page 39–48, New York, NY, USA. Associa… our card
  23. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancouver, Canada. Association for Computational Linguistics. find
  24. Guozheng Li, Peng Wang, and Wenjun Ke. 2023. Revisiting large language models as zero-shot relation extractors. In Proceeding of Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6877–6892, Singapore. Association for Computational Linguistics. find
  25. Shilong Li, Ge Bai, Zhang Zhang, Ying Liu, Chenji Lu, Daichi Guo, Ruifang Liu, and Sun Yong. 2024a. Fusion makes perfection: An efficient multi-grained matching approach for zero-shot relation extraction. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Comp… find
  26. Zehan Li, Fu Zhang, and Jingwei Cheng. 2024b. AlignRE: An encoding and semantic alignment approach for zero-shot relation extraction. In Proceedings of Findings of the Association for Computational Linguistics: ACL 2024, pages 2957–2966, Bangkok, Thailand. Association for Computational Linguistics. find
  27. Siyi Liu, Yang Li, Jiang Li, Shan Yang, and Yunshi Lan. 2024. Unleashing the power of large language models in zero-shot relation extraction via self-prompting. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13147–13161, Miami, Florida, USA. Association for Computati… find
  28. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, Lousiana, USA. OpenReview.net. find
  29. Marco Naguib, Xavier Tannier, and Aurélie Névéol. 2024. Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6829–6852, Miami, Florida, USA. A… find
  30. Liu Pai, Wenyang Gao, Wenjie Dong, Lin Ai, Ziwei Gong, Songfang Huang, Li Zongsheng, Ehsan Hoque, Julia Hirschberg, and Yue Zhang. 2024. A survey on open information extraction from rule-based model to large language model. In Proceedings of Findings of the Association for Computational Linguistics:… find
  31. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… our card
  32. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD’10), pages 148–163, Barcelona, Spain. Springer-Verlag. find
  33. Ofer Sabo, Yanai Elazar, Yoav Goldberg, and Ido Dagan. 2021. Revisiting few-shot relation classification: Evaluation data and classification schemes. Transactions of the Association for Computational Linguistics, 9:691–706. find
  34. Oscar Sainz, Oier Lopez de Lacalle, Gorka Labaka, Ander Barrena, and Eneko Agirre. 2021. Label verbalization and entailment for effective zero and few-shot relation extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021), pages 1199–1212, On… find
  35. Mourad Sarrouti, Carson Tao, and Yoann Mamy Randriamihaja. 2022. Comparing encoder-only and encoder-decoder transformers for relation extraction from biomedical texts: An empirical study on ten benchmark datasets. In Proceedings of the 21st Workshop on Biomedical Language Processing (BioNLP 2021), p… find
  36. Lawrence Stewart, Francis Bach, Felipe Llinares-Lopez, and Quentin Berthet. 2023. Differentiable clustering with perturbed spanning forests. In Advances in Neural Information Processing Systems (NeurIPS 2023), volume 36, pages 31158– 31176. Curran Associates, Inc. find
  37. George Stoica, Emmanouil Antonios Platanios, and Barnabás Póczos. 2020. Re-TACRED: A new relation extraction dataset. In Proceedings of the 4th Knowledge Representation and Reasoning Meets Machine Learning Workshop (KR2ML 2020, at NeurIPS’20), Online. find
  38. Hugo Thomas, Guillaume Gravier, and Pascale Sébillot. 2023. Derrière les plongements de relations. In Proceedings of the 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN 2023), pages 311–322, Paris, France. ATALA. find
  39. Hugo Thomas, Guillaume Gravier, and Pascale Sébillot. 2024. One-shot relation retrieval in news archives: adapting n-way k-shot relation classification for efficient knowledge extraction. Procedia Computer Science, 246:1060–1069. find
  40. Jiaxin Wang, Lingling Zhang, Jun Liu, Xi Liang, Yujie Zhong, and Yaqiang Wu. 2022a. Match-Prompt: Prompt-based open relation extraction with semantic consistency guided clustering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), pages 7875–7888… find
  41. Shusen Wang, Bosen Zhang, Yajing Xu, Yanan Wu, and Bo Xiao. 2022b. RCL: Relation contrastive learning for zero-shot relation extraction. In Proceedings of Findings of the Association for Computational Linguistics: NAACL 2022, pages 2456–2468, Seattle, Washington, USA. Association for Computational L… find
  42. Yang Yang, Zhilei Wu, Yuexiang Yang, Shuangshuang Lian, Fengjie Guo, and Zhiwei Wang. 2022. A survey of information extraction based on deep learning. Applied Sciences, 12(19):9691. find
  43. Zongqiang Yang, Junbo Fei, Zhen Tan, Jiuyang Tang, and Xiang Zhao. 2024. CL&CD: Contrastive learning and cluster description for zero-shot relation extraction. Knowledge-Based Systems, 293:111652. find
  44. Fu Zhang, He Liu, Zehan Li, and Jingwei Cheng. 2025. CE-DA: Custom embedding and dynamic aggregation for zero-shot relation extraction. In Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), pages 9814–9823, Abu Dhabi, United Arab Emirates. Association for Co… find
  45. Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. 2023. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 794–812, Toronto, Canada. Association for Computational Linguistics. find
  46. Jun Zhao, WenYu Zhan, Xin Zhao, Qi Zhang, Tao Gui, Zhongyu Wei, Junzhe Wang, Minlong Peng, and Mingming Sun. 2023. RE-Matching: A fine-grained semantic matching method for zero-shot relation extraction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2… find
  47. Xiaoyan Zhao, Yang Deng, Min Yang, Lingzhi Wang, Rui Zhang, Hong Cheng, Wai Lam, Ying Shen, and Ruifeng Xu. 2024. A comprehensive survey on relation extraction: Recent advances and new frontiers. ACM Computing Surveys, 56(11):293. find
  48. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. How does NLP benefit legal system: A summary of legal artificial intelligence. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), pages 5218–5230, Online. As… our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/thomas-2026-study-building-efficient-zero-shot/