Elamur Research⌕
☰
home / papers / 2026 / Prompt Optimization for Relation Extraction using Reinforcement Learning

Prompt Optimization for Relation Extraction using Reinforcement Learning

Paper · 2026 · Ying Liu, Dong Shuai, Cui Zibo et al.

Summary

Prompting LLMs for relation extraction (RE) avoids the large annotated corpora that supervised models need. However, results depend heavily on how the prompt is written, and hand-crafting prompts for specialised domains takes expertise and trial-and-error. The authors present REPO, which treats prompt construction as a sequential decision problem solved with deep reinforcement learning against a black-box LLM. It works in two stages. First, a heuristic procedure uses the LLM to generate seed prompts from example instances and keeps the best ones. Second, a DRL agent iteratively edits these prompts using a restricted, domain-aware set of actions. The reward is a composite score, REPQS, that mixes exact-match triple F1 with SBERT-based semantic similarity, which gives a denser learning signal than accuracy alone. REPO was tested on four datasets covering legal, financial, medical and news text. With GPT-4o, it beat the prompt optimizers APE, OPRO and SPO as well as a supervised CasRel baseline. This suggests that automated, structured prompt search can make LLM-based RE practical where labelled data is scarce.

Key points

  • Datasets and F1 scores: Using GPT-4o, REPO scored an F1 of 0.72 for LexEval, with FinCUGE at 0.60, CMeIE at 0.58 and LCN at 0.56. This is roughly 4–8% better than APE, OPRO and SPO, with the largest margins on FinCUGE and LCN.
  • Comparison with CasRel: The supervised CasRel (BERT) baseline was strong in precision on the data-rich LexEval but fell to 0.32–0.45 F1 on the other three datasets.
  • Reward design: The REPQS reward combines F1 and SBERT cosine similarity over entities and relations through weights α and β.
  • Fine-tuned variant: REPO-FT applies LoRA fine-tuning to Qwen2.5-7B-Instruct-1M. It added 0.05–0.08 F1 on three datasets but dropped slightly on LCN, for an average gain of about 4.7%.
  • Ablation and cost: Removing either stage ("Only Init" or "Only RL") cut F1 by 0.02–0.07. The authors report that optimization on most public datasets needs fewer than 3M tokens and fewer than 5K API calls. Code is available at github.com/dddong2-star/REPO.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MetaMenlo Park · United States183 works · 228,558 citations🎓Northeastern UniversityShenyang · China42 works · 1,940 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 167 works

Cite

Cite as Ying Liu, Dong Shuai, Cui Zibo, TengQi Ye and Gang Wu. 2026. Prompt Optimization for Relation Extraction using Reinforcement Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2100

@inproceedings{liu2026prompt,
  title = {{Prompt Optimization for Relation Extraction using Reinforcement Learning}},
  author = {Ying Liu, Dong Shuai, Cui Zibo, TengQi Ye and Gang Wu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2100},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2100}
}

References

33 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  2. Xiang Chen, Ningyu Zhang, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, and Huajun Chen. 2022. Hybrid transformer with multi-level fusion for multimodal knowledge graph completion. In Proceedings of the 45th international ACM SIGIR conference on research and development in info… find
  3. Kalpit Dixit and Yaser Al-Onaizan. 2019. Span-level model for relation extraction. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 5308–5314. find
  4. Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun, and Yu Su. 2022. Thinking about gpt-3 in-context learning for biomedical ie? think again. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4497–4512.
  5. Edward J. Hu, Yelong Shen, Phil Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Swabha Wang, Lu Wang, Weizhu Chen, and Denny Zhou. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR). Accepted to ICLR 2022.
  6. Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423–438.
  7. Guozheng Li, Peng Wang, and Wenjun Ke. 2023a. Revisiting large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6877– 6892. find
  8. Haitao Li, You Chen, Qingyao Ai, Yueyue Wu, Ruizhe Zhang, and Yiqun Liu. 2024. Lexeval: A comprehensive chinese legal benchmark for evaluating large language models. Advances in Neural Information Processing Systems, 37:25061–25094. find
  9. Junpeng Li, Zixia Jia, and Zilong Zheng. 2023b. Semi-automatic data enhancement for document-level relation extraction with distant supervision from large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5495–5505. find
  10. Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, and Muhao Chen. 2022. Summarization as indirect supervision for relation extraction. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6575–6594. find
  11. Kangqi Luo, Fengli Lin, Xusheng Luo, and Kenny Zhu. 2018. Knowledge base question answering via encoding of complex query graphs. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2185–2194. find
  12. Wenxin Luo, Weirui Wang, Xiaopeng Li, Weibo Zhou, Pengyue Jia, and Xiangyu Zhao. 2025. Tapo: Task-referenced adaptation for prompt optimization. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  13. Angrosh Mandya, Danushka Bollegala, and Frans Coenen. 2020. Graph convolution over multiple dependency sub-graphs for relation extraction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6424–6435. International Committee on Computational Linguistics. find
  14. Tapas Nayak and Hwee Tou Ng. 2020. Effective modeling of encoder-decoder architecture for joint entity and relation extraction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8528–8535. find
  15. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  16. Hado Van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double qlearning. In Proceedings of the AAAI conference on artificial intelligence, volume 30. find
  17. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1340–1350.
  18. Somin Wadhwa, Silvio Amir, and Byron C Wallace. 2023. Revisiting relation extraction in the era of large language models. In Proceedings of the conference. association for computational linguistics. meeting, volume 2023, page 15566. find
  19. Siyi Liu, Yang Li, Jiang Li, Shan Yang, and Yunshi Lan. 2024. Unleashing the power of large language models in zero-shot relation extraction via self-prompting. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13147–13161. find
  20. Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, and Sadao Kurohashi. 2023. Gpt-re: In-context learning for relation extraction using large language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 3534–3547.
  21. Dakuan Lu, Hengkui Wu, Jiaqing Liang, Yipei Xu, Qianyu He, Yipeng Geng, Mengkun Han, Yingsi Xin, and Yanghua Xiao. 2023. Bbt-fin: Comprehensive construction of chinese financial domain pre-trained language model, corpus and benchmark. arXiv preprint arXiv:2302.09432. — [arXiv:2302.09432](https://arxiv.org/abs/2302.09432) find
  22. Zhepei Wei, Jianlin Su, Yue Wang, Yuan Tian, and Yi Chang. 2020. A novel cascade binary tagging framework for relational triple extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1476– 1488.
  23. Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, and Yuyu Luo. 2025. Self-supervised prompt optimization. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 9017–9041. find
  24. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations.
  25. Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1785–1794.
  26. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. In The Twelfth International Conference on Learning Representations.
  27. Zuoxi Yang. 2020. Biomedical information retrieval incorporating knowledge graph for explainable precision medicine. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2486– 2486. find
  28. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers, pages 2335–2344.
  29. Daojian Zeng, Haoran Zhang, and Qianying Liu. 2020. Copymtl: Copy mechanism for joint extraction of entities and relations with multi-task learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9507–9514. find
  30. Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, and 1 others. 2022a. Cblue: A chinese biomedical language understanding evaluation benchmark. In Proceedings of the 60th annual meeting of the association for computational linguisti…
  31. Rui Zhang, Bayu Distiawan Trisedya, Miao Li, Yong Jiang, and Jianzhong Qi. 2022b. A benchmark and comprehensive survey on knowledge graph entity alignment via representation learning. The VLDB Journal, 31(5):1143–1168. find
  32. Tianyang Zhao, Zhao Yan, Yunbo Cao, and Zhoujun Li. 2021. Asking effective and diverse questions: A machine reading comprehension based framework for joint entity-relation extraction. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelli… find
  33. Xiaoyan Zhao, Min Yang, Qiang Qu, and Ruifeng Xu. 2024. Few-shot relation extraction with automatically generated prompts. IEEE Transactions on Neural Networks and Learning Systems, 36(3):4971– 4983. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-prompt-optimization-relation-extraction-reinforcement/