Elamur Research⌕
☰
home / papers / 2026 / ReX-GG: A LLM Ensemble Pipeline for Relation-extraction and Graph Generation

ReX-GG: A LLM Ensemble Pipeline for Relation-extraction and Graph Generation

Paper · 2026 · Giacomo Magnifico and Eduard Barbu

Summary

Ensembles of several LLMs tend to become bulky, because they add ranking components, fine-tuned modules or external knowledge sources. Simpler setups risk spreading hallucinations and producing malformed output. Magnifico and Barbu (University of Tartu) present ReX-GG, a compact, configurable workflow. In it, several coordinated LLMs independently extract pairwise relations from a text and return them as schema-constrained JSON. The candidate answers are then anonymised, and every model scores all of them. The best extraction is therefore chosen by peer grading rather than by a dedicated ranker. The extracted triplets pass through an embedding model that merges semantically similar nodes. The result is an interactive HTML graph with colour-coded edges. The authors demonstrate the system on English popular-science articles, using four ontology-grounded relation categories. In an anonymous user study, participants compared outputs with a human-annotated gold standard and gave an average perceived accuracy of 0.778. The design is modular and openly licensed, and it works with either API-based or locally hosted models. This makes it a practical building block for knowledge-graph construction.

Key points

  • Relation types: strong causation (an explicit mechanism or causal chain), weak causation (correlational or probabilistic links), contrastive (A vs. B comparisons) and compositional (whole → part or definition). Few-shot prompts guide extraction. Head and tail spans should be minimal noun phrases, with pronouns and anaphora resolved.
  • Structured output: each model returns a JSON dictionary keyed by "document". Its value is a list of entries with the fields "bond", "span 1" and "span 2". In this first pass, models do not interact.
  • Anonymous peer ranking: candidate outputs are relabelled A, B, C… to prevent self-preference bias. Each model then grades every candidate on a 0–10 scale, using the JSON fields "answer", "points" and "thinking".
  • Evaluation: the user study gave an average perceived accuracy of 0.778 against the gold standard. Anecdotal feedback favoured the colour-coded edges.
  • Release and venue: the complete framework is made publicly available with an open licence. The paper appears in the KG-LLM Workshop @ LREC26 (2026), pp. 93–101.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TartuTartu · Estonia12 works · 2,874 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksInformation extraction · 167 works

Cite

Cite as Giacomo Magnifico and Eduard Barbu. 2026. ReX-GG: A LLM Ensemble Pipeline for Relation-extraction and Graph Generation. Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2qkh8idwebsv

@inproceedings{magnifico2026rex,
  title = {{ReX-GG: A LLM Ensemble Pipeline for Relation-extraction and Graph Generation}},
  author = {Giacomo Magnifico and Eduard Barbu},
  booktitle = {Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26},
  year = {2026},
  doi = {10.63317/2qkh8idwebsv},
  url = {https://doi.org/10.63317/2qkh8idwebsv}
}

References

22 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Introducing claude sonnet 4.5. find
  2. Amazon AWS. 2024. Amazon nova foundation models. find
  3. Pawel Bujnowski, Tomasz Dryjanski, Christian Goltz, Bartosz Swiderski, Natalia Paszkiewicz, Bartlomiej Kuzma, Jacek Rutkowski, Jakub Stepka, Milosz Dudek, Wojciech Siemiatkowski, Weronika Plichta, Bartłomiej Paziewski, Maciej Grabowski, Katarzyna Beksa, Zuzanna Bordz-icka, Filip Ostrowski, and Grzeg… find
  4. DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jia… find
  5. Google. 2025. Gemini 3 flash documentation. find
  6. Xinrui He, Yikun Ban, Jiaru Zou, Tianxin Wei, Curtiss Cook, and Jingrui He. 2025. LLM-forest: Ensemble learning of LLMs with graph-augmented prompts for data imputation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6921–6936, Vienna, Austria. Association for Computat… find
  7. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, Toronto, Canada. Assoc…
  8. Zehan Li, Ruhua Pan, and Xinyu Pi. 2025. Beyond LLMs a linguistic approach to causal graph generation from narrative texts. In Proceedings of the The 7th Workshop on Narrative Understanding, pages 36–51, Albuquerque, New Mexico. Association for Computational Linguistics. find
  9. Giacomo Magnifico. 2025. Automated classification of causal relations. evaluating different LLM performances. In Proceedings of the 9th Student Research Workshop associated with the International Conference Recent Advances in Natural Language Processing, pages 27–36, Varna, Bulgaria. INCOMA Ltd., Sh… find
  10. Meta. 2024. Llama 3.3: Model cards & prompt formats. find
  11. OpenAI. 2025. Introducing gpt 5.2. find
  12. Angelina Parfenova and Jürgen Pfeffer. 2025. Measuring what matters: Evaluating ensemble LLMs with label refinement in inductive coding. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10803–10816, Vienna, Austria. Association for Computational Linguistics. find
  13. Nguyen Pham Hoang Le, An Dinh Thien, Son T. Luu, and Kiet Van Nguyen. 2025. DocIE@XLLM25: ZeroSemble - robust and efficient zero-shot document information extraction with heterogeneous large language model ensembles. In Proceedings of the 1st Joint Workshop on Large Language Models and Structure Mod… find
  14. Lauren N. Ross. 2025. Explanation in Biology. Elements in the Philosophy of Biology. Cambridge University Press. find
  15. Selim Furkan Tekin, Fatih Ilhan, Tiansheng Huang, Sihao Hu, and Ling Liu. 2024. LLM-TOPLA: Efficient LLM ensemble by maximising diversity. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11951–11966, Miami, Florida, USA. Association for Computational Linguistics. find
  16. Junde Wu, Jiayuan Zhu, Yunli Qi, Jingkun Chen, Min Xu, Filippo Menolascina, Yueming Jin, and Vicente Grau. 2025. Medical graph RAG: Evidence-based medical large language model via graph retrieval-augmented generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin…
  17. Zhen Xiong, Yujun Cai, Zhecheng Li, and Yiwei Wang. 2025. Mapping the minds of LLMs: A graph-based analysis of reasoning LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 17751–17763, Suzhou, China. Association for Computational Linguistics.
  18. Yangyifan Xu, Jianghao Chen, Junhong Wu, and Jiajun Zhang. 2025. Hit the sweet spot! span-level ensemble for large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 8314–8325, Abu Dhabi, UAE. Association for Computational Linguistics.
  19. Dun Zhang, Ziyang Zeng, Yudong Zhou, and Shuyang Lu. 2025. Jasper-token-compression-600m technical report. find
  20. Mo Zhiqiang, Yang Hua, Jiahui Li, Yuan Liu, Shawn Wong, and Jianmin Huang. 2025. Judge and improve: Towards a better reasoning of knowledge graphs with large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5303–5320, Suzhou, China. As… find
  21. Yigeng Zhou, Wu Li, Yifan Lu, Jing Li, Fangming Liu, Meishan Zhang, Yequan Wang, Daojing He, Honghai Liu, and Min Zhang. 2025. Reflection on knowledge graph for large language models reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 23840–23857, Vienna, Austria… find
  22. Xiangrong Zhu, Yuexiang Xie, Yi Liu, Yaliang Li, and Wei Hu. 2025. Knowledge graph-guided retrieval augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/magnifico-2026-rex-gg-llm-ensemble-pipeline/