Elamur Research⌕
☰
home / papers / 2026 / Optimizing Efficiency in Multi-Stage Semantic Re-ranking Architectures

Optimizing Efficiency in Multi-Stage Semantic Re-ranking Architectures

Paper · 2026 · Artur M. A. Novais, Anna P. V. L. B. Moreira, Maria C. X. de Almeida et al.

Summary

Cross-encoder rerankers give strong ranking quality in legal search, but they are slow. A single heavy reranker is often too costly for large collections, and a single light one loses accuracy. The authors do not propose a new method. Instead, they test a known cascade design, a "semantic funnel", on Portuguese legal data. A small bi-encoder retrieves candidates. Off-the-shelf rerankers of growing size and cost then re-score smaller and smaller candidate lists, so the most expensive model sees only a few documents.

The task is document-to-document similarity search over a collection of 300,000 documents belonging to the Goiás State Court of Accounts (TCE-GO). Evaluation uses 250 documents, each paired with a generated summary that serves as its relevant match. The best three-stage cascade (BGE-v2-m3 → Qwen3-0.6B → Qwen3-4B) cuts latency per query by about 60% compared with the strongest single reranker, with only small losses in recall and MRR. This shows that cascades can make high-quality semantic legal search practical for interactive use.

Key points

  • Corpus and test set: 300,000 documents from 16,000 TCE-GO lawsuits, cleaned of headers, signatures and metadata. The test set has 250 document–summary pairs, and each summary is treated as the single correct match for its source document.
  • Best single-stage baseline: Qwen3-Reranker-4B reached R@avg 93.90% and MRR@avg 0.8505 → 0.9124 over the retriever alone. It took 11.75 s per query.
  • Fastest cascade: BGE-v2-m3 → Qwen3-0.6B → Qwen3-4B, with stage inputs of 30 → 20 → 10 candidates. It took 4.66 s per query (−60.3%), with R@avg 91.90% and MRR@avg 0.8900, about 2 pp and 0.0224 below the baseline.
  • Three stages beat two: the three-stage cascade was faster than the two-stage Qwen-only cascade (6.15 s). The authors credit the cheap BGE stage, which removes easy negatives before the costly Qwen models run.
  • Limitations:
    • Only one institution's data was used.
    • Tests ran on an NVIDIA A100 (80GB) GPU.
    • Juru was excluded because of its 512-token limit.
    • No statistical significance testing was done.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 2Research subjects and languages the work is about.Information retrieval & search · 473 worksLegal AI · 134 works

Cite

Cite as Artur M. A. Novais, Anna P. V. L. B. Moreira, Maria C. X. de Almeida, João P. C. Presa, Fernando M. Federson and Sávio S. T. de Oliveira. 2026. Optimizing Efficiency in Multi-Stage Semantic Re-ranking Architectures. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{novais2026optimizing,
  title = {{Optimizing Efficiency in Multi-Stage Semantic Re-ranking Architectures}},
  author = {Artur M. A. Novais, Anna P. V. L. B. Moreira, Maria C. X. {de Almeida}, João P. C. Presa, Fernando M. Federson and Sávio S. T. {de Oliveira}},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.55/}
}

References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S Althammer, A Askari, S Verberne, and AH Hanbury. 2021. Dossier@ coliee 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval. Proceedings of the eighth international competition on legal information extraction/entailment (COLIEE 2021), pages 8–14. find
  2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  3. G. Capannini, C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, and N. Tonellotto. 2016. Quality versus efficiency in document scoring with learning-to-rank models. Information Processing & Management, 52(5):902–918. find
  4. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. LEGAL-BERT: The muppets straight out of law school. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898– 2910, Online. Association for Computational Linguisti…
  5. Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, and Nikolaos Aletras. 2022. LexGLUE: A benchmark dataset for legal language understanding in English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:…
  6. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bang…
  7. Jun Guo, Bojian Chen, Zhichao Zhao, Jindong He, Shichun Chen, Donglan Hu, and Hao Pan. 2024. Bkrag: A bge reranker rag for similarity analysis of power project requirements. In Proceedings of the 2024 6th International Conference on Pattern Recognition and Intelligent Systems, PRIS ’24, page 14–20,… find
  8. Roseval Malaquias Junior, Ramon Pires, Roseli Romero, and Rodrigo Nogueira. 2024. Juru: Legal brazilian large language model from reputable sources. arXiv preprint arXiv:2403.18140. find
  9. C. Nguyen, P. M. Nguyen, and N. Le. 2025. Retrieve– Revise–Refine: A novel framework for retrieval of concise entailing legal article set. Information Processing & Management, 62:103949. find
  10. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-Stage Document Ranking with BERT. ArXiv preprint arXiv:1910.14424. find
  11. C. Sansone and G. Sperlí. 2021. Legal Information Retrieval systems: State-of-the-art and open issues. Information Systems, 101:101967. find
  12. Yoshihiko Sasazawa, Kazuya Yokote, Osamu Imaichi, and Yusuke Sogawa. 2023. Text Retrieval with Multi-Stage Re-Ranking Models. ArXiv preprint arXiv:2311.07994. — [arXiv:2311.07994](https://arxiv.org/abs/2311.07994) find
  13. M. van Opijnen and C. Santos. 2017. On the concept of relevance in legal information retrieval. Artificial Intelligence and Law, 25(1):65–87. find
  14. Te-Lun Yang, Jyi-Shane Liu, Yuen-Hsien Tseng, and Jyh-Shing Roger Jang. 2025. Knowledge Retrieval Based on Generative AI. Preprint, arXiv:2501.04635. — [arXiv:2501.04635](https://arxiv.org/abs/2501.04635) find
  15. Qin Zhang, Shangsi Chen, Dongkuan Xu, Qingqing Cao, Xiaojun Chen, Trevor Cohn, and Meng Fang. 2023. A survey for efficient open domain question answering. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 14447–14465. find
  16. K. Zheng, H. Zhao, R. Huang, B. Zhang, N. Mou, Y. Niu, Y. Song, H. Wang, and K. Gai. 2024. Full Stage Learning to Rank: A Unified Framework for Multi-Stage Systems. In Proceedings of the ACM on Web Conference 2024. find
  17. J. Zhu, X. Luo, and J. Wu. 2022. A BERT-Based Two-Stage Ranking Method for Legal Case Retrieval. In Challenge of AI in Law (CAIL 2021), pages 534–546. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/novais-2026-optimizing-efficiency-multi-stage-semantic/