Elamur Research⌕
☰
home / papers / 2025 / LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning

LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning

Paper · 2025 · Weijie Shi, Han Zhu, Jiaming Ji et al.

Summary

Legal judgment prediction (LJP) asks a model to produce a court-like ruling from a plaintiff's claim and the case facts. LLMs often make logical mistakes when a case needs multi-step legal analysis. The authors propose LegalReasoner, which first identifies a case's dispute points and then reasons toward a verdict one step at a time. A process verifier rates every step on correctness, progressiveness and potential. When a step falls short, a classifier assigns the flaw to an error type defined by legal experts, such as misapplying a legal principle or reasoning that conflicts with the facts. A matching remedy then runs, for example retrieving provisions and precedents or tracking facts. To train the system, the authors built LegalHK, a corpus of Hong Kong court decisions annotated with disputes, reasoning chains and step-level verification labels. The annotations were extracted mainly with GPT-4 and screened by judicial experts. On LLAMA-3.1-70B, agreement with real court outcomes rises from 72.37 to 80.27. The work shows that step-level supervision, previously used mostly for math and code, can carry over to legal reasoning.

Key points

  • LegalHK contains 58,130 Hong Kong cases, filtered from 185,495 decisions dated 1900–2023 and scraped from HKLII. It is released on Hugging Face as weijiezz/LegalHK.
  • Decisions from the Court of Final Appeal and the Court of Appeal were excluded, since the official descriptions of such cases tend to be shortened and lack detail. Before GPT-4 structuring, documents were compressed to roughly one-third to one-half of their original length.
  • Fact descriptions were checked by GPT-4 and by judicial experts so that they do not reveal the final verdict.
  • The verifier scores each step on three criteria: correctness, progressiveness and potential. Detected errors are routed to expert-designed attribution and resolution strategies.
  • The authors list three limitations:
    • Step-wise checking is computationally expensive, though they report it beats Best-of-N sampling at lower cost.
    • The automatically generated verifier labels are noisy.
    • How well the approach transfers to other legal systems remains open.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Hong Kong University of Science and TechnologyHong Kong · China142 works · 25,136 citations🎓Soochow UniversitySuzhou · China63 works · 4,507 citations🎓Zhejiang Normal UniversityJinhua · China4 works · 16 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPolitical Science and International Relations · 182 worksLaw · 77 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksLegal AI · 134 works

Cite

Cite as Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han and Yike Guo. 2025. LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.361

@inproceedings{shi2025legalreasoner,
  title = {{LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning}},
  author = {Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han and Yike Guo},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.361},
  url = {https://doi.org/10.18653/v1/2025.acl-long.361}
}

References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adel Ammar, Anis Koubaa, Bilel Benjdira, Omer Nacar, and Serry Sibaee. 2024. Prediction of arabic legal rulings using large language models. Electronics, 13(4):764. ↗
  2. Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in english. arXiv.
  3. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. Legal-bert: The muppets straight out of law school. arXiv.
  4. Jiaxi Cui, Munan Ning, Zongjian Li, Bohua Chen, Yang Yan, Hao Li, Bin Ling, Yonghong Tian, and Li Yuan. 2024. Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model. arXiv. find
  5. Junyun Cui, Xiaoyu Shen, and Shaochun Wen. 2023. A survey on legal judgment prediction: Datasets, metrics, models and challenges. IEEE Access. ↗
  6. Francisco de Arriba-Pérez, Silvia García-Méndez, Francisco J González-Castaño, and Jaime González-González. 2022. Explainable machine learning multi-label classification of spanish legal judgements. Journal of King Saud University-Computer and Information Sciences, 34(10):10180–10192. ↗
  7. Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh. 2024. Ensemble methods for improving extractive summarization of legal case judgements. Artificial Intelligence and Law, 32(1):231–289. ↗
  8. Yi Feng, Chuanyi Li, and Vincent Ng. 2022a. Legal judgment prediction: A survey of the state of the art. In IJCAI, pages 5461–5469. find
  9. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv.
  10. Dugang Liu, Weihao Du, Lei Li, Weike Pan, and Zhong Ming. 2022. Augmenting legal judgment prediction with contrastive case relations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2658–2667. find
  11. Yifei Liu, Yiquan Wu, Yating Zhang, Changlong Sun, Weiming Lu, Fei Wu, and Kun Kuang. 2023. Ml-ljp: multi-law aware legal judgment prediction. In SIGIR, pages 1023–1034. find
  12. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. NIPS, 36. find
  13. Mihai Masala, Traian Rebedea, and Horia Velicu. 2024. Improving legal judgement prediction in romanian with long text encoders. arXiv. find
  14. Cristian Millan-Arias, Marie Chantelle C Medina, Jean Felipe C Ferreira, Lucas M Da Silva Oliveira, Al-lana Rocha, Leandro Honorato de S Silva, Cley-ton Mario O Rodrigues, João Fausto L de Oliveira, Paulo Christiano Sobral, Nathalia Cauás, et al. 2023. Legal judgement prediction: A comparative analysis of legal documents in brazil. In 2023 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pages 1–6. IEEE. find
  15. Yi Feng, Chuanyi Li, and Vincent Ng. 2022b. Legal judgment prediction via event extraction with constraints. In ACL, pages 648–664. find
  16. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, et al. 2024. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv. ↗
  17. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv. ↗
  18. Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2023. Refiner: Reasoning feedback on intermediate representations. arXiv. ↗
  19. TYSS Santosh, Mohamed Hesham Elganayni, Stanisław Sójka, and Matthias Grabmair. 2024. Incorporating precedents for legal judgement prediction on european court of human rights cases. arXiv. ↗
  20. Jingyun Sun, Shaobin Huang, and Chi Wei. 2024. Chinese legal judgment prediction via knowledgeable prompt learning. Expert Systems with Applications, 238:122177. ↗
  21. Dietrich Trautmann, Alina Petrova, and Frank Schilder. 2022. Legal prompt engineering for multilingual legal judgement prediction. arXiv. ↗
  22. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9426–9439. ↗
  23. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv. ↗
  24. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022. Re3: Generating longer stories with recursive reprompting and revision. arXiv. find
  25. Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Rest-mcts*: Llm self-training via process reward guided tree search. arXiv.
  26. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. NIPS, 36:46595–46623.
  27. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shi-2025-legalreasoner-step-wised-verification-correction/