Elamur Research
☰
home / papers / 2023 / Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture

Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-ND 4.0 (arXiv)" does not allow redistributing the text itself. Original — arxiv.org/pdf/2306.16092

Summary

General-purpose LLMs handle Chinese law poorly. They know too little about the national legal system and often hallucinate, for example by inventing statutes. Chatlaw addresses this with a multi-agent legal assistant modeled on how law firms divide work under Standard Operating Procedures (SOP). Four agents share the work: Legal Assistant (intake questioning), Legal Researcher (retrieving provisions and cases), Legal Editor (drafting) and Senior Lawyer (final review). They run on a Role-Aligned Mixture-of-Experts (RA-MoE) model. A standard MoE router assigns tokens only by data distribution. RA-MoE instead conditions routing on the agent's role and adds knowledge-aware gating, so each workflow step activates experts trained for that function. A curated legal dataset, built by deduplicating and denoising multi-source data, supports the system. A knowledge graph guides user interviews. Chatlaw beats GPT-4 both on LawBench and in China's unified examination that certifies legal practitioners, and legal experts rated it highest on real cases. The work suggests that aligning model internals with professional roles can make legal AI more reliable and accessible.

Key points

  • Compared with GPT-4, Chatlaw gains 7.73% accuracy on LawBench and earns 11 more points in China's unified examination that certifies legal practitioners.
  • Expert assessment of real cases covered four criteria: completeness, correctness, guidance and authority. Chatlaw received the best scores on all four.
  • RA-MoE combines role-conditioned routing with knowledge-aware gating, tying each expert to a task such as inquiry, analysis or drafting.
  • The Legal Researcher agent fact-checks outputs against external legal sources. The Legal Assistant uses the knowledge graph to resist users who conceal or distort facts.
  • Training took about 23.14 hours on eight 80GB A100 GPUs, with an estimated 42.81 kg of CO2 emitted. Future work includes compression for on-device use to address privacy concerns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Law

Cited by

152 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 144 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/j.fmre.2026.03.026 · arXiv 2306.16092

@misc{tian2023chatlaw,
  title = {{Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture}},
  author = {Tian;, Jiaxi Cui; Munan Ning; Zongjian Li; Bohua Chen; Yang Yan; Hao Li; Bin Ling; Yonghong and Yuan, Li},
  year = {2023},
  doi = {10.1016/j.fmre.2026.03.026},
  eprint = {2306.16092},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.fmre.2026.03.026}
}

References

77 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aminabadi, R.Y., Rajbhandari, S., et al., 2022. Deepspeed-inference: Enabling efficient inference of transformer models at unprecedented scale. arXiv preprint arXiv:2207.03669 . ↗
  2. Anthropic, 2025. Claude 3.7: Model card and performance benchmarks. https://www.anthropic.com/news/claude-3-7-sonnet . ↗
  3. Auerbach, J.S., 1977. Unequal justice: Lawyers and social change in modern America. Oxford University Press, New York. find
  4. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al., 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 . ↗ our card
  5. Boiko, D.A., MacKnight, R., Kline, B., Gomes, G., 2023. Autonomous chemical research with large language models. Nature 624, 570–578. find our card
  6. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al., 2020. Language models are few-shot learners, in: Advances in neural information processing systems, pp. 1877–1901. find
  7. Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., et al., 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297 . ↗ our card
  8. Chalkidis, I., Jana, A., Hartung, D., Bommarito, M., Androutsopoulos, I., Katz, D.M., Aletras, N., 2021. Lexglue: A benchmark dataset for legal language understanding in english. arXiv preprint arXiv:2110.00976 . ↗ our card
  9. Choi, J., Lee, B., 2024. Accelerating materials language processing with large language models. Communications Materials 5, 13. find
  10. Clusmann, J., Kolbinger, F.R., Muti, H.S., Carrero, Z.I., Eckardt, J.N., Laleh, N.G., Löffler, C.M.L., Schwarzkopf, S.C., Unger, M., Veldhuizen, G.P., et al., 2023. The future landscape of large language models in medicine. Communications medicine 3, 141. find
  11. Dao, T., 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691 . ↗
  12. Dao, T., Fu, D.Y., Ermon, S., et al., 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems . find
  13. Deng, W., Pei, J., Kong, K., Chen, Z., Wei, F., Li, Y., Ren, Z., Chen, Z., Ren, P., 2023. Syllogistic reasoning for legal judgment analysis, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13997–14009. find
  14. Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L., 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314 . ↗
  15. Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., Weston, J., 2023. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495 . ↗ our card
  16. Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., Tang, J., 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360 . ↗ our card
  17. Fan, W., Yang, Y., Qi, J., Zhang, Q., Liao, C., Wen, L., Wang, S., Wang, G., Xia, Y., Wu, Q., et al., 2024. A deep-learning-based framework for identifying and localizing multiple abnormalities and assessing cardiomegaly in chest x-ray. Nature Communications 15, 1347. find
  18. Farquhar, S., Kossen, J., Kuhn, L., Gal, Y., 2024. Detecting hallucinations in large language models using semantic entropy. Nature 630, 625–630. find
  19. Fedus, W., Zoph, B., Shazeer, N., 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23, 1–39. find
  20. Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Huang, A., Zhang, S., Chen, K., Yin, Z., Shen, Z., et al., 2024. Lawbench: Benchmarking legal knowledge of large language models, in: Proceedings of the 2024 conference on empirical methods in natural language processing, pp. 7933–7962. find our card
  21. Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Zhang, S., Chen, K., Shen, Z., Ge, J., 2023. Lawbench: Benchmarking legal knowledge of large language models. arXiv preprint arXiv:2309.16289 . ↗ our card
  22. Ferruz, N., Schmidt, S., Höcker, B., 2022. Protgpt2 is a deep unsupervised language model for protein design. Nature communications 13, 4348. find
  23. Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D., 2023. Gptq: Accurate post-training quantization for generative pre-trained transformers, in: ICLR. find
  24. Gan, B., Zhao, Y., Zhang, T., Huang, J., Li, Y., Teo, S.X., Zhang, C., Shi, W., 2025. Master: A multi-agent system with llm specialized mcts. arXiv preprint arXiv:2501.14304 . ↗
  25. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., Wang, H., 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 2. ↗ our card
  26. Gilbert, S., Harvey, H., Melvin, T., Vollebregt, E., Wicks, P., 2023. Large language model ai chatbots require approval as medical devices. Nature Medicine 29, 2396–2398. find
  27. Heinz, J.P., 2005. Urban lawyers: The new social structure of the bar. University of Chicago Press. find
  28. Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., et al., 2023. Metagpt: Meta programming for a multi-agent collaborative framework, in: The Twelfth International Conference on Learning Representations. find our card
  29. Hua, Y., Cao, H., Tao, Z., Li, B., Wu, Z., Liu, C., Xu, L., 2025. Input domain aware moe: Decoupling routing decisions from task optimization in mixture of experts. arXiv preprint arXiv:2510.16448 . ↗
  30. Huang, Q., Tao, M., An, Z., Zhang, C., Jiang, C., Chen, Z., Wu, Z., Feng, Y., 2023a. Lawyer llama technical report. ArXiv abs/2305.15062. ↗ our card
  31. Huang, Q., Tao, M., An, Z., Zhang, C., Jiang, C., Chen, Z., Wu, Z., Feng, Y., 2023b. Lawyer llama technical report. arXiv preprint arXiv:2305.15062 . ↗ our card
  32. Ji, Z., Lee, N., Frieske, R., et al., 2023. Survey of hallucination in natural language generation. ACM Computing Surveys . find our card
  33. Jiang, A.Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., Casas, D.d.l., Hanna, E.B., Bressand, F., et al., 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088 . ↗
  34. Kirk, H.R., Vidgen, B., Röttger, P., Hale, S.A., 2024. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence , 1–10. find
  35. Kiyasseh, D., Cohen, A., Jiang, C., Altieri, N., 2024. A framework for evaluating clinical artificial intelligence systems without ground-truth annotations. Nature Communications 15, 1808. find
  36. Krajewski, J., Ludziejewski, J., Adamczewski, K., Pióro, M., Krutul, M., Antoniak, S., Ciebiera, K., Król, K., Odrzygóźdź, T., Sankowski, P., et al., 2024. Scaling laws for fine-grained mixture of experts. arXiv preprint arXiv:2402.07871 . ↗
  37. Kuenneth, C., Ramprasad, R., 2023. polybert: a chemical language model to enable fully machine-driven ultrafast polymer informatics. Nature Communications 14, 4099. find
  38. Kuhn, L., Gal, Y., Farquhar, S., 2023. Semantic entropy offers robust measure of hallucination in large language models. Nature . find
  39. Kwon, W., Li, Z., Zhuang, S., et al., 2023. Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th Symposium on Operating Systems Principles. find
  40. Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., Chen, Z., 2020a. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 . ↗
  41. Lepikhin, D., Loh, H.J., Fedus, W., Chen, Y., et al., 2020b. Gshard: Scaling giant models with conditional computation and automatic sharding, in: ICLR. find
  42. Li, D., Ma, Y., Wang, N., Ye, Z., Cheng, Z., Tang, Y., Zhang, Y., Duan, L., Zuo, J., Yang, C., et al., 2024. Mixlora: Enhancing large language models fine-tuning with lora-based mixture of experts. arXiv preprint arXiv:2404.15159 . ↗
  43. Lin, S., Hilton, J., Evans, O., 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 . ↗ our card
  44. Lin, S., Hilton, J., Evans, O., 2022. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334 . ↗
  45. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al., 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 . ↗
  46. Lv, L., Lin, Z., Li, H., Liu, Y., Cui, J., Chen, C.Y.C., Yuan, L., Tian, Y., 2024. Prollama: A protein large language model for multi-task protein language processing. arXiv preprint arXiv:2402.16445 . ↗
  47. Maynez, J., Narayan, S., Bohnet, B., McDonald, R., 2020. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661 . ↗
  48. McCoy, R.T., Yao, S., Friedman, D., Hardy, M., Griffiths, T.L., 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638 . ↗
  49. Mik, E., 2024. Caveat lector: Large language models in legal practice. arXiv preprint arXiv:2403.09163 . ↗
  50. Nguyen, H.T., 2023. A brief report on lawgpt 1.0: A virtual legal assistant based on gpt-3. arXiv preprint arXiv:2302.05729 . ↗
  51. Niebla Zatarain, J.M., 2018. Artificial intelligence and legal analytics: New tools for law practice in the digital age. find
  52. Nigo, M., Rasmy, L., Mao, B., Kannadath, B.S., Xie, Z., Zhi, D., 2024. Deep learning model for personalized prediction of positive mrsa culture using time-series electronic health records. Nature Communications 15, 2036. find
  53. OpenAI, 2023. Gpt-4 technical report. arXiv:2303.08774 . ↗ our card
  54. OpenAI, 2025. Gpt-4.5 technical report. https://openai.com/index/introducing-gpt-4-5 . ↗
  55. Pei, C., Wang, Z., Liu, F., Li, Z., Liu, Y., He, X., Kang, R., Zhang, T., Chen, J., Li, J., et al., 2025. Flow-of-action: Sop enhanced llm-based multi-agent system for root cause analysis, in: Companion Proceedings of the ACM on Web Conference 2025, pp. 422–431. find
  56. Peng, D., Zheng, L., Liu, D., Han, C., Wang, X., Yang, Y., Song, L., Zhao, M., Wei, Y., Li, J., et al., 2024. Large-language models facilitate discovery of the molecular signatures regulating sleep and activity. Nature Communications 15, 3685. find
  57. Ren, R., Cai, S., Fang, X., Wang, X., Zhang, Z., Damiani, M., Hudlerova, C., Rosa, A., Hope, J., Cook, N.J., et al., 2023. Multiplexed detection of viral antigen and rna using nanopore sensing and encoded molecular probes. Nature Communications 14, 7362. find
  58. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., Dean, J., 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 . ↗
  59. Singh, C., Askari, A., Caruana, R., Gao, J., 2023. Augmenting interpretable models with large language models during training. Nature Communications 14, 7913. find
  60. Strachan, J.W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al., 2024. Testing theory of mind in large language models and humans. Nature Human Behaviour , 1–11. find
  61. Sun, X., Chen, Y., Huang, Y., Xie, R., Zhu, J., Zhang, K., Li, S., Yang, Z., Han, J., Shu, X., et al., 2024. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint arXiv:2411.02265 . ↗
  62. Suvarna, M., Vaucher, A.C., Mitchell, S., Laino, T., Pérez-Ramírez, J., 2023. Language models and protocol standardization guidelines for accelerating synthesis planning in heterogeneous catalysis. Nature Communications 14, 7964. find
  63. Tayebi Arasteh, S., Han, T., Lotfinia, M., Kuhl, C., Kather, J.N., Truhn, D., Nebelung, S., 2024. Large language models streamline automated machine learning for clinical studies. Nature Communications 15, 1603. find
  64. Team, I., 2023. Internlm: A multilingual language model with progressively enhanced capabilities. https://github.com/InternLM/InternLM . ↗
  65. Theodorou, B., Xiao, C., Sun, J., 2023. Synthesize high-dimensional longitudinal electronic health records via hierarchical autoregressive language model. Nature communications 14, 5305. find
  66. Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., Ting, D.S.W., 2023. Large language models in medicine. Nature medicine 29, 1930–1940. find
  67. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al., 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 . ↗
  68. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al., 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . ↗
  69. Tushnet, M.V., 2009. The rights revolution in the twentieth century. American Historical Association. find
  70. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al., 2024. Autogen: Enabling next-gen llm applications via multi-agent conversations, in: First Conference on Language Modeling. find
  71. Wu, S., Liu, Z., Zhang, Z., Chen, Z., Deng, W., Zhang, W., Yang, J., Yao, Z., Lyu, Y., Xin, X., Gao, S., Ren, P., Ren, Z., Chen, Z., 2023. fuzi.mingcha. https://github.com/irlab-sdu/fuzi.mingcha . ↗
  72. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al., 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 . ↗ our card
  73. Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al., 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 . ↗ our card
  74. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y., 2022. React: Synergizing reasoning and acting in language models, in: The eleventh international conference on learning representations. find our card
  75. Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al., 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414 . ↗ our card
  76. Zhou, Y., Lei, T., Liu, H., Du, N., et al., 2022. Mixture-of-experts with expert choice routing. Advances in Neural Information Processing Systems . find
  77. Zhou, Z., Shi, J.X., Song, P.X., Yang, X.W., Jin, Y.X., Guo, L.Z., Li, Y.F., 2024. Lawgpt: A chinese legal knowledge-enhanced large language model. arXiv preprint arXiv:2406.04614 . Table 6 Inference Performance and Throughput of Chatlaw-MoE under Various Configurations Metric Total Parameters Backbone Base Hardware Node Avg. Throughput (Single User) ↑ Avg. Throughput (100 CCU) ↑ Avg. Throughput (1000 CCU) ↑ (InternLLM) Avg. Throughput (100 CCU) ↑ (InternLLM) Avg. Throughput (1000 CCU) ↑ 6. Supplementary Material 6.1. Accelerating Deployment and Inference ↗