Elamur Research⌕
☰
home / papers / 2026 / LegalSim-PT: Building a Dataset for Legal Document Simplification in Portuguese Leveraging Linguistic Metrics

LegalSim-PT: Building a Dataset for Legal Document Simplification in Portuguese Leveraging Linguistic Metrics

Summary

Most document-level text simplification research targets English. Portuguese has had no sizable resource for simplifying whole documents, especially legal ones, whose dense syntax and specialized vocabulary hinder public access to justice. The authors address this gap with LegalSim-PT, a synthetic parallel corpus of complex and simplified Brazilian legal documents. They took source texts from the Brazilian portion of LegalPT and had the compact Qwen3-4B model produce simplified versions. They then kept only pairs that passed automatic checks for semantic consistency, readability gain and diversity, which reduces the need for manual review.

The corpus was assessed in several ways:

  • surface statistics compared against existing simplification corpora;
  • human ratings;
  • two LLM judges;
  • fine-tuning experiments with open models, evaluated with D-SARI, semantic similarity and Flesch.

Models fine-tuned on LegalSim-PT produced more readable outputs that kept more of the source meaning than their base versions. This makes the dataset a useful public benchmark and training resource for Portuguese legal simplification.

Key points

  • Construction: The source is LegalPT (about 12M documents in 11 sub-datasets), of which the six Brazilian-law sub-datasets were used. Qwen3-4B generated the simplifications with an 8,192-token context and a 4,096-token output cap, and only about 2% of documents went over this limit. Filtering relied on chunk-level Sentence-BERT similarity, readability and diversity. The data is released on GitHub (scalercio/legal-doc-simplification-data).
  • Human evaluation: 40 document pairs were rated.
  • LLM judges: GPT-OSS-120B and LLaMA-70B both ranked the outputs Ours > Qwen2.5 > Bode. Weighted Cohen's κ between the judges was 0.74 overall and 0.85 on hard samples.
  • Fine-tuning on the LegalSim-PT test set: Qwen2.5 rose from 35.74 to 50.78 in D-SARI, from 31.8 to 42.7 in Flesch, and from .653 to .810 in SIM. Qwen3 rose from 32.16 to 37.22 in D-SARI.
  • Other baselines and cross-dataset results: Bode and Tucano served as Portuguese LLM baselines and struggled to preserve content in long documents. On the sentence-level GovLang-BR set, fine-tuning raised Flesch but lowered D-SARI, mostly because the models deleted less.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing · Law

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{scalercio2026legalsim,
  title = {{LegalSim-PT: Building a Dataset for Legal Document Simplification in Portuguese Leveraging Linguistic Metrics}},
  author = {Scalercio, Arthur and Finatto, Maria José and Paes, Aline},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.39/}
}

References

57 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. This research was supported by CNPq (National Council for Scientific and Technological Development), grant 307088/2023-5, FAPERJ - Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro, processes SEI-260003/002930/2024, SEI-260003/000614/2023, and CAPES. We also thank the support of the CNPq National Institutes of Science and Technology, IAIA (grant 406417/2022-9), TILD-IAR (grant 408490/2024-1) and IAPROBEM (grant 408589/2024-8). find
  2. Suha S Al-Thanyyan and Aqil M Azmi. 2021. Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1–36. find
  3. Rodrigo Alarcón, Lourdes Moreno, and Paloma Martínez. 2021. Lexical simplification system to improve web accessibility. IEEE Access, 9:58755– 58767. ↗
  4. Sandra Aluísio and Caroline Gasperin. 2010. Fostering digital inclusion and accessibility: the porsimples project for simplification of portuguese texts. In Proc. of the NAACL HLT 2010 Young Investigators Workshop on Computational Approaches to Languages of the Americas, pages 46–53. find
  5. Sandra M Aluísio, Lucia Specia, Thiago AS Pardo, Erick G Maziero, and Renata PM Fortes. 2008. Towards brazilian portuguese automatic text simplification systems. In Proc. of the 8th ACM symposium on Document engineering, pages 240–248. find
  6. Alexandre Alves, Péricles B. C. Miranda, Rafael Fe Mello, and André C. A. Nascimento. 2023. Automatic simplification of legal texts in portuguese using machine learning. In Legal Knowledge and Information Systems - JURIX 2023: The 36th Annual Conference, volume 379 of Frontiers in Artificial Intelligence and Applications, pages 281–286. IOS Press. ↗
  7. Miriam Anschütz, Joshua Oehms, Thomas Wimmer, Bartłomiej Jezierski, and Georg Groh. 2023. Language models for german text simplification: Overcoming parallel data scarcity through style-specific pre-training. In Findings of the ACL: ACL 2023, pages 1147–1158. find
  8. Farid Ariai, Joel Mackenzie, and Gianluca Demartini. 2024. Natural language processing for the legal domain: A survey of tasks, datasets, models, and challenges. arXiv preprint arXiv:2410.21306. ↗
  9. Gabriel Assis, Cláudia Freitas, and Aline Paes. 2025. Exploring brazil’s llm fauna: Investigating the generative performance of large language models in portuguese. Journal of the Brazilian Computer Society, 31(1):940–972. ↗
  10. Luis Cabrera. 2024. Babel fish democracy? prospects for addressing democratic language barriers through machine translation and interpretation. American Journal of Political Science, 68(2):767–782. find
  11. M Charles. 2013. Active and passive voice in research articles: An interdisciplinary study. International Journal of Corpus Linguistics, 18(3):279–318. find
  12. Jenny Chim, Julia Ive, and Maria Liakata. 2025. Evaluating synthetic data generation from user generated text. Computational Linguistics, 51(1):191–233. ↗
  13. Letícia MS Corrêa, Erica dos S Rodrigues, and Renê Forster. 2019. On the processing of object relative clauses. ExLing 2019, 25:57. ↗
  14. Kambhatla. 2022. Text simplification for legal domain: Insights and challenges. In Proc. of the Natural Legal Language Processing Workshop 2022, pages 296–304. find
  15. Edward Gibson. 1998. Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1):1–76. ↗
  16. Koki Horiguchi, Tomoyuki Kajiwara, Takashi Ninomiya, Shoko Wakamiya, and Eiji Aramaki. 2025. Multimsd: A corpus for multilingual medical text simplification from online medical references. In Findings of the Association for Computational Linguistics, ACL 2025, pages 9248–9258. ACL. find
  17. Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, and Shiza Fatimah. 2025. Tucano: Advancing Neural Text Generation for Portuguese. Patterns. ↗
  18. Liam Cripwell, Joël Legrand, and Claire Gardent. 2023a. Context-aware document simplification. In Findings of the ACL: ACL 2023, pages 13190–13206. ACL. find
  19. Liam Cripwell, Joël Legrand, and Claire Gardent. 2023b. Document-level planning for text simplification. In 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 993– 1006. ACL. find
  20. Liam Cripwell, Joël Legrand, and Claire Gardent. 2024. Evaluating document simplification: On the importance of separately assessing simplicity and meaning preservation. CoRR, abs/2404.03278. ↗
  21. Michael Han Daniel Han and Unsloth team. 2023. Unsloth. find
  22. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: efficient finetuning of quantized llms. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc. find
  23. Ashwin Devaraj, William Sheffield, Byron C Wallace, and Junyi Jessy Li. 2022. Evaluating factuality in text simplification. In Proc. of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7331–7345. find
  24. Eduardo A. S. Garcia, Nadia F. F. Silva, Felipe Siqueira, Hidelberg O. Albuquerque, Juliana R. S. Gomes, Ellen Souza, and Eliomar A. Lima. 2024a. RoBER-TaLexPT: A legal RoBERTa model pretrained with deduplication for Portuguese. In Proc. of the 16th Int. Conf. on Computational Processing of Portuguese, pages 374–383. ACL. find
  25. Gabriel Lino Garcia, Pedro Henrique Paiola, Luis Henrique Morelli, Giovani Candido, Arnaldo Cândido Júnior, Danilo Samuel Jodas, Luis C. S. Afonso, Ivan Rizzo Guilherme, Bruno Elias Penteado, and João Paulo Papa. 2024b. Introducing bode: A fine-tuned large language model for portuguese prompt-based task. Preprint, arXiv:2401.02909. ↗
  26. Aparna Garimella, Abhilasha Sancheti, Vinay Aggarwal, Ananya Ganesh, Niyati Chhaya, and Nanda Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. ↗
  27. Sebastian Joseph, Kathryn Kazanas, Keziah Reina, Vishnesh J. Ramanathan, Wei Xu, Byron C. Wallace, and Junyi Jessy Li. 2023. Multilingual simplification of medical texts. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, pages 16662–16692. ACL. find
  28. Hong Jin Kang, Muhammad Ali Gulzar, Nanyun Peng, Miryung Kim, and 1 others. 2024. Human-in-the-loop synthetic text data inspection with provenance tracking. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3118–3129. find
  29. J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical report. ↗
  30. Sidney Evaldo Leal, Magali Sanchez Duran, Carolina Evaristo Scarton, Nathan Siegle Hartmann, and Sandra Maria Aluísio. 2023. Nilc-metrix: assessing the complexity of written and spoken language in brazilian portuguese. Lang Resources & Evaluation. ↗
  31. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017. ↗
  32. Louis Martin, Angela Fan, Éric de la Clergerie, Antoine Bordes, and Benoît Sagot. 2022. MUSS: multilingual unsupervised sentence simplification by mining paraphrases. In Proc. of the 13th Language Resources and Evaluation Conference, LREC 2022, pages 1651– 1664. ELRA. find our card
  33. Paloma Martínez, Lourdes Moreno, Hiram Ochoa, Alberto Ramos, and Mario Pérez-Enríquez. 2024. A tool suite for cognitive accessibility leveraging easy-to-read resources and simplification strategies. CEUR-WS. org. find
  34. Yoshinari Nagai, Teruaki Oka, and Mamoru Komachi. 2024. A document-level text simplification dataset for Japanese. In Proc. of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 459–476. ELRA and ICCL. find
  35. Francielle Vasconcellos Pereira, Ana Frazão, and Viviane P. Moreira. 2024. Automatic text simplification for the legal domain in brazilian portuguese. In Intelligent Systems - 34th Brazilian Conference, BRACIS 2024, Proceedings, Part IV, volume 15415 of LNAI, pages 31–45. Springer. find
  36. Nelson Perez-Rojas, Saúl Calderón Ramírez, Martin Solis-Salazar, Mario Romero-Sandoval, Monica Arias-Monge, and Horacio Saggion. 2023. A novel dataset for financial education text simplification in spanish. CoRR, abs/2312.09897. ↗
  37. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proc. of the 2019 Conference on Empirical Methods in Natural Language Processing. ACL. find
  38. Michael J Ryan, Tarek Naous, and Wei Xu. 2023a. Revisiting non-english text simplification: A unified multilingual benchmark. In Proc. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4898–4927. find
  39. Michael J. Ryan, Tarek Naous, and Wei Xu. 2023b. Revisiting non-english text simplification: A unified multilingual benchmark. In Proc. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, pages 4898–4927. ACL. find
  40. Horacio Saggion, Elena Gómez-Martínez, Esteban Etayo, Alberto Anula, and Lorena Bourg. 2011. Text simplification in simplext: Making texts more accessible. Procesamiento del lenguaje natural, (47):341– 342. find
  41. Arthur Mariano Rocha De Azevedo Scalercio, Elvis A. De Souza, Maria José Bocorny Finatto, and Aline Paes. 2025. Evaluating LLMs for Portuguese sentence simplification with linguistic insights. In Proc. of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24452–24477. ACL. find
  42. Milan Straka, Jan Hajic, and Jana Straková. 2016. Udpipe: trainable pipeline for processing conll-u files performing tokenization, morphological analysis, pos tagging and parsing. In Proc. of the 10th International Conference on Language Resources and Evaluation (LREC’16), pages 4290–4297. find our card
  43. Renliang Sun, Hanqi Jin, and Xiaojun Wan. 2021. Document-level text simplification: Dataset, criteria and baseline. In Proc. of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7997–8013. ACL. find
  44. Teerapaun Tanprasert and David Kauchak. 2021. Flesch-kincaid is not a text simplification evaluation metric. In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021), pages 1–14. ↗
  45. Verónica M Vargas. 2023. Analysis of barriers and proposals for inclusive access to justice for vulnerable groups. Journal of Law and Epistemic Studies, 1(2):20–24. ↗
  46. Laura Vásquez-Rodríguez, Nhung T. H. Nguyen, Piotr Przybyla, Matthew Shardlow, and Sophia Ananiadou. 2024. Simple is not enough: Document-level text simplification using readability and coherence. CoRR, abs/2412.18655. ↗
  47. Laura Vásquez-Rodríguez, Matthew Shardlow, Piotr Przybyła, and Sophia Ananiadou. 2023. Document-level text simplification with coherence evaluation. In Proc. of the 2nd Workshop on Text Simplification, Accessibility and Readability, pages 85–101. INCOMA Ltd., Shoumen. Leandro Wives and Maria José Finatto. Usando llms para simplificar e representar documentos médicos. In Anais Estendidos do XL Simpósio Brasileiro de Bancos de Dados, pages 415–425. SBC. find
  48. Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015. Problems in current text simplification research: New data can help. Transactions of the Association for Computational Linguistics, 3:283–297. ↗ our card
  49. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. In Transactions of the Association for Computational Linguistics, volume 4, pages 401–415. ↗ our card
  50. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. ↗ our card
  51. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. ↗
  52. Ziyu Yang. 2024. Enhancing the Comprehension: Text Simplification Approaches and the Role of Large Language Models. Temple University. find
  53. Xingxing Zhang and Mirella Lapata. 2017. Sentence simplification with deep reinforcement learning. In Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 595–605. ACL. Type Question PT: O texto simplificado é mais simples que o texto original, sob a condição de garantia de qualidade? Ou seja, ele também deve ser General Simplicity fluido na leitura e preservar o significado principal do texto original. EN: Is the simplified text simpler than the original text while maintaining quality? That is, it should be fluent and preserve the main meaning of the original text. PT: Qual é o grau de simplificação lexical, ou seja, o quanto as palavras originais foram Lexical Simplicity substituídas por termos mais simples? EN: What is the degree of lexical simplification, that is, to what extent were original words replaced with simpler terms? PT: Qual é o grau de simplificação estrutural, ou seja, o quanto a organização e Syntactic a complexidade sintática das sentenças Simplicity foram reduzidas? EN: What is the degree of structural simplification, that is, to what extent were sentence organization and syntactic complexity reduced? PT: O documento simplificado preservou o significado principal do texto original? Fatores que podem impactar nessa métrica são a remoção de conteúdo indispensável ou Content Preservation a inserção de informações novas não contidas no documento original e que não são de senso comum. EN: Did the simplified document preserve the main meaning of the original text? Factors that may impact this metric include the removal of essential content or the insertion of new information not contained in the original document and not considered common knowledge. PT: O texto simplificado é gramaticalmente correto e fluente, garantindo que as Grammaticality sentenças sejam naturais e bem formadas? EN: Is the simplified text grammatically correct and fluent, ensuring that sentences are natural and well formed? find our card
  54. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623. find
  55. Yang Zhong, Chao Jiang, Wei Xu, and Junyi Jessy Li. 2020. Discourse level factors for sentence deletion in text simplification. In Proc. of the AAAI conference on artificial intelligence, volume 34, pages 9709– 9716. find
  56. Terry Yue Zhuo, Qiongkai Xu, Xuanli He, and Trevor Cohn. 2023. Rethinking round-trip translation for machine translation evaluation. In Annual Meeting of the Association for Computational Linguistics (61st: 2023), pages 319–337. ACL. find
  57. Daniel Zwillinger and Stephen Kokoska. 1999. CRC standard probability and statistics tables and formulae. Crc Press. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/scalercio-2026-legalsim-pt-building-dataset-legal/