Elamur Research⌕
☰
home / papers / 2026 / Parallel Corpus Filtering Based on Semantic Similarity and Surface Dissimilarity for Japanese Text Simplification with LLMs

Parallel Corpus Filtering Based on Semantic Similarity and Surface Dissimilarity for Japanese Text Simplification with LLMs

Summary

LLMs can often be adapted to a task with very little supervised data, but it was unclear how many sentence pairs Japanese text simplification needs. The authors first show that the usual filtering approach, which keeps pairs with high LaBSE cosine similarity, backfires. Those pairs are mostly near-identical, so the fine-tuned model learns to copy its input. Random subsampling has the same problem, because near-copies dominate the corpora. They therefore score each complex–simple pair by how far it lies from an ideal point: full semantic agreement with zero surface overlap, measured with LaBSE and SacreBLEU. The worst-scoring pairs are discarded first. Swallow (with LoRA) and Japanese BART were fine-tuned on filtered subsets of MATCHA and SNOW, then evaluated on those corpora plus JADES with SARI and BERTScore. The method beat the alternative filters in most settings. Only 16–64 selected pairs gave the LLM basic simplification ability, although larger subsets were needed to learn domain knowledge. This makes cheap adaptation feasible for low-resource simplification.

Key points

  • Preliminary finding: Swallow was tuned on the 20% of SNOW's 82,300 pairs with the highest LaBSE similarity. It then returned the input unchanged for 90% of evaluation sentences.
  • Quality score: Q(x, y) = √((1 − LaBSE(x, y))² + BLEU(x, y)²), which favors high semantic similarity and low surface similarity. Pairs are removed in descending order of Q.
  • Data and models:
    • MATCHA: expert-annotated, 14,000 training pairs.
    • SNOW: non-expert annotations, 82,300 training pairs.
    • JADES: 3,907 pairs, used for evaluation only.
    • Models: Llama-3.1-Swallow-8B-Instruct-v0.2 with LoRA (r = 16, α = 16) and bart-large-japanese, both using greedy decoding.
  • Baselines: w/o Filtering, 0-shot, Random, Bicleaner-AI, COS (semantic similarity only), and BLEU (surface dissimilarity only).
  • Main result (SARI, in-domain, LLM):
    • MATCHA: 4,000 filtered pairs (about 70% fewer) outperformed full fine-tuning.
    • SNOW: 64,000 filtered pairs (about 20% fewer) outperformed full fine-tuning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2o26gctx8fej

@inproceedings{maekawa2026parallel,
  title = {{Parallel Corpus Filtering Based on Semantic Similarity and Surface Dissimilarity for Japanese Text Simplification with LLMs}},
  author = {Maekawa, Daisuke and Kajiwara, Tomoyuki and Ninomiya, Takashi},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2o26gctx8fej},
  url = {https://doi.org/10.63317/2o26gctx8fej}
}

References

30 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fernando Alva-Manchego, Carolina Scarton, and Lucia Specia. 2020. Data-driven Sentence Simplification: Survey and Benchmark. Computational Linguistics, 46(1):135–187. ↗ our card
  2. Mikel Artetxe and Holger Schwenk. 2019. Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. Transactions of the Association for Computational Linguistics, 7:597–610. ↗ our card
  3. Akshay Batheja and Pushpak Bhattacharyya. 2022. Improving Machine Translation with Phrase Pair Injection and Corpus Filtering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5395–5400. ↗
  4. Vishrav Chaudhary, Yuqing Tang, Francisco Guzmán, Holger Schwenk, and Philipp Koehn. 2019. Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings. In Proceedings of the Fourth Conference on Machine Translation, pages 261–266. find our card
  5. Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. 2024. AlpaGasus: Training A Better Alpaca with Fewer Data. In Proceedings of the Twelfth International Conference on Learning Representations. ↗ our card
  6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451. find
  7. Jan De Belder and Marie-Francine Moens. 2010. Text Simplification for Children. In Proceedings of the SIGIR 2010 Workshop on Accessible Search Systems, pages 19–26. find
  8. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT Sentence Embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 878–891. find our card
  9. Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities. In Proceedings of the First Conference on Language Modeling. ↗ our card
  10. Koki Hatagaki, Tomoyuki Kajiwara, and Takashi Ninomiya. 2022. Parallel Corpus Filtering for Japanese Text Simplification. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability, pages 12–18. find
  11. Akio Hayakawa, Tomoyuki Kajiwara, Hiroki Ouchi, and Taro Watanabe. 2022. JADES: New Text Simplification Dataset in Japanese Targeted at Non-Native Speakers. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability, pages 179–187. find
  12. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the Tenth International Conference on Learning Representations. find
  13. Akihiro Katsuta and Kazuhide Yamamoto. 2018. Crowdsourced Corpus of Sentence Simplification with Core Vocabulary. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, pages 461–466. find
  14. Philipp Koehn, Vishrav Chaudhary, Ahmed El-Kishky, Naman Goyal, Peng-Jen Chen, and Francisco Guzmán. 2020. Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment. In Proceedings of the Fifth Conference on Machine Translation, pages 726–742. find our card
  15. Philipp Koehn, Francisco Guzmán, Vishrav Chaudhary, and Juan Pino. 2019. Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions. In Proceedings of the Fourth Conference on Machine Translation, pages 54–72. find our card
  16. Philipp Koehn, Huda Khayrallah, Kenneth Heafield, and Mikel L. Forcada. 2018. Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 726–739. find our card
  17. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880. find our card
  18. Llama Team. 2024. The Llama 3 Herd of Models. arXiv:2407.21783. ↗
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceedings of the Seventh International Conference on Learning Representations. find
  20. Takumi Maruyama and Kazuhide Yamamoto. 2018. Simplified Corpus with Core Vocabulary. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, pages 1153–1160. find
  21. Rina Miyata, Hinata Koretaka, Hiroki Yamauchi, Daiki Yanamoto, Tomoyuki Kajiwara, Takashi Ninomiya, and Yasuhiro Nishiaki. 2024. MATCHA: Parallel Corpus for Japanese Text Simplification Based on Professionally Simplified Articles. Journal of Natural Language Processing, 31(2):590–609. ↗
  22. Naoaki Okazaki, Kakeru Hattori, Hirai Shota, Hiroki Iida, Masanari Ohi, Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Rio Yokota, and Sakae Mizuki. 2024. Building a Large Japanese Web Corpus for Large Language Models. In Proceedings of the First Conference on Language Modeling. ↗
  23. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318. ↗ our card
  24. Sarah E. Petersen and Mari Ostendorf. 2007. Text Simplification for Language Learners: A Corpus Analysis. In Proceedings of the 1st Workshop on Speech and Language Technology in Education, pages 69–72. find
  25. Víctor M. Sánchez-Cartagena, Marta Bañón, Sergio Ortiz-Rojas, and Gema Ramírez. 2018. Prompsit‘s submission to WMT 2018 Parallel Corpus Filtering shared task. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 955–962. find our card
  26. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing Statistical Machine Translation for Text Simplification. Transactions of the Association for Computational Linguistics, 4:401–415. ↗ our card
  27. Jaume Zaragoza-Bernabeu, Gema Ramírez-Sánchez, Marta Bañón, and Sergio Ortiz Rojas. 2022. Bicleaner AI: Bicleaner Goes Neural. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 824–831. find
  28. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. ↗
  29. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. LIMA: Less Is More for Alignment. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 55006–55021. find
  30. Dawei Zhu, Pinzhen Chen, Miaoran Zhang, Barry Haddow, Xiaoyu Shen, and Dietrich Klakow. 2024. Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 388–409. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/maekawa-2026-parallel-corpus-filtering-based-semantic/