Elamur Research⌕
☰
home / papers / 2026 / Paraphrasing as Zero-shot Translation with Feature-guided Diversity Enhancement

Paraphrasing as Zero-shot Translation with Feature-guided Diversity Enhancement

Summary

Training paraphrasers on parabanks built by machine-translating one side of a parallel corpus has two weaknesses. The MT system has usually seen that same corpus, so its outputs stay close to the references and lack variety. The paraphraser also learns only from the derived parabank, which holds just part of the knowledge in the original data. The authors propose ParaMNMT, a single-stage alternative. They train a bidirectional multilingual NMT model on English-Czech WMT 2016 data, with target-language tokens prepended to each sentence. To paraphrase, they ask the model for a zero-shot "translation" into the input's own language. To counter outputs that simply echo the input, each source token gets a "copy" or "not-copy" feature embedding during training, marking whether it reappears in the target. At inference, labelling more tokens "not-copy" pushes the model to rewrite them. In manual and automatic evaluation, the resulting paraphrases beat existing parabanks and LLMs on meaning preservation, fluency and diversity. As augmentation data, they improve low-resource GLUE tasks and low-resource MT, which matters for low-resource NLP settings.

Key points

  • Training data and model:
    • Data: the WMT 2016 English-Czech corpus (Europarl v7, Common Crawl, News Commentary v11, CzEng 1.6pre), reversed to cover both directions, with shared BPE.
    • Model: Transformer Base (6 layers, 512-dimensional embeddings) with LALN and LALT.
  • Copy tags: In the training data, 18.7% of source tokens carry the "copy" tag. At inference, 30% of tokens are tagged "not-copy." Two ways of choosing them were compared: by high frequency, or by high BERT-based entropy.
  • Tag compliance: The model disregards a "not-copy" instruction in about 7.17% of cases.
  • GLUE gains: Augmentation helps even on top of a LoRA-tuned Qwen 3 8B baseline, for example CoLA 63.4→65.5, SST-2 95.6→97.6 and MNLI 90.9→92.4.
  • Translation gains: BLEU with Qwen 3 8B rises from prompting to LoRA to LoRA plus ParaMNMT:
    • En-De: 27.27 → 30.49 → 31.30
    • En-Tr: 20.03 → 22.61 → 23.54
    • En-Fi: 17.58 → 18.95 → 19.63

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.783

@inproceedings{yan2026paraphrasing,
  title = {{Paraphrasing as Zero-shot Translation with Feature-guided Diversity Enhancement}},
  author = {Yan, Ziyue and Zan, Hongying and Lyu, Xinglin and Xu, Hongfei},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.783},
  url = {https://doi.org/10.18653/v1/2026.acl-long.783}
}

References

69 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Lillian Lee. 2003. Learning to paraphrase: an unsupervised approach using multiple-sequence alignment. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 16–23. ↗ our card
  2. Rahul Bhagat and Eduard H. Hovy. 2013. Squibs: What is a paraphrase? Computational Linguistics, 39:463– 472. find our card
  3. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. 2016a. Findings of the 2016 conference on machine translation (wmt16). In First conference on machine translation, pages 131–198. Association for Computational Linguistics. find our card
  4. Ondřej Bojar, Ondřej Dušek, Tom Kocmi, Jindřich Libovický, Michal Novák, Martin Popel, Roman Sudarikov, and Dušan Variš. 2016b. CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered. In Text, Speech and Dialogue: 19th International Conference, TSD 2016, Brno, Czech Republic, September 12-16, 2016, Proceedings. Springer Verlag. In press. find
  5. Yue Cao and Xiaojun Wan. 2020. Divgan: towards diverse paraphrase generation via diversified generative adversarial network. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2411–2421. ↗
  6. Jan Cegin, Branislav Pecher, Jakub Simko, Ivan Srba, Maria Bielikova, and Peter Brusilovsky. 2024. Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13148– 13171, Bangkok, Thailand. Association for Computational Linguistics. find
  7. Daniel Cera, Mona Diabb, Eneko Agirrec, Inigo Lopez-Gazpioc, Lucia Speciad, and Basque Country Donostia. Semeval-2017 task 1: Semantic textual similarity multilingual and cross-lingual focused evaluation. find
  8. Andong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai, Yang Xiang, Muyun Yang, Tiejun Zhao, and Min Zhang. 2025. Benchmarking LLMs for translating classical Chinese poetry: Evaluating adequacy, fluency, and elegance. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33019–33036, Suzhou, China. Association for Computational Linguistics. find our card
  9. John Chung, Ece Kamar, and Saleema Amershi. 2023. Increasing diversity while maintaining accuracy: Text data generation with large language models and human interventions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 575–593, Toronto, Canada. Association for Computational Linguistics. find
  10. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer. find
  11. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023. Bidirectional generative framework for cross-domain aspect-based sentiment analysis. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12272–12285, Toronto, Canada. Association for Computational Linguistics. find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  13. Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005). find our card
  14. William B. Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In International Conference on Computational Linguistics. find our card
  15. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
  16. Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. Multi-way, multilingual neural machine translation with a shared attention mechanism. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 866–875, San Diego, California. Association for Computational Linguistics. find our card
  17. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. Ppdb: The paraphrase database. In North American Chapter of the Association for Computational Linguistics. find our card
  18. Tanya Goyal and Greg Durrett. 2020. Neural syntactic preordering for controlled paraphrase generation. In Annual Meeting of the Association for Computational Linguistics. find our card
  19. Wenjie Hao, Hongfei Xu, Deyi Xiong, Hongying Zan, and Lingling Mu. 2022. Parazh-22m: A large-scale chinese parabank via machine translation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3885–3897. find
  20. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations. find
  21. J. Edward Hu, Rachel Rudinger, Matt Post, and Benjamin Van Durme. 2019a. Parabank: Monolingual bitext generation and sentential paraphrasing via lexically-constrained neural machine translation. In AAAI Conference on Artificial Intelligence. ↗
  22. J. Edward Hu, Abhinav Singh, Nils Holzenberger, Matt Post, and Benjamin Van Durme. 2019b. Large-scale, diverse, paraphrastic bitexts via sampling and clustering. In Conference on Computational Natural Language Learning. find
  23. Kuan-Hao Huang and Kai-Wei Chang. 2021. Generating syntactically controlled paraphrases without using annotated parallel pairs. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1022–1033. ↗
  24. Kuan-Hao Huang, Varun Iyer, I-Hung Hsu, Anoop Kumar, Kai-Wei Chang, and Aram Galstyan. 2023. ParaAMR: A large-scale syntactically diverse paraphrase dataset by AMR back-translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8047–8061, Toronto, Canada. Association for Computational Linguistics. ↗ our card
  25. Kuan-Hao Huang, Varun Iyer, Anoop Kumar, Sriram Venkatapathy, Kai-Wei Chang, and A. G. Galstyan. 2022. Unsupervised syntactically controlled paraphrase generation with abstract meaning representations. In Conference on Empirical Methods in Natural Language Processing. find our card
  26. Shankar Iyer, Nikhil Dandekar, Kornél Csernai, et al. 2017a. First quora dataset release: Question pairs. data. quora. com. find
  27. Shankar Iyer, Nikhil Dandekar, and Kornél Csernai. 2017b. First quora dataset release: Question pairs. find
  28. David Kauchak and Regina Barzilay. 2006. Paraphrasing for automatic evaluation. In North American Chapter of the Association for Computational Linguistics. find our card
  29. Huda Khayrallah, Brian Thompson, Matt Post, and Philipp Koehn. 2020. Simulated multiple reference training improves low-resource machine translation. In Conference on Empirical Methods in Natural Language Processing. find
  30. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Machine Translation Summit. find
  31. Ashutosh Kumar, Kabir Ahuja, Raghuram Vadapalli, and Partha Pratim Talukdar. 2020. Syntax-guided controlled generation of paraphrases. Transactions of the Association for Computational Linguistics, 8:330– 345. ↗ our card
  32. Fei-Tzin Lee, Miguel Ballesteros, Feng Nan, and Kathleen McKeown. 2022. Using structured content plans for fine-grained syntactic control in pretrained language model generation. In International Conference on Computational Linguistics. find
  33. Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thirteenth international conference on the principles of knowledge representation and reasoning. find
  34. Zichao Li, Xin Jiang, Lifeng Shang, and Qun Liu. 2019. Decomposable neural paraphrase generation. In Annual Meeting of the Association for Computational Linguistics. find
  35. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  36. Nitin Madnani, Joel R. Tetreault, and Martin Chodorow. 2012. Re-examining machine translation metrics for paraphrase identification. In North American Chapter of the Association for Computational Linguistics. find our card
  37. Kathleen McKeown. 1983. Paraphrasing questions using given and new information. Am. J. Comput. Linguistics, 9:1–10. ↗ our card
  38. Xingyuan Pan, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, and Shanbo Cheng. 2024. G-DIG: Towards gradient-based DIverse and hiGh-quality instruction data selection for machine translation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15395–15406, Bangkok, Thailand. Association for Computational Linguistics. find
  39. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392. ↗ our card
  40. Aurko Roy and David Grangier. 2019. Unsupervised paraphrasing without translation. In Annual Meeting of the Association for Computational Linguistics. find
  41. Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou, and Iryna Gurevych. 2024. Systematic task exploration with LLMs: A study in citation text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4832–4855, Bangkok, Thailand. Association for Computational Linguistics. ↗
  42. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725. find
  43. Eetu Sjöblom, Mathias Creutz, and Yves Scherrer. 2020. Paraphrase generation and evaluation on colloquial-style sentences. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1814–1822. find
  44. Aviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster, and Ido Dagan. 2024. Attribute first, then generate: Locally-attributable grounded text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3309–3344, Bangkok, Thailand. Association for Computational Linguistics. find
  45. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642. find
  46. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  47. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. ↗ our card
  48. Brian Thompson and Matt Post. 2020a. Automatic machine translation evaluation in many languages via zero-shot paraphrasing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 90–121. find our card
  49. Brian Thompson and Matt Post. 2020b. Paraphrase generation as zero-shot multilingual translation: Disentangling semantic similarity from lexical and syntactic diversity. In Proceedings of the Fifth Conference on Machine Translation, pages 561–570. find our card
  50. John Wieting and Kevin Gimpel. 2017. Paranmt-50m: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. In Annual Meeting of the Association for Computational Linguistics. find our card
  51. Yufei Tian, Arvind krishna Sridhar, and Nanyun Peng. 2021. Hypogen: Hyperbole generation with commonsense and counterfactual knowledge. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1583–1593. find
  52. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of NAACL-HLT, pages 1112–1122. find
  53. Jörg Tiedemann and Yves Scherrer. 2019. Measuring semantic abstraction of multilingual nmt with paraphrase recognition and generation tasks. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 35–42. ↗
  54. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  55. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019. Glue: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019. ↗ our card
  56. Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, and Michael Zeng. 2022. Paratag: A dataset of paraphrase tagging for fine-grained labels, nlg evaluation, and data augmentation. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 7111–7122. find
  57. Yasong Wang, Mingtong Liu, Yujie Zhang, Yufeng Chen, et al. 2021. Research on the construction and application of paraphrase parallel corpus. Beijing Da Xue Xue Bao, 57(1):68–74. find
  58. Zaitian Wang, Jinghan Zhang, Xinhao Zhang, Kunpeng Liu, Pengfei Wang, and Yuanchun Zhou. 2025a. Diversity-oriented data augmentation with large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22265– 22283, Vienna, Austria. Association for Computational Linguistics. find
  59. Zhilin Wang, Yafu Li, Jianhao Yan, Yu Cheng, and Yue Zhang. 2025b. Unveiling attractor cycles in large language models: A dynamical systems view of successive paraphrasing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12740– 12755, Vienna, Austria. Association for Computational Linguistics. ↗
  60. Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. ↗
  61. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find our card
  62. Ziyao Xu and Houfeng Wang. 2024. SPOR: A comprehensive and practical evaluation method for compositional generalization in data-to-text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 604–621, Bangkok, Thailand. Association for Computational Linguistics. ↗
  63. Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. Prompting large language model for machine translation: A case study. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 41092–41110. PMLR. ↗ our card
  64. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628– 1639, Online. Association for Computational Linguistics. ↗ our card
  65. Liang Zhang, Qin Jin, Haoyang Huang, Dongdong Zhang, and Furu Wei. 2024. Respond in my language: Mitigating language inconsistency in response generation based on large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4177–4192, Bangkok, Thailand. Association for Computational Linguistics. find
  66. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. ↗
  67. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 1097–1100. ↗ our card
  68. Chengqing Zong, Yujie Zhang, Kazuhide Yamamoto, Masashi Sakamoto, and Satoshi Shirai. 2001. Approach to spoken chinese paraphrasing based on feature extraction. In Natural Language Processing Pacific Rim Symposium. find
  69. Fei Zuo, Kehai Chen, Yu Zhang, Zhengshan Xue, and Min Zhang. 2025. InImageTrans: Multimodal LLM-based text image machine translation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20256–20277, Vienna, Austria. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/yan-2026-paraphrasing-as-zero-shot-translation/