Elamur Research⌕
☰
home / papers / 2018 / Unsupervised Text Style Transfer using Language Models as Discriminators

Unsupervised Text Style Transfer using Language Models as Discriminators

Summary

Unsupervised text style transfer rewrites a sentence so that it carries a different attribute, such as sentiment, cipher orthography or a closely related language, while its content stays the same. The model must learn this without paired examples. Earlier encoder-decoder systems judged the outputs with binary classifiers, either style classifiers or real/fake CNN discriminators. That feedback is coarse, can be unstable under adversarial training, and often fails to produce fluent text.

The authors replace the classifier with a language model trained on target-domain sentences. The generator is optimized to lower the NLL that this LM assigns to its outputs, which yields a signal for every token rather than one score per sentence. A continuous approximation of discrete sampling keeps the pipeline differentiable, so everything trains end to end with back-propagation. Because a normalized LM already gives low probability to out-of-domain text, the authors find that negative samples and the adversarial step can be dropped, which makes training steadier.

Across three tasks (cracking word-substitution ciphers, changing the sentiment of a sentence, and translating between closely related languages), the method outperforms CNN-discriminator baselines such as Shen et al. (2017). This makes it a practical way to inject monolingual target-domain knowledge into a generator.

Key points

  • Training objective. The model combines an auto-encoder reconstruction loss with LM-based discriminator losses for each direction (LM_x, LM_y). A weight γ on the negative-sample term lets the authors test whether adversarial negatives are needed at all.

  • Serbian–Bosnian (sr/bs) data. About 200k news sentences per language, each shorter than 20 tokens, come from the Leipzig Corpora Collection. The split is 80/10/10, the vocabulary is 25k, and reference translations for validation and test were created with the Google Translation API.

  • Simplified–Traditional Chinese (zh-CN/zh-TW) data. 300k news headlines per variant come from the Chinese Gigaword corpus. The model is character-based with a vocabulary of roughly 5k.

  • Serbian–Bosnian results, in BLEU1 (BLEU4 was near zero because the vocabularies do not overlap):

    DirectionLM discriminatorShen et al.
    sr–bs31.029.1
    bs–sr31.730.3

  • Chinese results, in BLEU. Simply copying the input already scores 32.3.

    DirectionLM discriminatorShen et al.
    cn–tw81.660.1
    tw–cn85.560.7

Interesting ideas

  • Use a target-domain LM as the critic. A language model scores each token, so it shows which words make an output sound off-domain. A sentence-level classifier cannot do this. The same feedback could help any generator that must sound native to a particular domain or language variety.
  • Rely on normalization instead of negative samples. A locally normalized LM already penalizes text that does not look like its training data. This makes it possible to skip the unstable min-max game, a useful lesson for anyone who finds GAN-style training on text fragile.
  • Make sampling differentiable. A continuous relaxation of token sampling lets gradients from the LM flow straight into the generator. This avoids REINFORCE-style fine-tuning and could carry over to other discrete-output tasks with a differentiable scorer.
  • Build cheap evaluation sets for related languages. The authors collected monolingual news, filtered it by sentence length, and created parallel validation and test sets with machine translation. They also reported a plain copy baseline as a floor. This setup could be reused to benchmark conversion between close language varieties when no parallel corpus exists.
  • Match the granularity to the task. For script conversion such as zh-CN/zh-TW, which behaves almost like decipherment, a character-level model with a small vocabulary worked well. Choosing the unit to fit the nature of the mapping is a simple design choice worth copying.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation

Cited by

113 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 107 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1805.11749 · arXiv 1805.11749

@misc{yang2018unsupervised,
  title = {{Unsupervised Text Style Transfer using Language Models as Discriminators}},
  author = {Yang, Zichao and Hu, Zhiting and Dyer, Chris and Xing, Eric P. and Berg-Kirkpatrick, Taylor},
  year = {2018},
  doi = {10.48550/arxiv.1805.11749},
  eprint = {1805.11749},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1805.11749}
}

References

47 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al. Tensorflow: A system for large-scale machine learning. In OSDI, volume 16, pages sr–bs bs–sr cn–tw tw–cn 0 29.1 0 30.3 32.3 60.1 32.3 60.7 31.0 31.7 81.6 85.5 265–283, 2016. find
  2. M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862, 2017. ↗
  3. M. Artetxe, G. Labaka, E. Agirre, and K. Cho. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041, 2017. ↗ our card
  4. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. ↗ our card
  5. S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, and S. Bengio. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349, 2015. ↗ our card
  6. T. Brants, A. C. Popat, P. Xu, F. J. Och, and J. Dean. Large language models in machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), 2007. find our card
  7. T. Che, Y. Li, R. Zhang, R. D. Hjelm, W. Li, Y. Song, and Y. Bengio. Maximum-likelihood augmented discrete generative adversarial networks. arXiv preprint arXiv:1702.07983, 2017. ↗
  8. J. Chen, P.-S. Huang, X. He, J. Gao, and L. Deng. Unsupervised learning of predictors from unpaired input-output samples. arXiv preprint arXiv:1606.04646, 2016a. ↗
  9. X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2172–2180, 2016b. find
  10. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014. ↗
  11. E. L. Denton, S. Chintala, R. Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in neural information processing systems, pages 1486–1494, 2015. find
  12. Q. Dou and K. Knight. Large scale decipherment for out-of-domain machine translation. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 266–275. Association for Computational Linguistics, 2012. find
  13. Z. Fu, X. Tan, N. Peng, D. Zhao, and R. Yan. Style transfer in text: Exploration and evaluation. arXiv preprint arXiv:1711.06861, 2017. ↗ our card
  14. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, pages 2414–2423. find
  15. A. N. Gomez, S. Huang, I. Zhang, B. M. Li, M. Osama, and L. Kaiser. Unsupervised cipher cracking using discrete gans. arXiv preprint arXiv:1801.04883, 2018. ↗
  16. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014. find
  17. C. Gulcehre, O. Firat, K. Xu, K. Cho, L. Barrault, H.-C. Lin, F. Bougares, H. Schwenk, and Y. Bengio. On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535, 2015. ↗ our card
  18. D. He, Y. Xia, T. Qin, L. Wang, N. Yu, T. Liu, and W.-Y. Ma. Dual learning for machine translation. In Advances in Neural Information Processing Systems, pages 820–828, 2016. ↗
  19. Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing. Toward controlled generation of text. In International Conference on Machine Learning, pages 1587–1596, 2017a. ↗ our card
  20. Z. Hu, Z. Yang, R. Salakhutdinov, and E. P. Xing. On unifying deep generative models. arXiv preprint arXiv:1706.00550, 2017b. ↗
  21. Z. Hu, Z. Yang, R. Salakhutdinov, X. Liang, L. Qin, H. Dong, and E. Xing. Deep generative models with learnable knowledge constraints. arXiv preprint arXiv:1806.09764, 2018a. ↗
  22. Z. Hu, Z. Yang, T. Zhao, H. Shi, J. He, D. Wang, X. Ma, Z. Liu, X. Liang, L. Qin, et al. Texar: A modularized, versatile, and extensible toolbox for text generation. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 13–22, 2018b. find
  23. X. Huang and S. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. CoRR, abs/1703.06868, 2017. ↗
  24. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. arXiv preprint, 2017. find
  25. E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016. ↗ our card
  26. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. ↗
  27. D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. ↗ our card
  28. A. M. Lamb, A. G. A. P. GOYAL, Y. Zhang, S. Zhang, A. C. Courville, and Y. Bengio. Professor forcing: A new algorithm for training recurrent networks. In Advances In Neural Information Processing Systems, pages 4601–4609, 2016. ↗
  29. G. Lample, L. Denoyer, and M. Ranzato. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043, 2017. ↗ our card
  30. J. Li, W. Monroe, T. Shi, A. Ritter, and D. Jurafsky. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547, 2017. ↗
  31. J. Li, R. Jia, H. He, and P. Liang. Delete, retrieve, generate: A simple approach to sentiment and style transfer. arXiv preprint arXiv:1804.06437, 2018. ↗ our card
  32. K. Lin, D. Li, X. He, Z. Zhang, and M.-T. Sun. Adversarial ranking for language generation. In Advances in Neural Information Processing Systems, pages 3155–3165, 2017. ↗
  33. Y. Liu, J. Chen, and L. Deng. Unsupervised sequence classification using sequential output statistics. In Advances in Neural Information Processing Systems, pages 3550–3559, 2017. ↗
  34. N. Pourdamghani and K. Knight. Deciphering related languages. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2513–2518, 2017. find
  35. S. Prabhumoye, Y. Tsvetkov, R. Salakhutdinov, and A. W. Black. Style transfer through back-translation. arXiv preprint arXiv:1804.09000, 2018. ↗ our card
  36. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015. ↗ our card
  37. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2234–2242, 2016. ↗ our card
  38. T. Shen, T. Lei, R. Barzilay, and T. Jaakkola. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Information Processing Systems, pages 6833–6844, 2017. ↗ our card
  39. R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems, pages 1057–1063, 2000. find
  40. O. Vinyals and Q. Le. A neural conversational model. arXiv preprint arXiv:1506.05869, 2015. ↗ our card
  41. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, pages 3156–3164. ↗ our card
  42. T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young. A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562, 2016. ↗
  43. Z. Yang, W. Chen, F. Wang, and B. Xu. Improving neural machine translation with conditional sequence generative adversarial nets. arXiv preprint arXiv:1703.04887, 2017a. ↗
  44. Z. Yang, Z. Hu, R. Salakhutdinov, and T. Berg-Kirkpatrick. Improved variational autoencoders for text modeling using dilated convolutions. arXiv preprint arXiv:1702.08139, 2017b. ↗
  45. L. Yu, W. Zhang, J. Wang, and Y. Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, pages 2852–2858, 2017. find
  46. J. Zhao, M. Mathieu, and Y. LeCun. Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126, 2016. ↗
  47. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593, 2017. ↗

↑ back to top