Elamur Research
☰
home / papers / 2018 / Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P18-1090.pdf

Summary

Changing a sentence's sentiment while keeping its topic and other content is hard without parallel positive/negative sentence pairs. Earlier style-transfer models mix sentiment and meaning in one dense hidden vector, so their outputs often drift off-topic. Xu et al. split the task between two cooperating components.

  • Neutralization module: an LSTM that tags each word as neutral or polar and removes the polar ones. It is pre-trained with attention weights from a self-attention based sentiment classifier (SASC). Words whose weight is above the sentence average are treated as emotional.
  • Emotionalization module: adds the desired sentiment back to the remaining content.

In cycled training, a review is neutralized and then rebuilt with its original sentiment. This turns unpaired data into supervised reconstruction pairs. Word selection is discrete, so the neutralizer is trained with policy gradient. Its reward combines how well outputs match the target sentiment and how much content they preserve.

On Yelp and Amazon reviews, the approach greatly improves content retention over prior systems. This shows that explicitly separating sentiment words is an effective strategy for unpaired sentiment translation.

Key points

  • Data: two review corpora, Yelp and Amazon. Only sentences and sentiment labels are needed, with no parallel pairs. Code is released at github.com/lancopku/unpaired-sentiment-translation.

  • BLEU rises from 1.64 to 22.46 on Yelp and from 0.56 to 14.06 on Amazon compared with earlier state-of-the-art systems.

  • The full model (neutralization + cycled RL + pre-training) achieves the following:

    DatasetSentiment accuracyG-score
    Yelp80.0042.38
    Amazon70.3731.45

  • The SASC classifier used for pre-training reaches about 89% and 90% accuracy on the two datasets.

  • Ablation shows both parts are needed. Cycled RL without pre-training collapses (Yelp BLEU 1.08). Remaining errors include sentiment-conflicted outputs and outputs that stay neutral.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

117 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 109 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1090

@inproceedings{xu2018unpaired,
  title = {{Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach}},
  author = {Xu, Jingjing and Sun, Xu and Zeng, Qi and Zhang, Xiaodong and Ren, Xuancheng and Wang, Houfeng and Li, Wenjie},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1090},
  url = {https://doi.org/10.18653/v1/p18-1090}
}

References

24 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tao Chen, Ruifeng Xu, Yulan He, and Xuan Wang. 2017. Improving sentiment analysis via sentence type classification using bilstm-crf and CNN. Expert Syst. Appl., 72:221–230. find
  2. Shuming Ma, Xu Sun, Wei Li, Sujian Li, Wenjie Li, and Xuancheng Ren. 2018a. Query and output: Generating words by querying distributed word representations for paraphrase generation. CoRR, abs/1803.01465. find
  3. Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. 2017. Learning to generate product reviews from attributes. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 623–632. find
  4. Shuming Ma, Xu Sun, Junyang Lin, and Xuancheng Ren. 2018b. A hierarchical end-to-end model for jointly improving text summarization and sentiment classification. CoRR, abs/1805.01089. — [arXiv:1805.01089](https://arxiv.org/abs/1805.01089) find
  5. John C. Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159. find
  6. Julian John McAuley and Jure Leskovec. 2013. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In 22nd International World Wide Web Conference, WWW ’13, Rio de Janeiro, Brazil, May 13-17, 2013, pages 897– 908. find
  7. Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In AAAI 2018. find
  8. Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2414–2423. find
  9. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA., pages 311–318. find our card
  10. Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. 2016. Dual learning for machine translation. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 820–82… find
  11. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2015, Denver, Colorado, USA, June 4-5, 2015, pages 4… find
  12. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Controllable text generation. In ICML 2017. find
  13. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016, pages 694–711. find
  14. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouver, Canada, August 3-4, 2017, pages 502–518. find our card
  15. Yanghao Li, Naiyan Wang, Jiaying Liu, and Xiaodi Hou. 2017. Demystifying neural style transfer. In Proceedings of the Twenty-Sixth International Joint Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In NIPS 2017. find our card
  16. Richard Socher, Cliff Chiung-Yu Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, page… find
  17. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 3104– 3112. find
  18. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA., pages 3097–3103. find
  19. Joao Paulo Albuquerque Vieira and Raimundo Santos Moura. 2017. An analysis of convolutional neural networks for sentence classification. In XLIII 2017, pages 1–5. find
  20. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229–256. find
  21. Jingjing Xu, Xu Sun, Xuancheng Ren, Junyang Lin, Bingzhen Wei, and Wei Li. 2018. DP-GAN: diversity-promoting generative adversarial network for generating informative and diversified text. CoRR, abs/1802.01345. — [arXiv:1802.01345](https://arxiv.org/abs/1802.01345) find
  22. Hongyu Zang and Xiaojun Wan. 2017. Towards automatic generation of product reviews from aspect-sentiment scores. In Proceedings of the 10th International Conference on Natural Language Generation, pages 168–177. find
  23. Zhiyuan Zhang, Wei Li, and Xu Sun. 2018. Automatic transferring between ancient chinese and contemporary chinese. CoRR, abs/1803.01557. — [arXiv:1803.01557](https://arxiv.org/abs/1803.01557) find
  24. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 2242–2251. find