Elamur Research
☰
home / papers / 2017 / Controlling Linguistic Style Aspects in Neural Language Generation

Controlling Linguistic Style Aspects in Neural Language Generation

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W17-4912.pdf

Summary

Neural text generators are usually steered toward content rather than style, and earlier style-control work typically handled one attribute at a time. Ficler and Goldberg ask whether several stylistic dimensions can be controlled jointly alongside content. They train an LSTM language model with a BPE subword vocabulary. The model is conditioned on a vector built by concatenating learned embeddings of the desired attribute values, and this vector is appended to the input at every time step. The data are Rotten Tomatoes movie reviews split into sentences. Labels come either from site meta-data, such as reviewer scores and critic versus audience origin, or from simple heuristics over word lists, function words and part-of-speech distributions. Six attributes are used: two content ones (theme, sentiment) and four stylistic ones (length, descriptiveness, personal voice, professional style). The conditioned model reaches lower perplexity than unconditioned baselines, mostly produces sentences matching the requested settings, and copes with attribute combinations absent from training. The work shows that a plain conditioned LM trained on noisy, automatically derived labels can control multiple attributes on naturally long sentences.

Key points

  • Corpus: 1,002,625 Rotten Tomatoes reviews covering 7,500 movies, segmented into sentences.
  • Sentiment labels: derived from scores normalized to a 0–5 scale. Grades 0–2 count as negative, 3 as neutral and 4–5 as positive, with "none" used when no score is available.
  • Attributes and values:
    • Theme: Plot, Acting, Production, Effects or Other.
    • Length: ≤10, 11–20, 21–40 or >40 words.
    • Professional, Personal and Descriptive: true or false.
  • Generalization test: the authors removed the 75,421 training sentences labeled both theme:plot and personal:true and retrained the model.
    • All generated sentences for this unseen combination contained personal pronouns.
    • 82.4% matched the plot theme, compared with 97.8% for a model trained on the full data.
  • Comparison with Hu et al. (2017): that VAE-plus-discriminator approach was limited to sentences of up to 16 words and two aspects (sentiment, tense). This model needs no discriminator and handles six aspects on full-length sentences.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

159 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 143 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-4912 · arXiv 1707.02633

@inproceedings{ficler2017controlling,
  title = {{Controlling Linguistic Style Aspects in Neural Language Generation}},
  author = {Ficler, Jessica and Goldberg, Yoav},
  booktitle = {Proceedings of the Workshop on Stylistic Variation},
  year = {2017},
  doi = {10.18653/v1/w17-4912},
  eprint = {1707.02633},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/w17-4912}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fadi Abu Sheikha and Diana Inkpen. 2011. Generation of formal and informal sentences. In Proceedings of the 13th European Workshop on Natural Language Generation. Association for Computational Linguistics, Nancy, France, pages 187–193. http://www.aclweb.org/anthology/W11-2826. ↗
  2. John A Bateman and Cecile Paris. 1989. Phrasing a text in terms the user can understand. In IJCAI. pages 1511–1517. find
  3. Douglas Biber and Susan Conrad. 2009. Register, genre, and style. Cambridge University Press. find
  4. Ondřej Dušek and Filip Jurcicek. 2016a. A context-aware natural language generator for dialogue systems. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Association for Computational Linguistics, Los Angeles, pages 185–190. http://www.aclweb.org/a… ↗
  5. Ondřej Dušek and Filip Jurcicek. 2016b. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume David D McDonald and James D Pustejovsky. 1985. A computational theory o… find
  6. Cong Duy Vu Hoang, Gholamreza Haffari, and Trevor Cohn. 2016. Incorporating side information into recurrent neural network language models. In Proceedings of NAACL-HLT. pages 1250–1255. find
  7. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2016. What to talk about and how? selective generation using lstms with coarse-to-fine alignment. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Asso… find
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. MIT Press, volume 9, pages 1735–1780. find
  9. Eduard Hovy. 1987. Generating natural language under pragmatic constraints. Journal of Pragmatics, volume 11, pages 689–719. find
  10. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Controllable text generation. In Proc. of ICML. find
  11. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Interspeech. volume 2, page 3. find
  12. Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. Globally coherent text generation with neural checklist models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 329–339. https://aclweb.or… ↗
  13. Kate G Niederhoffer and James W Pennebaker. 2002. Linguistic style matching in social interaction. Journal of Language and Social Psychology, volume 21, pages 337–360. find
  14. Ellie Pavlick and Ani Nenkova. 2015. Inducing lexical style properties for paraphrase and genre differentiation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistic… find
  15. Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. Controlling output length in neural encoder-decoders. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1328… find
  16. Ellie Pavlick and Joel Tetreault. 2016. An empirical analysis of formality in online communication. Transactions of the Association for Computational Linguistics, volume 4, pages 61–74. find our card
  17. Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1203–… find our card
  18. Richard Power, Donia Scott, and Nadjet Bouayad-Agha. 2003. Generating texts with style. In Proc. of CiCLING. Springer, pages 444–452. find
  19. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444. find our card
  20. Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics… find our card
  21. Ehud Reiter and Sandra Williams. 2010. Generating texts in different styles. In The Structure of Style, Springer, pages 59–75. find
  22. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In Proceedings of NAACL-HLT. pages 35–40. find our card
  23. Zachary C Lipton, Sharad Vikram, and Julian McAuley. 2015. Capturing meaning in product reviews with character-level generative text models. arXiv preprint arXiv:1511.03683. — [arXiv:1511.03683](https://arxiv.org/abs/1511.03683) find
  24. Jian Tang, Yifan Yang, Sam Carton, Ming Zhang, and Qiaozhu Mei. 2016. Context-aware natural language generation with recurrent neural networks. arXiv preprint arXiv:1611.09900. — [arXiv:1611.09900](https://arxiv.org/abs/1611.09900) find
  25. François Mairesse and Marilyn A Walker. 2011. Controlling user perceptions of linguistic style: Trainable generation of personality traits. MIT Press, volume 37, pages 455–488. find
  26. Tsung-Hsien Wen, Milica Gasic, Nikola Mrkšić, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned lstm-based natural language generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Co… find
  27. Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In Proceedings of COLING 2012. find our card
  28. The COLING 2012 Organizing Committee, Mumbai, India, pages 2899– 2914. http://www.aclweb.org/anthology/C12-1177. ↗ our card
  29. Yinfei Yang and Ani Nenkova. 2014. Detecting information-dense texts in multiple news domains. In AAAI. pages 1650–1656. find