Elamur Research
☰
home / papers / 2020 / Politeness Transfer: A Tag and Generate Approach

Politeness Transfer: A Tag and Generate Approach

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.169.pdf

Summary

The authors frame politeness transfer as a new text style transfer task: rewriting non-polite action-directives, such as requests and commands, into polite versions without changing their meaning. This differs from typical style transfer, which moves between two clearly marked styles like positive and negative sentiment. Here the source side is neutral, so there are often no stylistic words to remove. To support the task, they built a corpus of about 1.39 million sentences from the Enron email collection, scored automatically for politeness. They also hand-curated a test set of 800 request sentences. Their two-stage "tag and generate" model first tags positions where style markers should be replaced or inserted, and a second model then writes the final sentence. Every intermediate output is readable text, which makes the pipeline interpretable. Across politeness and five other transfer tasks, the system preserves content better than prior methods, including DRG, while matching or improving transfer accuracy. It also scores higher in human ratings, giving a strong general baseline for neutral-to-styled transfer.

Key points

  • Dataset construction: The Enron corpus was cleaned down to about 1.39M sentences and scored with the Niu and Bansal (2018) politeness classifier. Sentences were grouped into ten buckets (P0–P9) and split 80:10:10. The P9 bucket (score above 90%) supplies more than 270K polite training sentences.
  • Test set: The 800 test sentences were first filtered as action-directives using a Switchboard-trained AWD-LSTM dialog-act classifier, then selected by human annotators. Annotator agreement was Fleiss's κ = 0.77.
  • Tagger variants: The tagger comes in "replace" and "add" versions. The add-tagger suits neutral sources: on Captions it reaches 93.17 accuracy versus 84.5 for the replace-tagger. The replace-tagger suits polar data: on Yelp it reaches 86.6 accuracy, while the add-tagger gets only about 53.
  • Combined tagger: A tagger trained on both kinds of data partly learns which operation fits each dataset. It does about 20% more replacements on Yelp and about 50% more insertions on Captions.
  • Human evaluation: Ten judges rated outputs from 1 to 5 for content preservation, grammaticality, and attribute match. This covered 200 politeness samples and 100 samples each for Yelp, Gender, and Political. The authors also caution that automatic metrics may not reflect real task success.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

96 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 90 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.169 · arXiv 2004.14257

@inproceedings{madaan2020politeness,
  title = {{Politeness Transfer: A Tag and Generate Approach}},
  author = {Madaan, Aman and Setlur, Amrith and Parekh, Tanmay and Poczos, Barnabas and Neubig, Graham and Yang, Yiming and Salakhutdinov, Ruslan and Black, Alan W and Prabhumoye, Shrimai},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.169},
  eprint = {2004.14257},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.acl-main.169}
}

References

44 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Philip Bramsen, Martha Escobar-Molano, Ami Patel, and Rafael Alonso. 2011. Extracting social power relationships from natural language. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT ’11, pages 773–782, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  2. Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. find
  3. Khyathi Chandu, Shrimai Prabhumoye, Ruslan Salakhutdinov, and Alan W Black. 2019. my way of telling a story: Persona based grounded story generation. In Proceedings of the Second Workshop on Storytelling, pages 11–21. find
  4. Liz Coppock. 2005. Politeness strategies in conversation closings. unpublished paper available online at http://www.stanford. edu/˜ coppock/face. pdf (last accessed 23 December 2007). ↗
  5. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250–259, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  6. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the sixth workshop on statistical machine translation, pages 85–91. Association for Computational Linguistics. find
  7. Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  8. Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, and Li Deng. 2017. Stylenet: Generating attractive visual captions with styles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3137–3146. find
  9. Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pages 507–517. International World Wide Web Conferences Steering Committee. find
  10. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
  11. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1587–1596, International Convention Centre, Sydney, Australia. PMLR. find our card
  12. Daniel Im Jiwoong Im, Sungjin Ahn, Roland Memisevic, and Yoshua Bengio. 2017. Denoising criterion for variational auto-encoding framework. In Thirty-First AAAI Conference on Artificial Intelligence. find
  13. Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. 2017. Shakespearizing modern language using copy-enriched sequence to sequence models. In Proceedings of the Workshop on Stylistic Variation, pages 10–19, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  14. Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 424–434, Florence, Italy. Association for Computational Linguistics. find our card
  15. Daniel Jurafsky, Elizabeth Shriberg, and Debra Bi-asca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. Technical Report 97-02, University of Colorado, Boulder Institute of Cognitive Science, Boulder, CO. find
  16. Dániel Z Kádár and Sara Mills. 2011. Politeness in East Asia. Cambridge University Press. find
  17. Dongyeop Kang and Eduard Hovy. 2019. xslue: A benchmark and analysis platform for cross-style language understanding and evaluation. find
  18. Bryan Klimt and Yiming Yang. 2004. Introducing the enron corpus. In CEAS. find
  19. Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865–1874, New Orleans, Louisiana. Association for Computational Linguistics. find our card
  20. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. find
  21. Andrew McCallum, Xuerui Wang, and Andrés Corrada-Emmanuel. 2007. Topic and role discovery in social networks with experiments on enron and academic email. J. Artif. Int. Res., 30(1):249–272. find
  22. Ardith J Meier. 1995. Defining politeness: Universality in appropriateness. Language Sciences, 17(4):345– 356. find
  23. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2017. Regularizing and optimizing lstm language models. arXiv preprint arXiv:1708.02182. ↗ our card
  24. Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373–389. ↗
  25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. find our card
  26. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958. find
  27. Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Ma-heswaran. 2019. “transforming” delete, retrieve, generate approach for controlled text style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3267– 3277, Hong Kong, China. Association for Computational Linguistics. find our card
  28. Kelly Peterson, Matt Hohensee, and Fei Xia. 2011. Email formality in the workplace: A case study on the Enron corpus. In Proceedings of the Workshop on Language in Social Media (LSM 2011), pages 86–95, Portland, Oregon. Association for Computational Linguistics. find
  29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  30. Vinodkumar Prabhakaran, Emily E. Reid, and Owen Rambow. 2014. Gender and power: How gender and gender environment affect manifestations of power. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1965–1976, Doha, Qatar. Association for Computational Linguistics. find
  31. Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, and Yulia Tsvetkov. 2018. RtGender: A corpus for studying differential responses to gender. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  32. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. Style transfer through back-translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 866–876, Melbourne, Australia. Association for Computational Linguistics. find our card
  33. Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Mask and infill: Applying masked language model for sentiment transfer. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5271–5277. International Joint Conferences on Artificial Intelligence Organization. find
  34. Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129–140. find our card
  35. Ruochen Xu, Tao Ge, and Furu Wei. 2019. Formality style transfer with hybrid textual annotations. arXiv preprint arXiv:1903.06353. ↗
  36. Sravana Reddy and Kevin Knight. 2016. Obfuscating gender in social media writing. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 17–26. find
  37. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. ↗
  38. Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in neural information processing systems, pages 6830–6841. find our card
  39. Jitesh Shetty and Jafar Adibi. 2004. The enron email dataset database schema and brief statistical report. Information sciences institute technical report, University of Southern California, 4(1):120–128. find
  40. Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In Proceedings of COLING 2012, pages 2899–2914, Mumbai, India. find our card
  41. The COLING 2012 Organizing Committee. find
  42. Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. In Advances in Neural Information Processing Systems, pages 7287–7298. find
  43. Jen-Yuan Yeh and Aaron Harnly. 2006. Email thread reassembly using similarity matching. In Conference on Email and Anti-Spam. Conference on Email and Anti-Spam. find
  44. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204– 2213, Melbourne, Australia. Association for Computational Linguistics. Non-polite Input DRG Our Model jon - - please use this resignation letter in lieu of the one sent on friday . - i think this would be a good idea if you could not be a statement that harry ’s signed in one of the schedule . jon - sorry - please use this resignation letter in lieu of the one event sent on if you have a few minutes today, give me a call i’ll call today to discuss this. if you have a few minutes today, please give me a call at anyway you can let me know. anyway, i’m sure i’m sure. anyway please let me know as soon as possible yes, go ahead and remove it. yes, please go to the link below and delete it. yes, we can go ahead and remove it. can you explain a bit more about how those two coexist ? also ..... i can explain how the two more than <unk> i can help with mike ? can you explain a bit more about how those two coexist ? also thanks go ahead and sign it - i did . go away so we can get it approved . we could go ahead and sign it - i did look at *Table 6: Additional Qualitative Examples of outputs from our Model and DRG for the Politeness Transfer Task* Task Non-polite Input DRG Our Model Fem → Male my husband ordered the brisket . my wife had the best steak . my wife ordered the brisket . Fem → Male i ’ m a fair person . i ’ m a good job of the <unk> . i ’ m a big guy . Male → Fem my girlfriend and i recently stayed at this sheraton . i recently went with the club . my husband and i recently stayed at this office . Male → Fem however , once inside the place was empty . however , when the restaurant was happy hour for dinner . however , once inside the place was super cute . Pos → Neg good drinks , and good company . horrible company . terrible drinks , terrible company. Pos → Neg i will be going back and enjoying this great place ! i will be going back and enjoying this great ! i will not be going back and enjoying this garbage ! Neg → Pos this is the reason i will never go back . this is the reason i will never go back . so happy i will definitely be back . Neg → Pos salsa is not hot or good . salsa is not hot or good . salsa is always hot and fresh . Dem → Rep i am confident of trumps slaughter . i am mia love i am confident of trumps administration . Dem → Rep we will resist trump we will impeach obama we will be praying for trump Rep → Dem video : black patriots demand impeachment of obama video : black police show choose video : black patriots demand to endorse obama Rep → Dem mr. trump is good ... but mr. marco rubio is great ! ! thank you mr. good ... but mr. kaine is great senator ! ! mr. schumer is good ... but mr. pallone is great ! ! Fact → Rom a woman is sitting near a flower bed overlooking a tunnel . a woman is sitting near a flower overlooking a tunnel, determined to a woman is sitting near a brick rope , excited to meet her boyfriend . Fact → Rom two dogs play with a tennis ball in the snow . two dogs play with a tennis ball in the snow . two dogs play with a tennis ball in the snow celebrating their friendship . Fact → Hum three kids play on a wall with a green ball . three kids on a bar on a field of a date . three kids play on a wall with a green ball fighting for supremacy . Fact → Hum a black dog plays around in water . a black dog plays in the water . a black dog plays around in water looking for fish . *Table 7: Additional Qualitative Examples of our Model and DRG for other Transfer Tasks* find our card