📄 PoliSe: Reinforcing Politeness Using User Sentiment for Customer Care Response Generation

Summary

The paper addresses customer-care dialogue. A neutral, generic agent reply, such as "Send us a dm", is rewritten into a courteous one, and the kind of courtesy depends on how the customer feels. An upset customer should get apology or reassurance. A happy customer should get thanks or a greeting. Because no sentiment-labelled customer-care dialogues existed, the authors added sentiment labels to the Twitter-based Courteously Yours Customer Care Dataset (CYCCD).

Their model is a multi-task, end-to-end network built on a hierarchical Transformer:

• a sentence encoder handles individual utterances;
• a second Transformer encodes the dialogue context on top of it;
• a sentiment classifier predicts the user's sentiment;
• the decoder is initialised with that predicted sentiment plus the context.

Reinforcement-learning rewards push the output toward politeness that fits the sentiment while keeping the original content. In human evaluation, the jointly trained model beat Seq2Seq, HRED, Polite-RL, PT-TGA and PG-RL on fluency, relevance and politeness appropriateness. The work shows that politeness is not one uniform style. The right strategy depends on the listener's affective state.

Key points

• Data: CYCCD (Golchha et al., 2019) of Twitter customer-care conversations, newly annotated with user sentiment (negative / positive / neutral).
• Polite behaviours by sentiment: apology, assurance and empathy for negative users; appreciation and acknowledgement for positive users; greeting for neutral users.
• Architecture:
  • hierarchical Transformer encoder-decoder with GloVe word embeddings and sine-cosine positional embeddings;
  • a BERT-based sentiment classifier;
  • task-specific RL rewards for politeness and content preservation.
• Human evaluation (fluency / relevance / politeness appropriateness):
  • jointly trained HT + RL + SE: 4.23 / 4.17 / 75%;
  • strongest baseline, PG-RL: 4.11 / 4.06 / 67%;
  • Seq2Seq: 3.82 / 3.73 / 48%.
• Ablation and error analysis:
  • the ladder climbs step by step: HT (65%) → HT + RL (71%) → HT + RL + SE (73%) → joint training (75%);
  • automatic metrics (BLEU-4, Rouge-L, perplexity, politeness accuracy) are reported to follow the same trend;
  • error types found: repetition or unk tokens, loss of information, and sentiment-inconsistent politeness caused by misclassified sentiment.

Interesting ideas

• I1. Politeness is chosen as a strategy (apology, assurance, appreciation, greeting, acknowledgement) based on the listener's state, not applied as one uniform "polite" style; this explicit mapping is easy to inspect and adapt.
• I2. Treating the generic reply as fixed content and adding only a courteous wrapper separates "what to say" from "how to say it"; a reward guards the content side.
• I3. Courtesy acts such as apologising, reassuring, thanking and greeting are a compact label set for speech acts tied to the speaker's attitude toward the hearer.
• I4. The error taxonomy separates fluency failures (repetition), content failures (lost information) and pragmatic failures (politeness that clashes with sentiment); the last is traced back to an upstream classifier mistake.
• I5. The ablation adds components one at a time (hierarchy, then RL, then sentiment, then joint training) and shows each step's gain on the same human metrics.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (back): the generator could pick a politeness move from the reader's detected stance in the world model, as an explainable rule rather than a learned style vector.
• I2 (back): in the text → graph → text compressor, content could be generated from the meaning graph first, with a separate, checkable layer adding register and courtesy that must not change the graph.
• I3 (frames): crate prag could tag these acts in dialogue and link each one to the stance of the previous turn, giving material for implicature and irony checks (e.g. thanks after a complaint).
• I4 (learn): evaluation could report these three classes separately and attribute pragmatic errors to the earlier pipeline stage that caused them.
• I5 (learn): the same incremental ladder could be used when adding witness parsers or repair rules, so each component's contribution is measured rather than assumed.