Elamur Research⌕
☰
home / papers / 2017 / End-to-End Task-Completion Neural Dialogue Systems

End-to-End Task-Completion Neural Dialogue Systems

Summary

Classic task-completion dialogue systems chain separately trained modules (LU, dialogue manager, NLG), so mistakes made early propagate and pile up downstream. The authors propose a neural dialogue system trained end to end that queries a structured database to complete tasks, tested on movie-ticket booking. Its parts are:

  • a single LSTM that jointly predicts the intent and IOB slot tags;
  • a dialogue manager whose policy is first warm-started by supervised imitation of a rule-based agent, then refined with reinforcement learning (a DQN agent);
  • an agenda-based user simulator with a hybrid NLG.

Unlike earlier end-to-end RL work, users may ask their own questions during the dialogue and are not limited to yes/no answers. An error model controller injects LU noise of different types and rates, which allows a systematic study of how understanding errors affect dialogue success. The RL agent beats rule-based baselines with both simulated and real users, measured by success rate and user ratings. Errors in slots hurt more than errors in intents, and wrong slot values do the most damage. These findings can guide how joint LU models are designed with error control in mind.

Key points

  • Language understanding: one LSTM performs intent classification and slot filling together. Its output sequence is the IOB slot tags followed by a sentence-level intent tag, trained by maximum likelihood.
  • Dialogue state: the state is kept implicit rather than supervised with state-tracking labels. A rule-based agent bootstraps the policy, which RL then improves.
  • User simulator:
    • User goals contain inform slots (constraints) and request slots (information wanted from the agent).
    • NLG mixes predefined templates with a seq2seq LSTM generator. The generator uses beam search and fills slot placeholders afterwards.
  • Noise experiments:
    • Slot error types tested were slot deletion, incorrect slot value and incorrect slot; incorrect value hurt most.
    • Slot error rates of 0%, 10% and 20% were tested with a fixed 10% intent error rate.
    • Performance is more sensitive to slot error rate than to intent error rate.
  • Human evaluation and code: 110 sessions with 8 real users showed the RL agent clearly ahead of the rule-based agent. Code is released as TC-Bot (github.com/MiuLab/TC-Bot).

Interesting ideas

  • I1. Controlled noise injection. An error model controller corrupts LU output by error type and rate, turning robustness into a measurable experimental variable. The same trick could stress-test any pipeline whose components feed noisy predictions downstream.
  • I2. Rule-based warm start before RL. Imitating a handcrafted agent first gives RL a reasonable starting policy. This is useful whenever exploration from scratch is expensive but a simple heuristic system already exists.
  • I3. Hybrid NLG with templates as overrides. Templates are used when they cover a dialogue act, and a learned generator handles everything else. Developers can patch known weak spots without retraining, which suits low-resource languages and domains.
  • I4. Goal-driven user simulator. An agenda-based simulator built from a labeled dataset gives reproducible, identical evaluation conditions for comparing dialogue agents. This is valuable where recruiting human testers is costly.
  • I5. Error priorities for joint LU. Since slot values matter more than intents for task success, joint LU models could weight slot accuracy more heavily or flag uncertain values. Confirmation behaviour learned by the agent then compensates at the cost of slightly longer dialogues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

92 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 91 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{li2017end,
  title = {{End-to-End Task-Completion Neural Dialogue Systems}},
  author = {Li, Xiujun and Chen, Yun-Nung and Li, Lihong and Gao, Jianfeng and Celikyilmaz, Asli},
  booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2017},
  url = {https://aclanthology.org/I17-1074/}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jost Schatzmann, Blaise Thomson, and Steve Young. 2007. Error simulation for training statistical dialogue systems. In IEEE Workshop on Automatic Speech Recognition & Understanding. find
  2. Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. In Proceedings of ICLR. find
  3. Jost Schatzmann and Steve Young. 2009. The hidden agenda user simulation model. IEEE transactions on audio, speech, and language processing 17(4):733–747. find
  4. Yun-Nung Chen, Dilek Hakanni-Tür, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, and Li Deng. 2016. Syntax or semantics? knowledge-guided joint semantic frame parsing. In Proceedings of the 6th IEEE Workshop on Spoken Language Technology. pages 348–355. find
  5. Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. 2015. Prioritized experience replay. arXiv:1511.05952 . find
  6. Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Continuously learning neural dialogue management. arXiv:1606.02689 . find
  7. Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng. 2017. Towards end-to-end reinforcement learning of dialogue agents for information access. In Proceedings of the 55th Annual Meeting of the Association also explicitly show the user goal at the head of the… find our card
  8. Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, David Vandyke, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. pages 438–449. find
  9. Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pages 1711–1721. find our card
  10. Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference. pages 404–413. find
  11. Tiancheng Zhao and Maxine Eskenazi. 2016. Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. pages 1–10. find
  12. Victor Zue, Stephanie Seneff, James R Glass, Joseph Polifroni, Christine Pao, Timothy J Hazen, and Lee Hetherington. 2000. JUPITER: a telephone-based conversational interface for weather information. IEEE Transactions on speech and audio processing 8(1):85–96. find
  13. Victor W Zue and James R Glass. 2000. Conversational interfaces: Advances and challenges. Proceedings of the IEEE 88(8):1166–1180. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2017-end-end-task-completion-neural/