Elamur Research⌕
☰
home / papers / 2018 / Towards Universal Dialogue State Tracking

Towards Universal Dialogue State Tracking

Paper · 2018 · Liliang Ren, Kaige Xie, Lu Chen et al.

Summary

The paper tackles dialogue state tracking (DST), which means estimating a user's goal as slot-value pairs at each turn of a task-oriented spoken dialogue. Many existing trackers have three scaling problems. Some assume a fixed ontology. Others need a separate set of parameters for each slot. Others rely on hand-built semantic dictionaries for delexicalisation. The authors propose StateNet to address all three. It builds n-gram user utterance features weighted by ASR confidence and passes them through a "multi-scale receptors" layer. It combines these with machine act features and gates the result with a slot embedding, then tracks the dialogue over turns with an LSTM. Instead of classifying over a fixed list of values, it outputs a vector the size of a word embedding. Each candidate value is scored by its 2-norm distance to that vector. As a result, values can be added or changed freely and one model can serve every slot. On DSTC2 and WOZ 2.0, the parameter-shared and pre-initialized variant achieved the best joint goal accuracy reported at the time. This points toward trackers that can grow to larger and changing domains.

Key points

  • Three variants were tested: StateNet with separate per-slot models, StateNet PS with parameters shared across slots, and StateNet PSI, which is shared and initialized from a single-slot pre-trained model. StateNet PS uses about a third of the parameters of the separate-model version.
  • Joint goal accuracy for StateNet PSI was 75.5 on DSTC2 and 88.9 on WOZ 2.0. The strongest prior baseline, GLAD, scored 74.5 and 88.1.
  • Both benchmarks use the same Cambridge restaurant ontology with three slots: food, pricerange and area. WOZ 2.0 has typed rather than spoken input, so it is free of ASR errors.
  • Pre-training on the hardest slot, food, worked best (75.5 / 88.9). Pre-training on pricerange gave 73.6 / 88.2, and area gave 73.5 / 87.8.
  • The setup uses fixed Paragram-SL999,300-dimensional embeddings, N_c = 128, 4 receptors, bigrams and 3 ASR hypotheses. It was implemented in MXNet 1.1.0 and needs no manually tagged user utterances.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China63 works · 15,047 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 401 works

Cite

Cite as Liliang Ren, Kaige Xie, Lu Chen and Kai Yu. 2018. Towards Universal Dialogue State Tracking. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1299

@inproceedings{ren2018universal,
  title = {{Towards Universal Dialogue State Tracking}},
  author = {Liliang Ren, Kaige Xie, Lu Chen and Kai Yu},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1299},
  url = {https://doi.org/10.18653/v1/d18-1299}
}

References

32 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
  2. Lu Chen, Bowen Tan, Sishan Long, and Kai Yu. 2018. Structured dialogue policy with graph neural networks. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1257–1268. find
  3. Zhehuai Chen, Yimeng Zhuang, and Kai Yu. 2017. Confidence measures for ctc-based phone synchronous decoding. In IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pages 4850–4854. find
  4. Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014a. The second dialog state tracking challenge. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 263–272, Philadelphia, PA, U.S.A. Association for Computational Linguistics. find
  5. Matthew Henderson, Blaise Thomson, and Steve Young. 2014b. Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In Proceedings of IEEE Spoken Language Technology Workshop (SLT). find
  6. Matthew Henderson, Blaise Thomson, and Steve Young. 2014c. Word-based dialog state tracking with recurrent neural networks. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 292–299, Philadelphia, PA, U.S.A. Association for Computation…
  7. Mettew Henderson, Blaise Thomson, and Jason D. Williams. 2014d. The third dialog state tracking challenge. In Proceedings of IEEE Spoken Language Technology Workshop (SLT). find
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  9. (Volume 1: Long Papers), volume 1, pages 1777– 1788. find
  10. Julien Perez and Fei Liu. 2017. Dialog state tracking, a machine reading approach using memory network. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 305–314. find
  11. Osman Ramadan, Paweł Budzianowski, and Milica Gasic. 2018. Large-scale multi-domain belief tracking with knowledge sharing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 432–437.
  12. Abhinav Rastogi, Dilek Hakkani-Tur, and Larry Heck. 2017. Scalable multi-domain dialogue state tracking. arXiv preprint arXiv:1712.10224. — [arXiv:1712.10224](https://arxiv.org/abs/1712.10224) find
  13. Kai Sun, Lu Chen, Su Zhu, and Kai Yu. 2014a. A generalized rule based tracker for dialogue state tracking. In Proceedings of IEEE Spoken Language Technology Workshop (SLT). find
  14. Kai Sun, Lu Chen, Su Zhu, and Kai Yu. 2014b. The SJTU system for dialog state tracking challenge 2. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 318–326, Philadelphia, PA, U.S.A. Association for Computational Linguistics. find
  15. Kai Sun, Qizhe Xie, and Kai Yu. 2016a. Recurrent polynomial network for dialogue state tracking. Dialogue & Discourse, 7(3):65–88. find
  16. Kai Sun, Su Zhu, Lu Chen, Siqiu Yao, Xueyang Wu, and Kai Yu. 2016b. Hybrid dialogue state tracking for real world human-to-human dialogues. In Proc. InterSpeech, pages 2060–2064, San Francisco, America. find
  17. Blaise Thomson and Steve Young. 2010. Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems. Computer Speech & Language, 24(4):562–588. find
  18. Sungjin Lee. 2013. Structured discriminative model for dialog state tracking. In Proceedings of the SIGDIAL 2013 Conference, pages 442–451, Metz, France. Association for Computational Linguistics. find
  19. Zhuoran Wang and Oliver Lemon. 2013. A simple and generic belief tracking mechanism for the dialog state tracking challenge: On the believability of observed information. In Proceedings of the SIGDIAL 2013 Conference, pages 423–432, Metz, France. Association for Computational Linguistics. find
  20. Sungjin Lee and Maxine Eskenazi. 2013. Recipe for building robust spoken dialog state trackers: Dialog state tracking challenge system description. In Proceedings of the SIGDIAL 2013 Conference, pages 414–422, Metz, France. Association for Computational Linguistics. find
  21. Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M. Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In EACL, pages 438–449, Valencia, Spain. Association for Computational Linguistics. find
  22. Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics John Wieting, Mohit Bansal, Kevin Gimpel, Karen Lives… find
  23. Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference, pages 404–413, Metz, France. Association for Computational Linguistics. find
  24. Kaige Xie, Cheng Chang, Liliang Ren, Lu Chen, and Kai Yu. 2018. Cost-sensitive active learning for dialogue state tracking. In Proceedings of the 19th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 209– 213, Melbourne, Australia. Association for Computational… find
  25. Qizhe Xie, Kai Sun, Su Zhu, Lu Chen, and Kai Yu. 2015. Recurrent polynomial network for dialogue state tracking with mismatched semantic parsers. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 295–304, Prague, Czech Republic. Association for… find
  26. Puyang Xu and Qi Hu. 2018. An end-to-end approach for handling unknown slot values in dialogue state tracking. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1.
  27. Steve Young, Milica Gašić, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu. 2010. The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language, 24(2):150–174. find
  28. Steve Young, Milica Gasic, Blaise Thomson, and Jason D. Williams. 2013. POMDP-based statistical spoken dialog systems: A review. Proceedings of the IEEE, 101(5):1160–1179. find
  29. Kai Yu, Lu Chen, Bo Chen, Kai Sun, and Su Zhu. 2014. Cognitive technology in task-oriented dialogue systems: Concepts, advances and future. Chinese Journal of Computers, 37(18):1–17. find
  30. Kai Yu, Lu Chen, Kai Sun, Qizhe Xie, and Su Zhu. 2016. Evolvable dialogue state tracking for statistical dialogue management. Frontiers of Computer Science, 10(2):201–215. find
  31. Kai Yu, Kai Sun, Lu Chen, and Su Zhu. 2015. Constrained markov bayesian polynomial for efficient dialogue state tracking. IEEE/ACM Transactions on Audio, Speech and Language Processing, 23(12):2177–2188. find
  32. Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive encoder for dialogue state tracking. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1458–1467. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ren-2018-universal-dialogue-state-tracking/