Word-Based Dialog State Tracking with Recurrent Neural Networks

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W14-4340.pdf

Summary

Spoken dialog systems must keep a probability distribution over what the user wants, a task called dialog state tracking. Most trackers consume the output of a separate SLU semantic decoder and depend on hand-designed features. Henderson, Thomson and Young skip the SLU stage and update the belief state directly from ASR N-best hypotheses and the last system act. Their model uses one recurrent neural network per slot. It takes simple weighted n-gram features and keeps an internal memory across turns. To cope with rare or unseen slot values, part of the network works on "tagged" features, where specific values and slots are replaced by generic placeholders. This lets it learn patterns that hold for any value. Training relies on careful initialisation, namely denoising autoencoders and shared initialisation. Several trackers are combined by averaging their scores. Evaluated blind on the DSTC2 restaurant-domain corpus, the word-based RNN was among the strongest entries, especially for the quality of its probability estimates. This suggests that removing the explicit semantic layer can reduce information loss without heavy feature engineering.

Key points

The data is DSTC2 (Cambridge restaurant search). It has 1612 training dialogs and 506 dev dialogs, plus 1117 test dialogs collected with a dialog manager that does not appear in the labelled data.
Four variants were submitted under "team4". They cross two choices: word-based (ASR) vs. SLU input, and with vs. without the value-specific sub-network h. Each variant is a score-averaged ensemble of 12 trackers.
The best system (h-ASR) reached 0.768 joint goal accuracy with an L2 of 0.346 on the test set. It ranked first for joint goal L2 and for requested-slot accuracy and L2.
Across 318 DSTC2 metrics, h-ASR came top in 89, more than any other tracker. The ASR variant without h was second with 33.
Variants without h do not depend on specific values, so they can handle new values at runtime. An unoptimised Python implementation processes about 50 turns per second.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

94 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 89 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-4340

@inproceedings{henderson2014word,
  title = {{Word-Based Dialog State Tracking with Recurrent Neural Networks}},
  author = {Henderson, Matthew and Thomson, Blaise and Young, Steve},
  booktitle = {Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)},
  year = {2014},
  doi = {10.3115/v1/w14-4340},
  url = {https://doi.org/10.3115/v1/w14-4340}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Dan Bohus and Alex Rudnicky. 2006. A K-hypotheses+ Other Belief Updating Model. Proc. of the AAAI Workshop on Statistical and Empirical Methods in Spoken Dialogue Systems. find
Léon Bottou. 1991. Stochastic gradient learning in neural networks. In Proceedings of Neuro-Nı̂mes 91, Nı̂mes, France. EC2. find
Yun-Nung Chen, William Yang Wang, and Alexander I Rudnicky. 2013. An empirical investigation of sparse log-linear models for improved dialogue act classification. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. find
Matthew Henderson, Milica Gašić, Blaise Thomson, Pirros Tsiakoulis, Kai Yu, and Steve Young. 2012. find
Discriminative Spoken Language Understanding Using Word Confusion Networks. In Spoken Language Technology Workshop, 2012. IEEE. find
Matthew Henderson, Blaise Thomson, and Jason Williams. 2013a. Dialog State Tracking Challenge 2 & 3 Handbook. camdial.org/˜mh521/dstc/ . find
Matthew Henderson, Blaise Thomson, and Steve Young. 2013b. Deep Neural Network Approach for the Dialog State Tracking Challenge. In Proceedings of SIGdial, Metz, France, August. find
Matthew Henderson, Blaise Thomson, and Jason Williams. 2014. The second dialog state tracking challenge. In Proceedings of the SIGdial 2014 Conference, Baltimore, U.S.A., June. find
Sungjin Lee and Maxine Eskenazi. 2013. Recipe for building robust spoken dialog state trackers: Dialog state tracking challenge system description. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. find
Sungjin Lee. 2013. Structured discriminative model for dialog state tracking. In Proceedings of the SIG-DIAL 2013 Conference, Metz, France, August. find
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2012. Understanding the exploding gradient problem. CoRR. find
Hang Ren, Weiqun Xu, Yan Zhang, and Yonghong Yan. 2013. Dialog state tracking using conditional random fields. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. find
Gökhan Tür, Anoop Deoras, and Dilek Hakkani-Tür. 2013. Semantic parsing using word confusion networks with conditional random fields. In INTER-SPEECH. find
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland. find
Jason Williams. 2013. Multi-domain learning and generalization in dialog state tracking. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. find
Steve Young, Milica Gašić, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu. 2009. The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language. find

↑ back to top