Elamur Research⌕
☰
home / papers / 2018 / A Unified Neural Architecture for Joint Dialog Act Segmentation and Recognition in Spoken Dialog System

A Unified Neural Architecture for Joint Dialog Act Segmentation and Recognition in Spoken Dialog System

Paper · 2018 · Tianyu Zhao and Tatsuya Kawahara

Summary

Spoken dialog systems need to know both where each dialog act (DA) begins and ends inside a turn and which act it is, such as Statement, Backchannel or Yes-No-Question. Most earlier DA recognition work assumed hand-segmented transcripts, but ASR output has no punctuation or boundaries. Zhao and Kawahara propose a unified neural architecture with two parts. A BiLSTM word tagger marks segment ends using I/E labels. A hierarchical LSTM classifier then labels each predicted segment, drawing on encodings of the k preceding sentences. They compare three ways of sharing parameters between the parts: nothing shared, only word embeddings shared, or embeddings plus the BiLSTM encoder shared. They also build a single-step LSTM baseline that predicts combined labels. On the SwDA corpus, the jointly trained models beat both independent training and the single-step tagger. Sharing only the embeddings gave the lowest joint error. This shows that a cascaded design keeps the benefit of dialog context while joint training reduces the harm from segmentation errors.

Key points

  • Data: the Switchboard Dialog Act (SwDA) corpus. Segmentation is framed as I/E tagging over a whole speaker turn, and recognition as classification of a segment given preceding segment encodings.
  • Model 1 shares no layers, Model 2 shares word embeddings, and Model 3 shares embeddings and BiLSTM encoders. Their best word-level joint error rates are 31.8%, 30.6% and 31.0%, so Model 2 achieves a statistically significant 1.2-point reduction.
  • The single-step neural tagger with combined labels such as "E Statement" reaches 33.5% joint error. The authors attribute this mainly to its inability to use preceding-sentence context.
  • When punctuation and slash marks are kept, Model 2 reaches 20.5% joint error with 0.3% segmentation error. A punctuation-using single-step CRF from Quarteroni et al. (2011) scored 29.1% and 1.4% on the same measures.
  • Model 3 showed a reverse trend: recognition accuracy fell as segmentation error decreased. The authors suspect that heavy sharing lets recognition gradients disrupt the segmenter.

Interesting ideas

  • I1. Keep segmentation and labelling as separate cascaded components that share low-level layers, rather than merging them into one combined tag set, so the labeller can still look at previous units as context.
  • I2. During multi-task training, sum both task losses on every mini-batch instead of alternating tasks by epoch, so shared parameters receive both signals together.
  • I3. Report results under three regimes: gold boundaries, predicted boundaries, and a word-level joint error. This separates the downstream model's own skill from the errors passed down by upstream segmentation.
  • I4. Sharing more layers is not automatically better. Sharing only embeddings helped most, while deeper sharing let one task's gradients disturb the other.
  • I5. Evaluate with and without punctuation to reveal how much a segmenter relies on surface cues that ASR or informal text will not supply.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act detection in prag could first split unpunctuated or run-on input into act units and then classify each unit using the acts before it, keeping both steps inspectable.
  • I2 (learn): when training a model with several heads on silver data, combined per-batch losses could prevent one objective from repeatedly overwriting the other.
  • I3 (learn): pipeline stages could each be scored with gold versus predicted input from the previous stage, showing exactly how much error each step passes downstream.
  • I4 (tree): if gparse gains auxiliary heads such as tagging or segmentation, test how deep the sharing goes and check that parsing quality does not move against the auxiliary task.
  • I5 (tokens): the sentence splitter and tokenizer could be benchmarked on punctuation-stripped copies of the same texts to measure robustness to unpunctuated input.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Kyoto UniversityKyoto · Japan27 works · 8,049 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Tianyu Zhao and Tatsuya Kawahara. 2018. A Unified Neural Architecture for Joint Dialog Act Segmentation and Recognition in Spoken Dialog System. Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-5021

@inproceedings{zhao2018unified,
  title = {{A Unified Neural Architecture for Joint Dialog Act Segmentation and Recognition in Spoken Dialog System}},
  author = {Tianyu Zhao and Tatsuya Kawahara},
  booktitle = {Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2018},
  doi = {10.18653/v1/w18-5021},
  url = {https://doi.org/10.18653/v1/w18-5021}
}

References

11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai, and Xiaofei He. 2017. Dialogue act recognition via Matthias Zimmermann, Andreas Stolcke, and Elizabeth Shriberg. 2006. Joint segmentation and classification of dialog acts in multiparty meetings. In 2006 IEEE International Conference on Acoustics Speech and Signal Processing, ICASSP 2006, Toulouse, France, May 14-19, 2006, pages 581–584. find
  2. Wei Li and Yunfang Wu. 2016. Multi-level gated recurrent neural network for dialog act classification. In COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan, pages 1970–1979.
  3. Ramesh R. Manuvinakurike, Maike Paetzel, Cheng Qu, David Schlangen, and David DeVault. 2016. Toward incremental dialogue act segmentation in fast-paced interactive dialogue systems. In Proceedings of the SIGDIAL 2016 Conference, The 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 13-15 September 2016, Los Angeles, CA, USA, pages 252–262.
  4. Silvia Quarteroni, Alexei V. Ivanov, and Giuseppe Riccardi. 2011. Simultaneous dialog act segmentation and classification from human-human spoken conversations. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, May 22-27, 2011, Prague Congress Center, Prague, Czech Republic, pages 5596–5599. find
  5. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. ↗
  6. Sheng-syun Shen and Hung-yi Lee. 2016. Neural attention models for sequence classification: Analysis and application to key term extraction and dialogue act detection. In Interspeech 2016, 17th Annual Conference of the International Speech Communication Association, San Francisco, CA, USA, September 8-12, 2016, pages 2716–2720. find
  7. Vivek Kumar Rangarajan Sridhar, Srinivas Bangalore, and Shrikanth Narayanan. 2009. Combining lexical, syntactic and prosodic cues for improved online dialog act tagging. Computer Speech & Language, 23(4):407–422. ↗
  8. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
  9. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. ↗
  10. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding, volume 4, page 5. find
  11. Matthias Zimmermann. 2009. Joint segmentation and classification of dialog acts using conditional random fields. In INTERSPEECH 2009, 10th Annual Conference of the International Speech Communication Association, Brighton, United Kingdom, September 6-10, 2009, pages 864–867. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2018-unified-neural-architecture-joint-dialog/