Elamur Research⌕
☰
home / papers / 2021 / What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition

What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition

Summary

This TACL 2021 study tackles joint dialog act segmentation and classification of English conversation transcripts. Earlier work usually assumed gold segment boundaries or relied on CRF/RNN segmenters that made many errors. The authors adapt two long-input pre-trained transformers, XLNet (windowed processing with propagated activations) and Longformer (local attention), to label whole dialogs. They concatenate speaker turns with a special TURN token and compare the results against a BiGRU baseline on the SWDA and MRDA corpora. Controlled experiments vary three factors: how much dialog context the model sees, whether transcripts keep punctuation and casing, and how fine-grained the label set is. Broad context brings the largest gains for segmentation and for rare classes. Removing punctuation hurts performance more than any other factor, and models without it tend to merge adjacent segments. Label granularity has no effect on segmentation quality. These findings matter for spoken language understanding pipelines that take ASR output, which often lacks formatting. They also suggest that annotation could use simpler label sets.

Key points

  • Dialog-level XLNet reaches a dialog act segmentation error rate (DSER) of 8.4% on SWDA and 14.2% on MRDA with original transcripts. On lowercased, unpunctuated text these rise to 17.5% and 32.9%.
  • Without punctuation, even the strongest broad-context model (XLNet+prop) falls behind a simple turn-level BiGRU trained on formatted text. Commas and discourse markers are frequent sources of missed boundaries.
  • Adding broad context reduces the share of SWDA classes that are never correctly recognized from over 50% to under 10%.
  • On MRDA, segmentation quality stays the same across the 51-act full, 12-act general and 5-act basic sets, and also for pure segmentation without labels.
  • XLNet makes the best use of context, while Longformer runs inference about 5× faster on CPU. The code is released under Apache 2.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00420 · arXiv 2107.02294

@article{zelasko2021what,
  title = {{What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition}},
  author = {Żelasko, Piotr and Pappagari, Raghavendra and Dehak, Najim},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00420},
  eprint = {2107.02294},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1162/tacl_a_00420}
}

References

56 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In Proceedings.(ICASSP’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., volume 1, pages I–1061. IEEE. find
  2. John L. Austin. 1962. How to Do Things with Words. find
  3. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 [v1]. ↗
  4. Tony Bergstrom and Karrie Karahalios. 2009. Conversation clusters: Grouping conversation topics through human-computer dialog. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2349–2352. https://doi.org/10 .1145/1518701.1519060 https://github.com/pzelasko/daseg/tree/version/tacl2021. ↗
  5. Chandrakant Bothe, Cornelius Weber, Sven Magg, and Stefan Wermter. 2018. A context-based approach for dialogue act recognition using simple recurrent neural networks. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  6. Harry Bunt, Jan Alexandersson, Jae-Woong Choe, Alex Chengyu Fang, Koiti Hasida, Volha Petukhova, Andrei Popescu-Belis, and David R. Traum. 2012. ISO 24617-2: A semantically-based standard for dialogue annotation. In LREC, pages 430–437. find our card
  7. Harry Bunt, Volha Petukhova, and Alex Chengyu Fang. 2017. Revisiting the ISO standard for dialogue act annotation. In Proceedings of the 13th Joint ISO-ACL Workshop on Interoperable Semantic Annotation (ISA-13). find our card
  8. Harry Bunt, Volha Petukhova, Emer Gilmartin, Catherine Pelachaud, Alex Fang, Simon Keizer, and Laurent Prevot. 2020. The ISO standard for dialogue act annotation. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 549–558. find
  9. Lou Burnard. 2000. The British National Corpus Users Reference Guide. Oxford University Computing Services Oxford. find
  10. Eugene Charniak and Mark Johnson. 2001. Edit detection and parsing for transcribed speech. In Second Meeting of the North American Chapter of the Association for Computational Linguistics. https://doi.org/10.3115/1073336.1073352 ↗
  11. Chloe Clavel and Zoraida Callejas. 2015. Sentiment analysis: From opinion mining to human-agent interaction. IEEE Transactions on Affective Computing, 7(1):74–93. https://doi.org/10.1109/TAFFC.2015.2444846 ↗
  12. Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. 2020. Guiding attention in sequence-to-sequence models for dialogue act prediction. In AAAI, pages 7594–7601. https://doi.org/10.1609/aaai.v34i05.6259 ↗
  13. Mark G. Core and James Allen. 1997. Coding dialogs with the DAMSL annotation scheme. In AAAI Fall Symposium on Communicative Action in Humans and Machines, volume 56, pages 28–35. Boston, MA. find
  14. Zhigang Dai, Jinhua Fu, Qile Zhu, Hengbin Cui, Yuan Qi, et al. 2020. Local contextual attention with hierarchical structure for dialogue act recognition. arXiv preprint arXiv:2003. 06044 [v1]. ↗
  15. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988. find
  16. Viet-Trung Dang, Tianyu Zhao, Sei Ueno, Hirofumi Inaguma, and Tatsuya Kawahara. 2020. End-to-end speech-to-dialog-act recognition. Proceedings of Interspeech 2020, pages 3910–3914. find
  17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186. ↗
  18. Arash Eshghi, Christine Howes, Eleni Gregoromichelaki, Julian Hough, and Matthew Purver. 2015. Feedback in conversation as incremental semantic update. In Proceedings of the 11th International Conference on Computational Semantics, pages 261–271, London, UK. Association for Computational Linguistics. find
  19. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. find
  20. Goran Glavas and Swapna Somasundaran. 2020. Two-level transformer and auxiliary coherence modeling for improved text segmentation. ArXiv, abs/2001.00891 [v1]. ↗
  21. Ramón Granell, Stephen Pulman, Carlos Martı́nez-Hinarejos, and José Miguel Benedı́. 2010. Dialogue act tagging and segmentation with a single perceptron. In Eleventh Annual Conference of the International Speech Communication Association. find
  22. Dan Jurafsky, Elizabeth Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL Labeling Project Coder’s Manual. Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 383–392. find
  23. Daniel Jurafsky, Rebecca Bates, Noah Coccaro, Rachel Martin, Marie Meteer, Klaus Ries, Elizabeth Shriberg, Andreas Stolcke, Paul Taylor, and Carol Van Ess-Dykema DoD. 1998. Johns Hopkins LVCSR Workshop-97 Switchboard Discourse Language Modeling Project Final Report. find
  24. Yang Liu, Kun Han, Zhao Tan, and Yun Lei. 2017a. Using context information for dialog act classification in DNN framework. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2170–2178, Copenhagen, Denmark. Association for Computational Linguistics. https://doi.org/10.18653/v1/D17 -1231 ↗ our card
  25. Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. 2020. Transformers are RNNs: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning, pages 5156–5165. PMLR. ↗
  26. Ruth Kempson, Ronnie Cann, Eleni Gregoromichelaki, and Stergios Chatzikyriakidis. 2016. Language as mechanisms for interaction. Theoretical Linguistics, 42(3-4):203–276. https://doi.org/10.1515/tl-2016 -0011 ↗
  27. Ruth Kempson, Wilfried Meyer-Viol, and Dov M. Gabbay. 2000. Dynamic Syntax: The Flow of Language Understanding. Wiley-Blackwell. find
  28. Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The efficient transformer. Proceedings of International Conference on Learning Representations (ICLR). ↗ our card
  29. Taku Kudo and John Richardson. 2018. Sentence-Piece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71. https://doi.org/10.18653/v1/D18-2012 ↗ our card
  30. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, and Sachindra Joshi. 2018. Dialogue act sequence labeling using hierarchical encoder with crf. In Thirty-Second AAAI Conference on Artificial Intelligence. ↗ our card
  31. Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen. 2019. A dual-attention hierarchical recurrent neural network for dialogue act classification. In Yang Liu, Kun Han, Zhao Tan, and Yun Lei. 2017b. Using context information for dialog act classification in dnn framework. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2170–2178. https://doi.org/10 .18653/v1/D17-1231 ↗ our card
  32. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 [v1]. ↗
  33. Daniel Ortega and Ngoc Thang Vu. 2017. Neural-based context representation learning for dialog act classification. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 247–252. https://doi.org/10.18653/v1/W17-5530 ↗ our card
  34. Daniel Ortega and Ngoc Thang Vu. 2018. Lexico-acoustic neural-based models for dialog act classification. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6194–6198. IEEE. https://doi.org/10.1109/ICASSP.2018 .8461371 ↗
  35. Silvia Pareti and Tatiana Lando. 2018. Dialog intent structure: A hierarchical schema of linked dialog acts. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  36. Matthew Purver, Julian Hough, and Christine Howes. 2018. Computational models of miscommunication phenomena. Topics in Cognitive Science, 10(2):425–451. https://doi.org/10.1111/tops.12324 ↗
  37. Matthew Purver, Christine Howes, Eleni Gregoromichelaki, and Patrick Healey. 2009. Split utterances in dialogue: A corpus study. In Proceedings of the SIGDIAL 2009 Conference, pages 262–271, London, UK. Association for Computational Linguistics. https://doi.org/10.3115/1708376.1708413 ↗
  38. Silvia Quarteroni, Alexei V. Ivanov, and Giuseppe Riccardi. 2011. Simultaneous dialog act segmentation and classification from human-human spoken conversations. In 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5596–5599. IEEE. https://doi.org/10.1109/ICASSP.2011.5947628 ↗
  39. Vipul Raheja and Joel Tetreault. 2019. Dialogue act classification with context-aware self-attention. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3727–3733. ↗ our card
  40. Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1998. Dialogue act tagging with transformation-based learning. In COLING 1998 Volume 2: The 17th International Conference on Computational Linguistics. https://doi.org/10.3115/980432.980757 ↗ our card
  41. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725. https://doi.org/10.18653/v1/P16-1162 ↗
  42. Igor Shalyminov, Arash Eshghi, and Oliver Lemon. 2018. Multi-task learning for domain-general spoken disfluency detection in dialogue systems. The 22nd Workshop on the Semantics and Pragmatics of Dialogue SEMDIAL. ↗
  43. Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. 2021. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3531–3539. https://doi.org/10 .1109/WACV48630.2021.00357 ↗
  44. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The ICSI meeting recorder dialog act (mrda) corpus, International Computer Science Inst, Berkeley, CA. https://doi.org/10.21236/ADA460980 ↗ our card
  45. Y. Si, L. Wang, J. Dang, M. Wu, and A. Li. 2020. A hierarchical model for dialog act recognition considering acoustic and lexical context information. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7994–7998. https://doi.org/10.1109/ICASSP40776 .2020.9053061 ↗
  46. Ingo Siegert and Julia Krüger. 2018. How do we speak with Alexa: Subjective and objective assessments of changes in speaking style between HC and HH conversations. Kognitive Systeme, 2018(1). find
  47. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. https://doi.org/10.1162/089120100561737 ↗ our card
  48. Yi Tay, Dara Bahri, Liu Yang, Donald Metzler, and Da-Cheng Juan. 2020. Sparse sinkhorn attention. In International Conference on Machine Learning, pages 9438–9447. PMLR. ↗
  49. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. find
  50. Daan Verbree, Rutger Rienks, and Dirk Heylen. 2006. Dialogue-act tagging using smart feature selection; results on multiple corpora. In 2006 IEEE Spoken Language Technology Workshop, pages 70–73. IEEE. https://doi.org/10 .1109/SLT.2006.326819 ↗
  51. Sinong Wang, Belinda Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768 [v3]. ↗
  52. Zhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin, and Song Han. 2020. Lite transformer with long-short range attention. Proceedings of International Conference on Learning Representations (ICLR). ↗
  53. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, pages 5754–5764. ↗ our card
  54. Zihao Ye, Qipeng Guo, Quan Gan, Xipeng Qiu, and Zheng Zhang. 2019. BP-Transformer: Modelling long-range context via binary partitioning. arXiv preprint arXiv: 1911.04070 [v1]. ↗
  55. Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big bird: Transformers for longer sequences. Advances in Neural Information Processing Systems, 33. ↗ our card
  56. Tianyu Zhao and Tatsuya Kawahara. 2019. Joint dialog act segmentation and recognition in human conversations using attention to dialog context. Computer Speech & Language, 57:108–127. https://doi.org/10 .1016/j.csl.2019.03.001 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/elasko-2021-what-helps-transformers-recognize-conversational/