Elamur Research⌕
☰
home / papers / 2014 / Investigating the Contribution of Distributional Semantic Information for Dialogue Act Classification

Investigating the Contribution of Distributional Semantic Information for Dialogue Act Classification

Paper · 2014 · Dmitrijs Milajevs and Matthew Purver

Summary

The paper asks whether word meaning learned from co-occurrence statistics helps label utterances with dialogue acts such as statement, question, opinion or backchannel, and which kinds of information actually drive accuracy. The state of the art, a Recurrent Convolutional Neural Network, trains everything on the labelled corpus, which hides where its gains come from. The authors instead take word vectors from an external unlabelled resource, the Google Books Ngrams corpus. They build utterance vectors with simple composition (vector addition and pointwise multiplication) and use the labelled Switchboard data only for tagging.

They compare four setups:

  • a bag-of-unigrams baseline modelled on Serafin et al. (2003);
  • a bag of bigrams;
  • composed distributional vectors;
  • variants that also concatenate the previous utterance's vector.

Bigrams and distributional vectors both beat plain bag of words. Adding the previous utterance did not help. Overall accuracy stays below Webb et al. (2005) and Kalchbrenner and Blunsom (2013). The work matters as a controlled ablation that separates the effects of word order, lexical semantics and discourse context. It argues that richer, order- or grammar-aware composition is needed.

Key points

  • Data: the Switchboard corpus (about 2500 telephone conversations, 500 US speakers, 42 dialogue act tags, 1115 training conversations). The Spanish CallHome37 set used by Serafin et al. was not available to the authors.
  • Word vectors come from Google Books Ngrams 5-grams. The middle word is the target and its neighbours are contexts (window size 2). Context dimensions are the 3000 most frequent words, minus the top 100.
  • Three vector spaces were tried: raw counts, tf-idf weighting, and an NMF-reduced space. Addition beat multiplication, contrary to some earlier tasks.
  • Concatenating the previous utterance's vector doubled the dimensionality and gave no accuracy gain. Only reduced spaces were tested because of memory limits.
  • Reference points: Serafin et al. reached 65.36% on 37 tags with LSA (SVD to 50 components) and nearest-neighbour cosine. Webb et al. reached 0.719 using only intra-utterance n-grams. Kalchbrenner and Blunsom reached 0.739.

Interesting ideas

  • I1. Measure each information source separately (word identity, word order, external lexical meaning, previous-turn context) instead of reporting one end-to-end number, so each gain can be attributed to its cause.
  • I2. A tiny minimal pair (a name followed by "are there cookies" versus "there are cookies") shows that word order alone can flip a question into a statement, which unordered representations cannot see.
  • I3. Lexical meaning is learned from a large, general, unlabelled resource, while the labelled data is reserved for the task decision only, keeping the two contributions separable.
  • I4. Opinion acts tend to carry hedging frames such as "I think" or "I guess", so a small set of stance verbs is a strong cue for the act type.
  • I5. The negative result on context suggests that dialogue history should be modelled as an explicit relation (an answer following a question) rather than glued onto the representation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation runs could switch tree features, lexical knowledge and discourse context on one at a time, producing an explainable account of what each stage contributes.
  • I2 (frames): Speech act detection in prag should read the UD tree (auxiliary inversion, subject position), and a regression suite of such minimal pairs could guard it.
  • I3 (world): Compiled knowledge (verb classes, WordNet/FrameNet ontology) can stay a separate, inspectable layer, so pragmatic decisions trace back to explicit lexical facts rather than corpus-specific patterns.
  • I4 (frames): A cognition or hedge verb with a first-person subject over a clausal complement could mark the frame as an opinion rather than an asserted fact, which also helps implicature handling.
  • I5 (frames): Store adjacency links between turns, such as question-answer or statement-acknowledgement, as explicit discourse relations that later stages can inspect.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of LondonLondon · United Kingdom7 works · 790 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 2 Dialogue systems · 401 works Speech act theory · 160 works

Cite

Cite as Dmitrijs Milajevs and Matthew Purver. 2014. Investigating the Contribution of Distributional Semantic Information for Dialogue Act Classification. Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-1505

@inproceedings{milajevs2014investigating,
  title = {{Investigating the Contribution of Distributional Semantic Information for Dialogue Act Classification}},
  author = {Dmitrijs Milajevs and Matthew Purver},
  booktitle = {Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC)},
  year = {2014},
  doi = {10.3115/v1/w14-1505},
  url = {https://doi.org/10.3115/v1/w14-1505}
}

References

26 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Paşca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 19–27. Association for Computational Linguistics. our card
  2. Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1183–1193. Association for Computational Linguistics. find
  3. Christos Boutsidis and Efstratios Gallopoulos. 2008. Svd based initialization: A head start for nonnegative matrix factorization. Pattern Recognition, 41(4):1350–1362. ↗
  4. Bob Coecke, Mehrnoosh Sadrzadeh, and Stephen Clark. 2010. Mathematical foundations for a compositional distributional model of meaning. CoRR, abs/1003.4394. ↗
  5. Mark Core. 1998. Analyzing and predicting patterns of damsl utterance tags. In Proceedings of the AAAI spring symposium on Applying machine learning to discourse processing. find
  6. G. Dinu and M. Lapata. 2010. Measuring distributional similarity in context. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1162–1172. Association for Computational Linguistics. find
  7. John R. Firth. 1957. A Synopsis of Linguistic Theory, 1930-1955. Studies in Linguistic Analysis, pages 1– 32. find
  8. John J Godfrey, Edward C Holliman, and Jane Mc-Daniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517–520. IEEE. ↗
  9. Edward Grefenstette and Mehrnoosh Sadrzadeh. 2011. Experimental support for a categorical compositional distributional model of meaning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1394–1404. Association for Computational Linguistics. ↗
  10. Patrik O Hoyer. 2004. Non-negative matrix factorization with sparseness constraints. The Journal of Machine Learning Research, 5:1457–1469. ↗
  11. Daniel Jurafsky, Elizabeth Shriberg, and Debra Biasca. 1997. Switchboard swbd-damsl shallow-discourse-function annotation coders manual, draft 13. Technical Report 97-02, University of Colorado, Boulder. Institute of Cognitive Science. find
  12. Daniel Jurafsky, Elizabeth Shriberg, Barbara Fox, and Traci Curl. 1998. Lexical, prosodic, and syntactic cues for dialog acts. In Proceedings of the ACL-COLING Workshop on Discourse Relations and Discourse Markers. our card
  13. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 119–126, Sofia, Bulgaria, August. Association for Computational Linguistics. our card
  14. Yuri Lin, Jean-Baptiste Michel, Erez Lieberman Aiden, Jon Orwant, Will Brockman, and Slav Petrov. 2012. Syntactic annotations for the Google Books ngram corpus. In Proceedings of the ACL 2012 System Demonstrations, pages 169–174. Association for Computational Linguistics. find
  15. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗ our card
  16. Jeff Mitchell and Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL-08: HLT, pages 236–244. Association for Computational Linguistics. find
  17. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  18. Riccardo Serafin, Barbara Di Eugenio, and Michael Glass. 2003. Latent semantic analysis for dialogue act classification. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003–short papers-Volume 2, pages 94–96. Association for Computational Linguistics. find
  19. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. Association for Computational Linguistics. our card
  20. Richard Socher, John Bauer, Christopher D Manning, and Andrew Y Ng. 2013. Parsing with compositional vector grammars. In In Proceedings of the ACL conference. Citeseer. find
  21. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Carol Van Ess-Dykema, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. ↗ our card
  22. Dinoj Surendran and Gina-Anne Levow. 2006. Dialog act tagging with support vector machines and hidden markov models. In INTERSPEECH. find
  23. Stefan Thater, Hagen Fürstenau, and Manfred Pinkal. 2010. Contextualizing semantic representations using syntactically enriched vector models. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, pages 948–957, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  24. Peter D Turney, Patrick Pantel, et al. 2010. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research, 37(1):141–188. ↗ our card
  25. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding. Citeseer. find
  26. M. Zhitomirsky-Geffet and I. Dagan. 2009. Bootstrapping distributional feature vector quality. Computational Linguistics, 35(3):435–461. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/milajevs-2014-investigating-contribution-distributional-semantic-information/