Elamur Research⌕
☰
home / papers / 2021 / Rare-Class Dialogue Act Tagging for Alzheimer's Disease Diagnosis

Rare-Class Dialogue Act Tagging for Alzheimer's Disease Diagnosis

Paper · 2021 · Shamila Nasreen, Julian Hough and Matthew Purver

Summary

Dialogue act (DA) taggers are usually tuned for average accuracy, so they handle frequent tags well and largely miss rare ones. For Alzheimer's Disease (AD), the most telling interactional cues are exactly these rare tags: signals of non-understanding, clarification requests, and particular question and answer types. The authors build a tagger aimed at these classes. They reduce the DAMSL scheme to 20 tags: 17 clinically motivated tags are kept, two new subtypes are added (clarification-request qc and statement-answer sa), and everything else becomes "other". The model is a hierarchical BiLSTM with attention, trained with a class-weighted loss. It sees only left context (earlier utterances and earlier DA tags), so it can run incrementally. The tagger is evaluated on SwDA and on the AD-focused Carolinas Conversation Collection (CCC). Its outputs, used as unigram and bigram DA features, then feed an SVM that separates AD from non-AD speakers. This shows that interpretable DA profiles carry diagnostic signal and may transfer across languages better than lexical or syntactic cues.

Key points

  • Prior state of the art (Raheja and Tetreault, 82.9% overall on SwDA) reaches only about 25% on br, which makes up just 0.1% of SwDA utterances. Its results on qy, qw and b are also weak.
  • Results with context:
    • SwDA: 88% accuracy and macro F1 0.58.
    • CCC: 66% accuracy and F1 0.45.
  • Adding the previous utterance and its DA lifts SwDA F1 from 0.46 to 0.58 and recall from 0.55 to 0.64. Gold context tags beat predicted ones, but a longer context window narrows the gap.
  • Embeddings compared: random, pretrained GloVe, GloVe trained on SwDA+CCC, and ELMo. The upper LSTM is unidirectional so that processing stays online.
  • AD vs non-AD classification with a linear SVM on CCC (random baseline 0.50):
FeaturesAccuracy
DA unigrams0.65
+ interviewer→patient bigrams + confusion ratios0.70
+ turn, speech-rate and overlap features0.80

Interesting ideas

  • I1. Start from a standard tagset, keep only the task-relevant tags, split a few into finer subtypes, and merge the rest into "other". This keeps the label space small and general while still serving a specific purpose.
  • I2. Judge the tagger by macro F1 and per-class recall on rare labels, and train with a class-weighted loss, because high overall accuracy can hide near-zero performance on the classes that matter.
  • I3. Feed the previous predicted labels back in as context, and use a longer window to absorb the noise they bring when gold labels are unavailable. Prediction stays strictly left-to-right.
  • I4. Turn labels into interpretable downstream features: cross-speaker act bigrams (a question followed by a clarification request) and ratios such as confusion acts over all questions. This beats feeding opaque embeddings to a classifier.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): The speech-act inventory in prag could follow this pattern, adding subtypes like clarification-request under non-understanding instead of inventing a new scheme.
  • I2 (learn): Evaluation reports for the tagger and parser could list scores for rare labels and relations separately, so averages do not mask failures on uncommon constructions.
  • I3 (frames): Speech-act and frame assignment for a sentence could condition on the acts already assigned to earlier sentences, which keeps the reasoning incremental and traceable.
  • I4 (world): Dialogue-level states such as "the listener did not understand" could be derived explicitly from act sequences and stored as explainable facts about the participants.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Queen Mary University of LondonLondon · United Kingdom39 works · 3,307 citations🔬Jožef Stefan InstituteLjubljana · Slovenia10 works · 1,959 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksMedicine · 80 worksPsychiatry and Mental health · 3 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Shamila Nasreen, Julian Hough and Matthew Purver. 2021. Rare-Class Dialogue Act Tagging for Alzheimer's Disease Diagnosis. Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.sigdial-1.32

@inproceedings{nasreen2021rare,
  title = {{Rare-Class Dialogue Act Tagging for Alzheimer's Disease Diagnosis}},
  author = {Shamila Nasreen, Julian Hough and Matthew Purver},
  booktitle = {Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2021},
  doi = {10.18653/v1/2021.sigdial-1.32},
  url = {https://doi.org/10.18653/v1/2021.sigdial-1.32}
}

References

42 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. Proceedings. (ICASSP ’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., 1:I/1061– I/1064 Vol. 1. find
  2. Dina Demner-Fushman, Wendy W Chapman, and Clement J McDonald. 2009. What can natural language processing do for clinical decision support? Journal of biomedical informatics, 42(5):760–772. ↗
  3. Mark Dingemanse, Seán G Roberts, Julija Baranova, Joe Blythe, Paul Drew, Simeon Floyd, Rosa S Gis-ladottir, Kobin H Kendrick, Stephen C Levinson, Elizabeth Manrique, et al. 2015. Universal principles in the repair of communication problems. PloS one, 10(9):e0136100. ↗
  4. Christopher Elsey, Paul Drew, Danielle Jones, Daniel Blackburn, Sarah Wakefield, Kirsty Harkness, An-nalena Venneri, and Markus Reuber. 2015. Towards diagnostic conversational profiles of patients presenting with dementia or functional memory disorders to memory clinics. Patient Education and Counseling, 98(9):1071–1077. ↗
  5. Shahla Farzana, Mina Valizadeh, and Natalie Parde. 2020. Modeling dialogue in conversational cognitive health screening interviews. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 1167–1177. find
  6. Kathleen C. Fraser, Jed A. Meltzer, and Frank Rudzicz. 2016. Linguistic features identify Alzheimer’s disease in narrative speech. Journal of Alzheimer’s Disease, 49(2):407–422. ↗
  7. Simone Fuscone, Benoit Favre, and Laurent Prévot. 2020. The contribution of dialogue act labels for convergence studies in naturalconversations. find
  8. Harold Goodglass, Edith Kaplan, Sandra Weintraub, and Barbara Barresi. 2001. The boston diagnostic aphasia examination. find
  9. Dalia Gottlieb-Tanaka, Jeff Small, and Annalee Yassi. 2003. A programme of creative expression activities for seniors with dementia. Dementia, 2(1):127–133. find
  10. Heidi Ehernberger Hamilton. 2005. Conversations with an Alzheimer’s patient: An interactional sociolinguistic study. Cambridge University Press. find
  11. Daniel Jurafsky, Elizabeth Shriberg, and Debra Bi-asca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual. URL http://web.stanford. edu˜ jurafsky/ws97/manual. august1. html. ↗
  12. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584. ↗
  13. Chandrakant Bothe, Cornelius Weber, Sven Magg, and Stefan Wermter. 2018. A context-based approach for dialogue act recognition using simple recurrent neural networks. arXiv preprint arXiv:1805.06280. ↗
  14. Brianna Marlene Broderick, Si Long Tou, and Emily Mower Provost. 2018. TD-P-014: Cogid: A speech recognition tool for early detection of Alzheimer’s disease. Alzheimer’s and Dementia, 14(7S). ↗
  15. Harry Bunt. 2011. Multifunctionality in dialogue. Computer Speech & Language, 25(2):222–245. ↗
  16. Ali Khodabakhsh, Fatih Yesil, Ekrem Guner, and Cenk Demiroglu. 2015. Evaluation of linguistic and prosodic features for detection of alzheimer’s disease in turkish conversational speech. EURASIP Journal on Audio, Speech, and Music Processing, 2015(1):1–15. ↗
  17. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  18. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827. ↗
  19. Matthew Purver, Jonathan Ginzburg, and Patrick Healey. 2001. On the means for clarification in dialogue. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. find
  20. Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen. 2018. A dual-attention hierarchical recurrent neural network for dialogue act classification. arXiv preprint arXiv:1810.09154. ↗
  21. Vipul Raheja and Joel Tetreault. 2019. Dialogue act classification with context-aware self-attention. arXiv preprint arXiv:1904.02594. ↗
  22. Saturnino Luz, Sofia de la Fuente, and Pierre Albert. 2018. A method for analysis of patient speech in dialogue for dementia detection. In Proceedings of the LREC 2018 Workshop Resources and Processing of linguistic, para-linguistic and extra-linguistic Data from people with various forms of cognitive/psychiatric impairments (RaPID-2). ↗
  23. Bahman Mirheidari, Daniel Blackburn, Traci Walker, Markus Reuber, and Heidi Christensen. 2019. Dementia detection using automatic analysis of conversations. Computer Speech & Language, 53:65–79. ↗
  24. Shamila Nasreen, Matthew Purver, and Julian Hough. 2019. A corpus study on questions, responses and misunderstanding signals in conversations with Alzheimer’s patients. In Proceedings of the 23rd Workshop on the Semantics and Pragmatics of Dialogue-Full Papers. find
  25. SEMDIAL, London, United Kingdom (Sep 2019), http://semdial.org/anthology/Z19-Nasreen semdial, volume 13. ↗
  26. Nithin Ramacandran. 2013. Dialogue act detection from human-human spoken conversations. International Journal of Computer Applications, 67(5). ↗
  27. Brian Roark, Margaret Mitchell, John-Paul Hosom, Kristy Hollingshead, and Jeffrey Kaye. 2011. Spoken language derived measures for detecting mild cognitive impairment. IEEE transactions on audio, speech, and language processing, 19(7):2081–2090. find
  28. Kepa Rodrı́guez and David Schlangen. 2004. Form, intonation and function of clarification requests in German task-oriented spoken dialogues. In Proceedings of the 8th Workshop on the Semantics and Pragmatics of Dialogue (Catalog), pages 101–108, Barcelona. find
  29. Jeff A Small and JoAnn Perry. 2005. Do you remember? how caregivers question their spouses who have Alzheimer’s disease and the impact on communication. Journal of Speech, Language, and Hearing Research, 48(1):125–136. ↗
  30. John B Orange, Rosemary B Lubinski, and D Jeffery Higginbotham. 1996. Conversational repair by individuals with dementia of the Alzheimer’s type. Journal of Speech, Language, and Hearing Research, 39(4):881–895. ↗
  31. Vivek Kumar Rangarajan Sridhar, Srinivas Bangalore, and Shrikanth Narayanan. 2009. Combining lexical, syntactic and prosodic cues for improved online dialog act tagging. Computer Speech & Language, 23(4):407–422. ↗
  32. Sylvester O Orimaye, Jojo SM Wong, Karen J Golden, Chee P Wong, and Ireneous N Soyiri. 2017. Predicting probable Alzheimer’s disease using linguistic deficits and biomarkers. BMC bioinformatics, 18(1):34. ↗
  33. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. ↗
  34. Daniel Ortega and Ngoc Thang Vu. 2017. Neural-based context representation learning for dialog act classification. arXiv preprint arXiv:1708.02561. ↗
  35. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. ↗
  36. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. ↗
  37. Charlene Pope and Boyd H Davis. 2011. Finding a balance: The Carolinas conversation collection. Corpus Linguistics and Linguistic Theory, 7(1):143– 161. ↗
  38. Ana Varela Suárez. 2018. The question-answer adjacency pair in dementia discourse. International Journal of Applied Linguistics, 28(1):86–101. ↗
  39. Sumithra Velupillai, Hanna Suominen, Maria Liakata, Angus Roberts, Anoop D Shah, Katherine Mor-ley, David Osborn, Joseph Hayes, Robert Stewart, Johnny Downs, et al. 2018. Using clinical natural language processing for health outcomes research: Overview and actionable suggestions for future advances. Journal of biomedical informatics, 88:11– 19. ↗
  40. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding, volume 4, page 5. Citeseer. find
  41. MAH Zahid, Ankush Mittal, Ramesh Chandra Joshi, and G Atluri. 2018. Cliniqa: A machine intelligence based clinical question answering system. arXiv preprint arXiv:1805.05927. ↗
  42. Matthias Zimmermann. 2009. Joint segmentation and classification of dialog acts using conditional random fields. In Tenth Annual Conference of the International Speech Communication Association. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nasreen-2021-rare-class-dialogue-act-tagging/