Elamur Research⌕
☰
home / papers / 2021 / Dialogue Act Classification for Augmentative and Alternative Communication

Dialogue Act Classification for Augmentative and Alternative Communication

Paper · 2021 · E. Margaret Perkoff

Summary

Augmentative and Alternative Communication (AAC) devices help people with complex communication needs take part in conversations, but users often abandon them. Most NLP work for AAC has focused on word prediction, and the paper reviews evidence that this has not improved communication rates. Perkoff proposes a different direction: Dialogue Act classification, which labels the intent of each conversational turn. The predicted acts could guide reply suggestions on the device and could also support speech pathologists during evaluation and intervention.

As a pilot, the author took an existing state-of-the-art classifier and tested it on a small set of transcribed conversations between a person with ALS and their usual conversation partners. Three training setups were compared:

  • AAC data for both training and validation
  • Switchboard for both training and validation
  • Switchboard for training with AAC data for validation

The AAC-trained model scored best (38.6% accuracy), ahead of the Switchboard-trained model (34.1%). Accuracy is low in every setup. The author argues that the field needs larger labeled AAC corpora, tag sets suited to AAC use, and models that can be personalized.

Key points

  • The tag inventory is the Switchboard DAMSL set (Jurafsky et al., 1997), which has 42 classes.
  • The AAC transcripts were split at random by sentence: 80% training, 10% validation, 10% test. All three model variants were evaluated on the same AAC test set, using categorical crossentropy and categorical accuracy.
  • Using AAC data only for validation (hyperparameter setting) had little effect. The author suggests two possible reasons: the validation set was very small, or Switchboard simply does not transfer well to AAC speech.
  • The review identifies just one conversational corpus with AAC users in it: Friginal et al.'s (2013) Workplace Corpus, which contains speech from both AAC users and non-users. It covers more than 200 hours from eight participants and carries part-of-speech coding. Other clinical resources are audio only, such as the CSLU Autism Speech Corpus (45 children, 1.5 hours, 1,022 utterances).
  • Earlier context-aware prediction systems gave no significant gains in conversational rate. These include SmartPredict, which draws on suggestions from the conversation partner, and location-aware language models.

Interesting ideas

  • I1. The experiment fixes one in-domain test set and varies only where the training and validation data come from, which separates the effect of training data from the effect of model selection.
  • I2. The dialogue act of the partner's turn can determine the shape of a good reply. For example, a Wh-question about plans calls for a statement-style partial phrase that the user then completes.
  • I3. The paper argues that the right tag inventory depends on the purpose: clinical coding and live device suggestions need different classes, even for the same data.
  • I4. The author recommends having domain-expert annotators (speech pathologists) label the data and measure inter-annotator agreement, rather than relying on generic labels.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): We could evaluate parsers on fable and tale sentences with a fixed test set, then compare training on the standard treebank, on in-domain silver data, and on a mix, to see where domain shift actually costs accuracy.
  • I2 (back): The generator could use the classified speech act of the incoming question to choose an answer frame template before filling in content from the world model.
  • I3 (frames): The speech-act layer in prag could expose a coarse inventory for answering and a finer one for analysis, with an explicit mapping between them so the results stay explainable.
  • I4 (learn): Agreement among witness parsers or LLM teacher tiers could be reported per domain, flagging categories where even experts would be unlikely to agree, so we avoid trusting silver labels there.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Colorado BoulderBoulder · United States52 works · 49,647 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Health Professions · 14 works Occupational Therapy · 4 works

🧭 Topics 2 Dialogue systems · 401 works Speech act theory · 160 works

Cite

Cite as E. Margaret Perkoff. 2021. Dialogue Act Classification for Augmentative and Alternative Communication. Proceedings of the 1st Workshop on NLP for Positive Impact.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.nlp4posimpact-1.12

@inproceedings{perkoff2021dialogue,
  title = {{Dialogue Act Classification for Augmentative and Alternative Communication}},
  author = {E. Margaret Perkoff},
  booktitle = {Proceedings of the 1st Workshop on NLP for Positive Impact},
  year = {2021},
  doi = {10.18653/v1/2021.nlp4posimpact-1.12},
  url = {https://doi.org/10.18653/v1/2021.nlp4posimpact-1.12}
}

References

26 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ali Ahmadvand, Jason Ingyu Choi, and Eugene Agichtein. 2019. Contextual dialogue act classification for Open-Domain conversational agents. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’19, pages 1273–1276, New York, NY, USA. Association for Computing Machinery. our card
  2. David R Beukelman and Janice C Light. 2020. Augmentative et alternative communication : supporting children and adults with complex communication needs. Paul H. Brookes Publishing Co. find
  3. Matthew W Brault. 2012. Americans with disabilities: 2010. Technical Report P70-131, U.S. Census Bureau, Washington, D.C. find
  4. Yasmin Elsahar, Sijung Hu, Kaddour Bouazza-Marouf, David Kerr, and Annysa Mansor. 2019. Augmentative and alternative communication (AAC) advances: A review of configurations for individuals with a speech disability. Sensors, 19(8). ↗
  5. Fried-Oken, M., Jakobs, T., & Jakobs, E. 2018. Smart-Predict: AAC app that integrates partner knowledge into word prediction. Annual Conference of the Assistive Technology Industry Association (ATIA). find
  6. Eric Friginal, Pamela Pearson, Laura Di Ferrante, Lucy Pickering, and Carrie Bruce. 2013. Linguistic characteristics of AAC discourse in the workplace. Discourse Studies, 15(3):279–298. ↗
  7. Robert Gale, Liu Chen, Jill Dolata, Jan van Santen, and Meysam Asgari. 2019. Improving ASR systems for children with autism and language impairment using Domain-Focused DNN transfer techniques. Interspeech, 2019:11–15. find
  8. Luı́s Filipe Garcia, Luı́s Caldas de Oliveira, and David Martins de Matos. 2016. Evaluating pictogram prediction in a location-aware augmentative and alternative communication system. Assist. Technol., 28(2):83–92. ↗
  9. Luı́s Filipe Garcia, Luı́s Caldas De Oliveira, and David Martins De Matos. 2015. Measuring the performance of a Location-Aware text prediction system. ACM Trans. Access. Comput., 7(1):1–29. ↗
  10. D Jeffery Higginbotham, Gregory W Lesher, Bryan J Moulton, and Brian Roark. 2011. The application of natural language processing to augmentative and alternative communication. Assist. Technol., 24(1):14–24. ↗
  11. J Higginbotham. 2021. Communication and assistive device laboratory - AAC shared narrative corpora:ALS. find
  12. Hagen Jaeger, Dhaval Trivedi, and Michael Stadtschnitzer. 2019. Mobile device voice recordings at king’s college london (MDVR-KCL) from both early and advanced parkinson’s disease patients and healthy controls. find
  13. Dan Jurafsky, Liz Shriberg, and Debra Biasca. 1997. WS-97 Switchboard DAMSL Coders Manual. find
  14. Janice Light, David McNaughton, David Beukelman, Susan Koch Fager, Melanie Fried-Oken, Thomas Jakobs, and Erik Jakobs. 2019. Challenges and opportunities in augmentative and alternative communication: Research and technology development to enhance communication and participation for individuals with complex communication needs. Augment. Altern. Commun., 35(1):1–12. ↗
  15. Max A Little, Patrick E McSharry, Stephen J Roberts, Declan A E Costello, and Irene M Moroz. 2007. Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomed. Eng. Online, 6:23. ↗
  16. Barry S Oken, Umut Orhan, Brian Roark, Deniz Er-dogmus, Andrew Fowler, Aimee Mooney, Betts Peters, Meghan Miller, and Melanie B Fried-Oken. 2014. Brain-computer interface with language model-electroencephalography fusion for locked-in syndrome. Neurorehabil. Neural Repair, 28(4):387– 394. ↗
  17. Juan Rafael Orozco, Julian D Arias-Londoño, J Francisco Vargas-Bonilla, and Elmar Noeth. 2014. New spanish speech corpus database for the analysis of people suffering from parkinson’s disease. In International Conference on Language Resources and Evaluation. unknown. find
  18. Sujith Ravi and Zornitsa Kozareva. 2018. Self-Governing neural networks for On-Device short text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 887–893, Brussels, Belgium. Association for Computational Linguistics. find
  19. Fabien Ringeval, Julie Demouy, György Szaszak, Mohamed Chetouani, Laurence Robel, Jean Xavier, David Cohen, and Monique Plaza. 2011. Automatic intonation recognition for the prosodic assessment of Language-Impaired children. IEEE Trans. Audio Speech Lang. Processing, 19(5):1328–1342. ↗
  20. Sebastian Ruder. 2021. NLP-progress. find
  21. Frank Rudzicz, Aravind Kumar Namasivayam, and Talya Wolff. 2010. The TORGO database of acoustic and articulatory speech from speakers with dysarthria. Language Resources and Evaluation, 46(4):1–19. ↗
  22. Andres Suarez. 2021. SGNN. find
  23. Athanasios Tsanas, Max A Little, Cynthia Fox, and Lorraine O Ramig. 2014. Objective automatic assessment of rehabilitative speech treatment in parkinson’s disease. IEEE Trans. Neural Syst. Re-habil. Eng., 22(1):181–190. ↗
  24. Stephanie Valencia, Amy Pavel, Jared Santa Maria, Seunga (gloria) Yu, Jeffrey P Bigham, and Henny Admoni. 2020. Conversational agency in augmentative and alternative communication. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI ’20, pages 1–12, New York, NY, USA. Association for Computing Machinery. find
  25. Annalu Waller. 2019. Telling tales: unlocking the potential of AAC technologies. Int. J. Lang. Commun. Disord., 54(2):159–169. ↗
  26. Bruce Wisenburn and D Jeffery Higginbotham. 2009. Participant evaluations of rate and communication efficacy of an AAC application using natural language processing. Augment. Altern. Commun., 25(2):78–89. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/perkoff-2021-dialogue-act-classification-augmentative-alternative/