Elamur Research⌕
☰
home / papers / 1998 / Dialogue Act Tagging with Transformation-Based Learning

Dialogue Act Tagging with Transformation-Based Learning

Summary

The paper addresses automatic dialogue act recognition: labeling each utterance with an abstract intention such as SUGGEST, ACCEPT or REJECT. The task is hard because the intended act often differs from what the utterance literally says. The authors are the first to apply Brill's Transformation-Based Learning (TBL) to a discourse-level problem. TBL learns an ordered list of human-readable relabeling rules from tagged dialogues in the appointment-scheduling VERBMOBIL corpora.

To limit sparse data, the system relies on abstract utterance features: change of speaker, punctuation, utterance length, and the tags of surrounding utterances. It also introduces a new feature, dialogue act cues. These are one- to three-word substrings chosen automatically because the dialogue acts they co-occur with have low entropy. The cue set is then refined by filtering and by semantic clustering of words.

Two additions address TBL's weaknesses. A Monte Carlo rule-sampling scheme speeds up training, and a committee of models yields confidence scores.

The resulting tagger matches the best reported accuracy (Reithinger and Klesen). This shows that automatically selected cues beat both hand-made cue phrase lists and exhaustive word n-grams.

Key points

  • With Monte Carlo sample size R = 14, the full system averaged 75.12% accuracy over five runs (σ = 1.34%), with a best run of 77.44%. Performance was also comparable to Decision Trees.
  • On held-out data, accuracy varied with the set of word substrings used:
    • no word substrings: 41.16%
    • literature cue phrases (936 items): 61.74%
    • all word n-grams (16,271 items): 69.21%
    • entropy-minimized cues with filtering and clustering (only 811 items): 71.22%
  • Cues are kept if the entropy H(D|s) falls below a threshold θ1 and their frequency exceeds a threshold θ2. They fall into three useful types:
    • traditional cues (56, e.g. "but", "so")
    • potential cues (71, e.g. "thanks", "see you")
    • domain cues (42, e.g. "what time", "busy")
  • Filtering trims redundant superstring cues such as "but the" from 690 to 472.
  • Planned future work includes WordNet-based clustering and weakly-supervised TBL driven by high-confidence committee tags.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

32 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 25 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/980691.980757

@inproceedings{samuel1998dialogue,
  title = {{Dialogue Act Tagging with Transformation-Based Learning}},
  author = {Samuel, Ken and Carberry, Sandra and Vijay-Shanker, K.},
  booktitle = {36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2},
  year = {1998},
  doi = {10.3115/980691.980757},
  url = {https://doi.org/10.3115/980691.980757}
}

References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Toine Andernach. 1996. A machine learning approach to the classification of dialogue utterances. In Proceedings of NeMLaP-2. ↗
  2. Eric Brill. 1995a. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4):543-566. find our card
  3. Eric Drill. 1995b. Unsupervised learning of disambiguation rules for part of speech tagging. In Proceedings of the Very Large Corpora Workshop. find
  4. Kuang-Hua Chen. 1995. Topic identification in discourse. In Proceedings of the Seventh Meeting of the European Association for Computational Linguistics, pages 267-271. find
  5. Jennifer Chu-Carroll. 1998. A statistical model for discourse act recognition in dialogue interactions. In Applying Machine Learning to Discourse Processing: Papers from the 1998 AAAISpring Symposium, pages 12-17. Technical Report ~SS-98-01. find
  6. Ido Dagan and Sean P. Engelson. 1995. Committee-based sampling for training probabilistic classifiers. In Proceedings of the 12th International Conference on Machine Learning, pages 150-157. ↗
  7. Barbara Di Eugenio, Johanna D. Moore, and Massimo Paolucci. 1997. Learning features that predict cue usage. In Proceedings of the 35th Annual Meeting of the A CL, pages 80-87. ↗
  8. Yoav Freund and Robert E. Schapire. 1996. Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning. find
  9. Barbara Grosz and Candace Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175-204. ↗ our card
  10. Julia Hirschberg and Diane Litman. 1993. Empirical studies on the disambiguation of cue phrases. Computational Linguistics, 19(3):501-530. find
  11. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh. find
  12. Lynn Lambert. 1993. Recognizing Complex Discourse Acts: A Tripartite Plan-Based Model of Dialogue. Ph.D. thesis, The University of Delaware. Technical Report #93-19. find
  13. Diane J. Litman. 1994. Classifying cue phrases in text and speech using machine learning. In Proceedings of the 12th National Conference on Artificial Intelligence, pages 806-813. ↗
  14. Norbert Reithinger and Martin Klesen. 1997. Dialogue act classification using language models. In Proceedings of EuroSpeech-97, pages 2235-2238. find
  15. Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1998a. Computing dialogue acts from features with transformation-based learning. In Applying Machine Learning to Discourse Processing: Papers from the 1998 A A A I Spring Symposium, pages 90-97. Technical Report #SS-98-01. ↗
  16. Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1998b. An investigation of transformation-based learning in discourse. In Machine Learning: Proceedings of the Fifteenth International Conference. ↗
  17. Kenneth B. Samuel. 1996. Using statistical learning algorithms to compute discourse information. Technical Report #97-11, The University of Delaware. Dissertation proposal. find
  18. Janyce Wiebe, Tom O'Hara, Kenneth McKeever, and Thorsten Oehrstroem-Sandgren. 1997. An empirical approach to temporal reference resolution. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 174-186. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/samuel-1998-dialogue-act-tagging-transformation-based/