Elamur Research⌕
☰
home / papers / 2004 / FLSA: Extending Latent Semantic Analysis with Features for Dialogue Act Classification

FLSA: Extending Latent Semantic Analysis with Features for Dialogue Act Classification

Summary

The paper addresses a known weakness of Latent Semantic Analysis: it builds its semantic space from word co-occurrence alone and ignores other annotation a corpus may carry. The authors propose Feature LSA (FLSA). In FLSA, each value of an extra categorical feature, such as speaker identity, dialogue game, or previous dialogue act, becomes an additional pseudo-word row in the Word–Document matrix before SVD is applied. Unlabeled utterances are then assigned the DA of the most cosine-similar training utterance. The method was evaluated on dialogue act classification over three corpora: Spanish CallHome (in collapsed CallHome37 and CallHome10 tagsets), HCRC MapTask, and the authors' small DIAG-NLP tutoring corpus. Compared with a majority-class baseline, plain LSA already cuts errors substantially, and FLSA cuts them further, with especially large gains on MapTask. These results are on par with or better than previously published systems. Dialogue-level features such as Game proved useful, while syntactic features and DA history did not. The work shows that a simple, easily trained statistical model can benefit from rich corpus annotation.

Key points

  • Relative to the most-frequent-DA baseline, LSA lowers error rates by 33–52%, while FLSA lowers them by 60–78%. Evaluation used 5-fold cross-validation, with the reduced dimension k tuned between 25 and 350.
  • The data covers three corpora:
    • Spanish CallHome: 120 calls, 44,628 DAs, 232 original tags collapsed to 37 and 10.
    • MapTask: 128 dialogues, 27,084 DAs, 13 moves.
    • DIAG-NLP: 23 dialogues, 660 DAs, 4 tutor moves.
  • On DIAG-NLP, FLSA brought no significant gain, likely because the corpus is small or because Consult Type was the only usable feature. A separate variant, Clustered LSA, reached 79.24% there.
  • Agreement between FLSA and the gold annotation is κ = 0.676 (CallHome37), 0.721 (CallHome10), and 0.740 (MapTask). For comparison, expert human coders reached κ = 0.83 on MapTask with access to audio. The main confusions of FLSA largely mirror those of human annotators.
  • Information gain helps rank or discard candidate features, though it does not predict accuracy perfectly. For example, the previous DA added a little information on CallHome37 yet lowered accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1218955.1219043

@inproceedings{serafin2004flsa,
  title = {{FLSA: Extending Latent Semantic Analysis with Features for Dialogue Act Classification}},
  author = {Serafin, Riccardo and Eugenio, Barbara Di},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1219043},
  url = {https://doi.org/10.3115/1218955.1219043}
}

References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In COLT98, Proceedings of the Conference on Computational Learning Theory. ↗ our card
  2. Jean Carletta, Amy Isard, Stephen Isard, Jacqueline C. Kowtko, Gwyneth Doherty-Sneddon, and Anne H. Anderson. 1997. The reliability of a dialogue structure coding scheme. Computational Lingustics, 23(1):13–31. ↗
  3. Jean Carletta. 1996. Assessing agreement on classification tasks: the Kappa statistic. Computational Linguistics, 22(2):249–254. ↗ our card
  4. Peter W. Foltz, Walter Kintsch, and Thomas K. Landauer. 1998. The measurement of textual coherence with Latent Semantic Analysis. Discourse Processes, 25:285–308. ↗
  5. Peter W. Foltz, Darrell Laham, and Thomas K. Landauer. 1999. The intelligent essay assessor: Applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2). find
  6. Michael Glass and Barbara Di Eugenio. 2002. MUP: The UIC standoff markup tool. In The Third SigDIAL Workshop on Discourse and Dialogue, Philadelphia, PA, July. find
  7. Michael Glass, Heena Raval, Barbara Di Eugenio, and Maarika Traat. 2002. The DIAG-NLP dialogues: coding manual. Technical Report UIC-CS 02-03, University of Illinois - Chicago. find
  8. Dharmendra Kanejiya, Arun Kumar, and Surendra Prasad. 2003. Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA. In HLT-NAACL Workshop on Building Educational Applications using Natural Language Processing, pages 53–60, Edmonton, Canada. find
  9. Walter Kintsch. 2001. Predication. Cognitive Science, 25:173–202. find
  10. Klaus Krippendorff. 1980. Content Analysis: an Introduction to its Methodology. Sage Publications, Beverly Hills, CA. find
  11. T. Lager and N. Zinovjeva. 1999. Training a dialogue act tagger with the µ-TBL system. In The Third Swedish Symposium on Multimodal Communication, Linköping University Natural Language Processing Laboratory (NLPLAB). find
  12. Thomas K. Landauer and S.T. Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104:211–240. ↗
  13. Lori Levin, Ann Thymé-Gobbel, Alon Lavie, Klaus Ries, and Klaus Zechner. 1998. A discourse coding scheme for conversational Spanish. In Proceedings ICSLP. ↗
  14. J. Ross Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann. find our card
  15. Klaus Ries. 1999. HMM and Neural Network Based Speech Act Detection. In Proceedings of ICASSP 99, Phoenix, Arizona, March. ↗
  16. Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1998. Dialogue act tagging with transformation-based learning. In ACL/COLING 98, Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics (joint with the 17th International Conference on Computational Linguistics), pages 1150–1156. ↗ our card
  17. John R. Searle. 1975. Indirect Speech Acts. In P. Cole and J.L. Morgan, editors, Syntax and Semantics 3. Speech Acts. Academic Press. Reprinted in Pragmatics. A Reader, Steven Davis editor, Oxford University Press, 1991. ↗
  18. Riccardo Serafin. 2003. Feature Latent Semantic Analysis for dialogue act interpretation. Master’s thesis, University of Illinois - Chicago. find
  19. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. ↗ our card
  20. Douglas M. Towne. 1997. Approximate reasoning techniques for intelligent diagnostic instruction. International Journal of Artificial Intelligence in Education. find
  21. Peter Wiemer-Hastings. 2001. Rules for syntax, vectors for semantics. In CogSci01, Proceedings of the Twenty-Third Annual Meeting of the Cognitive Science Society, Edinburgh, Scotland. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/serafin-2004-flsa-extending-latent-semantic-analysis/