Elamur Research⌕
☰
home / papers / 2012 / Syntactic annotation of spontaneous speech: application to call-center conversation data

Syntactic annotation of spontaneous speech: application to call-center conversation data

Summary

The authors address syntactic parsing of spontaneous speech. Tools trained on written texts handle it poorly because of repetitions, broken-off phrases, discourse markers and ungrammatical constructions. The work is carried out on the DECODA corpus, which consists of manual transcriptions of French telephone conversations with the call center of the Paris transport operator RATP. The corpus is annotated at three levels: POS tags, syntactic chunks and dependencies. The approach is semi-automatic. First, the annotation is produced by the MACAON toolkit, whose models were trained on the French TreeBank. Annotators then correct errors through a web interface, including bulk corrections using regular expressions. After that, the models are retrained, and quality is evaluated on the fully manually annotated subset DECODA-GOLD. The cycle is repeated until the error rate becomes acceptable. To save effort, dependencies were annotated manually between chunks and then transferred automatically to the word level using templates. The adaptation reduced the number of POS tagging errors by more than half. The MATE parser trained on this corpus achieved accuracy comparable to that obtained when parsing newspaper texts. The work thus demonstrates a relatively inexpensive way to build a treebank of spoken language for speech analytics and dialogue understanding.

Key points

  • DECODA contains 1514 dialogues, about 74 hours of recordings, with an average conversation length of approximately 3 minutes. The corpus is fully anonymized, and segmentation and transcription were done manually.
  • POS tagging error rate after adaptation: on DECODA, from 21.0% to 8.5%; on the EPAC corpus of radio conversations, from 13.3% to 4.5%.
  • The dependency annotation has 15 types based on the French TreeBank guidelines. About 195 templates were used to transfer relations from chunks to words. More than 200 lexicalized templates were added for ambiguous cases.
  • Approximately 40.8% of utterances had multiple roots. The Governor before and Governor after strategies add an artificial arc to the neighboring tree. They eliminate this problem entirely and reduce the proportion of non-projective trees from 8.21% to 7.83% or 6.49%, respectively. At the same time, the proportion of overlapping trees increases slightly.
  • The MATE parser (graph-based, second-order features) was trained and tested on reference transcriptions and tags. The turns were split in an 80/10/10 ratio. On the test set, LAS is about 87.4–87.9% and UAS is about 90.8–91.4%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Treebanks · Part-of-speech tagging

Cited by

8 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{bazillon2012syntactic,
  title = {{Syntactic annotation of spontaneous speech: application to call-center conversation data}},
  author = {Bazillon, Thierry and Deplano, Melanie and Bechet, Frederic and Nasr, Alexis and Favre, Benoit},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/43xtqkc3yiw8},
  url = {https://doi.org/10.63317/43xtqkc3yiw8}
}

References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Abeillé, L. Clément, and F. Toussenel. 2003. Building a treebank for french. In Anne Abeillé, editor, Treebanks. Kluwer, Dordrecht. find
  2. Bernd Bohnet. 2010. Top Accuracy and Fast Dependency Parsing is not a Contradiction. In Proceedings of COL-ING. find our card
  3. Frédéric Béchet and Alexis Nasr. 2009. Robust dependency parsing for spoken language understanding of spontaneous speech. In Interspeech, Brighton. find
  4. X. Carreras. 2007. Experiments with a higher-order projective dependency parser. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL, volume 7, pages 957–961. find
  5. K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. 2006. Online passive-aggressive algorithms. The Journal of Machine Learning Research, 7:551–585. find
  6. R. De Mori, F. Bechet, D. Hakkani-Tur, M. McTear, G. Riccardi, and G. Tur. 2008. Spoken language understanding. Signal Processing Magazine, IEEE, 25(3):50–58, May. find
  7. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online Large-Margin Training of Dependency Parsers. In Association for Computational Linguistics. find our card
  8. Alexis Nasr, Frederic Bechet, Jean-Francois Rey, and Joseph Le Roux. 2011. Macaon:a linguistic tool suite for processing word lattices. In The 49th Annual Meeting of the Association for Computational Linguistics: demonstration session. find
  9. J. Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies. find our card
  10. Slav Petrov and Dan Klein. 2007. Improved Inference for Unlexicalized Parsing. In HLT-NAACL, pages 404–411. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bazillon-2012-syntactic-annotation-spontaneous-speech-application/