Elamur Research⌕
☰
home / papers / 2016 / Annotating Discourse Relations in Spoken Language: A Comparison of the PDTB and CCR Frameworks

Annotating Discourse Relations in Spoken Language: A Comparison of the PDTB and CCR Frameworks

Summary

Most discourse relation frameworks were designed for and tested on written text. It is therefore unclear whether different schemes yield compatible annotations and whether they suit spoken language. The authors annotated two spoken genres from the SPICE Ireland corpus, private telephone conversations and public broadcast interviews, using two very different schemes. PDTB 3.0 assigns hierarchical end labels. CCR instead describes every relation through a set of four cognitively motivated features: basic operation, source of coherence, polarity and order.

Labelling the same text in both frameworks provides an empirical check of proposed cross-framework mappings. For most relations the correspondence held. The remaining mismatches stem partly from annotator interpretation and partly from differing guideline tests and granularity, which produce many-to-many mappings.

The study also found that spoken data differ from written data in the balance of explicit and implicit relations and in the distribution of causal relation types. Finally, the authors discuss spoken-specific phenomena that the schemes handle poorly. The resulting corpus is roughly as large as the CoNLL 2015 Shared Task test set, making it useful for evaluating automatic discourse relation labellers.

Key points

  • This is the first larger text annotated with both PDTB 3.0 and CCR. About 70% of PDTB-labelled relations fell into the CCR categories predicted by the mapping. In an analysed subset of the mismatches, roughly half came from framework-independent annotator disagreement.
  • In the spoken data, explicit relations occur about twice as often as implicit ones, whereas written genres show a roughly equal proportion.
  • Causal relations are distributed differently across the two spoken genres, yet because and so mark most explicit causal relations in both.
  • PDTB 3.0's added pragmatic labels (epistemic and speech-act) proved useful for speech, while CCR does not separate speech-act relations from other subjective relations. The authors propose new PDTB categories, AltTop (Alternative Topicalisation) and AltStress (Alternative Stress), for contrast expressed through word order or prosody.
  • Implicit relations were not restricted to adjacent sentences, because disfluencies, backchannels and fragments often separate the arguments in conversation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

29 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{rehbein2016annotating,
  title = {{Annotating Discourse Relations in Spoken Language: A Comparison of the PDTB and CCR Frameworks}},
  author = {Rehbein, Ines and Scholman, Merel and Demberg, Vera},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/3zroe3z9ppok},
  url = {https://doi.org/10.63317/3zroe3z9ppok}
}

References

20 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Afantenos, S., Asher, N., Benamara, F., Bras, M., Fabre, C., Ho-Dac, M., Draoulec, A. L., Muller, P., Pery-Woodley, M.-P., Prevot, L., Rebeyrolles, J., Tanguy, L., Vergez-Couret, M., and Vieu, L. (2012). An empirical resource for discovering cognitive principles of discourse organisation: the ANNODIS corpus. In Proceedings of the Eight International Conference on Language Resources and Evaluation, LREC’12. find
  2. Benamara, F. and Taboada, M. (2015). Mapping different rhetorical relation annotations: A proposal. In Proceedings of the 4th Joint Conference on Lexical and Computational Semantics, *SEM. find
  3. Bunt, H., Prasad, R., and Joshi, A. (2012). First steps towards an ISO standard for annotating discourse relations. In Proceedings of the Joint ISA-7, SRSL-3, and I2MRT LREC 2012 Workshop on Semantic Annotation and the Integration and Interoperability of Multimodal Resources and Tools. find
  4. Carlson, L., Marcu, D., and Okurowski, M. E. (2002). RST discourse treebank, ldc2002t07. Web Download. Philadelphia: Linguistic Data Consortium. find
  5. Chiarcos, C. and Sukhareva, M. (2012). Ontologies of linguistic annotation. Semantic Web, 1(0). find
  6. Chiarcos, C. (2014). Towards interoperable discourse annotation. discourse features in the ontologies of linguistic annotation. In Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC’14, Reykjavik, Iceland. find
  7. da Cunha, I., Torres-Moreno, J., and Sierra, G. (2011). On the development of the RST spanish treebank. In Proceedings of the Fifth Linguistic Annotation Workshop, LAW 2011, pages 1–10, Portland, Oregon. find
  8. Demirşahin, I. and Zeyrek, D. (2014). Annotating discourse connectives in spoken Turkish. In Proceedings of LAW VIII - The 8th Linguistic Annotation Workshop, pages 105–109, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. find our card
  9. Kallen, J. L. and Kirk, J. M., (2008). ICE-Guide. Belfast: Cló Ollscoil na Banrı́ona. find
  10. Kallen, J. and Kirk, J., (2012). SPICE-Ireland: A User’s Guide. Belfast: Cló Ollscoil na Banrı́ona. find
  11. Knott, A. and Sanders, T. (1998). The classification of coherence relations and their linguistic markers: An exploration of two languages. Journal of pragmatics, 30(2):135–175. ↗
  12. Prasad, R. and Bunt, H. (2015). From semantic relations in discourse: The current state of iso 24617-8. In Proceedings of the Eleventh Joint ACL - ISO Workshop on Interoperable Semantic Annotation. find
  13. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., and Webber, B. (2008). The penn discourse treebank 2.0. In Proceedings of the 6 International Conference on Language Resources and Evaluation, LREC’08. European Language Resources Association. find our card
  14. Sanders, T., Spooren, W., and Noordman, L. (1992). Toward a taxonomy of coherence relations. Discourse Processes, 15:1–35. ↗
  15. Sanders, T., Vis, K., and Broeder, D. (2012). Project notes on the dutch project discan. In Eighth Joint ACL - ISO Workshop on Interoperable Semantic Annotation. find
  16. Scholman, M. C., Evers-Vermeul, J., and Sanders, T. J. (2016). A step-wise approach to discourse annotation: Towards a reliable categorisation of coherence relations. Dialogue and Discourse, 2(7). find
  17. Stoyanchev, S. and Bangalore, S. (2015). Discourse in customer care dialogues. Poster presented at the DiSpoL 2015 workshop ”Identification and Annotation of Discourse Relations in Spoken Language”, October 2015 in Saarbrücken, Germany. find
  18. Sweetser, E. (1990). From Etymology to Pragmatics: Metaphorical and Cultural Aspects of Semantic Structure. Cambridge University Press. find
  19. Tonelli, S., Prasad, R., and Joshi, A. (2010). Annotation of discourse relations for conversational spoken dialogs. In Proceedings of the 7 International Conference on Language Resources and Evaluation, LREC’10. find our card
  20. Webber, B. (2009). Genre distinctions for discourse in the penn treebank. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics, ACL’09. find

↑ back to top