Elamur Research⌕
☰
home / papers / 2017 / Towards Full Text Shallow Discourse Relation Annotation: Experiments with Cross-Paragraph Implicit Relations in the PDTB

Towards Full Text Shallow Discourse Relation Annotation: Experiments with Cross-Paragraph Implicit Relations in the PDTB

Summary

The Penn Discourse Treebank (PDTB) leaves a notable hole in its inter-sentential coverage. Implicit relations are labeled only between adjacent sentences inside a paragraph, so a paragraph-opening sentence with no explicit connective gets no link to the preceding text. This hole blocks full-text discourse parsing, which needs complete relation sequences. The authors ran a three-phase pilot over PDTB texts of varied sub-genre and length, using the PDTB Annotation Tool and the PDTB-3 sense hierarchy (slightly extended). They measured two things: how often a paragraph-initial sentence attaches to the immediately preceding sentence versus some earlier material, and how reliably each case can be annotated. Long-distance attachments turned out to be about as common as local ones but were much harder to annotate consistently. The final guidelines therefore keep PDTB's adjacency requirement for implicits, mark non-adjacent cases in an underspecified way, and add refinements for relation inter-dependency, new senses and entity-based coherence. These guidelines reach 62% agreement. The study gives the first quantitative estimate of how hard long-range discourse dependencies are to annotate, and it lays groundwork for a fuller PDTB extension.

Key points

  • Size of the gap: PDTB-2 contains over 12K unannotated cross-paragraph implicit contexts. These make up 30% of all inter-sentential contexts and 87% of the cross-paragraph ones.
  • Pilot data: The pilot covered 440 paragraph-initial sentence (CPFS) tokens lacking an explicit inter-sentential connective. Phase One used 10 texts (130 tokens) and Phase Two used 10 texts (103 tokens).
  • Adjacency split: Adjacent and non-adjacent attachments were nearly balanced (51% vs. 47%). This mirrors the pattern previously reported for cross-paragraph explicit relations.
  • Agreement: Annotators agreed on 68% of adjacent relations but only 47% of non-adjacent ones. The final guidelines achieved 62% overall agreement.
  • Planned resource: The authors plan to annotate about 200 PDTB texts (around 700 tokens) and release them on GitHub. They estimate roughly 3 minutes per token, or about 35 hours per annotator.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-5502

@inproceedings{prasad2017full,
  title = {{Towards Full Text Shallow Discourse Relation Annotation: Experiments with Cross-Paragraph Implicit Relations in the PDTB}},
  author = {Prasad, Rashmi and Riley, Katherine Forbes and Lee, Alan},
  booktitle = {Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2017},
  doi = {10.18653/v1/w17-5502},
  url = {https://doi.org/10.18653/v1/w17-5502}
}

References

14 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stergos Afantenos, Nicholas Asher, Farah Benamara, Myriam Bras, Ccile Fabre, Lydia-Mai Ho-Dac, Anne Le Draoulec, Philippe Muller, Marie-Paule Pry-Woodley, Laurent Prevot, Josette Rebeyrolle, Ludovic Tanguy, Marianne Vergez-Couret, and Laure Vieu. 2012. An empirical resource for discovering cognitive… find
  2. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind Joshi. 2008. Easily identifiable discourse relations. In Proceedings of the 22nd International Conference on Computational Linguistics. Manchester, UK, pages 87–90. the CoNLL-16 shared task. Berlin, Germany, pages 1–19… find our card
  3. Fan Zhang, Diane Litman, and Katherine Forbes Riley. 2016. Inferring discourse relations from PDTB-style discourse labels for argumentative revision classification. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan, pages 2615–2624. find
  4. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation. Marrakech, Morocco, pages 2961–2968. find our card
  5. Rashmi Prasad, Aravind Joshi, and Bonnie Webber. 2010. Exploiting scope for shallow discourse parsing. In Proceedings of the 7th International Conference on Language Resources and their Evaluation. Valletta, Malta, pages 2076–2083. find our card
  6. Rashmi Prasad, Bonnie Webber, and Aravind Joshi. 2014. Reflections on the Penn Discourse TreeBank, comparable corpora and complementary annotation. Computational Linguistics 40(4):921–950. find our card
  7. Swapna Somasundaran, Janyce Wiebe, and Josef Ruppenhofer. 2008. Discourse level opinion interpretation. In Proceedings of the 22nd International Conference on Computational Linguistics, Volume 1. Manchester, U.K., pages 801–808. find
  8. Manfred Stede. 2012. Discourse processing. Synthesis Lectures on Human Language Technologies (series editor, Graeme Hirst). Morgan & Claypool Publishers. find
  9. Jianxiang Wang and Man Lan. 2015. A refined end-to-end discourse parser. In Proceedings of the CoNLL-2015 Shared Task. Beijing, China, pages 17–24. find our card
  10. Bonnie Webber. 2009. Genre distinctions for discourse in the Penn TreeBank. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Suntec, Singapore, page 674–682. find
  11. Bonnie Webber, Aravind Joshi, Matthew Stone, and Alistair Knott. 2003. Anaphora and discourse structure. Computational Linguistics 29(4):545–587. find our card
  12. Bonnie Webber, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2016. A discourse-annotated corpus of conjoined VPs. In Proceedings of the 10th Linguistic Annotation Workshop. ACL, Berlin, Germany, pages 22–31. find
  13. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the CoNLL-15 shared task. Beijing, China, pages 1–16. find our card
  14. Nianwen Xue, Hwee Tou Ng, Attapol Rutherford, Bonnie Webber, Chuan Wang, and Hongmin Wang. 2016. The CoNLL-2016 shared task on multilingual shallow discourse parsing. In Proceedings of find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/prasad-2017-full-text-shallow-discourse-relation/