Elamur Research⌕
☰
home / papers / 2021 / DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection

DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection

Paper · 2021 · Luke Gessler, Shabnam Behzad, Yang Janet Liu et al.

Summary

DisCoDisCo is the Georgetown Corpling Lab entry to DISRPT 2021. It covers three discourse tasks across many languages and frameworks (RST, SDRT, PDTB): splitting text into elementary discourse units, detecting discourse connectives, and labeling the relation between two units. The authors argue that contextualized word embeddings (CWEs) cannot easily see some signals, such as where a sentence sits in a document or how many words two units share. They therefore add dense embeddings of hand-crafted categorical and numeric features to a Transformer core. Segmentation and connective detection are handled by a bi-LSTM tagger over character, fastText and CWE inputs, plus a summary of the neighboring sentences. Relations are labeled by a BERT-style sentence-pair classifier enriched with features. The system beats the best 2019 shared-task scores on segmentation and connective detection and sets strong first numbers on the new relation benchmark. Ablations show that the extra features usually help, sometimes a lot (speaker information in chat data), and occasionally hurt. Models pretrained with Next Sentence Prediction work best for relations.

Key points

  • Tagger input per word joins three embeddings: char bi-LSTM (d=64), frozen fastText (d=300) and a fine-tuned CWE (768/1024) at a lower learning rate. Subword vectors are average-pooled to the word level.
  • Each categorical feature gets an embedding whose dimensionality equals the square root of how many distinct labels that feature has, and numeric features are log-scaled. The two neighboring sentences are compressed by a bi-LSTM into a 400-dim vector attached to every token.
  • The decoder choice depends on the task:
    • connective detection (BIO) uses a CRF;
    • segmentation uses a plain linear layer, because the CRF slightly hurt it.
  • The CWE was chosen per language on validation data, for example:
    • xlm-roberta-large for German segmentation;
    • bert-base-german-cased for German relations;
    • google/electra-large-discriminator for English segmentation.
  • The plain-text scenario uses automatic sentence splits and UD parses from the AMALGUM splitter. The error analysis found that the dominant labels (ELABORATION in GUM, COMMENT in STAC) get over-predicted, while rare and implicit relations such as RESULT and JUSTIFY are poorly recognized.

Interesting ideas

  • I1. Add explicit features for things a pretrained encoder is structurally blind to, such as document position, word overlap between units, length ratio and speaker, instead of trusting the embeddings to carry everything.
  • I2. Size each categorical feature's embedding by the square root of its number of values, and log-scale counts, as a cheap default for mixing heterogeneous symbolic features into a neural model.
  • I3. Pick the decoder per task rather than by habit: a CRF helped BIO-style connective spans but slightly hurt binary boundary tagging.
  • I4. Use a confusion matrix to separate "default" labels the model falls back on from labels it gets right through reliable lexical cues (question mark, attribution verbs, conditional markers).
  • I5. An ambiguous marker gets mapped to its most prototypical relation ("then" read as SEQUENCE when the gold label is RESULT), and the authors note that two relations could hold at once.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): When linking clauses into discourse or speech-act structure, prag could combine parse-derived cues with simple countable signals like shared lemmas and clause position, keeping each signal inspectable.
  • I2 (tree): gparse could take UD morphological features or deprel-of-head as embedded inputs using this sizing rule, avoiding hand-tuning a dimension per feature.
  • I3 (tokens): Sentence or token boundary taggers in crate en could stay with independent per-token decisions, keeping CRF-style decoding for multi-token span labels like multiword expressions.
  • I4 (learn): Evaluation could report which frame or relation labels act as fallbacks, and these could serve as negative controls showing that a predicted label is cue-driven rather than a majority guess.
  • I5 (world): Event ordering could keep both a temporal and a causal link for a marker like "then", instead of forcing one label, and let later reasoning choose.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgetown UniversityWashington · United States69 works · 4,417 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Discourse analysis · 394 works

Cite

Cite as Luke Gessler, Shabnam Behzad, Yang Janet Liu, Siyao Peng, Yilun Zhu and Amir Zeldes. 2021. DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection. Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2021).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.disrpt-1.6

@inproceedings{gessler2021discodisco,
  title = {{DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection}},
  author = {Luke Gessler, Shabnam Behzad, Yang Janet Liu, Siyao Peng, Yilun Zhu and Amir Zeldes},
  booktitle = {Proceedings of the 2nd Shared Task on Discourse Relation Parsing and Treebanking (DISRPT 2021)},
  year = {2021},
  doi = {10.18653/v1/2021.disrpt-1.6},
  url = {https://doi.org/10.18653/v1/2021.disrpt-1.6}
}

References

61 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Our error analysis suggests unsurprisingly that class imbalances, especially in the case of relations that tend to be implicit (i.e. lack overt lexical signals), lead to over-prediction of majority classes, suggesting a need for more training data for the minority ones. However, we are encouraged by improvements on datasets that were featured in the 2019 Shared Task (Zeldes et al., 2019), and the overall high scores obtained by the system across a range of datasets, all while including some correct predictions for relatively rare relations. We hope that the growing availability of annotated data, coupled with architectures that can harness pre-trained models, will lead to further improvements in the near future. Then she lets go and falls . You scream . [GUM_fiction_falling] find
  2. In fact, if it were not the case that RST mandates a single outgoing relation per discourse unit, it would be possible to claim a concurrent sequential relation (likely to a previous unit) next to the annotated RESULT relation for the pair in the example (see Stede 2008 on concurrent relations in RST).
  3. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. HILDA: A discourse parser using support vector machine classification. Dialogue and Discourse, 1(3):1–33.
  4. Giuseppe Attardi, Daniele Sartiano, and Yu Zhang. 2021. DiaParser attentive dependency parser. find
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  6. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  7. Peter Bourgonje and Robin Schäfer. 2019. Multilingual and cross-genre discourse unit segmentation. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 105–114, Minneapolis, MN. Association for Computational Linguistics. find
  8. Dan Jurafsky and James H. Martin. 2020. Speech and Language Processing. https://web.stanford.edu/ jurafsky/slp3/. ↗
  9. Yusuke Kido and Akiko Aizawa. 2016. Discourse relation sense classification with two-step classifiers. In Proceedings of the CoNLL-16 shared task, pages 129–135, Berlin, Germany. Association for Computational Linguistics. find
  10. Chloé Braud, Maximin Coavoux, and Anders Søgaard. 2017a. Cross-lingual RST discourse parsing. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 292–304, Valencia, Spain. Association for Computational Linguistics. ↗
  11. Najoung Kim, Song Feng, Chulaka Gunasekara, and Luis Lastras. 2020. Implicit discourse relation classification: We need to talk about evaluation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5404– 5414, Online. Association for Computational Linguistics. ↗
  12. Chloé Braud, Ophélie Lacroix, and Anders Søgaard. 2017b. Does syntax help discourse segmentation? Not So Much. In Proceedings of EMNLP 2017, pages 2432–2442, Copenhagen. find
  13. Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top-down rst parsing utilizing granularity levels in documents. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8099–8106. find
  14. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Current and New Directions in Discourse and Dialogue, Text, Speech and Language Technology 22, pages 85–112. Kluwer, Dordrecht. ↗
  15. Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2021. Improving neural RST parsing model with silver agreement subtrees. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1600–1612, Online. Association for Computational Linguistics. find
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  17. Li Liang, Zheng Zhao, and Bonnie Webber. 2020. Extending implicit discourse relation recognition to the PDTB-3. In Proceedings of the First Workshop on Computational Approaches to Discourse, pages 135–147, Online. Association for Computational Linguistics. ↗
  18. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of ACL 2018, pages 1–6, Melbourne, Australia.
  19. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A pdtb-styled end-to-end discourse parser. Natural Language Engineering, 20(2):151–184. find
  20. Luke Gessler, Siyao Peng, Yang Liu, Yilun Zhu, Shabnam Behzad, and Amir Zeldes. 2020. AMALGUM - a free, balanced, multilayer English web corpus. In Proceedings of LREC 2020, pages 5267–5275, Marseille, France. ↗
  21. Yang Liu. 2019. Beyond the Wall Street Journal: Anchoring and comparing discourse signals across genres. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 72– 81, Minneapolis, MN. Association for Computational Linguistics.
  22. Grigorii Guz and Giuseppe Carenini. 2020. Coreference for discourse parsing: A neural approach. In Proceedings of the First Workshop on Computational Approaches to Discourse, pages 160–167, Online. Association for Computational Linguistics. ↗
  23. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, page 2750–2756. AAAI Press.
  24. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 13–16, Suntec, Singapore. find
  25. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. find
  26. Balaji Polepalli Ramesh, Rashmi Prasad, Tim Miller, Brian Harrington, and Hong Yu. 2012. Automatic discourse connective detection in biomedical text. Journal of the American Medical Informatics Association, 19(5):800–808. ↗
  27. William C Mann and Sandra A Thompson. 1988. Rhetorical Structure Theory: Toward a Functional Theory of Text Organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. ↗
  28. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008), pages 2961–2968, Marrakesh, Morocco.
  29. Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge, MA. ↗
  30. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Special Issue on Using Large Corpora, Computational Linguistics, 19(2):313–330. ↗
  31. Rashmi Prasad, Susan McRoy, Nadya Frid, Aravind Joshi, and Hong Yu. 2011. The Biomedical Discourse Relation Bank. BMC bioinformatics, 12(1):188. ↗
  32. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  33. Rashmi Prasad, Bonnie Webber, Alan Lee, and Aravind Joshi. 2019. Penn Discourse Treebank Version 3.0. LDC2019T05. ↗
  34. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101– 108, Online. Association for Computational Linguistics. ↗
  35. Philippe Muller, Chloé Braud, and Mathieu Morey. 2019. ToNy: Contextual embeddings for accurate multilingual discourse segmentation of full documents. In Proceedings of Discourse Relation Treebanking and Parsing (DISRPT 2019), pages 115– 124, Minneapolis, MN. find
  36. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842–866. ↗
  37. Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, and Xiaoli Li. 2021. RST parsing from scratch. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1613–1625, Online. Association for Computational Linguistics.
  38. Attapol Rutherford, Vera Demberg, and Nianwen Xue. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 281–291, Valencia, Spain. Association for Computational Linguistics. ↗
  39. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc. ↗
  40. Niko Schenk, Christian Chiarcos, Kathrin Donandt, Samuel Rönnqvist, Evgeny Stepanov, and Giuseppe Riccardi. 2016. Do we really need all those rich linguistic features? a neural network-based approach to implicit sense labeling. In Proceedings of the CoNLL-16 shared task, pages 41–49, Berlin, Germany. Association for Computational Linguistics. find
  41. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of HLT-NAACL 2003, pages 149–156, Edmonton.
  42. Gary Patterson and Andrew Kehler. 2013. Predicting the presence of discourse connectives. In Proceedings of EMNLP 2013, pages 914–923.
  43. Caroline Sporleder and Mirella Lapata. 2005. Discourse chunking and its application to sentence compression. In Proceedings of EMNLP 2005, pages 257–264, Vancouver. find
  44. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text.
  45. Manfred Stede. 2008. Disambiguating rhetorical structure. Research on Language and Computation, 6(3):311–332. language understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. ↗
  46. Manfred Stede. 2011. Discourse Processing. Synthesis Lectures on Human Language Technologies, 4(3):1– 165. find
  47. Yue Yu, Yilun Zhu, Yang Liu, Yan Liu, Siyao Peng, Mackenzie Gong, and Amir Zeldes. 2019. Gum-Drop at the DISRPT2019 shared task: A model stacking approach to discourse unit segmentation and connective detection. In Proceedings of Discourse Relation Treebanking and Parsing (DISRPT 2019), pages 133–143, Minneapolis, MN. find
  48. Huong Le Thanh, Geetha Abeysinghe, and Christian Huyck. 2004. Generating discourse structures for written text. In Proceedings of COLING 2004, pages 329–335, Geneva, Switzerland. find
  49. Amir Zeldes. 2018. Multilayer Corpus Studies. Routledge Advances in Corpus Linguistics 22. Routledge, London. ↗
  50. Jianxiang Wang and Man Lan. 2016. Two end-to-end shallow discourse parsers for English and Chinese in CoNLL-2016 shared task. In Proceedings of the CoNLL-16 shared task, pages 33–40, Berlin, Germany. Association for Computational Linguistics.
  51. Amir Zeldes, Debopam Das, Erick Galani Maziero, Juliano Antonio, and Mikel Iruskieta. 2019. find
  52. The DIS-RPT 2019 shared task on elementary discourse unit segmentation and connective detection. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 97–104, Minneapolis, MN. Association for Computational Linguistics. find
  53. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 184–188, Vancouver, Canada. Association for Computational Linguistics. ↗
  54. Yingxue Zhang, Fandong Meng, Peng Li, Ping Jian, and Jie Zhou. 2021. Context tracking network: Graph-based context modeling for implicit discourse relation recognition. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1592–1599, Online. Association for Computational Linguistics.
  55. Gregor Weiss and Marko Bajec. 2016. Discourse sense classification from scratch using focused RNNs. In Proceedings of the CoNLL-16 shared task, pages 50– 54, Berlin, Germany. Association for Computational Linguistics. find
  56. Yuping Zhou, Jill Lu, Jennifer Zhang, and Nianwen Xue. 2014. Chinese Discourse Treebank 0.5. LDC2014T21. find
  57. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  58. Yuping Zhou and Nianwen Xue. 2012. PDTB-style discourse annotation of Chinese text. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 69–77, Jeju Island, Korea. Association for Computational Linguistics. find
  59. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 1–16, Beijing, China. Association for Computational Linguistics. ↗
  60. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Attapol Rutherford, Bonnie Webber, Chuan Wang, and Hongmin Wang. 2016. CoNLL 2016 shared task on multilingual shallow discourse parsing. In Proceedings of the CoNLL-16 shared task, pages 1– 19, Berlin, Germany. Association for Computational Linguistics. ↗
  61. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gessler-2021-discodisco-disrpt2021-shared-task-system/