Elamur Research⌕
☰
home / papers / 2021 / Adversarial Learning for Discourse Rhetorical Structure Parsing

Adversarial Learning for Discourse Rhetorical Structure Parsing

Summary

Text-level discourse rhetorical structure (DRS) parsing builds RST-style trees over whole documents. Training data is scarce: RST-DT has 385 WSJ articles and CDTB has 500 Chinese newswire articles. Earlier parsers, whether bottom-up or top-down, assemble the tree from step-by-step local choices and never judge the finished structure as a whole. As a starting point, the authors take Zhang et al.'s (2020) top-down parsing model and streamline it by removing dummy split points and combining nuclearity and relation labels into joint N-R tags. They then add a global training signal. Each gold or predicted tree is rendered as a small two-channel "image", with one channel for structure and one for nuclearity-relation. An LSGAN-style discriminator learns to tell real tree images from generated ones. Because the images are continuous, this avoids the usual difficulty of applying GANs to discrete outputs. Evaluated with the original Parseval on RST-DT and CDTB, the parser reports new state-of-the-art results on both corpora. This suggests that whole-tree adversarial supervision can partly offset small discourse treebanks.

Key points

  • The baseline is the top-down encoder-decoder parser of Zhang et al. (2020), modified as follows:
    • It uses a biaffine split-point ranker.
    • Sentence- and paragraph-boundary features are added.
    • Only two classifiers are used: structure and combined N-R. The N-R classifier has 41 classes for RST-DT and 46 for CDTB.
  • Trees are encoded as m × (n+2) matrices, where m is tree height and n is the number of split points. The two channels are ST (structure) and NR (nuclearity-relation).
  • The discriminator uses a least-squares (LSGAN) loss with a learning rate of 1e-4. Sigmoid cross-entropy performed clearly worse.
  • Experiments use GloVe, ELMo and XLNet representations. The XLNet-based parser beats JE2017-updated by 18.8% on relation ("R") scores. Ablations confirm that the adversarial component contributes to the gains.
  • After converting predicted trees into discourse dependencies, the parser also improves UAS and LAS over previous systems, so the global optimization helps dependency analysis too.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.acl-long.305

@inproceedings{zhang2021adversarial,
  title = {{Adversarial Learning for Discourse Rhetorical Structure Parsing}},
  author = {Zhang, Longyin and Kong, Fang and Zhou, Guodong},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.305},
  url = {https://doi.org/10.18653/v1/2021.acl-long.305}
}

References

40 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chloé Braud, Maximin Coavoux, and Anders Søgaard. 2017. Cross-lingual RST discourse parsing. In EACL, pages 292–304, Valencia, Spain. Association for Computational Linguistics. find our card
  2. Chloé Braud, Barbara Plank, and Anders Søgaard. 2016. Multi-view and multi-task training of RST discourse parsers. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1903–1913, Osaka, Japan. find
  3. The COLING 2016 Organizing Committee. find
  4. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurovsky. 2001. Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. find our card
  5. Francine Chen and Yan-Ying Chen. 2019. Adversarial domain adaptation using artificial titles for abstractive title generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2197–2203, Florence, Italy. Association for Computational Linguistics. find
  6. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 657–668, Online. Association for Computational Linguistics. find our card
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  8. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In The 5th International Conference on Learning Representations, ICLR2017. find our card
  9. Yanai Elazar and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11–21, Brussels, Belgium. Association for Computational Linguistics. find our card
  10. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511–521, Baltimore, Maryland. Association for Computational L… find our card
  11. Hugo Hernault, Helmut Prendinger, Mitsuru Ishizuka, et al. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue and Discourse, 1(3). find our card
  12. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13–24, Baltimore, Maryland. Association for Computational Linguistics. find our card
  13. Yangfeng Ji and Noah A. Smith. 2017. Neural discourse structure for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996–1005, Vancouver, Canada. Association for Computational Linguistics. find
  14. Shafiq Joty, Giuseppe Carenini, Raymond Ng, and Yashar Mehdad. 2013. Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 486–496, Sofi… find
  15. Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top-down rst parsing utilizing granularity levels in documents. In Association for the Advancement of Artificial Intelligence 2020, AAAI2020. find
  16. Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021. Top-down discourse parsing via sequence labelling. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 715–726, Online. Association for Computational Linguistics. find our card
  17. Jiwei Li, Rumeng Li, and Eduard Hovy. 2014a. Recursive deep models for discourse parsing. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2061– 2069, Doha, Qatar. Association for Computational Linguistics. find our card
  18. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 362–371, Austin, Texas. Association for Computational Linguistics. find our card
  19. Yancui Li, wenhe Feng, jing Sun, Fang Kong, and Guodong Zhou. 2014b. Building chinese discourse corpus with connective-driven dependency tree structure. In Proceedings of EMNLP 2014, pages 2105– 2114. find
  20. Xiang Lin, Shafiq Joty, Prathyusha Jwalapuram, and M Saiful Bari. 2019. A unified linear-time framework for sentence-level discourse parsing. In ACL, pages 4190–4200, Florence, Italy. Association for Computational Linguistics. find our card
  21. Linlin Liu, Xiang Lin, Shafiq Joty, Simeng Han, and Lidong Bing. 2019. Hierarchical pointer net parsing. In EMNLP-IJCNLP, pages 1006–1016, Hong Kong, China. Association for Computational Linguistics. find
  22. Amandla Mabona, Laura Rimell, Stephen Clark, and Andreas Vlachos. 2019. Neural generative rhetorical structure parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),… find our card
  23. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. find our card
  24. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, Baltimore, Ma… find our card
  25. Kenji Sagae and Alon Lavie. 2005. A classifier-based parser with linear run-time complexity. In Proceedings of the Ninth International Workshop on Parsing Technology, pages 125–132, Vancouver, British Columbia. Association for Computational Linguistics. find our card
  26. Ke Shi, Zhengyuan Liu, and Nancy F. Chen. 2020. An end-to-end document-level neural discourse parser exploiting multi-granularity representations. CoRR, abs/2012.11169. — [arXiv:2012.11169](https://arxiv.org/abs/2012.11169) find
  27. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 184–188, Vancouver, Canada. Association for Computational Linguist… find our card
  28. X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley. 2017. Least squares generative adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2813–2821. find
  29. Jiawei Wu, Xin Wang, and William Yang Wang. 2019. Self-supervised dialogue learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3857–3867, Florence, Italy. Association for Computational Linguistics. find
  30. Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge, MA, USA. find
  31. Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-aware neural extractive text summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5021–5031, Online. Association for Computational Linguistics. find our card
  32. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324, Copenhagen, Denmark. Ass… find our card
  33. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  34. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  35. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In LREC 2008. find our card
  36. Yuanyuan Qiu, Hongzheng Li, Shen Li, Yingdi Jiang, Renfen Hu, and Lijiao Yang. 2018. Revisiting correlations between intrinsic and extrinsic evaluations of word embeddings. In CCL & NLP-NABD 2017, pages 209–221. Springer. find
  37. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. find our card
  38. Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural RST parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559–570, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  39. Longyin Zhang, Yuqing Xing, Fang Kong, Peifeng Li, and Guodong Zhou. 2020. A top-down neural architecture towards text-level parsing of discourse rhetorical structure. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6386–6395, Online. Association for… find our card
  40. Wei Zou, Shujian Huang, Jun Xie, Xinyu Dai, and Jiajun Chen. 2020. A reinforced generation of adversarial examples for neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3486–3497, Online. Association for Computational Lingui… find

↑ back to top