Elamur Research⌕
☰
home / papers / 2014 / Dependency-based Discourse Parser for Single-Document Summarization

Dependency-based Discourse Parser for Single-Document Summarization

Summary

Discourse-aware extractive summarization depends on accurate discourse trees, and automatic RST parsers produce errors that lower summary quality. The strongest prior single-document method (Hirao et al., 2013) frames summarization in terms of a Tree Knapsack Problem (TKP) defined on a dependency-based discourse tree (DEP-DT). In that method the DEP-DT is converted heuristically from an automatically parsed RST-DT, so a single nucleus/satellite mistake can reshape the dependency tree and lose the most salient EDU.

The authors skip the RST step. They convert gold RST-DTs into DEP-DTs and train a discourse dependency parser on them directly. The parser uses a first-order MST algorithm, MIRA training and HILDA-style features. They also test a loss weighted by EDU depth, so that errors near the root count more, since the TKP mostly keeps EDUs close to the root.

On the RST-DT corpus, TKP with the new parser beats TKP with HILDA and comes close to TKP with gold DEP-DTs. This suggests the parser can serve as a practical replacement for RST parsing in summarization.

Key points

  • Data split: the RST-DT corpus (Carlson et al., 2002) has 385 Wall Street Journal articles. The parser is trained on 355 of them. Summarization is tested on the 30 that have human reference summaries, with summaries limited to about 10% of the source length.

  • ROUGE recall scores:

    SystemROUGE-1ROUGE-2
    TKP-DIS-DEP-LOSS0.3230.121
    TKP-DIS-DEP0.3190.109
    TKP-GOLD0.3210.112
    TKP-HILDA0.2840.093

  • Statistical significance: the ROUGE gains over HILDA appear not to be statistically significant, which the authors attribute to the small test set. The extracted passage on the Wilcoxon test is garbled, so this point should be checked against the original PDF.

  • Tree-quality metrics (both proposed variants vs. TKP-HILDA): EDU overlap with gold-tree summaries, as average F-value, was 0.532 vs 0.415. Root accuracy was 0.933 vs 0.733. Dependency Accuracy in Summary (DAS) was 0.847 and 0.843 vs 0.596. Table 2 marks these gains as significant at p < .05.

  • Loss function: the depth-weighted loss L_Depth gave the best ROUGE-2 score. It did not improve root accuracy or overlap compared with the standard loss, which counts EDUs with an incorrect parent.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

39 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1196

@inproceedings{yoshida2014dependency,
  title = {{Dependency-based Discourse Parser for Single-Document Summarization}},
  author = {Yoshida, Yasuhisa and Suzuki, Jun and Hirao, Tsutomu and Nagata, Masaaki},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1196},
  url = {https://doi.org/10.3115/v1/d14-1196}
}

References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2002. Rst discourse treebank, ldc2002t07. find
  2. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive-aggressive algorithms. The Journal of Machine Learning Research, 7:551–585. find
  3. Hal Daumé III and Daniel Marcu. 2002. A noisy-channel model for document compression. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 449 – 456, Philadelphia, PA, July 6 – 12. find
  4. Hugo Hernault, Helmut Prendinger, Mitsuru Ishizuka, et al. 2010. Hilda: a discourse parser using support vector machine classification. Dialogue and Discourse, 1(3). find our card
  5. Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda, and Masaaki Nagata. 2013. Single-document summarization as a tree knapsack problem. In Proceedings of the 2013 Conference on EMNLP, pages 1515–1520. find
  6. Yuta Kikuchi, Tsutomu Hirao, Hiroya Takamura, Manabu Okumura, and Masaaki Nagata. 2014. Single document summarization based on nested tree structure. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 315–320, Baltimore, Marylan… find
  7. Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014. Text-level discourse dependency parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25–35, Baltimore, Maryland, June. Association for Computational Linguistics. find
  8. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281. find our card
  9. Daniel Marcu. 1998. Improving summarization through rhetorical parsing tuning. In Proc. of The 6th Workshop on VLC, pages 206–215. find
  10. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005a. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ’05, pages 91–98, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  11. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajic. 2005b. Non-projective dependency parsing using spanning tree algorithms. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 523–530, Vancouver, British Columbi… find our card
  12. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. find our card
  13. Frank Wilcoxon. 1945. Individual Comparisons by Ranking Methods. Biometrics Bulletin, 1(6):80–83, December. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yoshida-2014-dependency-based-discourse-parser-single/