Elamur Research⌕
☰
home / papers / 2014 / Reducing Sparsity Improves the Recognition of Implicit Discourse Relations

Reducing Sparsity Improves the Recognition of Implicit Discourse Relations

Summary

Li and Nenkova investigate the effect of feature sparsity on recognizing implicit relations between sentences, using the Penn Discourse Treebank (PDTB) as their data. Implicit relations link adjacent sentences without an explicit connective. They focus on the two strongest feature families from earlier work: lexical word pairs and syntactic production rules. Using only lexical and syntactic information from gold Penn Treebank parses, they train binary SVM classifiers for seven relation classes.

They show that how features are encoded matters. Recording which words occur in each argument, instead of taking the cross-product of words across arguments, yields far fewer and denser features and better results. This explains part of the advantage earlier credited to syntax alone. They also propose a new, denser syntactic representation called production sticks, which significantly improves recognition. Removing lexical content from syntactic features lowers performance for most relations, so word-level information remains essential. Agreement analysis shows that different lexical encodings make quite different errors. This suggests that combining them in future discourse parsers could help.

Key points

  • Setup: Training uses PDTB sections 2–19 and testing uses the enlarged sections 20–24. The classifiers are SVMLight with a linear kernel, with a balanced (downsampled) negative class.
    • Seven relations are covered: Comparison, Contingency, Expansion, Temporal, and the Expansion subtypes Conjunction, Instantiation and Restatement.
  • Lexical features: Word pairs produce 92,128 features, compared with 12,116 for binary-lexical. Moving to binary-lexical raises average F from 29.46 to 31.79 and accuracy from 57.22 to 60.42. Instantiation gains 7.7 F points.
  • Sparsity: The average rate of feature occurrence is:
    • sticks: 0.00623
    • prodrules: 0.00374
    • binary-lexical: 0.00276
    • word-pairs: 0.00113
  • Lexicalization: A purely syntactic variant of the sticks representation (sticks-nolex) performs worse on most relations. Lexical items also make up many of the most predictive features.
  • Classifier agreement: Q-statistic analysis shows that word-pairs and binary-lexical disagree on at least about 25% of test instances. Sticks and prodrules, by contrast, agree closely.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-4327

@inproceedings{li2014reducing,
  title = {{Reducing Sparsity Improves the Recognition of Implicit Discourse Relations}},
  author = {Li, Junyi Jessy and Nenkova, Ani},
  booktitle = {Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)},
  year = {2014},
  doi = {10.3115/v1/w14-4327},
  url = {https://doi.org/10.3115/v1/w14-4327}
}

References

14 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Or Biran and Kathleen McKeown. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL): Short Papers, pages 69–73. find our card
  2. Sasha Blair-Goldensohn, Kathleen McKeown, and Owen Rambow. 2007. Building and refining rhetorical-semantic relation models. In Human Language Technologies: The Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 428–435. find
  3. Hugo Hernault, Danushka Bollegala, and Mitsuru Ishizuka. 2010. A semi-supervised approach to improve classification of infrequent discourse relations using feature vector extension. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 399–409. find our card
  4. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Advances in kernel methods, pages 169–184. find
  5. Ludmila I. Kuncheva and Christopher J. Whitaker. 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51(2):181–207, May. find
  6. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 343–351. find our card
  7. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A PDTB-styled end-to-end discourse parser. Natural Language Engineering, 20:151–184, 4. find
  8. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL), pages 368–375. find our card
  9. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics - Special issue on using large corpora, 19(2):313–330. find our card
  10. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 108–112. find our card
  11. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-IJCNLP), pag… find our card
  12. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC). find our card
  13. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering, 14(3):369–416, July. find
  14. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pages 1507–1514. find our card

↑ back to top