Elamur Research⌕
☰
home / papers / 2015 / Machine Comprehension with Discourse Relations

Machine Comprehension with Discourse Relations

Summary

Answering questions about a short story often means linking several sentences, such as recognizing that one event caused another. Earlier systems drew discourse information from off-the-shelf parsers trained on newswire treebanks. Those parsers' relation inventories and text genre may not suit comprehension. Narasimhan and Barzilay instead learn inter-sentence relations directly from the comprehension objective. They build a discriminative log-linear model with hidden variables. Given a question, it picks one or two relevant sentences, infers a distribution over relation types between them, and scores the answer options. Gold answers are the only supervision. A few seed discourse markers and question-type features push the relations toward linguistically sensible ones, and training uses standard gradient descent. On MCTest, where roughly half the questions need more than one sentence, the relation-aware model beats the best individual published systems by up to 5.7%. It is also competitive with a combination system. Its induced relations overlap considerably with human judgments, which suggests task-driven discourse modeling can replace annotation-heavy parser pipelines.

Key points

  • Three models of increasing complexity are compared:
    • Model 1 uses a single hidden sentence.
    • Model 2 uses sentence pairs within a window k.
    • Model 3 adds a hidden relation variable over Causal, Temporal, Explanation and Other.
  • Evaluation is on MCTest (Richardson et al., 2013). The authors had the MC160 test set manually annotated with relevant sentences and relations for analysis.
  • Features from an off-the-shelf discourse parser did not improve the model.
  • For sentence retrieval, Model 3 ranks the annotated sentence first about 51% of the time for single-sentence questions, and in the top 5 about 80% of the time. For multi-sentence questions these figures drop to about 25% and 50%.
  • Model 3's top predicted relation matches human annotation 51.45% of the time (R@2 is 65.04%), with Causal and Other predicted best. Code and data are released at people.csail.mit.edu/karthikn/mcdr.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 20 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1121

@inproceedings{narasimhan2015machine,
  title = {{Machine Comprehension with Discourse Relations}},
  author = {Narasimhan, Karthik and Barzilay, Regina},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1121},
  url = {https://doi.org/10.3115/v1/p15-1121}
}

References

22 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, and Christopher D. Manning. 2014. Modeling biological processes for reading comprehension. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP),… find
  2. Phil Blunsom, Krystle Kocik, and James R Curran. 2006. Question classification with log-linear models. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 615–616. ACM. find
  3. Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. 1995. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5):1190– 1208. find
  4. Lynn Carlson, Mary Ellen Okurowski, Daniel Marcu, Linguistic Data Consortium, et al. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania. find
  5. Joyce Y Chai and Rong Jin. 2004. Discourse structure for context question answering. In Proceedings of the Workshop on Pragmatics of Question Answering at HLT-NAACL, volume 2004, pages 23–30. Citeseer. find
  6. George Corliss. 2002. Automatic differentiation of algorithms: from simulation to optimization, volume 1. Springer Science & Business Media. find
  7. Vanessa Wei Feng and Graeme Hirst. 2012. Text-level discourse parsing with rich linguistic features. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 60–68. Association for Computational Linguistics. find our card
  8. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511–521, Baltimore, Maryland, June. Association for Computati… find our card
  9. Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daumé III. 2014. A neural network for factoid question answering over paragraphs. In Empirical Methods in Natural Language Processing. find
  10. Peter Jansen, Mihai Surdeanu, and Peter Clark. 2014. Discourse complements lexical semantics for non-factoid answer reranking. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 977–986, Baltimore, Maryland, June. Association for… find
  11. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 343–351. Association for Computational Linguistics. find our card
  12. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A pdtb-styled end-to-end discourse parser. Natural Language Engineering, pages 1–34. find
  13. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281. find our card
  14. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. find our card
  15. Daniel Marcu. 1997. The Rhetorical Parsing, Summarization and Generation of Natural Language Texts. Ph.D. thesis, University of Toronto. find
  16. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 108– 112. Association for Computational Linguistics. find our card
  17. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind K Joshi. 2008. Easily identifiable discourse relations. find our card
  18. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The penn discourse treebank 2.0. In LREC. Citeseer. find our card
  19. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, pages 193–203. find our card
  20. Asher Stern and Ido Dagan. 2011. A confidence model for syntactically-motivated entailment proofs. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 455–462, Hissar, Bulgaria, September. RANLP 2011 Organising Committee. find
  21. Suzan Verberne, Lou Boves, Nelleke Oostdijk, and Peter-Arno Coppen. 2007. Evaluating discourse-based answer extraction for why-question answering. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 735–736. ACM. find
  22. Florian Wolf and Edward Gibson. 2005. Representing discourse coherence: A corpus-based study. Computational Linguistics, 31(2):249–287. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/narasimhan-2015-machine-comprehension-discourse-relations/