Elamur Research
☰
home / papers / 2009 / Automatic sense prediction for implicit discourse relations in text

Automatic sense prediction for implicit discourse relations in text

What it is about

The paper predicts the sense (Comparison, Contingency, Expansion, Temporal) of implicit discourse relations, meaning adjacent sentences with no connective, in PDTB newswire. It tests whether word-pair features from earlier work help. Word pairs from artificial implicit data (explicit connectives removed) learn the connectives that were removed, and their top pairs are function-word pairs such as "but-but", so they do not capture semantic opposition. The authors add polarity tags, Levin verb classes, first and last words, modality, number features and context (a neighbouring explicit relation or a paragraph boundary). The best model reaches F about 47 for Contingency (16 points over the word-pair baseline) and about 76 for Expansion. Evaluation uses the natural class distribution, with sections 2-20 for training and 21-22 for testing.

Ideas

For Mova

The chapter is languages/en/36-discourse-relations.md, section 7 (implicit relations). The code is the link labels in prag and the word-pair and polarity features would live in a new module next to them. The paper's datasets (PDTB, TextRels, the Wilson polarity lexicon and the General Inquirer) need a licence check before use; the paper's own experiments are research-only, so use them as a reference, not as training data in a release.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/P09-1077.pdf

Summary

The paper looks at how to recognise the sense of implicit discourse relations. These are links between adjacent sentences that have no connective such as "but" or "because" to signal them. Using the Penn Discourse Treebank (PDTB), the authors predict the four top-level senses (Comparison, Contingency, Expansion, Temporal). Each sense gets its own one-against-the-rest classifier, tested on the natural, skewed distribution of senses. Earlier work used balanced or synthetic data instead. The authors first examine word-pair features drawn from TextRels, a set of explicit examples taken from Gigaword with the connective removed. They find that the most informative pairs are mostly function words and connectives, not the semantically opposed content words earlier work assumed. They then build linguistically motivated features: polarity tags, General Inquirer categories, Levin verb classes, verb-phrase length, modality, context, and the first and last words of each sentence. Together these features clearly beat the word-pair baseline, most of all for Contingency. The study set a realistic way to evaluate implicit relations and explained why cheap training data built from explicit relations transfers poorly.

Key points

  • Explicit relations are kept separate because the connective alone gives about 93% accuracy on the top-level senses. That makes implicit relations the real open problem.
  • Ranking TextRels word pairs by information gain (with MALLET) mostly surfaces stop-word and connective pairs. Word pairs containing "but" actually point to a relation other than Contrast.
  • Word pairs work best when they are selected on TextRels and their weights are estimated on PDTB implicit examples ("Wordpairs-selected"). This beats training on TextRels by about 4 F-score points for Comparison and 14 for Contingency. Training on PDTB explicit relations is also weaker than training on implicit ones.
  • Best F-scores with Naive Bayes: Contingency 47.13 (baseline 31.10, +16), Comparison 21.96 (baseline 17.13), Expansion 76.42 (word pairs 63.84), Temporal 16.76.
  • Context helps: paragraph position and nearby explicit relations are useful features. A CRF sequence model over a document's implicit relations reaches 44.58% 6-way accuracy, against 43.27% for Naive Bayes.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1690219.1690241

@inproceedings{pitler2009automatic,
  title = {{Automatic sense prediction for implicit discourse relations in text}},
  author = {Pitler, Emily and Louis, Annie and Nenkova, Ani},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  doi = {10.3115/1690219.1690241},
  url = {https://doi.org/10.3115/1690219.1690241}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. N. Asher and A. Lascarides. 2003. Logics of conversation. Cambridge University Press. find
  2. S. Blair-Goldensohn, K.R. McKeown, and O.C. Rambow. 2007. Building and Refining Rhetorical-Semantic Relation Models. In Proceedings of NAACL HLT, pages 428–435. find
  3. L. Carlson, D. Marcu, and M.E. Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue, pages 1–10. find our card
  4. B.J. Dorr. 2001. LCS Verb Database. Technical Report Online Software Database, University of Maryland, College Park, MD. find
  5. Y. Freund and R.E. Schapire. 1996. Experiments with a New Boosting Algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148–156. find
  6. R. Girju. 2003. Automatic detection of causal relations for Question Answering. In Proceedings of the ACL 2003 workshop on Multilingual summarization and question answering-Volume 12, pages 76–83. find
  7. D. Graff. 2003. English gigaword corpus. Corpus number LDC2003T05, Linguistic Data Consortium, Philadelphia. find
  8. J. Hobbs. 1979. Coherence and coreference. Cognitive Science, 3:67–90. find
  9. A. Knott and T. Sanders. 1998. The classification of coherence relations and their linguistic markers: An exploration of two languages. Journal of Pragmatics, 30(2):135–175. find
  10. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In International Conference on Machine Learning 2001, pages 282–289. find
  11. M. Lapata and A. Lascarides. 2004. sentence-internal temporal relations. NAACL 2004: Main Proceedings. Inferring In HLT-B. find
  12. Levin. 1993. English Verb Classes and Alternations: A Preliminary Investigation. Chicago, IL. find
  13. W.C. Mann and S.A. Thompson. 1988. Rhetorical structure theory: Towards a functional theory of text organization. Text, 8. find
  14. D. Marcu and A. Echihabi. 2001. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 368–375. find our card
  15. D. Marcu. 2000. The Theory and Practice of Discourse and Summarization. The MIT Press. find
  16. K. McKeown. 1985. Text Generation: Using Discourse strategies and Focus Constraints to Generate Natural Language Text. Cambridge University Press, Cambridge, England. find
  17. E. Pitler, M. Raghupathy, H. Mehta, A. Nenkova, A. Lee, and A. Joshi. 2008. Easily identifiable discourse relations. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING08), short paper. find
  18. R. Soricut and D. Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In HLT-NAACL. find our card
  19. C. Sporleder and A. Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering, 14:369–416. find
  20. P.J. Stone, J. Kirsh, and Cambridge Computer Associates. 1966. The General Inquirer: A Computer Approach to Content Analysis. MIT Press. find
  21. B. Wellner, J. Pustejovsky, C. Havasi, A. Rumshisky, and R. Sauri. 2006. Classification of discourse coherence relations: An exploratory study using multiple knowledge sources. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue. find
  22. T. Wilson, J. Wiebe, and P. Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 347–354. find
  23. F. Wolf and E. Gibson. 2005. Representing discourse coherence: A corpus-based study. Computational Linguistics, 31(2):249–288. find