Elamur Research⌕
☰
home / papers / 2016 / Learning Connective-based Word Representations for Implicit Discourse Relation Identification

Learning Connective-based Word Representations for Implicit Discourse Relation Identification

Summary

Implicit discourse relations lack an overt connective such as but, so they are much harder to classify than explicit ones. Braud and Denis ask whether word representations designed for this rhetorical task can beat generic embeddings. They use a large amount of automatically detected explicit connectives and their arguments. From this they build a simple distributional vector space in which each word is described by how often it co-occurs with each connective. The counts are reweighted with TF-IDF or PPMI-IDF. The vectors of the two discourse segments are combined either by concatenation or by a Kronecker product and passed to a classifier trained only on implicit examples from the Penn Discourse Treebank (PDTB). In multiclass level-1 experiments, these representations (called Bllip) match or outperform Brown clusters and off-the-shelf embeddings. They also reach state-of-the-art macro-F1 without hand-crafted features. The method is cheap to train, needs no lexical databases and should transfer to resource-poor languages wherever connectives can be reliably detected in raw text.

Key points

  • The best configuration, Bllip PPMI-IDF with concatenation, reaches a macro-F1 of 42.2. Adding traditional features raises it to 42.8.
  • For comparison:
    • Rutherford & Xue (2015) report a macro-F1 of 40.5 but a higher accuracy (57.1 vs. 52.5).
    • The best pre-trained embedding scores 41.6.
    • One-hot tokens score 39.0 to 40.2.
  • The best Bllip vectors keep all 96 dimensions, one per connective found in the raw data, with no dimensionality reduction. Concatenation generally beat the Kronecker product.
  • Hand-crafted features add little overall, but they substantially help the Comparison relation. The connective vectors alone give the best results for Temporal and Contingency.
  • Explicit data are used only to build the representations, not to train the classifier. This avoids distribution-mismatch and sample-selection problems and shortens training time. The resources are released at bitbucket.org/chloebt/discourse-data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

20 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 13 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1020

@inproceedings{braud2016learning,
  title = {{Learning Connective-based Word Representations for Implicit Discourse Relation Identification}},
  author = {Braud, Chloé and Denis, Pascal},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1020},
  url = {https://doi.org/10.18653/v1/d16-1020}
}

References

37 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. find
  2. Cambridge University Press. Or Biran and Kathleen McKeown. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of ACL. Chloé Braud and Pascal Denis. 2014. Combining natural and artificial examples to improve implicit discourse relation identification.… find our card
  3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. find our card
  4. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML. find
  5. Juliette Conrath, Stergos Afantenos, Nicholas Asher, and Philippe Muller. 2014. Unsupervised extraction of semantic relations using discourse cues. In Proceedings of Coling. find
  6. Zellig S. Harris. 1954. Distributional structure. Word, 10(23):146–162. find
  7. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of ACL. find our card
  8. Yangfeng Ji, Gongbo Zhang, and Jacob Eisenstein. 2015. Closing the gap: Domain adaptation from explicit to implicit discourse relations. In Proceedings of EMNLP. find our card
  9. Anders Johannsen and Anders Sgaard. 2013. Disambiguating explicit discourse connectives without oracles. In Proceedings of IJCNLP. find our card
  10. Ian Jolliffe. 2002. Principal component analysis. find
  11. Wiley Online Library. Man Lan, Yu Xu, and Zhengyu Niu. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In Proceedings of ACL. Rémi Lebret and Ronan Collobert. 2014. Word emded-dings through Hellinger PCA. In Proceedings of ACL. find our card
  12. Junyi Jessy Li and Ani Nenkova. 2014. Reducing sparsity improves the recognition of implicit discourse relations. In Proceedings of SIGDIAL. find our card
  13. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of EMNLP. find our card
  14. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2010. A PDTB-styled end-to-end discourse parser. Technical report, National University of Singapore. find
  15. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2011. Automatically evaluating text coherence using discourse relations. In Proceedings of ACL-HLT. find our card
  16. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A PDTB-styled end-to-end discourse parser. Natural Language Engineering, 20:151–184. find
  17. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of ACL. find our card
  18. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  19. Andriy Mnih and Geoffrey Hinton. 2007. Three new graphical models for statistical language modelling. In Proceedings of ICML. find
  20. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of SIGDIAL Conference. find our card
  21. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP. find our card
  22. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of ACL-IJCNLP. find our card
  23. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of LREC. find our card
  24. Attapol Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through Brown cluster pair representation and coreference patterns. In Proceedings of EACL. find our card
  25. Attapol Rutherford and Nianwen Xue. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In Proceedings of NAACL-HLT. find our card
  26. Claudia Soria and Giacomo Ferrari. 1998. Lexical marking of discourse relations - some experimental findings. In Proceedings of the ACL Workshop on Discourse Relations and Discourse Markers. find our card
  27. Caroline Sporleder and Alex Lascarides. 2005. Exploiting linguistic cues to classify rhetorical relations. In Proceedings of RANLP-05. find
  28. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering, 14:369–416. find
  29. Philip J. Stone and John Kirsh. 1966. The General Inquirer: A Computer Approach to Content Analysis. find
  30. MIT Press. Rajen Subba and Barbara Di Eugenio. 2009. An effective discourse parser that uses rich linguistic information. In Proceedings of ACL-HLT. find our card
  31. Maite Taboada. 2006. Discourse markers as signals (or not) of rhetorical relations. Journal of Pragmatics, 38:567–592. find
  32. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of ACL. find our card
  33. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning : Vector space models of semantics. Journal of Artificial Intelligence Research, pages 141–188. find
  34. Yannick Versley and Anna Gastel. 2013. Linguistic tests for discourse relations in the TüBa-D/Z corpus of written German. Dialogue & Discourse, 4(2):142–173. find our card
  35. Xun Wang, Sujian Li, Jiwei Li, and Wenjie Li. 2012. Implicit discourse relation recognition by selecting typical training examples. In Proceedings of COLING 2012: Technical Papers. find our card
  36. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of HLT-EMNLP. find
  37. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of CoNLL. find our card

↑ back to top