Elamur Research⌕
☰
home / papers / 2014 / Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification

Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification

Summary

This COLING 2014 paper presents the first experiments on recognizing implicit discourse relations in French, meaning relations not signalled by an explicit connective. Manually annotated data is scarce: the ANNODIS corpus has roughly 3,000 examples, compared with about 40,000 in the PDTB. French also lacks lexical resources such as polarity databases. To compensate, the authors follow Marcu and Echihabi (2002). They take explicit examples marked by unambiguous connectives, delete the connective, and use the result as artificially labelled implicit data. A classifier trained only on these artificial examples performs poorly on natural implicit relations, which confirms the earlier English finding of Sporleder and Lascarides (2008). The authors trace this to distribution mismatches between the two kinds of data, including prior probability shift and covariate shift. They then test simple combination schemes borrowed from domain adaptation, plus automatic filtering of noisy artificial examples. The best setup feeds the artificial model's predicted relation into the natural-data classifier as an extra feature. It reaches 41.7% accuracy, 4.4 points above a model trained only on manual data. The work shows that cheap artificial data helps only when integrated carefully.

Key points

  • Data: the French ANNODIS corpus, with four relations evaluated (contrast, result, continuation, explanation), plus artificial examples extracted from raw text using unambiguous connectives.
  • Combination methods were inspired by domain adaptation (Daumé III and Marcu, 2006). They include training on the union of both datasets, weighting natural examples (NATW), subsampling artificial data (ARTSUB), linear interpolation of the two models (LININT), and adding the artificial model's prediction as a feature (ADDPRED).
  • Automatic selection of artificial examples raised ARTONLY accuracy from 23.0 to 25.0 and UNION from 22.6 to 40.1.
  • The best system, ADDPRED+SELEC, reached 41.7% accuracy. By class, it obtained an F1 of 20.8 on contrast, with result at 51.0, continuation at 31.2 and explanation at 53.9.
  • Contrast and continuation gained most from artificial data; contrast's best F1 was 28.8, obtained with ARTONLY+SELEC. Heavy reliance on artificial data hurt result and explanation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{braud2014combining,
  title = {{Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification}},
  author = {Braud, Chloé and Denis, Pascal},
  booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers},
  year = {2014},
  url = {https://aclanthology.org/C14-1160/}
}

References

35 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stergos Afantenos, Nicholas Asher, Farah Benamara, Myriam Bras, Cécile Fabre, Lydia-Mai Ho-Dac, Anne Le Draoulec, Philippe Muller, Marie-Paule Péry-Woodley, Laurent Prévot, Josette Rebeyrolles, Ludovic Tanguy, Marianne Vergez-Couret, and Laure Vieu. 2012. An empirical resource for discovering cognit… find
  2. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. find
  3. Sasha Blair-Goldensohn, Kathleen R. McKeown, and Owen C. Rambow. 2007. Building and refining rhetorical-semantic relation models. In Proceedings of NAACL HLT. find
  4. Marie Candito, Joakim Nivre, Pascal Denis, and Enrique Henestroza Anguiano. 2010. Benchmarking of statistical dependency parsers for french. In Proceedings of ICCL (posters). find
  5. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. find our card
  6. Hal Daumé III and Daniel Marcu. 2006. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research. find
  7. Hal Daumé III. 2007. Frustratingly easy domain adaptation. In Proceedings of ACL. find
  8. Joost C.F. de Winter. 2013. Using the student’s t-test with extremely small sample sizes. Practical Assessment, Research & Evaluation. find
  9. Pascal Denis and Benoît Sagot. 2009. Coupling an annotated corpus and a morphosyntactic lexicon for state-ofthe-art POS tagging with less human effort. In Proceedings of PACLIC. find
  10. Anna Gastel, Sabrina Schulze, Yannick Versley, and Erhard Hinrichs. 2011. Annotation of implicit discourse relations in the tüba-d/z treebank. GSCL. find
  11. David J. Hand. 2006. Classifier technology and the illusion of progress. Statistical Science. find
  12. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue and Discourse. find our card
  13. Jing Jiang. 2008. A literature survey on domain adaptation of statistical classifiers. http://sifaka.cs.uiuc.edu/jiang4/domain_adaptation/survey/da_survey.html. ↗
  14. Shafiq R. Joty, Giuseppe Carenini, Raymond T. Ng, and Yashar Mehdad. 2013. Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis. In Proceedings of ACL. find
  15. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of EMNLP. find our card
  16. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2010. A PDTB-styled end-to-end discourse parser. Technical report, National University of Singapore. find
  17. Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. 2009. Domain adaptation: Learning bounds and algorithms. In Proceedings of COLT. find
  18. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of ACL. find our card
  19. Ryan McDonald and Fernando Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of EACL. find our card
  20. Jose G. Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez, Nitesh V. Chawla, and Francisco Herrera. 2012. A unifying view on dataset shift in classification. Pattern Recognition. find
  21. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of SIGDIAL. find our card
  22. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP (Short Papers). find our card
  23. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of ACL-IJCNLP. find our card
  24. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In Proceedings of LREC. find our card
  25. Charlotte Roze, Laurence Danlos, and Philippe Muller. 2012. Lexconn: A french lexicon of discourse connectives. Discours. find our card
  26. Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. Proceedings of IWPT. find
  27. Available from: Tobias Scheffer. 1999. Error Estimation and Model Selection. Ph.D. thesis, Technischen Universitet Berlin, School of Computer Science. find
  28. Anders Sogaard. 2013. Semi-supervised learning and domain adaptation in natural language processing. Morgan & Claypool. find
  29. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations, an assessment. Natural Language Engineering. find
  30. Caroline Sporleder. 2008. Lexical models to identify unmarked discourse relations: Does Wordnet help? Lexical-Semantic Resources in Automated Discourse Analysis. find
  31. Sudhir Varma and Richard Simon. 2006. Bias in error estimation when using cross-validation for model selection. BMC bioinformatics. find
  32. Yannick Versley. 2013. Subgraph-based classification of explicit and implicit discourse relations. In Proceedings of IWCS. find
  33. WenTing Wang, Jian Su, and Chew Lim Tan. 2010. Kernel based discourse relation recognition with temporal ordering information. In Proceedings of ACL. find our card
  34. Xun Wang, Sujian Li, Jiwei Li, and Wenjie Li. 2012. Implicit discourse relation recognition by selecting typical training examples. In Proceedings of COLING (Technical Papers). find our card
  35. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of COLING (Posters). find our card

↑ back to top

This page in Markdown →