Elamur Research⌕
☰
home / papers / 2015 / The UniTN Discourse Parser in CoNLL 2015 Shared Task: Token-level Sequence Labeling with Argument-specific Models

The UniTN Discourse Parser in CoNLL 2015 Shared Task: Token-level Sequence Labeling with Argument-specific Models

Summary

The paper presents the University of Trento (UniTN) system submitted to the 2015 CoNLL shared task, which focused on shallow parsing of discourse. The task is to find Penn Discourse Treebank (PDTB) style relations end to end: connectives, Argument 1 and Argument 2 spans, and senses, for both explicit and non-explicit (Implicit, AltLex, EntRel) relations. The authors build a pipeline that mirrors the PDTB annotation process. Span-finding steps for explicit relations, connective detection and argument extraction, are modeled as token-level sequence labeling with CRFs (CRF++). The argument models are specialized: separate models handle arguments of same-sentence (SS) and previous-sentence (PS) relations, and the SS models run in a cascade with Arg2 before Arg1. All other decisions use AdaBoost classifiers (icsiboost) with lexical, syntactic, dependency and Brown Cluster features. These decisions are connective sense, argument position, non-explicit relation detection, and sense. In the closed track, the parser placed second on the blind test set with a global F-measure of 0.2184. This suggests that token-level labeling is a workable strategy for explicit argument extraction.

Key points

  • The system was trained and evaluated on the PDTB data supplied by the organizers, including POS tags and constituency and dependency parses. Testing used WSJ Section 22 (dev), WSJ Section 23 (test), and a separately annotated blind test set.
  • On the blind test set, the parser reached a global F1 of 0.2184 (P 0.2094, R 0.2283), ranking 2nd. The top system, lan15, scored 0.2400, and the third-place li15b scored 0.1851. The authors note that their parser tends to have higher recall than other entries.
  • Explicit relations were handled much better than non-explicit ones (blind parser-level F1 0.2997 vs 0.1577). Non-explicit sense classification was the weakest component, with a macro-average of about 0.07.
  • Argument position classification reached 0.9868 accuracy on dev when given gold connectives. The PS Arg1 step relies on the Previous Sentence Heuristic, even though about 9% of PS Arg1s are not in the adjacent sentence.
  • Connective detection scored 0.8992 F1 on the blind set but ranked only 4th among participants. Because it is the first stage of the pipeline, the authors identify its robustness as a priority for improvement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k15-2003

@inproceedings{stepanov2015unitn,
  title = {{The UniTN Discourse Parser in CoNLL 2015 Shared Task: Token-level Sequence Labeling with Argument-specific Models}},
  author = {Stepanov, Evgeny and Riccardi, Giuseppe and Bayer, Ali Orkan},
  booktitle = {Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task},
  year = {2015},
  doi = {10.18653/v1/k15-2003},
  url = {https://doi.org/10.18653/v1/k15-2003}
}

References

17 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sabine Buchholz. 2000. chunklink.pl. http://ilk.uvt.nl/software/. ↗
  2. Benoit Favre, Dilek Hakkani-Tür, and Sebastien Cuen-det. 2007. Icsiboost. https://github.com/benob/icsiboost/. ↗
  3. Yoav Freund and Robert E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, August. find
  4. Sucheta Ghosh, Richard Johansson, Giuseppe Riccardi, and Sara Tonelli. 2011. Shallow discourse parsing with conditional random fields. In Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP 2011). find our card
  5. Taku Kudo. 2013. CRF++. http://taku910.github.io/crfpp/. ↗
  6. John Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of 18th International Conference on Machine Learning, pages 282–289. find
  7. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A PDTB-styled end-to-end discourse parser. Natural Language Engineering, 20:151 – 184. find
  8. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP Conference, pages 13–16. find our card
  9. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 683–691. find our card
  10. Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, Livio Robaldo, and Bonnie L Webber. 2007. The Penn Discourse Treebank 2.0 annotation manual. find
  11. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008). find our card
  12. Attapol T. Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through Brown Cluster pair representation and coreference patterns. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2014). find our card
  13. Evgeny A. Stepanov and Giuseppe Riccardi. 2013. Comparative evaluation of argument extraction algorithms in discourse relation parsing. In Proceedings of the 13th International Conference on Parsing Technologies (IWPT 2013), pages 36–44, Nara, Japan, November. find our card
  14. Evgeny A. Stepanov and Giuseppe Riccardi. 2014. Towards cross-domain PDTB-style discourse parsing. In EACL Workshops - Proceedings of the Louhi 2014: The Fifth International Workshop on Health Text Mining and Information Analysis, pages 30–37, Gothenburg, Sweden, April. ACL. find our card
  15. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semisupervised learning. In In ACL, pages 384– 394. find
  16. Bonnie L. Webber, Markus Egg, and Valia Kordoni. 2011. Discourse structure and language technology. Natural Language Engineering, pages 1–54. find
  17. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 Shared Task on Shallow Discourse Parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning: Shared Task. find our card

↑ back to top

This page in Markdown →