Elamur Research⌕
☰
home / papers / 2019 / Zero-shot transfer for implicit discourse relation classification

Zero-shot transfer for implicit discourse relation classification

Summary

Implicit discourse relations link two text spans without an explicit connective such as "because," and they are the hardest part of discourse parsing. Most languages have no annotated data for this task. Kurfalı and Östling offer what they describe as the first zero-shot approach. Each argument is encoded with frozen, pre-trained multilingual LASER sentence embeddings. The two vectors are combined DisSent-style (concatenation, mean, difference, element-wise product) and passed to a small feedforward network. Four one-versus-rest classifiers cover the top-level PDTB senses. Training uses English PDTB 3.0, optionally adding the Turkish Discourse Bank (TDB). Testing uses TED-MDB, a parallel corpus of TED talks annotated under PDTB 3.0 guidelines.

On TED-MDB, scores in languages with no training data are close to those for English, so transfer costs little accuracy. Pooling English and Turkish training data significantly beats English-only training for every test language. The method needs no discourse annotation in the target language, so it could in principle cover the roughly hundred languages LASER supports. This makes it relevant to low-resource languages such as Ukrainian.

Key points

  • Data: PDTB 2.0 serves as a sanity check against prior work, and PDTB 3.0 is the main training set. TDB (about 40,000 words, 649 implicit training relations) is added for pooling. TED-MDB (six TED talks) is used only for evaluation in English, German, Lithuanian, Polish, Portuguese, Russian and Turkish.
  • Same-language baseline (PDTB 2.0 test, F1): Comparison 28.19, Contingency 50.63, Expansion 64.07, Temporal 29.22. These scores are below state of the art but adequate as a starting point.
  • English-only training (PDTB 3.0): The average F1 is 45.23 on the PDTB 3.0 test set. On TED-MDB it ranges from 33.52 (Turkish) to 40.96 (English).
  • Pooled training (PDTB 3.0 + TDB): Average F1 rises for every TED-MDB language compared with English-only training, for example Russian 35.00 → 38.23 and German 39.22 → 41.44. Each setup was run 100 times, and the gains are significant (Mann-Whitney U, p < 0.001). Most of the improvement comes from Expansion.
  • Implementation: The code is written in PyTorch and released at github.com/MurathanKurfali/multilingual IDRC. It uses a 100-unit hidden layer, dropout 0.3, the AdaGrad optimizer and balanced negative sampling in each epoch.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-5927

@inproceedings{kurfal2019zero,
  title = {{Zero-shot transfer for implicit discourse relation classification}},
  author = {Kurfalı, Murathan and Östling, Robert},
  booktitle = {Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue},
  year = {2019},
  doi = {10.18653/v1/w19-5927},
  url = {https://doi.org/10.18653/v1/w19-5927}
}

References

17 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rashmi Prasad, Bonnie Webber, and Alan Lee. 2018. Discourse annotation in the pdtb: The next generation. In Proceedings 14th Joint ACL-ISO Workshop on Interoperable Semantic Annotation, pages 87–97. find our card
  2. Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464. find our card
  3. Attapol Rutherford, Vera Demberg, and Nianwen Xue. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 281–291. find our card
  4. Nicholas Asher. 2012. Reference to abstract objects in discourse, volume 50. Springer Science & Business Media. find
  5. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. arXiv preprint arXiv:1807.05154. find our card
  6. Attapol Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 645–654. find our card
  7. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. arXiv preprint arXiv:1804.05918. find our card
  8. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics, 3:329–344. find our card
  9. Attapol Rutherford and Nianwen Xue. 2016. Robust non-explicit neural discourse parser in english and chinese. Proceedings of the CoNLL-16 shared task, pages 55–59. find our card
  10. Man Lan, Jianxiang Wang, Yuanbin Wu, Zheng-Yu Niu, and Haifeng Wang. 2017. Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1299– 1308. find our card
  11. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Attapol Rutherford, Bonnie Webber, Chuan Wang, and Hongmin Wang. 2016. find
  12. Conll 2016 shared task on multilingual shallow discourse parsing. Proceedings of the CoNLL-16 shared task, pages 1–19. find our card
  13. Deniz Zeyrek and Murathan Kurfalı. 2017. Tdb 1.1: Extensions on turkish discourse bank. In Proceedings of the 11th Linguistic Annotation Workshop, pages 76–81. find
  14. Allen Nie, Erin D Bennett, and Noah D Goodman. 2017. Dissent: Sentence representation learning from explicit discourse relations. arXiv preprint arXiv:1710.04334. — [arXiv:1710.04334](https://arxiv.org/abs/1710.04334) find
  15. Deniz Zeyrek, Amália Mendes, Yulia Grishina, Murathan Kurfalı, Samuel Gibbon, and Maciej Ogrodniczuk. 2018. Ted multilingual discourse bank (ted-mdb): a parallel corpus annotated in the pdtb style. Language Resources and Evaluation, pages 1–27. find
  16. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2230–2235. find our card
  17. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 1507–1514. Association for Computational Linguis… find our card

↑ back to top

This page in Markdown →