Elamur Research⌕
☰
home / papers / 2020 / TransS-Driven Joint Learning Architecture for Implicit Discourse Relation Recognition

TransS-Driven Joint Learning Architecture for Implicit Discourse Relation Recognition

Summary

The paper tackles implicit discourse relation recognition: deciding how two neighbouring text spans (Arg1 and Arg2) relate logically when no explicit connective signals the link. The authors argue that earlier neural approaches underuse the gold relation labels and overlook geometric regularities in embedding space. They borrow the translation idea from TransE, originally built for knowledge-graph completion, and adapt it to whole sentences. The result is TransS, where the vector of Arg1 plus a learned relation vector should land near the vector of Arg2. Arguments are first encoded by a stack of attention-equipped BiLSTM layers whose outputs are average-pooled. Two objectives are then trained together. The first, geometric structure learning (GSL), uses a margin-based ranking loss. The second, semantic feature learning (SFL), is a standard softmax classifier. Each objective is meant to improve the shared argument representations used by the other. Experiments on the Penn Discourse TreeBank (PDTB) show results competitive with several state-of-the-art systems. The work matters because it introduces relation-as-translation modelling at the sentence level for discourse analysis.

Key points

  • TransS scoring: TransS scores an argument–relation triplet by the squared L2 distance between Arg2 and (Arg1 + relation vector). It is trained with a margin loss against corrupted triplets, built by randomly swapping either the head or the tail argument, never both.
  • Role of TransS: TransS cannot classify relations directly, because discourse arguments are not reused across instances the way knowledge-graph entities are. It therefore serves to inject geometric structure that guides the SFL classifier.
  • Architecture: The model has four parts:
    • an embedding layer;
    • a multi-level encoder made of stacked BiLSTM layers with an attention controller, followed by average pooling;
    • the GSL module;
    • the SFL module, a softmax over the concatenated Arg1/Arg2 vectors trained with cross-entropy.
  • Encoder depth: In an ablation on PDTB 4-way classification, F1 rose up to three encoder layers and dropped clearly at four and five layers, which the authors attribute to overfitting. Three layers were adopted as the baseline.
  • Evaluation and novelty: The model is evaluated on PDTB. The authors claim it is the first work to uncover the hidden geometric structure underlying argument–relation instances.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.14

@inproceedings{he2020transs,
  title = {{TransS-Driven Joint Learning Architecture for Implicit Discourse Relation Recognition}},
  author = {He, Ruifang and Wang, Jian and Guo, Fengyu and Han, Yugui},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.14},
  url = {https://doi.org/10.18653/v1/2020.acl-main.14}
}

References

23 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 EMNLP, pages 362–371. find our card
  2. Hongxiao Bai and Hai Zhao. 2018. Deep enhanced representation for implicit discourse relation recognition. In Proceedings of the 27th COLING, pages 571–583. find our card
  3. Yang Liu and Sujian Li. 2016. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proceedings of the 2016 EMNLP, pages 1224–1233. find our card
  4. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th NIPS, pages 2787–2795. find
  5. Yang Liu, Sujian Li, Xiaodong Zhang, and Zhifang Sui. 2016. Implicit discourse relation classification via multi-task neural networks. In Proceedings of the 30th AAAI, pages 2750–2756. find
  6. Chloé Braud and Pascal Denis. 2015. Comparing word representations for implicit discourse relation classification. In Proceedings of the 2015 EMNLP, pages 2201–2211. find our card
  7. Deng Cai and Hai Zhao. 2017. Pair-aware neural sentence modeling for implicit discourse relation classification. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pages 458–466. Springer. find
  8. Jifan Chen, Qi Zhang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Implicit discourse relation detection via a deep architecture with gated relevance network. In Proceedings of the 54th ACL, pages 1726–1735. find our card
  9. Zeyu Dai and Ruihong Huang. 2018. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In Proceedings of the 2018 NAACL, pages 141–151. find our card
  10. Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 EMNLP, pages 1602–1613. find
  11. Thomas Meyer and Andrei Popescu-Belis. 2012. Using sense-labeled discourse connectives for statistical machine translation. In Proceedings of the 13th EACL, pages 129–138. find our card
  12. Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. 2014. Recurrent models of visual attention. In Proceedings of the 27th NIPS, pages 2204–2212. find
  13. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 NAACL, pages 2227–2237. find
  14. Emily Pitler, Annie Louis, and Ani and Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the AFNLP, pages 683–691. find our card
  15. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016a. Implicit discourse relation recognition with context-aware character-enhanced embeddings. In Proceedings of the 26th COLING, pages 1914–1924. find our card
  16. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016b. Shallow discourse parsing using convolutional neural network. In CoNLL Shared Task, pages 70–77. find our card
  17. Lianhui Qin, Zhisong Zhang, and Hai Zhao. 2016c. A stacking gated neural architecture for implicit discourse relation classification. In Proceedings of the 2016 EMNLP, pages 2263–2270. find our card
  18. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric P. Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th ACL, pages 1006–1017. find our card
  19. Attapol Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of the 14th EACL, pages 645–654. find our card
  20. Attapol T Rutherford, Vera Demberg, and Nianwen Xue. 2016. Neural network models for implicit discourse relation classification in english and chinese without surface features. arXiv preprint arXiv:1606.01990. — [arXiv:1606.01990](https://arxiv.org/abs/1606.01990) find
  21. Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, and Guodong Zhou. 2019. Topic tensor network for implicit discourse relation recognition in chinese. In Proceedings of the 57th ACL, pages 608–618. find our card
  22. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of the 2015 EMNLP, pages 2230–2235. find our card
  23. Biao Zhang, Deyi Xiong, and Jinsong Su. 2016. Neural discourse relation recognition with semantic memory. arXiv preprint arXiv:1603.03873. — [arXiv:1603.03873](https://arxiv.org/abs/1603.03873) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/he-2020-transs-driven-joint-learning-architecture/