Elamur Research⌕
☰
home / papers / 2018 / Automatic Prediction of Discourse Connectives

Automatic Prediction of Discourse Connectives

Summary

The paper treats choosing a sentence-initial discourse connective (e.g., however, by then) as a standalone task. Earlier work mostly predicted connectives only as a step toward implicit discourse relation classification. The authors argue the task is useful in its own right for summarization, question answering and writing assistance, wherever separate sentences must be joined fluently.

They mined the English Wikipedia for adjacent sentence pairs in which the second sentence opens with one of 79 PDTB-derived connectives. They added pairs without any connective, giving about 2.9M examples. From these they built balanced splits over 19 frequent, semantically distinct connectives plus a [No connective] class.

They adapted the decomposable attention (DA) model, originally built for natural language inference, and compared it against a WORD-PAIRS baseline and human raters. DA clearly beat the baseline and slightly exceeded human F1 overall. Humans still did better on cases where most raters agreed on an explicit connective, which leaves room for improvement. The public dataset is far larger than PDTB, which makes it suitable for training neural models.

Key points

  • Data: The corpus comes from an English Wikipedia snapshot of 5 Sept 2016.
    • It holds 1.95M pairs separated by a connective and 0.91M [No connective] pairs, about 2.86M in total.
    • It is roughly 70× larger than PDTB, which has 40,600 examples.
    • It is released at github.com/ekQ/discourse-connectives, together with human-filled connectives for 10,000 sentences.
  • Task setup: There are 20 balanced classes (19 connectives plus [No connective]).
    • The training set has 400k examples, 20k per class, built by over- and under-sampling.
    • The development and test sets have 10k examples each, 500 per class.
    • No Wikipedia article contributes examples to more than one split.
  • Model changes: The DA variant differs from the original in three ways:
    • it drops self-attention;
    • it uses 100-dimensional word2vec embeddings that are fine-tuned during training;
    • it adds layer normalization to speed up convergence.
  • Results: DA reaches macro F1 31.80 and accuracy 32.71. WORD-PAIRS reaches 14.81 and 15.60, and random guessing scores 5.00.
    • The abstract reports human raters at roughly 30 F1, against about 32 for DA.
    • The extracted results table, however, lists a lower human score of 23.72. The evaluation details behind this gap fall in the part of the text that was not provided.
  • Analysis: Raters tend to prefer leaving the connective implicit.
    • DA's attention alignments capture lexical and tense cues, such as contrast words signalling however and past perfect signalling by then.
    • The model's confusion patterns closely resemble those of human raters.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 1702.00992

@inproceedings{malmi2018automatic,
  title = {{Automatic Prediction of Discourse Connectives}},
  author = {Malmi, Eric and Pighin, Daniele and Krause, Sebastian and Kozhevnikov, Mikhail},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/3ztvcigon75v},
  eprint = {1702.00992},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.63317/3ztvcigon75v}
}

References

28 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2015). Tensorflow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. find
  2. Ba, J. L., Kiros, J. R., and Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450. find our card
  3. Biran, O. and McKeown, K. (2013). Aggregated word pair features for implicit discourse relation disambiguation. In Proc. find our card
  4. ACL. Braud, C. and Denis, P. (2016). Learning connective-based word representations for implicit discourse relation identification. In Proc. find our card
  5. EMNLP. Di Eugenio, B., Moore, J. D., and Paolucci, M. (1997). Learning features that predict cue usage. In Proc. find
  6. EACL. Elhadad, M. and McKeown, K. R. (1990). Generating connectives. In Proc. the 13th Conference on Computational Linguistics. find
  7. Grote, B., Stede, M., et al. (1998). Discourse marker choice in sentence planning. In Proc. Ninth International Workshop on Natural Language Generation. find
  8. Li, J. J. and Nenkova, A. (2014). Reducing sparsity improves the recognition of implicit discourse relations. In Proc. find our card
  9. SIGDIAL. Liu, Y. and Li, S. (2016). Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention. In Proc. find our card
  10. EMNLP. Liu, Y., Li, S., Zhang, X., and Sui, Z. (2016). Implicit discourse relation classification via multi-task neural networks. In Proc. find
  11. AAAI. Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. In Proc. find our card
  12. ACL. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Proc. NIPS. Parikh, A. P., Täckström, O., Das, D., and Uszkoreit, J. (2016). A decomposable attention model for natural language inference… find
  13. EMNLP. Patterson, G. and Kehler, A. (2013). Predicting the presence of discourse connectives. In Proc. find our card
  14. EMNLP. Pitler, E., Raghupathy, M., Mehta, H., Nenkova, A., Lee, A., and Joshi, A. K. (2008). Easily identifiable discourse relations. In Proc. find our card
  15. Coling 2008: Companion volume: Posters and Demonstrations. find
  16. Pitler, E., Louis, A., and Nenkova, A. (2009). Automatic sense prediction for implicit discourse relations in text. In Proc. find our card
  17. ACL-IJCNLP. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A. K., and Webber, B. L. (2008). The penn discourse treebank 2.0. In Proc. find our card
  18. LREC. Qin, L., Zhang, Z., and Zhao, H. (2016). A stacking gated neural architecture for implicit discourse relation classification. In Proc. find our card
  19. EMNLP. Qin, L., Zhang, Z., Zhao, H., Hu, Z., and Xing, E. P. (2017). Adversarial connective-exploiting networks for implicit discourse relation classification. find our card
  20. Rohde, H., Dickinson, A., Clark, C., Louis, A., and Webber, B. (2015). Recovering discourse relations: Varying influence of discourse adverbials. In Proc. find our card
  21. LSDSem Workshop. Rohde, H., Dickinson, A., Schneider, N., Clark, C. N., Louis, A., and Webber, B. (2016). Filling in the blanks in understanding discourse adverbials: Consistency, conflict, and context-dependence in a crowdsourced elicitation task. In Proc. find
  22. LAW X Workshop. Rutherford, A. and Xue, N. (2014). Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proc. find our card
  23. EACL. Rutherford, A. and Xue, N. (2015). Improving the inference of implicit discourse relations via classifying explicit discourse connectives. In Proc. find our card
  24. NAACL-HLT. Wu, C., Shi, X., Chen, Y., Huang, Y., and Su, J. (2017). Leveraging bilingually-constrained synthetic data via multi-task neural networks for implicit discourse relation recognition. Neurocomputing, 243:69–79. find
  25. Xu, Y., Lan, M., Lu, Y., Niu, Z. Y., and Tan, C. L. (2012). Connective prediction using machine learning for implicit discourse relation classification. In Proc. find
  26. IJCNN. Yung, F., Duh, K., Komura, T., and Matsumoto, Y. (2017). A psycholinguistic model for the marking of discourse relations. Dialogue & Discourse, 8(1):106–131. find
  27. Zhang, B., Xiong, D., Su, J., Liu, Q., Ji, R., Duan, H., and Zhang, M. (2016). Variational neural discourse relation recognizer. In Proc. find our card
  28. EMNLP. Zhou, Z. M., Lan, M., Niu, Z. Y., Xu, Y., and Su, J. (2010). The effects of discourse connectives prediction on implicit discourse relation recognition. In Proc. SIGDIAL. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/malmi-2018-automatic-prediction-discourse-connectives/