📄 Improving Implicit Discourse Relation Recognition with Discourse-specific Word Embeddings

Summary

Recognizing implicit discourse relations is hard because sentence pairs have no connective such as but or and to signal the relation. Earlier neural approaches relied on generic pre-trained embeddings. Those vectors tend to place words like good and bad close together, which blurs exactly the contrasts that matter for discourse. Wu et al. instead learn discourse-specific word embeddings (DSWE) from explicit discourse data, where connectives serve as free, noisy labels.

They mined about 4.92M same-sentence explicit instances from the Xin and Ltw portions of the English Gigaword Corpus, using an existing PDTB parser to detect connectives. A simple network averages the word vectors of each argument, passes the concatenated arguments through hidden layers and predicts the connective. The word vectors it learns become DSWE. These are then frozen inside a classifier (CDRR) for 4-way top-level PDTB relation classification.

DSWE significantly outperforms GloVe and word2vec and improves F1 on every relation, especially the minority classes. Its accuracy is also higher than three earlier systems that exploit explicit data. The result shows that weak supervision from connectives can be absorbed at scale through embeddings rather than added directly as noisy training data.

Key points

• Training data and resource: about 4.92M explicit instances from Gigaword (3rd edition), with a vocabulary of 185,048 words and 300-dimensional vectors. Connective classification reaches about 53% accuracy on its dev set. 96 of the 100 PDTB connectives are used, and the DSWE vectors are publicly released.

• Comparison of embeddings (PDTB test, Sections 21–22):

  Embeddings	Accuracy	Macro F1
  GloVe	55.68	41.27
  word2vec	57.17	40.71
  DSWE	58.85	44.84

• Per-relation gains: F1 for Temporal, Comparison and Contingency rises by roughly 4–7 points over the best public embeddings.

• Comparison with earlier systems (accuracy): R&X2015 57.10, B&D2016 52.81, Liu2016 57.27, CDRR+DSWE 58.85. Liu2016 is slightly ahead on Macro F1 (44.98) but relies on extra labeled RST-DT data.

• Qualitative analysis: in DSWE, the nearest neighbours of good no longer include bad. The cosine similarity is 0.54 for good–great versus 0.33 for good–bad, and words close to not carry negative meaning (e.g., declined).