📄 Large Discourse Treebanks from Scalable Distant Supervision

Summary

Discourse parsers are usually trained on a few small, hand-annotated treebanks such as RST-DT, Instr-DT and PDTB. These cover narrow domains and are expensive to extend. The authors generate large "silver-standard" RST-style discourse treebanks automatically, using sentiment as an indirect training signal.

First, a neural Multiple-Instance Learning (MIL) model is trained on Yelp'13 document-level star ratings. From it they extract per-EDU sentiment and attention scores. A CKY-style dynamic program then searches for the tree whose aggregated sentiment best matches the document's gold rating. The plain version yields Yelp13-DT. A scaled version, MEGA-DT, adds:

• a beam per chart cell,
• stochastic exploration for lower subtrees,
• nuclearity prediction.

The Two-Stage parser trained on MEGA-DT outperforms parsers trained on a gold treebank from another domain when tested on RST-DT and Instr-DT. This shows that any sentiment-labelled corpus can be turned into a large, domain-specific discourse treebank without manual annotation.

Key points

• Pipeline. The MIL model of Angelidis and Lapata (2018) is trained on Yelp'13 document sentiment. Its EDU-level sentiment and attention scores drive a CKY tree search that minimises the gap to the gold document label.
• Complexity and beam. Exact CKY takes O(n³) time, but its space grows with the Catalan number. MEGA-DT keeps a beam of 10 trees per CKY cell and uses an RL-style exploration/exploitation trade-off for deep subtrees, so very long documents become tractable.
• Structure scores (Two-Stage parser, RST-DT / Instr-DT).
  • Trained on MEGA-DT: 77.82 / 75.18
  • Trained on Yelp13-DT: 76.41 / 74.14
  • Cross-domain gold training: 74.32 (trained on Instr-DT, tested on RST-DT) and 73.57 (trained on RST-DT, tested on Instr-DT)
• Nuclearity scores (trained on MEGA-DT).
  • 44.88 on RST-DT, against the 44.68 cross-domain gold baseline.
  • 54.87 on Instr-DT, against 49.78.
  • These remain well below in-domain results, e.g. Two-Stage at 72.40 on RST-DT.
• Evaluation protocol. Inter-domain results are averaged over 10 runs, with Bonferroni-adjusted significance tests. A sample generated tree with 72 EDUs looks balanced and non-trivial.

Interesting ideas

• I1. Hidden structure can be recovered by searching for the tree whose bottom-up aggregation of fine-grained predictions best reproduces a cheap, coarse label that already exists; this converts weak labels into structural supervision.
• I2. Capping each chart cell at a fixed number of best partial trees keeps cubic-time chart search usable on long inputs without enumerating the exponential space of trees.
• I3. Near the leaves, the global objective is a poor guide, so allowing random exploration there keeps silver data varied instead of collapsing to one greedy shape.
• I4. A silver resource is judged by training an off-the-shelf model on it and testing out-of-domain on gold data, against a baseline trained on gold data from a different domain.
• I5. Attention weights from the sentiment model double as an importance signal, which supports deciding which unit is the nucleus of a relation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Document-level answers or labels we already hold (e.g., test answers) could score competing candidate analyses, giving an extra witness when assembling silver data.
• I2 (tree): A per-span beam could let us combine sentence-level UD trees into paragraph- or document-level structures without blowing up memory.
• I3 (learn): When producing silver or self-play data, deliberately sampling non-top options for low-confidence local decisions could reduce systematic bias in what the student learns.
• I4 (learn): Our silver treebanks could be validated the same way, comparing against parsers trained on gold UD from another genre as an honest reference point.
• I5 (world): Salience scores like this could help rank which events or states in a text's world are central versus supporting, e.g. for summaries or answering "what is the story about".