📄 Rationalizing Neural Predictions

Summary

Neural text models rarely show why they reach a decision, which limits their use wherever outputs must be checked. Lei, Barzilay and Jaakkola make justification part of learning. The model must back each prediction with a rationale: a small set of input words that forms phrases and, used alone, yields nearly the same output as the full text. Two jointly trained modules do this. A generator gives each token a probability of being kept, and an encoder predicts the target from the kept tokens only. No rationale labels are used. Instead, penalties on the number of selected words and on gaps between selections shape what gets extracted. The authors test the approach on the multi-aspect beer review corpus (McAuley et al., 2012), whose test subset marks sentences relevant to each aspect. They also test it on similar-question retrieval, where rationales act as compressed questions. On sentiment, 96% of extracted words fall in the annotated sentences, well above a bigram SVM and an attention baseline. This shows that explanations can be learned without extra annotation.

Key points

• Rationale extraction accuracy on the beer review data is 96%, compared with 38% for a bigram SVM and 81% for a neural attention baseline.
• The training loss is the squared (L2) error between the encoder's output on the selected words and the gold multi-aspect rating vector.
• The regularizer has two weighted terms:
  • λ1 times the count of selected tokens, which keeps rationales short;
  • λ2 times the number of on/off transitions between neighbouring positions, which favours contiguous phrases.
• The basic generator is a bidirectional RNN that makes a selection decision for each token, conditioned on the whole input. The paper also describes a dependent-selection variant that can model phrases and avoid choosing the same word again.
• Code and data are released at github.com/taolei87/rcnn.

Interesting ideas

• I1. Treat the explanation as a latent variable trained only by the task signal. The model needs no human rationale annotations, so the setup transfers to any labelled text dataset, for example Ukrainian classification corpora that lack explanation markup.
• I2. Keep the selector and the predictor as separate modules. Separating them exposes direct controls over what counts as an acceptable rationale, such as length or contiguity. It also lets the generator be reused with a different encoder.
• I3. Require sufficiency, not just salience. The encoder sees only the selected words, so the rationale must actually support the prediction. Attention weights over the full text offer no such guarantee, which makes this design useful wherever explanations need to be auditable.
• I4. Score rationales indirectly through a downstream task. In retrieval, the extracted text is judged by how well it stands in for the full question. This is a cheap way to evaluate extractive summaries or keyword selection when no gold highlights exist.
• I5. Use a total-variation-style penalty on binary tags. Penalizing switches between selected and unselected positions is a simple trick for getting span-like outputs from per-token decisions. It could be reused in other tagging problems where coherent segments are preferred.